Digital workflow

A while ago when I got my camera (well, the “real” camera) and the first week of euphoria and countless of useless pictures passed, I realized that I need some sort of archiving mechanism. So I did some research and after much reading I arrived at the solution that I used ever since. It’s a combination of what I read on various sites and blogs (unfortunately I don’t have their addresses any more) and some of my own ideas.

I usually work either on my computer at home or on my laptop when I am away. I could either keep the images on the laptop/desktop and sync them when I am at home or just work on a portable hard drive. The portable disk made more sense so I got myself a 320GB Seagate FreeAgent Go USB drive, storage is quite cheap these days:

The drive is nice, physically small, fast and big enough for what I need. I just connect it to whatever computer I happen to be using, the desktop at home or the laptop while I am on the go.

To manage the photo collection I use Adobe Lightroom, which is in my opinion the best tool for the job. Lightroom organizes photos in “catalogs” and I personally have 4 of them:

  • Incubator: this is where I import the pictures from the camera. Once the pictures are imported, I go through them and delete the trash right away. Then, the most important step, I edit the pictures’ Metadata which is made very easy by Lightroom. On import I already insert common copyright info (my name, website, etc) but now I also update the image location, the city and country. Based on this info and the image timestamp, Adobe makes managing very large collections quite easy.

    After I do that, I work on the images and I edit them in Lightroom as needed (brightness, contrast, exposure, noise reduction, sharpening, color correction, etc). Depending on what I want to do, I might send them to an external editor for further processing: Photomatix for HDR or maybe Adobe Photoshop for more complex editing. Only after the processing is done I decide what to do with the image. I use 2 color labels for that:

    • Green: images which I consider to be “keepers”, this is where I choose the pictures I might publish on Deviantart or other similar websites.
    • Blue: personal images. They are things like portraits, family pictures, and so on.
  • Selected: This is where the images labeled as “Green” go.
  • Personals: This is where the images labeled as “Blue” go.
  • All Raw: This is where I copy every image from the Incubator catalog. It includes images of both blue and green labels as well as the raw images. This is a sort of backup catalog and I don’t usually look here often unless I need the raw version of a image to recreate something. This is also the biggest catalog since it contains all the pictures I made so far.

Each catalog has its own folder on the USB drive and images are organized based on the date: Year – Month – Day. That is done automatically by Lightroom when the images are imported from the camera. The powerful filtering makes it quite easy to sort through thousand of image:

I almost always shoot in RAW and not in JPEG. Yes, the resulting image can be up to 20MB, much bigger than a JPEG and you need to spend some time to post process it in Lightroom but you are much more flexible and the end result is far superior. I only export JPEG copies of the images I want to upload on the web. So far I don’t have problems with the space, the biggest catalogs are the AllRaw and Selected, with 42GB and 17GB.

Once the images in the Incubator are copied to the other 3 catalogs and I make sure they are correctly imported there, I empty the Incubator. This whole process works ok for me. The portable hard drive makes it easy to switch between laptop and desktop, where I have the same tools installed.

Of course, backup is important too. The catalogs themselves (which are actually the Lightroom databases) are backed up on my home NAS server, as well as the physical images. I may switch soon to backup to the cloud backup everything on the Jungledisk account. The whole portable drive is fully encrypted with Truecrypt  so even if I somehow loose it, nobody can really rummage through my personal collection.

That’s about it. I don’t know if this is the best solution and I am sure it does not work  for everybody or it might be overkill for some but it suits my purposes ;-)

10 Responses to “Digital workflow”

  1. aka says:

    I have the same USB hard drive with 500GB though and I’m very pleased with it as well. Your whole process seems very meticulous :).

  2. Stefan says:

    It does look like it on paper, doesn’t it ;-) It’s slightly more chaotic in reality.

    And yes, these drives are quite nice and at least mine, extremely quiet

  3. Radu says:

    Interesting workflow. I mainly use my desktop at home for image processing, mostly due to its 24″ screen, but I have used the laptop when traveling on vacation and in other circumstances.
    When I use the laptop I tend to mostly finish the processing on the laptop and then export the relevant files as a catalog, importing them in the main catalog on the home machine.
    I didn’t see the need for separate catalogs yet (and I use Lightroom since the first beta), even if my catalog is somewhat larger than yours.

    I’m interested in backing up the home computer (including the photos) and I was considering a NAS drive instead of an external HDD. The link to your previous post on the NAS drive is broken, so please let me know about your experience and recommendations in this area.

    • Mihai says:

      I have a Netgear DUO (www.readynas.com) and I’m extremely pleased with it. 2 1TB drives in XRAID5, enough to store all the junk on it (plus the nightly backups), and stream it through the house, as needed. Plus, the drives are hot-swappable, so upgrades can be made on the fly, with automatic resizing of the partitions.

      • Stefan says:

        2 Drives in XRAID5 ? How is that working ? I thought the DUO can only hold 2 drives so RAID5 is not really supported.

        You mean RAID1 with X-RAID hot-swappable feature implemented by Netgear ?

        • Mihai says:

          That’s exactly what it is. For whatever reason, they refer to this as XRAID5 on their documentation. But, the architecture is pretty nifty. Say you start with a 500Gb drive, and sometimes later you plug in a second. Poof, automatically you get mirroring. Then you take one out, replace with 1Tb. Now you still have mirroring @500Gb. Take out the second 500Gb and plug in a second 1Tb, and you automagically get resizing and mirroring to 1Tb. Rinse, repeat as necessary.

          • Stefan says:

            That is a nice feature indeed. I am really close the the 1TB limit on my NAS and there is no easy solution to upgrade the drives to 2TB.

            I mean I can just put in 2x 2TB drives, build the empty mirror then just manually copy the data from the old drive. But now that I think of it, time-wise it’s probably close to the solution in your NAS. I mean if you replace one of the 500GB drives with a 1TB drive, the whole mirror building process takes time. And then you have to do it again when you put in the second 1TB drive.

            *Note to self* You can nest comments only so long before it starts looking shitty. Need to adjust this.

  4. Stefan says:

    That’s odd, the link seems to work for me. Here it is again:

    http://www.nektulos.de/2009/12/viata-digitala-si-backupurile/

    I would highly recommend it as opposed to an external hard drive. The NAS server I use has 2 1TB drives inside in RAID1 configuration (mirror) so you have redundancy if one of the disks goes bad. There are many such servers on the market, it does not have to be from Dlink.

    • Radu says:

      I read your previous post and checked the prices on the Romanian market. I decided to go with an external hard drive for 2 reasons:
      1. I just wanted a backup for the internal hard disk of the home computer, thus having the data on 2 disks (1 internal & 1 external). The NAS with 2 drives would have given me the same data on 3 disks (1 internal and 2 external) thus little extra protection.
      2. Price: a simple 1TB external hard disk will cost me about 80EUR, while the NAS with 2 1TB drives would have cost me about 300EUR for the same result: 1 TB of external backup. I decided to buy an APC UPS for the difference.

      I decided a NAS would be a very interesting option when I’ll have more data to store and when we’ll have at least 2 computers used at the same time in the house (not far, but not here yet).
      But then I would spring for a unit with more disks. What I really like is the Drobo FS (5 drives, BeyondRAID technology, protection for failure of up to 2 drives, etc) but it’s expensive (about 700-800EUR currently – without drives, of course).

  5. Stefan says:

    Yes, in your case I guess it makes more sense to go with an external drive.

    In my case there are 4-5 computers in the house which uses the NAS. That’s also where we have our movie collection, which we watch via a network media player connected to a TV.

    But if you only want to back your internal drives up, then I guess this is a bit overkill for the time being.

Leave a Reply