Photo Storage Management


Photo storage management is a topic that comes up often. I have participated in numerous discussions on this topic on many photographer community sites. Regardless of what photo management software you use to edit, catalog and manage your photographs and videos, you need a good strategy for physically storing and preserving these valuable digital assets. I am a OS X and Apple Aperture user so you will see references to Aperture Libraries. This process works exactly the same for Lightroom Catalogs so Lightroom users on OS X should feel equally comfortable with this solution. Windows users too should have no problem finding tools for disk drive synchronization so this solution should apply equally to Windows as well as OS X users. While photo storage management is only a part of the larger subject of Digital Asset Management, it is of such importance that it deserves dedicated attention. In this article I share my method for photo storage management.

There are two components to my photo storage management solution: the physical storage media, and a disciplined process for keeping backup copies of my Libraries in sync with my master Libraries. I don't take tens of thousands of photographs a year, I don't shoot thousands of images at a given location, and I don't shoot a lot of video footage that consumes large amounts of high performance storage.  My photo storage management solution is driven by convenience and peace of mind. If you require large capacity (many terabytes) and/or high performance storage (7200 rpm+ drives), this may not be the solution for you. I believe this solution will work for the vast majority of photographers who will read this article.

Physical Storage Media

My physical storage media of choice is 2TB, USB-powered, portable, USB disk drives. I have four of them. I prefer these drives precisely because they are USB-powered, compact and portable, and cheap and easy to replace.

  • These drives don't require external power. They get their power from the USB cable that connects them to my laptop. I can sit in any airport or at any cafe, turn on my laptop, connect my USB drive, and I have access to my entire photo collection.

  • These drives are compact and portable. I can put two drives in my laptop bag along with my laptop and it's power chord, and I am ready to travel anywhere. I don't need space in my bag for disk drive power chords, I don't need power outlets to use my disk drives, and I don't need power converters for them in foreign countries.

  • These drives are cheap and easy to replace. They are commodity items stocked in most electronics stores. For about $100 USD, I can purchase a new 2TB Western Digital MyPassport USB3 drive almost anywhere on the globe. I carry two with me. I keep them in sync as I travel. If one fails, I can replace it on the spot, populate it from the second drive I have with me, and I am off and running again.

As I said, I do not require large capacity, high speed storage. These drives typically have 5400 rpm disks inside them. That speed is sufficient for most of my work. I don't do the majority of my editing and metadata tagging in my master library. The master library is primarily for searching and archiving.

Disciplined Storage Management

I use a disciplined storage management process for maintaining backups of my Libraries and files. This process encompasses ingesting new images from the camera, making selections, applying adjustments and metadata, merging the new images into my master Aperture Library on my master disk, and finally synchronizing that master Library disk to all my backup disks.

When I am ready to import photographs from my camera, I create a new temporary Aperture Library on my laptop's internal HD. I import all my new photographs into this temporary Library. This temporary Library is where I make my selections, delete all of the unwanted images, apply all of my adjustments and metadata (contact info, keywords, location info, captions, descriptions), and apply star ratings and color labels to my liking. The bulk of my work on new images takes place in this temporary Library. Once I am finished processing these new images, I am ready to merge them into my master Library.

To merge new photographs into my master Library, I open the master Library on my master disk in Aperture, then use the menu item File > Import > Library... to import the internal HD temporary Library into my master Library. After this import completes, I then move the new photographs into their permanent locations in the organization structure of my master Library. At this point my master Library is up-to-date so I exit Aperture. Exiting Aperture insures that all of the information in my Library is safely written to my master disk and nothing is going to change it. My master disk is now ready for synchronization to my backup disks.

There are many tools available for synchronizing data between Mac OS X disk drives. I am an old school techie who is comfortable working with Apple OS X Terminal so I use "rsync". The basic premise of this command is "rsync <source drive> <target drive>". So one-by-one, I connect my backup drives to my laptop, execute one "rsync" command per drive, and disconnect the backup drive. This gets all of my backup drives in sync with my master drive. Any changes in my Master Library are now reflected on all of my backup drives. For those that prefer to use the multiple-library photo management, my method still applies provided all of your Libraries are on the same master disk.

Actual 'rsync' screenshot while synchronizing Master drive to Backup drive
Actual 'rsync' screenshot while synchronizing Master drive to Backup drive

Some may find these products more comfortable: ChronoSyncSuperDuper! and Acronis. If you use one of these products, you will select your master disk as the source and one of your backup disks as the target, press a few buttons, and off it goes.

With rsync or any of the other tools I mentioned, the major benefit is that they look at individual file dates, sizes and disk blocks to determine what has changed, and they only copy over the changed bits. This is a very efficient and speedy process for keeping the backup disks in sync with the master disk.

Aperture provides the Vault process for maintaining backups of your Library(s). That works, but can be slow and inefficient. In addition, if you use a Referenced Library, it doesn't include backing up your actual photographs, so you still need a disciplined process for maintaining backups of your referenced files. I use a Managed Library on my master disk as my Master Library. All photographs and their metadata are in my Master Library on my master drive. My disciplined storage management process insures I always have everything on a single, easy-to-use disk, and that my backup disks are also wholly complete and ready to use should my master drive fail.

Benefits

One of the key benefits of my disk synchronization method over using Aperture Vaults is that everything on the master disk is synchronized to the backup disks in a single command per disk. With Aperture Vaults, you have to update the Vault for each Library individually and one at a time. Another key benefit if the disk synchronization method is that I can use a Library on any one of my backup drives at any time without having to restore an entire Vault. This is key for two reasons.

First, if my master drive fails, I have a ready-made backup drive I can begin to use immediately as my new master drive. You may recall I said above that I put two disk drives in my laptop bag when I travel. Because these are commodity disk drives, they do fail from time to time. For me, maybe once a year. Usage pattern dictates "mean time between failure" as it is known in the technology industry so your mileage may vary from mine. Since these drives can fail, I always keep a second drive with me. When I import images into my master drive catalog, I immediately synchronize the master drive to the second drive I have with me. This insures that I have two copies of all the new images I added to my master catalog while traveling. If my master or backup drive fail while traveling, I can pop into any electronics store and pick up another one. After an initial sync from the remaining good drive to the newly purchased drive, I am ready again.

Second, if I happen to delete something from a Library on my master drive and decide I want it back, I can open the same Library on a backup drive, export it to a temporary Library, and merge that temporary Library back into the master drive Library. Of course this only applies if I have not synchronized my master drive to my backup drives. Once synchronization takes place, accidental deletes are lost forever.

So what about those third and fourth disks I mentioned? One stays at my home and one stays at my office. When I return from traveling, I synchronize my master drive to the backup copy I keep at home. When I next go into the office, I synchronize my master drive to the backup disk I keep there. This process insures that I always have backups in three locations. I keep a master and a backup with me, and I maintain a backup at home and a backup at the office.

Conclusion

My choice of USB-powered, portable disk drives addresses the convenience portion of the drivers behind my photo storage management process. The disciplined storage management process I use addresses the peace of mind portion.

Some will ask why don't I use cloud based services like BackBlaze or Carbonite for offsite backups. I don't use them because I am impatient. It would take too long to seed them with an initial backup (days or weeks), and it would take too long to restore from them should I ever need to do so. I also don't want to be dependent upon internet access to retrieve them. With my solution, I can populate a new disk from an existing disk in a matter of hours, I have immediate access to a backup should I need it, and I require nothing I do not have at hand. And by keeping two drives with me when I travel I have multiple copies of all new work.

This is my solution. It works for me. That doesn't mean it is the right solution for you. I want to hear in the comments below what others do for photo storage management. What do you use as your primary storage media? What do you use for backups? How do you maintain them? How often do you update your backups from your primary storage? Do you keep copies in multiple locations? Do you use cloud services for offsite backups?


by Walter Rowe

Freelance Photographer
IT Professional

Share It!

Login required to comment
In the time since I originally published this article, Apple announced their discontinuation of Apple Aperture. I migrated all of my images into Adobe Photoshop Lightroom and then into PhaseOne Capture One Pro. My storage management solution and process has NOT changed one bit. The system described in this article is completely independent of what you use to import, edit, catalog and manage your photographs.