Spending Your Money Wisely on Digital Image Storage
[by Richard Anderson]
With every New Year, digital photographers, (which is pretty much all of us now), face decisions about how to make room for a new years worth of image files.
Most of us have been shooting digitally long enough to know approximately how many gigabytes we create in a year’s time. It has inched up as we traded up to higher megapixel cameras of course, but the good news is that hard drives have become larger and cheaper. So what is the least painful way to accommodate these growing collections?
There has always been a lot of buzz about RAID enclosures, but I would propose that buying the largest available drives and arranging them in JBOD enclosures is the easiest, cheapest, safest, and most workflow friendly arrangement for digital image storage.
I’ll make my case:
• Easiest—Each drive needs to be backed up. Take a 2 TB drive and fill it to approximately 75% capacity. Now do the same with a second 2 TB drive. Now your hard drive storage is properly backed up. Easy!
Take the backup drive offsite for maximum protection. If that isn’t an option, keep the backup drive disconnected and only connect it when you need to synchronize with the primary drive.
• Cheapest—In a JBOD setup- you get full value of the drive cost because you get to use the whole drive for storage. With any RAID other than RAID 0, you lose some portion of the drive space to redundancy. With RAID 5, you will lose the entire capacity of one of those 2 TB drives. On an energy saving note, RAID arrays keep all the drives running when they are on. A JBOD enclosure can have drives on or off independently of each other.
• Safest— If RAID 5 didn’t have to be backed up, you’d come out ahead moneywise, but unfortunately RAID 5 doesn’t protect you from data loss due to equipment failure, file corruption, Fire, theft, or other disasters, so you will still need to back up all the data on the RAID, the same as you would for JBOD. I have discovered that many people don’t back up their RAID 5 setups because they focus on the word “redundant” and figure they’ll take the risk. Since JBOD is cheaper, there is less temptation to “cheap out” and not make your setup truly redundant.
An important fact that impacts safety is that in a RAID 5 setup, when one disk fails, there is a high chance that a second one will fail. The theory behind the error correction in RAID assumes that failures of drives are independent. It is often the case that the drives making up the RAID are the same ages. Since all drives are on when the RAID is on, this means that they all have the same number of hours of use. This means that the chances of failure of all the drives in a RAID are statistically correlated. Occasionally, manufacturers have a run of bad drives. Think about what happens if the RAID is made up of these drives.
• Workflow friendly—RAIDs are great until you run out of space. At that point, you will need to offload ALL of the data on the RAID, rebuild it with larger drives (or get a bigger box), and then restore ALL of the data back. It can take 6 to 8 hours to migrate each 2TB of data using a validated transfer utility. If you skip the verification, you will save some time, but you can never be sure that every “bit” of your data has transferred and is uncorrupted. With JBOD, you can migrate your data to larger drives on a less hectic schedule because you only have to do it one drive at a time.
Data Robotics DROBO enclosures relieve some of the pain of scaling up your storage capacity because you can mix and match different drive sizes. DROBO enclosures still consume more drive space than JBOD however, and the data contained on a DROBO still needs to be backed up.
My advice is to stay away from the fancy boxes. Get a sturdy multi-bay drive enclosure and a double set of 2 TB drives. It’s Easier, cheaper, safer, and more workflow friendly IMHO.
For more information on hard drive storage go here.
12 Responses to 'Spending Your Money Wisely on Digital Image Storage'
Subscribe to comments with RSS or TrackBack to 'Spending Your Money Wisely on Digital Image Storage'.
Good quick overview of a simple data storage plan. Thanks. I would also throw in a DVD backup of your most important shots from a shoot, just in case. And after you take your drives and store them somewhere safe, don’t forget to spin them up to keep the data fresh! http://www.tweakdigital.com/2008/09/spin-your-disks.html
[...] With every New Year, digital photographers, (which is pretty much all of us now), face decisions about how to make room for a new years worth of image files. [...]
I have found that on top of my RAID enclosures, the best way to do off-site storage is to use bare drives and a Newer Tech Voyager Q. You can pop a bare drive in and use it like any other disk and then take them off-site.
It seems to be the cheapest way to buy drives and much safer than DVD’s. I set a schedule to bring in all the drives and spin them up 3 times per year and store them back off site.
For those readers who, like me, have no idea what “JBOD” means, I did the “look-it-up” for you…. it’s “Just A Bunch Of Disks (or Drives)”
I’ve been on the hunt for a perfect back up system and have found the closest thing to it. I back up all my photos right onto by DROBO (which is redundant, any drive fails and the data is still safe) and work off the drives using the Firewire 800 connection.
All my DROBO data is backed up to the cloud via BackBlaze, which runs in the background.
Finally, I use spare hard drives through those plug and play docks (they look like the old Super Nintendo) to back up my lightroom libraries and local data using Time Machine.
It sounds convoluted, but it all runs quietly in the background and I know all my data is safe.
Many thanks for all of your comments. I apologize for not defining JBOD, thanks Jeff for clarifying. I appreciate Jeremy Wilker’s link to information about the deterioration of data on stored drives. We will add this important info to dpBestflow-thanks Jeremy!
I do have a question for George Koroneos. How much data have you sent to the “cloud” and how long did it take to get it there? I would be concerned about the amount of time it would take to get all that data back down if your Drobo failed. Also- what legal liability does the internet backup service have in the event that they have a server problem? Hopefully this is an unlikely event, but there have been cases, like Digital Railroad, of companies giving very short notice that you have to get all your stuff because they are going out of business, or cases like T-Mobile having a server go down and losing thousands of customer’s phone data. I think I’d back that DROBO up with Blu-Ray, DVD, or another set of drives, especially if you have more than a few hundred megabytes on it.
This is not an ad nor do I get “Hit money”…but is this what you are talking about
http://www.cooldrives.com/sataenclosures.html
I am trying to set up something other than the external drives My Book eg.
Matthew-
Yes- good examples of multi-bay enclosures. Some are designed to use trays and some, as Michael Conner mentioned are designed to work with bare drives. If you choose one with bare drives, it is good to get some plastic cases to put the bare drives in for transport. Wiebetech makes some— http://www.wiebetech.com/products/cases.php
We use Burley enclosures with trays. The trays provide enough protection for carrying the drives from my studio to my house where I keep my set of off site backups. http://www.macgurus.com/productpages/sata/SATAEnclosures.php
By the way, a single MyBook external Firewire drive of sufficient capacity is an excellent choice for backing up the system drive of your Mac workstation. Since it is Firewire, it can be used as a boot drive, and it can be easily disconnected and taken offsite, or somewhere secure. —Richard
This is a very helpful post! I need to start using some type of a multi-bay enclosure and have decided to create a JBOD based on this info. Are there any opinions of this OWC unit: http://eshop.macsales.com/shop/hard-drives/RAID/Desktop/
I can’t determine how easy it would be to take a drive in or out. One reason I like it is the 4x interface.
Thanks, Alex
Hi Alex,
Unfortunately the OWC unit (and in fact all OWC units like this) don’t support JBOD- only some form of RAID. You can put 2 drives in it as RAID1 or 0, and the other 2 can be JBOD. We use the Burley enclosures since they are relatively inexpensive and bullet proof, but there are other JBOD options out there.
Best,
Richard Anderson
Richard Anderson,
It took me about 15 days to upload 1.5 TB of data to BackBlaze with a Fios connection running 24/7. If I ever need to reclaim the data. I can either download it, or pay a fee to have them send me the data on either a collection of DVDs or a hard drive.
Like any other cloud back up system, there’s always the chance it could go under or the servers can fail. But the chance of that happening the same time my studio burns down is very slim. Again, the cloud is a back up for my back up, so I’m not as concerned.
Hope that helps.
-glk-
This is why I like staying in touch with my best friend…google. Is because of it I find quality blogs with the info that I need. Thanx for sharing these info with us. Keep up the good work…