There is no 100 percent fail-safe image storage system; however there are some simple steps that put the odds resoundingly in favour of a digital image file collection surviving to a venerable old age…
While hard drives and ‘The Cloud’ (which is really just a sexy name to describe an offsite server, accessible only via an internet connection) have their weaknesses, so to do the other alternatives – CDs and DVDs, and flash memory (memory cards and USB flash drives).
The statistics outline the challenge: more than 1.5 million new photos are shared every day on Facebook, WhatsApp and Snapchat alone. That equates to about 550 billion a year – and the number is growing.
InfoTrends’ 2014 Worldwide Image Capture Forecast estimated consumers snapped 810 billion photos worldwide in 2014. This number will swell to 1 trillion photos in 2015 and 1.3 trillion photos by 2017. And according to Canon Australia, a full two-thirds of photographers surveyed have lost digital image files due to ‘the damage or failure of physical technology’.
The issue received some high profile attention earlier this year when Dr Vint Cerf, VP and ‘chief internet evangelist’ with Google, and one of the recognised pioneers of the internet, addressed the American Association for the Advancement of Science, on what he saw as a looming ‘Digital Dark Age’.
‘In our zeal to get excited about digitising we digitise photographs thinking it’s going to make them last longer, and we might turn out to be wrong,’ he said. ‘I would say if there are photos you are really concerned about create a physical instance of them. Print them out.’
He noted there were two threats to images which existed only as digital files:
– the evolution of technology and subsequent obsolescence of various storage media (imagine if your images were stored to floppy disc);
– and something called ‘bit rot’ – the slow deterioration in the performance and integrity of data recorded to storage media.
Cerf called for the development of a project to preserve old software and hardware so that out-of-date files could be recovered – no matter how old.
Georgia McCabe, CEO of the Photo Marketing Association in the US (herself a key player in the development of Kodak Photo CD technology back in the day) has talked about her concern that we’re creating a ‘digital landfill in the sky’ and how turning that around should be a priority for the photo industry.
The message was further reinforced at the time by a local academic, Dr Chris Chesher, senior lecturer in Digital Cultures at the University of Sydney, who said that computer hard drives had a life of around five years or less, while storing digital images in ‘The Cloud’, Facebook, Flickr, etc – offers no guarantee that your precious memories are safe.
– If banks had the same ‘all care and no responsibility’ policies as the image storage services businesses, well there would be a lot more cash stashed under mattresses.
‘When you look at the terms of service of a lot of the cloud companies, they say a lot of friendly things [about how] they’ll look after their data,’ he told 702 ABC Sydney.
‘But when it comes down to it – terms of service – basically, “we’re not going to take any responsibility at all”.’
‘The recommendation is to diversify the instances of that data, so printing out your photographs is one of the ways. Even if your hard drive goes down, your cloud service disappears or gets corrupted, you still have the hard copy. Paper is still one of the best archiving forms that we have.’
– So before we plunge into the digital storage alternatives, the message is that photographs represent an excellent ‘archival medium’ in themselves.
Memory cards
In addition to the sheer volume of digital image files, cameras are getting increasingly memory hungry. A camera like the Samsung NX1 is capable of nine frames per second at full 28-megapixel resolution, when shooting in the fat, data-rich Raw format. Canon has launched a couple of 50-megapixel DSLRs. Shooting at 5fps in RAW, they will generate around 250MB of image file data per second. And there are an increasing number of models offering 4K video, which also demands lots of storage space and fast transfer speeds.
So the new generation of high performance cameras need cards which are both high capacity and speedy in transferring data. It’s not something photographers should skimp on. Cheap cards are more liable to be faulty and not likely to have the fastest read-write speeds. Reputable brands like Lexar, Sandisk and Kingston tend to be first to market with high capacity, fast memory cards. They also offer file recovery software to retrieve images from a card if it fails.
Unfortunately the SD Association has made a dog’s dinner of describing memory card performance. Perhaps that’s a positive for photo specialists. The picture is so muddy that consumers need some guidance, and who better to provide that than a knowledgeable camera store staff member…
First, there’s the basic capacity issue. SD cards now pretty well obsolete – at a max of 2GB, they are too limited in capacity for modern cameras. They’ve been superseded first by SDHC which offers up to 32GB capacity and more recently by SDXC which can support capacities of up to 2TB (2000GB).
SD and SDHC-compatible cameras can’t use SDXC memory cards, although SD memory cards and SDHC memory cards will work in SDXC-compatible cameras.
For the shrinking number of DSLRs using CompactFlash cards, UDMA 7 cards deliver the best performance when capturing high resolution images and HD video. These cards are compatible with any camera that uses CompactFlash card storage.
CFast is the emerging version of card format from the CompactFlash Association and provides a higher maximum transfer rate than CompactFlash cards. CFast cards are similar in size to CompactFlash cards, but do not have any pins which can bend in the connector. The format was primarily designed to support the capabilities of the next-generation of camcorders and DSLR cameras.
Another new and improved version of CompactFlash is XQD, developed by Sony, Nikon and Sandisk. Only the Nikon D4 and D4s and some Sony broadcast cameras use the format. Only Sony and Lexar manufacture XQD cards.
CFast cards are not backward compatible with CompactFlash or CFast cards.
Apart from card capacity, the other key specification is speed – if the data can’t be written to the card sufficiently fast it will slow down a camera’s frame rate or result in glitchy video.
Cards sometimes have a multiplication factor written on them (such as 133x, 200x, 800x, 1066x, etc) which usually represents read speed. Sort of…when you multiply that number by 150 KB/sec, which is apparently the original speed of CD-ROM drives.
Then, having abandoned this obscure measure, the SD Association moved to a ‘Class rating’ (and proceeded to muck that up). Class rating focusses on the minimum data transfer rate. The read speed is how long it takes to open a file on the card. More importantly for photographers, write speed is how long it takes to save image files to the card. And the higher the Class number, the faster the read/write speed performance.So a Class 2 card can handle sustained writing of data at a rate of 2MB/sec; a Class 4 card achieves 4MB/sec; a Class 8 card 8MB/sec, and so on. But...this is the minimum rate rather than the actual rate. A Class 4 SDXC card may also claim it a speed of 15MB/s – which will be the maximum rather than sustainable speed.
So the SD Class system was going along nicely until you get to Class 10, at which stage the SD Association branched off to UHS Speed Class I and 3 (UHS-I and UHS-II) , which is focussed on video capture. (Who knows what happened to UHS 2!)
It’s an unappetising alphabet soup. Here’s how one leading memory card manufacture attempts to describe one of its top-of-the-line SDXC cards:
‘These high-speed cards leverage UHS-II technology (UHS Speed Class 3 (U3)) and provide a read transfer speed up to 2000x (300MB/s) when used with the included SD UHS-II reader. For versatility, the cards also work with UHS-I devices at UHS-I speeds, and they’re backwards compatible with older cameras and readers, performing at Class 10 speeds when used with non-UHS devices.
– Got that?
UHS cards offer minimum write speeds to successfully record HD-and-above video. They are not compatible with all cameras. And there’s a bit of an overlap between Class 10 and UHS-1. Confused? This is how the SD Association explains it: ‘It’s important to remember that Speed Class and UHS Speed Class are two different speed indication symbols for different devices. However, a UHS-I memory card may also indicate a Speed Class.’
…Now you must be confused – or you haven’t been paying attention!
Fortunately, camera manuals will recommend the minimum speed class card for a camera. Moving to a higher rated card may deliver some increase in speed, especially shooting RAW, or following a burst of continuous shooting. A lower speed card will drag down the overall responsiveness of a camera.
Spending $1000 on a high-resolution camera with fast continuous shooting or 4K video and then quibbling about spending an extra $50 or $100 on a card which will allow it to perform at its capacity is clearly a false economy.
The high capacity of memory cards can be a hindrance to digital image file storage because by the time the card is full, the task of sorting through the images is massive. According to Sandisk, a 16GB card will store 286 RAW images or 2861 lossless JPEGs from a 16-megapixel camera. Sorting through and organising 200-300 images is a chore; 2000 will go into the too-hard basket for most of us.
So delete early and delete often – don’t make the task harder by carting visual rubbish about in your camera for weeks or months on end. At least then when it comes to the organisational task, you’ve already sorted the potential keepers from the dross. The job is half done.
It’s also worth considering using several smaller capacity cards rather than a big whopper. That forces you to attend to the transfer to a storage medium more frequently. It also means that at any given time before transfer to storage, you haven’t got all your eggs in one basket.
Lexar has a Professional Workflow line of products which is effectively a flash memory-based external storage system. It includes compact four-bay reader and storage drive hubs, a selection of card readers, and storage drives. It can be used to speedily offload multiple memory cards to a PC or back up externally to a 256 or 512GB USB 3 storage drive.
Costs aren’t too prohibitive, and there’s something comforting about an achival system which doesn’t spin around at thousands of RPMs.
Hard Disk Drives
While a PC is the logical tool for organising a digital image file collection, it’s the worst possible place to store it.
Most personal computers are used on a daily basis, so there’s plenty of extra wear and tear on the disk drive. Notebooks can be damaged, lost or stolen. Many people upgrade their computers every two or three years, which makes them a fairly temporary spot to store a lifetime image collection anyway.
A study by US online backup company BackBlaze published last year and based on three year’s data from its installed base of 27,000 HDDs, indicated that in the first 18 months of their lives, consumer grade hard disk drives had an average failure rate of 5.1 percent.
For the next 18 months the failure rate drops to 1.4 percent. After three years, the failure rate explodes to 11.8 percent per year. So after six years you have around a 50-50 chance that the hard drive will have failed. (See chart at beginning of this article.)These are not good odds if a computer hard disk failure is all that stands between you and the loss of your precious digital image files.
External hard disk drives will have a longer life because they aren’t worked as hard. On the other hand, being small and portable, they have more chance of coming to a sticky end – being dropped, lost or stolen, for instance. With 2TB of storage for around $150, external HDDs represent great value. At those kind of prices, it’s worth considering having one external drive for everyday back-up for your PC, and a separate drive on which you build your image file collection. There will be an extra copy by default on the drive used for regular computer backups.
Just back to that BackBlaze study: there were some surprising differences in failure rates among the three brands being used: Hitachi (now HGST), Western Digital and Seagate. Overall, HGST was the most reliable, with Western Digital coming in a comfortable second and Seagate having more failures, especially with its 1.5TB drive. An update of the data from January this year showed HGST maintaining a healthy lead in reliability as BackBlaze moved to 4GB drives.
The new kid on the block slowly replacing hard disc drives are Solid State Drives (SSDs). They are a close relative of the USB drive or thumb drive, which in turn are part of the flash memory family SD and CompactFlash cards belong to.
They are more robust than HDDs, as they have no moving parts. Drop one and it’s still likely to work. HDDs not so much. SSDs are faster at transferring data than hard drives. SSDs also don’t have spinning disks, which draw a lot of power when reading and writing data. But on the other hand, hard drives provide more storage for the dollar.
Samsung’s 250GB external SSD is priced at $180, a 500GB drive is $300, and a 1TB drive is $600. By comparison, a 1TB magnetic drive (HDD) can be acquired for under $100. Maybe next year…
CDs and DVDs
Don’t be put off by stories of CDs deteriorating after two years or so. If you purchase good quality, premium-branded media and take some reasonable precautions, CDs and DVDs should last for a decade or more. Once again, this is not something you can afford to skimp on.
Alarming stories about CD decay are often generated by vendors of alternative technologies – so a ‘storage expert’ from, say, IBM (which is big in magnetic tape storage) will predict CD lifespans of between 2 to 5 years. On the other hand, Verbatim says that, stored properly, its Ultralife Archival Gold CDs and DVDs will last for 100 years.
Here are some estimated lifespans for various optical media from independent website CD-Info.com:
– Unrecorded CD-R and CD-RW: 5-10 years
– Recorded CD-R: 50-200 years
– Recorded CD-RW: 20-100 years
– Recorded DVD-R: 30-100 years
– Recorded DVD-RW: up to 30 years
– Recorded BD-R and BD-RE (Blu ray): 30-200 years
Having purchased premium discs with archival properties, here are some tips for extending optical disc storage lives:
– For maximum CD longevity, look for media with a gold reflective layer;
– Treat CDs and DVDs with care – hold them by the outer edges or the hole in the centre, don’t touch the surface, avoid scratches, and keep dirt from the disc;
– Keep them in a dry, dark, and cool place since humidity, sunlight, high temperatures, and pollutants can damage the different layers;
– Store them in jewel cases rather than paper slips;
– Use non solvent-based felt-tip permanent markers, suitable for writing on CD or DVD labels;
– Rewrite your rewritable discs as little as possible;
– Choose slow writing speeds to reduce errors and increase quality.
While optical discs have a reasonable lifespan, there are two other issues which are combining to reduce their usefulness: At under IGB and 4.7GB respectively, CDs and DVDs are lacking the capacity for the larger and larger files being generated by high resolution camera sensors; and disc drives and burners are gradually disappearing from PCs.
Beyond archival quality DVDs, there’s another format called M-Disc, developed by Millienniata, for which a 1000 year life is claimed. These are available in 4.7GBG DVD and 25GB Blu ray versions. Verbatim sells DVD and Blu ray M-Discs. The M-Disc media must be burned with a M-Ready drive, but can be read by any DVD or Blueray drive.
‘The Cloud’
Dropbox, Flickr, Bitcasa. Google Drive, Microsoft OneDrive, Smugmug, Canon Irista and Amazon are among the plethora of businesses who want your images stored on their servers.
Amazon put the cat among the pigeons in March by offering an ‘Unlimited Photos Plan’ for just US$11.99 per year. This allows storage of unlimited photos, including some RAW file types (Nikon NEF, Canon CR2 and Sony ARW). Other file types, such as JPEG, BMP, PNG and ‘most’ TIFF files are accepted, but any photo over 2GB is not.
This is just a fraction of what the other providers have been asking. For instance, Dropbox is asking $10.00 per month for up to 1TB or storage. However, for Windows 365 subscribers, Microsoft OneDrive offers 1TB of storage as part of the monthly subscription fee.
For those concerned about privacy and security, NZ-based Mega might be worth a look. It provides encryption in every part of the process, so anything you send to the cloud is encrypted locally, enroute, and on the destination server. What’s more, Mega itself doesn’t have any way of accessing your information, as you hold the encryption key so that anything stored on Mega is only able to be opened by you. Mega offers 50GB of storage free, with 500GB for 99 Euros ($140) per year, 2TB for 199 Euros per year, or 4TB for 299 Euros per year.
So is The Cloud The Holy Grail for long-term image storage? This is what Mike Ashenfelder, a digital archiving expert from the US Library of Congress had to say:
‘Cloud services relieve you of the responsibility of tending to storage hardware, but your collection becomes inaccessible if the network connection is disrupted.
‘No online backup service is as reliable as a storage device that you can see and touch. Cloud storage should only be a secondary backup option.’
Finally, there’s what’s being termed ‘personal cloud’ storage, which is a hybrid between an external HDD and a cloud service. Files are stored on your local networked drive so there is full control over your image files, yet they are remotely accessible (so long as you have an internet connection).
Rather than having your data stored at a third-party server farm in the US, it is right next to your router, on a Network-Attached Storage (NAS) device – generally a hard disk drive. The data is password protected and encrypted, and when you delete something you know it’s gone. And once you buy the equipment, there are no annual storage fees.
Unlike desktops and laptops, a NAS device offers a simple OS that focuses on doing a small subset of tasks as efficiently and securely as possible.
Once the Personal Cloud library is set up on the drive, it can be accessed from any networked computer or device, or remotely using Wi-Fi or Internet connections on your phone, tablet or laptop.
So what’s best? Here’s Mike Ashenfelder again: ‘There is no “best” storage medium. CDs, DVDs, flash drives, solid-state drives, spinning-disk hard drives, tape and networked cloud storage all have benefits and drawbacks. CDs and DVDs are light, flat and easy to store but they can be scratched and damaged. Flash drives do not have the moving parts that spinning disks do but they are more affected by extreme temperatures. Spinning-disk drives – the kind that “whir” when you turn them on – have a large storage capacity but you could easily damage them just by dropping them.
‘The best strategy is to backup copies onto at least two different types of media and store a copy in a different geographic location in case some disaster strikes your home or office.
‘In fact, professional photographers – who have a financial stake in the accessibility and safekeeping of their digital photographs – have a “3-2-1” rule: make three copies, store two on different types of media and one in a different location.
– But as that keen snapshooter General Patton once said: “A good plan violently executed now is better than a perfect plan executed next week.” In reality, many (most?) of us aren’t going to make three copies on two different types of media and store one in a safety deposit box – it’s just not going not happen.
But at the very least, delete the photos you don’t want, print the photos you love, and store the rest of your collection safely out of harm’s way.
Finally, here’s a comment from a reader responding to a Wall Street Journal article extolling the virtues of cloud storage: ‘Last year on my iMac, I opened iPhoto and found that four years of my photos had disappeared. This included our daughter’s wedding, the first year of our grandson, trips and family gatherings. As my iMac was out of warranty, I paid Apple $19 hoping we could get them back. My WD external hard drive had also failed. I spent hours talking with 5 different Apple Techs over several days. The last guy finally found them and worked with me to restore them.
‘The upshot: there is no perfect storage system. NONE. A week later, I spoke with an LA Times photographer; he and the Apple guy BOTH say, PRINT them to keep in a frame or a book. In iPhoto, they are not really photos (they are similes in essence); NO CD, DVD, Cloud, or computer will keep digital photos safe over time. Digital photos degrade! I was lucky.
‘NOT chancing it again, I make prints and photo books of those I really care about. Others that are a passing fancy, I put on FB or Instagram, then delete them. Be forewarned!’
Be First to Comment