IT managers have a hard enough time managing the data users need today, let alone the data they will need 100 years from now.
And yet, according to a recent study published by the Storage Networking Industry Association (SNIA), 80 per cent of organizations believe they have a need to retain electronic records for more than half a century, and more than half want some way of storing information permanently.
For most enterprises, it’s just not going to happen.
Data formats, storage technologies and user behaviours change too much over time to make permanent archiving of all data a realistic project. The industry has also tended to focus first on developing ways to produce, capture, share and (sometimes) secure data long before storage is addressed.
Back in 2002, for example, Claire Tristam wrote in MIT’s Technology Review about what the industry was doing then to ensure permanent storage.
“Proposed solutions include migration, which consists of updating or sometimes entirely rewriting old files to run on new hardware; emulation, a way of mimicking older hardware so that old software and files don’t have to be rewritten in order to run on new machines; and more recently, encapsulation, a way of wrapping an electronic document in a digital envelope that explains, in simple terms, how to recreate the software, hardware or operating systems needed to decode what’s inside,” she wrote, adding that none of these were guaranteed to work.
In a recent Computerworld U.S. online article, John Webster of research firm Illuminata made the case for long-term archiving in response to SNIA’s plans. “We rarely if ever think of saving our digitized thoughts for the sake of posterity,” Webster said. “But for the sake of historians, lawmakers, sociologists and scientists yet to be born, we should.”
Really? IT managers aren’t always in the best position to judge what needs to be preserved for years and what can be disposed of after a few days. This is one of the reasons compliance with regulations, which often demands storage of certain data for a period of several years, has been such a headache.
No one knows how future generations will use or require old data, but storing it provides some level of security and comfort that we won’t have to reinvent the wheel. IT managers, however, may need to provide their counterparts in the enterprise a reality check, explaining how moving data from one obsolete format to newer ones will add incremental costs, drain more resources and in some cases provide little benefit.
Instead, we need to develop a process of evaluating data more effectively to determine what is worth preservation and what can be discarded. That may become a much more valuable effort than pinning our hopes on an uber-vault that may never come to pass.