How do I read this? (Image: Mihail Glushkov/Getty)
A YOUNG woman walks through a desolate New York City 1000 years from now. Society has fallen. She comes upon the basement of the New York Public Library. Forcing the door she find caverns of hard drives. The world’s knowledge was once stored here, crucial information for getting by with ease: how to make medicines, how the internal combustion engine works, how to plough a field. But the hard drives are long dead, and she doesn’t have a way to read them anyway.
Advertisement
This scenario is the extreme end of a situation that archivists and data preservationists want to avoid, one in which humanity’s cumulative knowledge is lost. Now, new ways of storing digital information are giving us a shot at preserving our records, so that our descendants can know their past better than we know ours.
Until the 21st century, libraries preserved cultural knowledge in the form of books. But with humans and their computers now generating more data every year than the entire planet did up until 2003, how we store and preserve that data has to change. As time passes, hard drives fail, web pages disappear and valuable data stored by companies can vanish if the firm goes bankrupt. And that’s assuming we still know how to access obsolete formats.
Group 47 in Woodland Hills, California, is working on ways to get around the fact that our drives and discs have a limited lifespan. Instead of writing 1s and 0s as magnetic signals, they write them as microscopic dots onto metal tape, using a laser in a system called DOTS. The tape is then stored in cartridges. A high resolution digital camera can read the data back, but all a future human would need to retrieve the image is knowledge of binary code and a microscope. The firm says the tape should last hundreds of years without degrading and, crucially, doesn’t need any special climate-controlled storage.
The company received funding from a US intelligence agency last year to develop a prototype, and is now raising funds to build a commercial version. The medium is designed to guide someone with no knowledge of hard drives or computers to read its data – all 1.2 terabytes of it. “It’s just like the disc on the side of the Voyager spacecraft,” says Rob Hummel, Group 47’s president.
Hummel says he sees demand for this kind of offline storage that will last indefinitely. For example, the government agency he dealt with wanted to back up the data contained in 40 football fields’ worth of datacentres. “They want to keep it forever – they don’t actually say forever, they say for the life of the republic,” he says.
It’s not just government agencies that want to keep things on record indefinitely. Future historians will need a way of accessing the incredible explosion of information that has taken place over the past 15 years. Ian Milligan, a digital historian at the University of Waterloo in Canada, says digital records, like web pages and personal blogs from before the dotcom boom, give historians insights into the lives of normal people. “Starting from about 1996, we’re collecting billions of digital objects, things documenting the lives of everyday people, their fears, their loves, their thoughts,” he says.
“We’re collecting billions of digital objects, things documenting people’s fears, loves and thoughts”
That means finding ways to store and then search our history is vital. But the scale is daunting and goes beyond web pages. “We’re letting enormous amounts of human information be gathered,” says Jason Scott of the non-profit organisation Internet Archive. “Cellphones, writing, ubiquitous and constant photos.”
New history
Even the seemingly uninteresting data about our working lives that companies store – emails, databases, HR records – could one day be a fascinating source for a historian who wants to learn more about the early 21st century.
But having a medium that lasts without degrading is no good if people in the future can’t read what’s written on it. So Group 47 is also working on storing digital data in a format that will never become obsolete.
“Say we’ve solved the media problem,” says Hummel. “You can put this media on the shelf and it will be there decades later waiting for you. What about the file format issue? If I gave you a WordPerfect file from 1985 that is in perfect condition, you won’t be able to open it, because the software doesn’t exist anymore.”
So Group 47’s Dan Rosen has devised a system called bitplain that doesn’t save data in a specific format. An image, for instance, isn’t stored as a JPEG but is broken down into its smallest components – the 16 or 24 individual bits that make up each of its pixels. The value of each bit – a 1 or a 0 – and their arrangement, is then burned into the metal.
Writing files in bitplain does have a downside, however: it needs more data than other formats. A 4K movie . Each DOTS cartridge holds 1.2 terabytes, so that’s seven cartridges per movie. Archiving the same movie as bitplain would require 10 cartridges. “Two more cartridges, but now you are in a format that lasts forever and there is no fear about being able to retrieve it,” says Hummel.
But for David Rosenthal, who studies data storage at Stanford University in California, this kind of long-lasting medium is missing the point. He believes that we now live in an age where, if data isn’t online, it may as well not exist. DOTS isn’t designed to be searchable and easy to access, and for Rosenthal, that’s a problem.
Rosenthal is working on a way to preserve society’s data by networking many copies of it together. When one copy goes down, due to disc failure, human error or sabotage, the network can replenish that copy from another location. The idea is much like how Google backs up its data, but at a national level. One of the first such national digital archive networks is being built in the UK. Known as , it will ensure that the knowledge accumulated digitally by UK universities perseveres. “If those copies get lost or damaged, they can be replaced,” he says. SafeNet is due to go into service next year.
Relying on electronic data storage has a disadvantage when it comes to post-apocalyptic scenarios, or archaeological ones (see graphic). One problem with recovering data stored electronically is that fragments aren’t useful. Unlike books or DOTS, you either have the whole hard drive or you have nothing.FIG-mg30422901.jpg
Even with the best efforts of people like Scott, Hummel and Rosenthal, a struggle to preserve every last tweet, blog post and Instagram is futile. The data that underpins our digital lives is subject to the same laws of thermodynamics that ensure a broken glass won’t spontaneously reassemble. “I have long ago stopped disrespecting entropy,” says Scott. “Life’s capacity to break things down and take things away. It’s ubiquitous, it’s part of life.”
This article appeared in print under the headline “Archives live forever”
