An automated system for storing large amounts of information in glass could change the future of data centres.

Our world runs on data, from the internet and readouts of countless industrial sensors to scientific data from particle colliders, and all of it must be stored safely and efficiently.

In 2014, at the University of Southampton in the UK and his colleagues showed that lasers can be used to , thus creating a data storage method that could last longer than the age of the universe.

Their method was too impractical to be scaled up to industrial size, but and his colleagues at Microsoft’s have now demonstrated a similar glass-based technology that might lead to long-lasting glass data libraries in the near future.

“Glass can withstand extreme temperatures, humidity, particulates and electromagnetic fields. On top of that, glass has a great lifespan and doesn’t require replacing every couple of years. That makes it a more sustainable medium as well. It requires very little energy to make and it’s easy to recycle when we’re done with it,” says Black.

The team’s process starts by using femtosecond lasers, which emit light pulses lasting quadrillionths of a second, to convert data into tiny structures etched into thin glass layers. When turning bits of data into these structures, the team also added extra bits that ensured fewer reading and writing errors.

Free newsletter

Sign up to The Daily

Sign up to newsletter

The data could be read with a combination of a microscope and a camera, whose images were then passed to a neural network algorithm that converted the information back into bits. The whole process was easily repeatable and automated, making a case for robotically operated data facilities.

The researchers managed to store 4.8 terabytes of data in a square piece of glass 120 millimetres wide and 2 millimetres thick – equivalent to roughly 37 iPhones’ worth of storage in about a third of the volume of one.

Based on accelerated ageing experiments, such as heating the glass in a furnace, the team estimated that data could remain stable and readable for more than 10,000 years at 290°C and even longer at room temperature. Additionally, the researchers tested their method with borosilicate glass, which is cheaper than standard glass, but could only accommodate less complex data.

Kazansky says the main breakthrough of Project Silica is that it offers an end-to-end system that could be scaled up to the level of data centres. The physics principles behind glass-based data storage have been known for more than a decade, but the new work confirms that it can be turned into a viable technology, he says.

Microsoft isn’t the only firm interested in pushing this technology towards the mainstream. Kazansky co-founded a company called that has, for example, stored the human genome in a piece of glass. An Austrian start-up called similarly offers to store large amounts of data in ultra-thin layers of ceramic and glass.

Still, questions remain, for instance, about the cost of integrating glass libraries into existing data centres and whether the Project Silica team can increase the capacity of its glasses, which ought to reach up to 360 terabytes based on the work of Kazansky’s team.

Black says the clearest potential applications for Project Silica’s technology right now are anywhere data must survive for centuries, such as national libraries, scientific repositories or cultural records. Working with companies such as Warner Bros. and the Global Music Vault, his team has also begun to explore storing data that is meant to be kept indefinitely and currently resides in the cloud, he says.

Kazansky says that the technology was even featured in the film Mission: Impossible – The Final Reckoning, where the protagonist found it capacious and safe enough to trap a villainous artificial intelligence. “It is a rare moment where Hollywood’s sci-fi is actually based on our peer-reviewed reality,” he says.

An artificial intelligence model trained on the medical data of 57 million people who have used the National 91ɫ��Ƭ Service in England could one day assist doctors in predicting disease or forecast hospitalisation rates, its creators have claimed. However, other researchers say there are still significant privacy and data protection concerns around such large-scale use of health data, while even the AI’s architects say they can’t guarantee that it won’t inadvertently reveal sensitive patient data.

The model, called Foresight, was first developed in 2023. That initial version used OpenAI’s GPT-3, the large language model (LLM) behind the first version of ChatGPT, and trained on 1.5 million real patient records from two London hospitals.

Now, at University College London and his colleagues have scaled up Foresight to create what they say is the world’s first “national-scale generative AI model of health data” and the largest of its kind.

Foresight uses eight different datasets of medical information routinely collected by the NHS in England between November 2018 to December 2023 and is based on Meta’s open-source LLM Llama 2. These datasets include outpatient appointments, hospital visits, vaccination data and records, comprising a total of 10 billion different health events for 57 million people – essentially everyone in England.

Tomlinson says his team isn’t releasing information about how well Foresight performs because the model is still being tested, but he claims it could one day be used to do everything from making individual diagnoses to predicting broad future health trends, such as hospitalisations or heart attacks. “The real potential of Foresight is to predict disease complications before they happen, giving us a valuable window to intervene early, and enabling a shift towards more preventative healthcare at scale,” he told a press conference on 6 May.

While the potential benefits are yet to be supported, there are already concerns about people’s medical data being fed to an AI at such a large scale. The researchers insist all records were “de-identified” before being used to train the AI, but the risks of someone being able to use patterns in the data to re-identify the records are well-recorded, particularly when it comes to large datasets.

Free newsletter

Sign up to The Daily

Sign up to newsletter

“Building powerful generative AI models that protect patient privacy is an open, unsolved scientific problem,” says at the University of Oxford. “The very richness of data that makes it valuable for AI also makes it incredibly hard to anonymise. These models should remain under strict NHS control where they can be safely used.”

“The data that goes into the model is de-identified, so the direct identifiers are removed,” said at NHS Digital, speaking at the press conference. But Chapman, who oversees the data used to train Foresight, admitted that there is always a risk of re-identification: “It’s then very hard with rich health data to give 100 per cent certainty that somebody couldn’t be spotted in that dataset.”

To mitigate this risk, Chapman said the AI is operating within a custom-built “secure” NHS data environment to ensure that information isn’t leaked out of the model and is accessible only to approved researchers. Amazon Web Services and data company Databricks have also supplied “computational infrastructure”, but can’t access the data, said Tomlinson.

at Imperial College London says one way to check whether models can reveal sensitive information is to verify whether they can memorise data seen during training. When asked by New Scientist whether the Foresight team had conducted these tests, Tomlinson said it hadn’t, but that it was looking at doing so in the future.

Using such a vast dataset without communicating to people how the data has been used can also weaken public trust, says at the University of Oxford. “Even if it is being anonymised, it’s something that people feel very strongly about from an ethical point of view, because people usually want to keep control over their data and they want to know where it’s going.”

But existing controls give people little chance to opt out of their data being used by Foresight. All of the data used to train the model comes from nationally collected NHS datasets, and because it has been “de-identified”, , says an NHS England spokesperson, though people who have chosen not to share data from their family doctor won’t have this fed into the model.

Under the General Data Protection Regulation (GDPR), people must have the option to withdraw consent for the use of their personal data, but because of the way LLMs like Foresight are trained, it isn’t possible to remove a single record from an AI tool. The NHS England spokesperson says that “as the data used to train the model is anonymised, it is not using personal data and GDPR would therefore not apply”.

Exactly how the GDPR should address the impossibility of removing data from an LLM is an , but the UK Information Commissioner’s Office’s website states that “de-identified” data should not be used as a synonym for anonymous data. “This is because UK data protection law doesn’t define the term, so using it can lead to confusion,” it states.

The legal position is further complicated because Foresight is currently being used only for research related to covid-19, says Tomlinson. That means exceptions to data protection laws enacted during the pandemic still apply, says Sam Smith at , a UK data privacy organisation. “This covid-only AI almost certainly has patient data embedded in it, which cannot be let out of the lab,” he says. “Patients should have control over how their data is used.”

Ultimately, the competing rights and responsibilities around using medical data for AI leave Foresight in an uncertain position. “There is a bit of a problem when it comes to AI development, where the ethics and people are a second thought, rather than the starting point,” says Green. “But what we need is the humans and the ethics need to be the starting point, and then comes the technology.”

The famous marketing slogan about how a diamond is forever may only be a slight exaggeration for a diamond-based system capable of storing information for millions of years – and now researchers have created one with a record-breaking storage density of 1.85 terabytes per cubic centimetre.

Previous techniques have also used laser pulses to encode data into diamonds, but the higher storage density afforded by the new method means a diamond optical disc with the same volume as a standard Blu-ray could store approximately 100 terabytes of data – the equivalent of about 2000 Blu-rays – while lasting far longer than a typical Blu-ray’s lifetime of just a few decades.

“Once the internal data storage structures are stabilised using our technology, diamond can achieve extraordinary longevity – data retention for millions of years at room temperature – without requiring any maintenance,” says at the University of Science and Technology of China in Hefei.

Wang and his colleagues worked with small pieces of diamond only a few millimetres long, although they say future versions of the system could be in the form of larger storage discs. Their method used ultrafast laser pulses to knock some of a diamond’s carbon atoms out of place, leaving behind empty spaces the size of single atoms that each exhibited a stable brightness level.

By controlling the energy of the laser, the researchers could make multiple empty spaces at specific sites within the diamond, and the density of those spaces influenced each site’s overall brightness. “The number of empty spaces can be determined by looking at the brightness, which allows us to read the stored information,” says Wang.

The team then stored images – including Eadweard Muybridge’s 1878 sequence of photos showing a rider on a galloping horse – by mapping the brightness of each pixel to the brightness levels of specific sites inside the diamond. The system saved this data with more than 99 per cent accuracy and completeness.

Free newsletter

Sign up to The Weekly

Sign up to newsletter

This storage method isn’t yet commercially viable because it requires expensive lasers and high-speed fluorescence imaging cameras, along with other devices, says Wang. But he and his colleagues expect that their diamond-based system could eventually be miniaturised to fit within a space the size of a microwave oven.

“In the short term, government agencies, research institutes and libraries focused on archiving and data preservation would likely be eager to adopt this technology,” he says.