strode past the Pontifical Swiss Guards, in their Renaissance-era uniforms. She was headed not for the Sistine Chapel or St Peter’s Basilica, but the Vatican’s archives. Precious few people are allowed into this legendary collection of documents and letters spanning 12 centuries. Yet even in that context, Megyesi’s 2012 visit was unusually intriguing. She was here to see texts so secret that no living person, not even the pope, could tell you what they contain.
Megyesi, a linguist based at Uppsala University in Sweden at the time, had travelled to the Vatican to pore over a tranche of papers written in elaborate ciphers – the secret codes used by spymasters and others eager to send private messages. An expert in cracking historical codes, she had been invited after breaking the notorious Copiale cipher.
Megyesi had the opportunity to use the Vatican’s encrypted papers for a project with an audacious goal: to fully automate the process of decrypting historical ciphers so that many thousands of otherwise inaccessible letters could finally speak to us from down the centuries. “The dream is to be able to point your phone camera at a cipher and read it immediately,” she says.
Advertisement
In the decade since, Megyesi and her colleagues have developed software that expedites their painstaking cryptanalysis – and researchers associated with the project have notched some remarkable successes. These include the recent decryption of a particularly fiendish code employed by a 17th-century French nobleman and, most sensationally, the cipher encrypting letters written by Mary, Queen of Scots, during her imprisonment at the hands of Queen Elizabeth I.
Breaking historical ciphers is difficult because they can work in so many different ways. A useful place to start is to assume that whatever cipher you are confronted with contains homophones, or symbols that represent one sound or letter of the hidden message. In a very simple cipher, for example, every “a” in the original message – which is known as plaintext – could be replaced with a “b”, every “b” with “c” and so on. Cryptanalysts would try many combinations of substitutions until they start to get a message where a few fragments of words begin to make sense.
Elaborate ciphers
But unless your cipher is very basic, it won’t stop there. Many historical ciphers also contain elements called nomenclators, symbols that represent syllables, whole common words or names. These can be extremely hard to crack unless you have some sense of what the letter is about or who wrote it and can make an educated guess about what the nomenclators mean. Sometimes, these symbols can even be “nulls” – characters that have no meaning and should be discounted – just to throw adversaries off the scent.
Over the centuries, expert cipher creators developed ever-more sophisticated techniques. For example, not all codes are homophonic. Polyphonic ciphers sneakily use the same individual symbol to refer to more than one letter or sound and are harder to crack. Then there are ciphers that involve translations of letters, for example swapping the first and last letters of every word before you can read the plaintext. There is even super encryption, when plaintext is encrypted and then the encrypted version is encrypted again using a second cipher.
Take the , the code encrypting text inside a 105-page manuscript bound with green and gold brocade-backed paper, thought to date from around 1750 and discovered in an academic archive. Its name derives from one of just a few words written in plaintext. The rest of it is a bizarre mix of familiar-looking Roman letters and obscure symbols. It is obviously a code, and the expensive feel of the manuscript makes it seem as if it mattered to someone. But who wrote it, and what does it say?
Megyesi, now at Stockholm University in Sweden, began work on this cipher in 2011 with her colleagues at the University of Southern California. The first step was to devise a way of making the code machine-readable, which they managed by effectively encoding it so that each symbol was assigned a letter on a keyboard. They could then carry out a frequency analysis, to see how often each symbol was used and to look for statistical patterns. For example, the most commonly used letters in modern English are, roughly speaking, e, a, r, i and so on – so by working out the frequency of symbols, you can often guess what they mean.
After a few dead ends, the team arrived at two key hypotheses: that the underlying text was written in German and that each unaccented Roman letter marked the start of a new word, while the more obscure symbols encoded letters. With these potential breakthroughs in hand, they could use a computer algorithm to try out a range of cipher keys until they alighted on the correct one and had figured out the homophones. With that, .
The Copiale cipher turned out to have been written earlier than thought, in the 1730s, by a secret society known as the oculists operating in what is now Germany. It contained details of a bizarre initiation ritual that involved, among other things, new members being asked to read a blank piece of paper and having a single hair plucked from their eyebrow. (The oculists used sight as a metaphor for knowledge.)
Following this success, Megyesi began thinking about how cryptanalysis could be automated to make some of the most laborious aspects of the process less so. Advances in artificial intelligence were coming on apace, which suggested that computers could do much of the work. Rather than digitising the code symbols manually, perhaps computer vision AI could be used to recognise and sort them. It might also be possible to develop software to run frequency analysis and decipher most of the homophones. It would be a difficult undertaking to develop each of these tools and stitch them together into a seamless process. But it would be worth it if it allowed us to enter a vast, hidden world of historical secrets. Crucially, Megyesi would need a large collection of encrypted texts to use as a training ground.
The DECRYPT project
That was what took her to the Vatican. Once inside the archives, she ordered up boxes of documents and went through them by hand, identifying those written in ciphers. “I thought back to when they would have been written; the candles, the quills,” she says. In the end, she selected thousands of examples and the Vatican provided digital copies of them.
With that treasure trove in hand, in 2015, Megyesi gathered a small team and founded what would go on to become the to work on the documents and start trying to automate the decryption process. The first challenge was to develop an algorithm that cracks the homophones of a cipher by trying lots of different possibilities. In itself that wasn’t enough to completely automate the decipherment, but it did speed up the process. With several researchers on board to use it, the team was soon cracking dozens of ciphers every year.
The bulk of the documents from the Vatican turned out to be letters from nuncios, or papal ambassadors stationed in foreign countries. Spanning the 15th to 17th centuries, most were anything but sensational – generally they contained everyday gossip and requests for people to be paid. But when you are decrypting lots of documents, sooner or later you are bound to come across something explosive. And that is precisely what happened when another of the project’s researchers got his hands on an intriguing cipher held in France.
is a computer scientist at Google in Tel Aviv, Israel. Around a decade ago, he began playing around with cracking ciphers, inspired by on which readers were challenged to break the ones posted there. Eventually he got so good that he is now part of the DECRYPT project.
As well as working on the ciphers from the Vatican, Lasry would also check a website called maintained by Japanese physicist Satoshi Tomokiyo. In his spare time, Tomokiyo waded through the archives of national libraries and when he discovered historical ciphers, he would post them on the section of his site. In March 2021, noticing that Tomokiyo had posted an encrypted document called BNF fr.2988, which he had discovered in the collection of the National Library of France in Paris, Lasry began to study it. “It was just another puzzle,” he says. At least, that was how it seemed.
Decoding Mary, Queen of Scots
Once Lasry had discerned that the underlying plaintext was in a historical form of French, he applied his own cryptanalysis program to work out the homophones with relative ease. But this only revealed a small part of the message. The cipher used a series of “diacritics”, or symbols with dots next to them that modify what they represent, and a particularly sprawling series of nomenclators to symbolise names and common endings of French words, including “-ent”, “-oit” and “-endre”.
With many of the symbols defeating him, Lasry recruited his friend , at Berlin University of the Arts in Germany, to help. Biermann was a major asset, and not only because he is an expert cryptanalyst. He also happens to be a professional pianist and helps produce operas, some of which are written in Middle French, the language of the text. Working together, Lasry, Biermann and Tomokiyo gradually figured it out. One of the last pieces of the puzzle slotted into place when Biermann realised that a certain symbol meant “delete previous”, a particularly confusing trick.
The document wasn’t signed or addressed, but from the content the trio could deduce that it was a letter written by a woman imprisoned in England with a son. The other big clue was that it also mentioned Francis Walsingham, Queen Elizabeth I’s spymaster. From all this, they came to their breathtaking conclusion in February this year: , otherwise known as Mary, Queen of Scots. The documents Tomokiyo had discovered turned out to contain 57 letters, written between 1578 and 1584, and Lasry and Biermann went on to decipher them all.
The decryption made headlines around the world. Mary had a disputed claim to the English throne, which made her politically dangerous. As a result, she was caught in a web of 16th-century plotting and spent many years imprisoned. Historian John Guy at the University of Cambridge has called the work “stunning” and “the most important new find on Mary Stuart, Queen of Scots, for 100 years”. The reason for his excitement is that, now we know what they say, we can see that the letters offer fresh insights into how Mary negotiated regarding her and her son’s future with her contacts in France and Elizabeth I’s court.
For historical significance, the decoding of the letters of Mary, Queen of Scots, is hard to beat. Even so, when many of the world’s best historical codebreakers gathered at the in Munich, Germany, in June, they were buzzing with talk of other scintillating successes.
For instance, at the French National Institute for Computer Science Research in Nancy had heard a rumour about a mysterious letter stored somewhere in the city. Word on the grapevine said it was a missive from Charles V, the Holy Roman Emperor for most of the first half of the 16th century. Many academics knew the story, she says, but there were no details about the letter’s location.
Pierrot asked everyone she could think of and, after two years of searching, found the letter at Nancy’s library. It identified Charles V as the sender in plaintext, so it was obviously the real deal. Pierrot and her colleagues in late 2022. It turned out to be a communication to Jean de Saint-Mauris, one of the emperor’s ambassadors. Among other things, it mentions a rumour of an assassination plot, giving historians an insight into Charles V’s fears at the time.
Lasry also revealed another remarkable result at the conference. It concerns , a leader in a series of civil wars in 17th-century France known as The Fronde. Lasry showed that Bourbon used a cipher that is neither polyphonic nor homophonic, but polyhomophonic – in other words, some of its individual cipher symbols can represent several plaintext letters, but equally each plaintext letter can be represented by several different cipher symbols. Lasry says that, to his knowledge, it is the only cipher of this kind discovered so far.
How close are we to having an app that automatically decrypts historical ciphers? The DECRYPT project now has a piece of software called CrypTool 2, created by at the University of Siegen in Germany. If you feed cipher symbols into it, the program can do an automated cryptanalysis to find the homophones and often partially breaks the code. As things stand, the team has cracked more than 2500 ciphers, most of which were aided by its algorithms.
What has proved much more challenging is the first step in the process: having computer vision AI recognise and digitise enciphered texts. at the Autonomous University of Barcelona in Spain has been leading on this part of the project. She and her team have tested various AI models for the task, but it has proved extremely difficult to train them, because the handwriting from these letters written centuries ago is so variable. The other problem is that the symbols often touch each other, such that computers struggle to discern where one ends and another begins. The team has recently had more success by hand-labelling the symbols and using these datasets to train algorithms – but it remains tricky.
For her part, Megyesi is actually in no rush to get to the stage where you can point a smartphone at a historical cipher and read it instantly. That might well be the stated goal of the project, in her own words. “But frankly, it would be a bit sad to have an app that does this,” she says. “Because who doesn’t want to have the enjoyment of cracking codes?”
Joshua Howgego is a features editor at New Scientist
