As AI chatbots have taken off, you have probably found yourself wondering whether a piece of text you are reading was written by a human or a machine. Now, in response to European Union legislation, many AI firms are adding watermarks to text, images and video created by their models with the aim of letting you do just that. The hope is that these watermarks will help stem the spread of misinformation and bring transparency to the use of AI, but experts point out that, at least when it comes to text, watermarking is unlikely to work.
The requirement to watermark is part of the EU’s AI Act, and came into force on 2 August. OpenAI, the maker of ChatGPT, is already , and says it Anthropic, the firm behind Claude, has announced that future models . The AI Act allows a grace period for models that have already been deployed, but all models must provide watermarking by 2 December this year.
at , an AI safety firm, says that while image and video watermarks are more reliable because they are composed of dense and multi-layered data, watermarks in text will always be more difficult to implement.
Advertisement
One approach to watermarking text is to apply a mathematically detectable pattern to the choice of words. Large language models like ChatGPT create sentences by appending whichever word is statistically most likely to come next. By varying this selection of words, perhaps by choosing alternately the most likely next word and then the second, in repetition throughout a sentence, models can embed a pattern without changing the overall meaning of the text. Under the AI Act, AI companies will be required to offer tools that can look for these patterns, allowing people to identify AI-generated text.
But Padolsey points out that these watermarking patterns are susceptible to deletion by even light edits. “It’s not useful, and the way that people are going to try and use it will be incorrect,” he says. To drive home his point, Padolsey has created an online tool called , which takes AI-generated text and lightly changes it in such a way that existing AI detectors no longer work. The same process should work on watermarked text, he says.
Even without such tools, Padolsey says that some open-source AI models that don’t abide by the EU’s rules will always be available, because these aren’t controlled by any one tech firm. “Any bad actors will just use those. And they’ll probably be cheaper and easier to run,” he says. “If you’re a nasty individual or a bad actor and you’re trying to generate this stuff en masse, it’s not hard to do.”
“These rules will help people recognise when they are interacting with AI or when content has been generated or altered by AI,” says a spokesperson for the European Commission. “The adversarial robustness of marking and detection solutions must be assessed in terms of resilience to malicious behaviour, such as copying, removal, regeneration and modification attacks on the markings.”
While Padolsey’s concern is that the watermarks are easily removed, others are worried about false positives – when text is found to be AI-generated in error, or because a model had even a minor role in producing a piece of work. at the University of Reading, UK, says lots of companies have pitched products to his department claiming to be able to help spot student plagiarism, albeit using other methods than watermarking, while also including small print that says their systems shouldn’t be used to conclusively prove or punish incidents of plagiarism because they aren’t infallible.
“If it says there’s a 67.8 per cent chance that this text was generated in some way by AI, as an educator, how would I act upon that? I’m not really sure,” he says. “These systems don’t work perfectly, and because of that, they’re of limited use.”
Watermark-detection tools will work in a similar way and could flag text that is simply processed by AI, not just text entirely generated by it. A student could find themselves in hot water even if they only asked a model to scan their manually written work for grammatical mistakes prior to submission.
Scarfe says that AI can be used for a host of purposes and doesn’t inherently spread misinformation, so perhaps a blanket policy on watermarking is a blunt instrument, but points out that it can make mistakes and hallucinate facts, and can also be used to disseminate lies quickly – such as by powering disinformation bots on social media.
“Because these technologies came in so quickly, and they’ve accelerated so quickly, everyone in every area of life is kind of on the back foot,” he says. “[But] I don’t think [watermarking] is a magic bullet. I don’t think it’s going to suddenly solve what the EU maybe are hoping it’s going to solve.”
But perhaps watermarks may just create enough friction that they will put a brake on some nefarious uses of AI, such as mass creation of social media posts pushing disinformation. at Case Western Reserve University in Cleveland, Ohio, says that even if removing a watermark takes a trivial effort, this can, over time, make certain kinds of misuse impractical.
“Anybody can tamper with the watermark if they spend enough time on it, but then, when you spend enough time on it, then maybe you should have just generated the content yourself in the first place, right?” says Ayday.
