WATERMARKS IN LLM RESPONSES
There is something almost poetic about the idea of hiding a secret message inside plain text. For centuries, spies used invisible ink, microdots, and steganographic tricks to embed information that only the right reader, with the right key, could ever find. Now, in the summer of 2026, one of the world's most prominent artificial intelligence companies, Anthropic, has announced that every word its Claude models generate will carry exactly such a hidden signature -- invisible to the human eye, but perfectly legible to a machine. The announcement is not merely a corporate curiosity. It sits at the intersection of cutting-edge machine learning research, European law, academic integrity debates, and a genuinely fascinating arms race between those who want to mark AI-generated text and those who want to erase those marks. Buckle up, because this story has everything: clever mathematics, political regulation, philosophical tension, and more than a little drama.
WHY NOW? THE REGULATORY HAMMER FALLS
To understand why Anthropic made this move in August 2026, you need to understand what happened on August 2nd of that same year. That date marks the moment when Article 50 of the European Union's AI Act became enforceable. The EU AI Act is the world's first comprehensive legal framework for artificial intelligence, and Article 50 is its transparency chapter. In plain language, it tells every provider of a generative AI system -- any system that produces text, images, audio, or video -- that their outputs must be marked in a machine-readable format so that they are detectable as artificially generated. The law does not merely suggest this. It mandates it, under penalty of fines reaching up to fifteen million euros or three percent of a company's worldwide annual turnover, whichever is higher.
Anthropic, which has signed the EU AI Act's voluntary Code of Practice on Transparency of AI-Generated Content, announced that all Claude models released on or after August 2, 2026, will carry invisible watermarks in their text output and digitally signed provenance metadata for supported file formats such as images. Crucially, and this is the part that affects every Claude user on the planet regardless of where they live, Anthropic decided to apply these marking techniques globally, not just within the European Union. The company's reasoning is straightforward and pragmatic: maintaining separate regional model behaviors is technically complex, operationally costly, and ultimately inconsistent. If the watermark is going in for European users, it goes in for everyone.
This global application means that a student in Tokyo using Claude to brainstorm essay ideas, a software engineer in San Francisco using Claude Code to generate boilerplate, and a marketing professional in Berlin drafting ad copy are all, as of this writing, producing text that carries Anthropic's hidden signature. Whether they know it or not. Whether they like it or not.
THE SCIENCE OF HIDING A SIGNAL IN PLAIN SIGHT
Before we can appreciate what Anthropic is doing, we need to understand how a language model actually generates text, because the watermark lives inside that process. When you type a prompt into Claude and press send, the model does not retrieve a pre-written answer from a database. Instead, it performs a sophisticated probabilistic calculation at every single step of the response. At each step, it looks at everything written so far -- your prompt plus whatever it has already generated -- and it produces a probability distribution over its entire vocabulary, which typically contains tens of thousands of tokens. A token is roughly a word or a word-fragment; the word "watermarking" might be a single token, while "unbelievable" might be split into "un" and "believable."
The model assigns a probability to every token in its vocabulary at each step. It might decide that the next token is 40% likely to be "the," 15% likely to be "a," 8% likely to be "this," and so on, with the probabilities of all remaining tokens summing to the remaining 37%. It then samples from this distribution to pick the actual next token. This sampling step is where the watermark enters the picture.
The foundational academic work here comes from a 2023 paper by John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein, titled "A Watermark for Large Language Models." Their method, known in the research community as the KGW watermark, introduced the concept of green lists and red lists. The idea is elegant in its simplicity. Before the model samples the next token, the entire vocabulary is pseudorandomly divided into two halves: a "green list" and a "red list." The model then slightly increases the probability of tokens on the green list before sampling. The division is not fixed; it is recomputed at every single token step using a secret cryptographic key combined with a hash of the tokens that have already been generated. This means that whether a token is "green" or "red" depends entirely on what came before it in the text, making the pattern context-sensitive and extremely difficult to reverse-engineer without the secret key.
Let us make this concrete with a tiny illustration. Imagine the model is about to generate the sixth word of a sentence and its top candidates are:
Token Base Probability Color (this step)
"quickly" 22% GREEN
"rapidly" 21% RED
"swiftly" 19% GREEN
"fast" 18% RED
"briskly" 12% GREEN
Without watermarking, the model might pick any of these roughly in proportion to their probabilities. With the green-list bias applied, the effective probabilities of "quickly," "swiftly," and "briskly" are nudged upward, making it more likely that the model picks one of them. The nudge is small enough that a human reader would never notice any difference in the quality or meaning of the text. But across hundreds or thousands of tokens, the statistical pattern accumulates. A detector that knows the secret key can reconstruct the green and red lists for every position in the text and count what fraction of the chosen tokens were green. In unwatermarked text, you would expect roughly 50% green tokens by pure chance. In watermarked text, you would see a significantly higher fraction, and a standard statistical test called a z-score can determine with high confidence whether that elevation is due to the watermark or just random variation.
The detection logic works roughly like this. The detector takes the text, applies the same secret key and the same hashing procedure, reconstructs the green/red list for each token position, and counts how many of the actual tokens in the text fell on the green list. If the text has N tokens and G of them are green, the z-score is computed as:
z = (G - 0.5 * N) / sqrt(0.25 * N)
If z exceeds a threshold -- say, 4.0, which corresponds to an astronomically small false-positive probability -- the detector declares the text watermarked. The beauty of this approach is that it requires no access to the original model during detection, only the secret key and the algorithm.
GOOGLE'S SYNTHID: THE TOURNAMENT THAT PICKS YOUR WORDS
Google DeepMind, which has been watermarking its Gemini models' text output since 2024, took the green-list idea and refined it into a more sophisticated system called SynthID-Text. Anthropic's implementation, according to available information, is based on a version of SynthID-Text, so understanding SynthID is essential to understanding what Claude is actually doing under the hood.
SynthID-Text uses a technique called tournament sampling, and it is worth spending a moment on this because it is genuinely clever. In the standard green-list approach, the model's probability distribution is directly modified before sampling -- you literally add a bonus to the logits (the raw scores before probabilities are computed) of green tokens. Tournament sampling takes a different route. It draws multiple candidate tokens from the model's unmodified distribution and then runs them through a tournament bracket to pick the winner.
Here is how the tournament works. The model generates its normal probability distribution over the vocabulary. A pseudorandom seed is derived from the secret key and a hash of the preceding tokens. This seed is used to assign a pseudorandom "g-value" to every token in the vocabulary -- think of it as a hidden score that has nothing to do with the model's linguistic judgment. Multiple candidate tokens are drawn from the model's distribution, and they are then paired off in a bracket. In each pairing, the candidate with the higher g-value advances. The token that wins the tournament is the one actually output.
The crucial insight is that tokens with higher g-values are systematically more likely to win the tournament, even if their linguistic probability is not the highest. Over many steps, this creates a detectable statistical bias toward tokens that score well under the pseudorandom g-value scheme -- which is, of course, the watermark signal. The advantage of this approach over simple logit modification is that every candidate token is still drawn from the model's own linguistic distribution, so the output never feels forced or unnatural. The tournament merely tilts the playing field in a way that is invisible to the reader but legible to the detector.
SynthID also has a particularly important property: the watermark signal is strongest at high-entropy positions, meaning positions where the model is genuinely uncertain and has many plausible options. When the model is very confident -- say, it is generating the word "the" after "Eiffel" -- there is only one reasonable choice, and the watermark has little room to operate. But when the model is choosing between "quickly," "rapidly," and "swiftly," as in our earlier example, the watermark has real leverage. This is actually a feature, not a bug, because it means the watermark concentrates its signal in the parts of the text where it will cause the least perceptible distortion.
C2PA: THE SECOND LAYER OF ANTHROPIC'S MARKING STRATEGY
The invisible text watermark is only one half of Anthropic's marking strategy. For file-based outputs -- images in formats like SVG, PNG, and JPG -- Anthropic is attaching digitally signed provenance metadata following the C2PA standard. C2PA stands for the Coalition for Content Provenance and Authenticity, an organization founded in 2021 by Adobe, Arm, BBC, Intel, Microsoft, and Truepic, which by January 2026 had grown to over six thousand members and affiliates.
The C2PA approach is conceptually different from statistical watermarking. Rather than hiding a signal inside the content itself, C2PA attaches a cryptographically signed manifest to the file. This manifest, sometimes called a Content Credential, records who created the content, when it was created, what tools were used (including which AI model), and what edits were made. The manifest is signed with a cryptographic key, so any tampering with the content breaks the signature and makes the modification detectable. Think of it as a tamper-evident seal on a medicine bottle: you cannot open the bottle and reseal it without leaving visible evidence of the intrusion.
The limitation of C2PA is that the metadata lives outside the content itself, attached to the file container. When a file is re-uploaded to a platform that strips metadata -- which many social media platforms do automatically -- the Content Credential disappears. This is precisely why C2PA and statistical watermarking are complementary rather than competing approaches. The C2PA metadata provides rich, readable provenance information as long as it survives, while the statistical watermark is embedded in the content itself and persists even after metadata stripping, heavy editing, or format conversion. Anthropic is deploying both layers simultaneously, which is exactly what the EU AI Act's guidance recommends when it states that no single technique is considered sufficient and that a multi-layered approach is required.
WHAT DOES A WATERMARKED TEXT ACTUALLY LOOK LIKE?
This is the question that most people ask first, and the answer is simultaneously reassuring and slightly unsettling: it looks exactly like any other text. There is no visible marker, no footer saying "generated by AI," no subtle change in font or spacing. The watermark is a statistical property of the text as a whole, not a property of any individual word or sentence. You cannot point to a single sentence and say "that is the watermark." The watermark is the aggregate pattern of word choices across the entire document.
To illustrate this, consider two versions of the same paragraph. Both are grammatically correct, semantically identical, and would read identically to any human. The difference exists only at the level of which synonyms and phrasings were chosen at each step.
Version A (hypothetical unwatermarked):
"The experiment yielded results that were broadly consistent with
the theoretical predictions, although some deviation was observed
in the high-temperature regime."
Version B (hypothetical watermarked):
"The experiment produced outcomes that were largely consistent with
the theoretical predictions, though some deviation was noted in
the high-temperature regime."
"Yielded" versus "produced," "results" versus "outcomes," "broadly" versus "largely," "although" versus "though," "observed" versus "noted." Five small choices, each one nudged by the green-list mechanism. To you and me, these two paragraphs are interchangeable. To a watermark detector with the secret key, Version B contains a statistically significant excess of green-list tokens. Multiply this across a five-hundred-word document and the signal becomes unmistakable.
This is the fundamental genius and the fundamental limitation of statistical watermarking in a single example. The genius is that the watermark is truly invisible and does not degrade the quality of the text. The limitation is that it is also fragile in a very specific way, which brings us to the most contentious part of this story.
HOW USERS CAN FIND -- AND POTENTIALLY REMOVE -- THE WATERMARK
Let us be clear about something before diving into this section. The purpose of discussing watermark removal is not to encourage academic dishonesty or regulatory evasion. It is to give an honest, complete picture of the technology's actual robustness, because understanding the limitations of watermarking is essential to understanding what it can and cannot achieve. Researchers, journalists, and policymakers all need this information to make sound judgments.
Finding the watermark in the first place is not something an ordinary user can do by reading the text. Detection requires access to the secret key and the detection algorithm. Anthropic has stated that it plans to provide detection tools to users and third parties, but as of this writing, the detailed technical guidance has not yet been fully published. Google's SynthID detector for text is similarly not publicly available in the way that, say, a spam filter is. So in practice, the watermark is not something you can "find" by inspecting your own Claude output -- you would need Anthropic's cooperation to run the detection.
Removing the watermark, however, is a different matter, and this is where the arms race gets interesting. The statistical watermark works by creating a pattern across many token choices. If you change enough of those token choices, you destroy the pattern. The most straightforward way to do this is paraphrasing. If you take a watermarked text and rewrite it -- replacing synonyms, restructuring sentences, changing the order of ideas -- you disrupt the green-list pattern because the new word choices are no longer governed by Claude's watermarked sampling process. They are governed by whatever process you used to rewrite the text, whether that is your own brain or another language model.
Research has shown that using a second LLM to paraphrase watermarked text is particularly effective at removing the watermark, because the second model makes its own independent token choices, which are uncorrelated with the original green-list assignments. A 2025 study introduced what researchers called the Self-Information Rewrite Attack, or SIRA, which specifically targets the high-entropy positions in the text -- exactly the positions where the watermark signal is strongest -- and rewrites those positions while leaving the low-entropy, high-confidence positions alone. This targeted approach achieves high watermark removal rates with relatively few edits, making it computationally efficient.
Another documented attack is called the Color-Aware Substitution Attack, or SCTS. This method attempts to infer which tokens in the text are "green" by prompting the watermarked model itself to reveal its preferences, and then systematically replacing those green tokens with non-green alternatives. It is a clever exploit of the fact that the model's own behavior can sometimes be used to reverse-engineer the watermark structure.
More mundane removal methods also work to varying degrees. Translating the text into another language and then translating it back disrupts the token-level statistics because the translation process introduces its own independent word choices. Inserting or deleting words, swapping synonyms manually, or mixing the AI-generated text with substantial amounts of human-written text all dilute the watermark signal. The longer the original text, the more robust the watermark is, because the statistical signal accumulates over more tokens and requires more disruption to erase. Conversely, short texts -- a single paragraph, a tweet-length response -- are much more vulnerable to watermark removal because the signal has not had enough tokens to accumulate robustly.
There is also a class of removal that requires no effort at all: simply using a non-watermarked model. Open-source language models like Meta's LLaMA family, Mistral, and dozens of others can be run locally on consumer hardware and produce no watermarks whatsoever, because no one has implemented watermarking in their inference pipelines. The EU AI Act's obligations apply to commercial providers, not to individuals running open-source models on their own machines. This creates an obvious asymmetry: the watermarking regime applies to the most visible, most regulated commercial products, while leaving a wide-open lane for unregulated local inference.
The honest summary is this: Anthropic's watermark is robust against casual editing and copy-pasting, moderately robust against light paraphrasing, and significantly vulnerable to determined paraphrasing using another LLM. It is not a cryptographic lock. It is a probabilistic signal, and like all probabilistic signals, it can be overwhelmed by sufficient noise.
THE ADVANTAGES: WHY THIS IS STILL WORTH DOING
Given that the watermark can be removed with enough effort, one might reasonably ask whether it is worth implementing at all. The answer is yes, for several reasons that are worth examining carefully.
The first and most important advantage is that watermarking raises the cost of deception. If someone wants to pass off AI-generated text as human-written, they now have to do extra work: paraphrase it, rewrite it, run it through another model. This friction is not zero. It takes time, effort, and in some cases money. For casual misuse -- a student who wants to submit a Claude-generated essay without modification, a content farm that wants to flood the internet with unedited AI text -- the watermark provides a meaningful deterrent. Not every bad actor is sophisticated enough to mount a SIRA attack.
The second advantage is accountability at scale. Even if individual watermarks can be removed, the existence of a detection system creates a credible threat of detection that changes behavior. This is analogous to speed cameras on highways: not every speeder is caught, but the existence of cameras reduces average speeds because drivers know detection is possible. Watermarking creates a similar deterrent effect in the information ecosystem.
The third advantage is that watermarking works extremely well for the use cases where it matters most: detecting large-scale, automated AI content generation. If a state actor or a commercial operation is generating millions of pieces of AI content and publishing them without modification, the watermark will be present in essentially all of them, and a detector can identify the campaign with high statistical confidence. The watermark is weakest against a single, motivated individual who wants to remove it from one document. It is strongest against industrial-scale content generation where there is no time or incentive to paraphrase every output.
The fourth advantage is legal and regulatory clarity. The EU AI Act requires machine-readable marking, and Anthropic's implementation satisfies that requirement. This gives regulators, courts, and institutions a technical tool they can use when investigating suspected AI misuse. The watermark is not proof of authorship -- Anthropic itself is careful to note that a detected watermark indicates that Claude likely processed the content, not that Claude was the sole author -- but it is admissible evidence in an investigation, and it shifts the burden of explanation onto the party whose content was flagged.
THE DISADVANTAGES: THE PROBLEMS THAT KEEP RESEARCHERS UP AT NIGHT
The disadvantages of text watermarking are real, significant, and deserve to be taken seriously rather than dismissed as edge cases.
The most technically fundamental problem is false positives. A false positive occurs when the detector flags a piece of human-written text as AI-generated. This can happen because the statistical test is probabilistic, not deterministic. There is always some nonzero probability that a human writer, by pure chance, happens to use a pattern of words that looks like a green-list bias. The probability of this happening for any given short text is small, but across millions of documents being analyzed, false positives will occur. In academic settings, a false positive can destroy a student's reputation and career. A New York court case in early 2026 saw a student successfully sue their university after being falsely accused of AI use based on an AI detection tool's output. Non-native English speakers and people with certain writing styles that happen to be more formal or repetitive are disproportionately at risk of false positives from AI detection systems generally, and watermark detectors are not immune to this problem.
The second major problem is false negatives, which occur when genuinely AI-generated text evades detection. As we discussed in the previous section, a determined adversary with access to paraphrasing tools can remove the watermark. This means the watermark cannot be relied upon as definitive proof of AI non-involvement. If a student submits an essay that was generated by Claude, then paraphrased by GPT-4, the Claude watermark will likely be gone. The detector will say "no watermark found," which might be interpreted as "this text is human-written," when in fact it is doubly AI-generated. This is a dangerous failure mode.
The third problem is the coverage gap created by open-source models. As mentioned earlier, anyone running a local LLM generates unwatermarked text. The watermarking regime therefore creates a two-tier system where compliant commercial providers mark their outputs while non-compliant or unregulated systems do not. This does not make watermarking useless, but it does mean that watermark absence cannot be interpreted as evidence of human authorship.
The fourth problem is the chilling effect on legitimate users. Anthropic has acknowledged that some users have canceled their subscriptions in response to the watermarking announcement, citing concerns that their work will be identified as AI-generated even when they used Claude only for minor assistance, such as proofreading or brainstorming. This concern is legitimate. If a professional writer uses Claude to check the grammar of a paragraph and then publishes the corrected text, that text may carry a Claude watermark even though the substantive content is entirely human-written. The watermark does not distinguish between "Claude wrote this" and "Claude touched this," and that ambiguity has real consequences for authors, journalists, and anyone whose professional reputation depends on the perception of human authorship.
The fifth problem is the arms race dynamic. Every time a new watermarking technique is published, researchers begin working on attacks against it. Every time a new attack is published, watermarking researchers work on more robust schemes. This cycle is not going to end. It is the same dynamic that governs every security system, from password hashing to digital rights management. The question is not whether watermarks are perfect -- they are not -- but whether they provide sufficient value at their current level of robustness to justify their costs. Reasonable people disagree on the answer.
THE BROADER PICTURE: WHERE THIS IS ALL HEADING
Anthropic's announcement does not exist in isolation. Google has been watermarking Gemini's text output since 2024 using SynthID. OpenAI has stated that it has developed watermarking technology and plans to roll it out to comply with EU law, though a full public announcement on their text watermarking implementation had not been made as of this writing. Meta, which develops the open-source LLaMA models, faces a different situation: because LLaMA weights are publicly available and can be run by anyone, Meta cannot enforce watermarking at the inference level, and the EU AI Act's obligations for open-source providers are more limited.
The EU AI Act also mandates that watermark detection must be interoperable across providers by February 2, 2027. This means that eventually, a single detector should be able to identify whether text was generated by Claude, Gemini, GPT-5, or any other major commercial model, without needing separate tools for each. Achieving this interoperability while keeping the secret keys secret is a non-trivial cryptographic challenge, and the technical community is actively working on solutions.
One promising direction is the development of public watermarking schemes, where the detection algorithm is public but the secret key is held by the provider. This allows anyone to run a detector and get a yes/no answer about whether a specific provider's watermark is present, without being able to forge the watermark themselves. This is analogous to public-key cryptography: you can verify a signature without knowing the private key used to create it.
Another direction is multi-bit watermarking, where the watermark encodes not just a binary "this is AI-generated" signal but richer information: which model generated the text, at what time, under what API key, and potentially even which specific conversation. This would transform the watermark from a simple provenance signal into a full audit trail, with obvious implications for accountability -- and equally obvious implications for privacy.
The privacy dimension is one that deserves more attention than it typically receives in discussions of AI watermarking. If every Claude response carries a unique, traceable watermark, and if Anthropic or regulators can decode that watermark, then in principle every piece of Claude-generated text can be traced back to the conversation that produced it. This is a powerful tool for accountability. It is also a powerful tool for surveillance. The line between those two things depends entirely on who controls the key and what legal constraints govern its use.
A FINAL THOUGHT: INVISIBLE INK AND THE SOCIAL CONTRACT
There is something philosophically interesting about the fact that the most advanced AI systems in the world are now, in a sense, signing their work -- not with a visible signature, but with a hidden statistical fingerprint that only a machine can read. This is not how human authors sign their work. When a novelist publishes a book, the signature is visible on the cover. When a journalist publishes an article, their byline is at the top. The AI watermark is more like a manufacturer's serial number stamped on a component inside a machine: invisible in normal use, but traceable when someone opens the hood.
Whether you find this reassuring or unsettling probably depends on where you sit. If you are a regulator worried about AI-generated misinformation flooding democratic discourse, the watermark is a welcome tool. If you are a writer who uses Claude to polish your prose and resents the implication that your work is somehow less yours because a machine touched it, the watermark feels like an accusation. If you are a researcher studying the robustness of AI safety mechanisms, the watermark is a fascinating technical puzzle. If you are a student who genuinely wrote your own essay and is terrified of a false positive, the watermark is a source of anxiety.
Anthropic's decision to watermark Claude's outputs is, on balance, a reasonable and responsible one. It complies with a legitimate legal requirement, it provides a real (if imperfect) tool for detecting AI misuse at scale, and it does so without any perceptible impact on the quality of Claude's responses. The limitations are real, and they should be communicated honestly rather than papered over with marketing language about "robust" and "imperceptible" watermarks. The watermark is not a silver bullet. It is a probabilistic signal in an ongoing arms race, and anyone who tells you otherwise is overselling the technology.
What is certain is that this is only the beginning. The watermarks of 2026 are primitive compared to what will exist in 2030. The legal frameworks are still being written. The detection tools are still being built. The arms race between watermarkers and erasers is still in its early rounds. And somewhere in the middle of all of this, billions of words are being generated every day by machines that are now, quietly and invisibly, signing their names.
No comments:
Post a Comment