Watermarking is the most technically elegant of the provenance defences and the one whose limits are most consistently overstated in coverage.
The mechanism
At each generation step a language model produces a distribution over its vocabulary. A watermarking scheme intervenes there.
Using the preceding tokens as a seed, it pseudorandomly splits the vocabulary into two groups, commonly described as a green list and a red list, then adds a small bias toward the green list before sampling.
For any single token this is unnoticeable. The model still picks sensible words, because the bias is small and the green list is large. Across hundreds of tokens, though, the proportion of green-list choices drifts measurably above chance, and anyone who can regenerate the same partitions can measure it.
Why it is clever
- Invisible. No reader can detect it. The text is not degraded in any perceptible way.
- No model access needed to detect. The detector needs the key and the hash, not the weights.
- Statistically principled. Detection yields a p-value, so confidence is quantified rather than asserted.
- Cheap. Adds negligible cost to generation.
How it breaks
Paraphrasing. Rewriting the passage, including with another language model, replaces the token choices that carried the signal. This is the most effective attack and requires no understanding of the scheme.
Length. Detection is statistical, so it needs enough tokens to accumulate significance. Short passages carry too little signal to distinguish from chance.
Editing. Ordinary human revision attenuates the signal in proportion to how much was changed. Nobody has to be attacking anything.
Temperature and sampling. Aggressive sampling settings interact with the bias and can weaken it.
Translation. A round trip through another language destroys the token sequence entirely.
The failure that matters most here
For the distillation question specifically, there is a more fundamental problem than any of the above.
Watermarks indicate that a passage was generated by a watermarked model. They do not indicate that a model was trained on watermarked passages. Training on watermarked text does not reliably cause the student to emit the watermark in its own generations, because the student learns the language distribution rather than the specific token-level bias that encoded the mark.
So watermarking is a content provenance tool, not a model provenance tool. It can tell you where a document came from. It cannot tell you what a model was trained on, which is the actual question in every 2026 accusation. See How to Tell If a Model Was Distilled.
Where it is genuinely useful
Detecting machine-generated submissions, labelling synthetic media, and internal provenance tracking within a controlled pipeline. These are real applications where the text is unlikely to have been adversarially paraphrased.
What it cannot do is survive an opponent who wants it gone, which is the setting it keeps being invoked for.
Common questions
How does AI text watermarking work?
The most studied approach pseudorandomly partitions the vocabulary at each step, using the preceding tokens as a seed, and nudges the model toward one partition. Individually this is invisible, but across a long passage the statistical preference is detectable by anyone with the key.
Can AI watermarks be removed?
They degrade under paraphrasing, translation round-trips, heavy editing and further training. None of these were designed as attacks; they are ordinary text operations. Robustness against a determined adversary is substantially weaker than against incidental modification.
Does watermarking prove a model was distilled?
No. Training a student on watermarked text does not reliably cause the student to reproduce the watermark in its own outputs, which is the property that would be needed. Watermarks can indicate that a specific passage came from a model, not that a model was trained on such passages.