Distillation Technologies Request Access

Home  /  Learning Center  /  Article 18

How to tell if a model was distilled

Every available method is either unreliable, easily defeated, or both. This is the honest state of the art, and it explains why the 2026 accusations have been argued rather than proven.

Detection, defence and provenance 12 min read Article 18

This question moved from academic to consequential during 2026, when several laboratories and one government accused others of distilling frontier models. The short answer is that nobody can currently prove it from the model alone, and the reasons why are worth understanding.

Method 1: behavioural fingerprinting

The intuitive approach is to look for inherited habits. Models have idiosyncrasies: characteristic refusal phrasing, a particular hedging cadence, formatting preferences, recurring turns of phrase. A student trained on a teacher’s outputs tends to pick these up.

The most dramatic version is self-identification, where a model simply claims to be a different system.

Why it is weak evidence: the open web is now saturated with model-generated text. Any model trained on a recent web crawl absorbs other systems’ self-descriptions and stylistic tics without anyone deliberately distilling anything. During 2026 this cut in every direction at once, including an instance of Claude identifying itself as Alibaba’s Qwen. Contamination makes fingerprints suggestive at best.

Method 2: output distribution comparison

A more rigorous approach compares the probability distributions two models assign across many prompts. Genuine distillation optimises the student to match the teacher, so a distilled student should track its teacher unusually closely, including on the low-probability tokens where models normally diverge.

Why it is limited: it requires logit access to both models. Commercial APIs mostly do not expose full distributions, precisely because doing so makes white-box distillation easier. You are usually left comparing sampled text, which is a far noisier signal. Two models trained on similar data also converge naturally, so a similarity threshold that catches real cases also catches innocent ones.

Method 3: weight-space forensics

If you have both sets of weights, you can attempt to align them and look for structural correspondence.

Why it fails in practice: neural networks have enormous symmetry. Permuting neurons, rotating representations, or simply continuing to train destroys naive correspondence while preserving behaviour. A student with a different architecture from its teacher has no meaningful weight-level relationship to begin with, and most distillation targets a different architecture by design, since the point is usually to be smaller.

Method 4: watermark detection

Some providers watermark outputs statistically, biasing token selection in a way that is invisible to readers but detectable with the key. If watermarked text ends up in a student’s training data, traces might survive.

Why it degrades: watermarks are fragile. Paraphrasing, temperature changes, translation round-trips, and ordinary further training all attenuate the signal. Training a model on watermarked text does not reliably reproduce the watermark in that model’s own outputs, which is the property you would actually need.

Method 5: benchmark correlation

You can compare performance profiles. A student that inherited its teacher’s capability tends to inherit its teacher’s shape, including its weaknesses, which is a more distinctive signature than raw scores.

Why it is circumstantial: benchmarks are public, everyone optimises against them, and convergent scores are the expected outcome of a field optimising the same targets. A suspicious profile is a reason to investigate, not a finding.

What actually works: infrastructure evidence

The strongest evidence produced in any 2026 dispute did not come from examining models at all. It came from providers examining their own systems: account creation patterns, query volumes, traffic shaped unlike ordinary usage.

That evidence is genuinely strong for what it shows. Anthropic’s disclosures of 16 million exchanges through 24,000 fraudulent accounts and later 28.8 million exchanges through around 25,000 accounts are observations of their own infrastructure, not inferences about somebody else’s model.

But it establishes extraction, not construction. Knowing that someone collected millions of responses does not tell you what they trained, or whether they trained anything at all.

The honest conclusion

There is currently no reliable method for attributing a trained model to a specific teacher. Every technique above is either defeatable, contaminated, or merely circumstantial.

This is why the 2026 accusations have been resolved politically rather than evidentially, and why the fifteen-day gap in the Kimi K3 claim became so contentious. When attribution is unavailable, timelines and plausibility arguments are what remain.

If you need provenance guarantees today, the practical answer is contractual rather than technical: know where your training data came from, keep records, and do not rely on anyone being able to prove or disprove origin after the fact.

Common questions

Can you prove a model was distilled from another model?

Not reliably, with current methods. Every available technique produces suggestive evidence rather than proof. Behavioural fingerprints are contaminated by training data, weight-space forensics do not survive retraining, and watermarks degrade under paraphrase.

Why does one AI model sometimes say it is a different AI model?

Because models are trained on web text that now contains large volumes of other models' outputs. A model can absorb another system's self-descriptions without ever being trained on that system deliberately. This makes self-identification weak evidence in both directions.

What is the strongest evidence of distillation?

Infrastructure evidence held by the model provider: account creation patterns, query volume, and traffic shape. That evidence can establish that large-scale extraction occurred. It does not establish which model, if any, was built from the results.