Skip to content

OpenAI distillation defenses cut reasoning theft to under 1%

Comic-style photocopier scanning a chained, padlocked AI chip while unlocked glowing copies spill into gloved hands Big tech

OpenAI distillation defenses now block almost every attempt to pull hidden reasoning out of its newest models. In research published on November 20, OpenAI says attacks of that kind succeed less than 1% of the time against GPT-6.2 Astra. Against GPT-6 Astra with its original safeguards, the most common one worked 54% of the time.

Distillation means training one model on the outputs of another. OpenAI argues that it can copy dangerous skills, such as finding software exploits. The safety rules that limit how those skills are used stay behind. The company says it has disrupted multiple distillation campaigns on its platform this year.

What OpenAI found about distillation

OpenAI tested how much a model can learn from another model's answers. It trained student models on outputs from an Astra-level teacher on coding and defensive cyber tasks. One student saw only what users see: final answers, reasoning summaries and tool calls. The other saw the teacher's exact reasoning traces.

The student trained on full reasoning generally came out ahead after fine-tuning. After a short round of reinforcement learning, it still scored higher on the ExploitBench and SEC-bench Pro cyber tests at the largest compute budget. At smaller budgets, the summary-trained student did better. OpenAI did not publish the scores in the text of its post.

Anthropic reported a similar result in October. A student trained on Claude's full reasoning traces scored 80.4% on SWE-bench Verified, against 71.8% for one trained on summaries. Both companies draw the same conclusion. "Protecting reasoning traces can limit cyber capability transfer," OpenAI writes.

That is why OpenAI's new defenses focus on one thing.

How OpenAI protects its models' reasoning

OpenAI models think before they answer, but users see only a summary of that reasoning. The full trace is encrypted and hidden. Attackers have tried to trick the models into revealing it, either from an earlier message or from the current one.

OpenAI says it trained its models against successful attacks and added classifiers on both sides. Input classifiers catch extraction attempts, and output classifiers stop suspected leaks before they reach the caller. Protected reasoning is now tied to the conversation that produced it. If an earlier message is rewritten, the reasoning after it is dropped.

The results come from red-teaming with six attacker models. Average success at extracting reasoning from an earlier request fell from 54.0% on GPT-6 Astra to 0.2% on GPT-6.2 Astra. For the current request, it fell from 22.9% to 0.9%. These are OpenAI's own tests, which no outside group has checked yet.

OpenAI distillation defenses: reasoning extraction success fell from 54.0% to 0.2%; Anthropic counted 13M MiniMax and 3.4M Moonshot exchanges
Attack success rates from OpenAI's November 20, 2026 report; distillation exchanges by lab from Anthropic's February 23, 2026 report

The new protections cover GPT-6.2 Astra and GPT-6.2 Sol, which launched on November 17. OpenAI says it is bringing some of the fixes back to GPT-6 Astra. Blocking a single request is only half the job, though.

How OpenAI detects distillation campaigns

Distillers rarely work from one account. OpenAI says it looks for networks of coordinated accounts and acts against the whole network. Its weekly enforcement ranged from thousands to tens of thousands of accounts, and the number rose sharply over the summer.

The company described one campaign in detail in September. It began on July 1 and peaked on July 24 and 25, with 16,000 requests. More than 4,000 users drove the high-volume activity, and a related cluster covered over 15,000. The operators copied encrypted reasoning from one conversation and had another conversation decrypt it. "The operators did not break our encryption," OpenAI wrote then.

OpenAI attributed a core cluster of that activity to individuals associated with Moonshot AI, the maker of Kimi. In its new post, it also repeats that it briefed Congress in February on activity indicating that DeepSeek was trying to distill U.S. frontier models. OpenAI did not name any other company this time.

OpenAI shares indicators of compromise with other AI providers and through the Frontier Model Forum. It also wants resellers of its models to follow the same standards. Its rivals have reported much the same problem.

Anthropic and Google report the same attacks

Anthropic published the largest numbers so far. In February, it said three Chinese labs ran more than 16 million exchanges with Claude through about 24,000 fraudulent accounts. It counted more than 13 million from MiniMax, more than 3.4 million from Moonshot and about 150,000 from DeepSeek.

In October, Anthropic said its new defenses cut the reasoning data distillers collected by about 90% to 99%. It banned tens of thousands of accounts, often within two hours. Google's threat intelligence team reported a campaign of more than 100,000 prompts in February. It aimed at copying Gemini's reasoning, and Google did not name who was behind it.

Reuters added a wider view on November 17. After reviewing more than 300 Chinese papers, patents and filings, it found that distillation has become routine practice in China's AI industry. A U.S. official called it the main reason Chinese models have closed the gap. Two people familiar with Chinese developers' work disputed that.

China's government rejects the accusations. Its foreign ministry has urged the U.S. to stop making unfounded claims, and Moonshot denies building its models through distillation. Critics in the West raise a different objection.

The criticism OpenAI faces

The labs fighting distillation trained their own models on vast amounts of web text, books and code without asking. Critics see a double standard. When OpenAI published its September report, The Register called its story an "irony alert" in the headline.

OpenAI's answer is that the risk lies in what gets copied. Its post warns that distillation lets bad actors "reproduce these capabilities without appropriate safeguards" in their own models. Hacking skills in a model without safety rules are a security problem, whatever one thinks of training data.

The company is also asking Washington for help. It backs legislation that would give the U.S. government more tools to identify and sanction foreign actors who distill American models. It supports stronger export controls on AI chips too. In OpenAI's view, they "can limit adversaries' access to the hardware" for training.

Can OpenAI stop distillation

It can make distillation much harder, but it cannot stop it. Hiding the reasoning trace removes the most valuable training data, and Anthropic's figures suggest the effect is large. Final answers are still visible, and models trained on them still learn a lot.

What OpenAI's report shows is a shift in tactics. Distillation is becoming an arms race between account networks and classifiers, much like spam or fraud. The defenses buy U.S. labs time and raise the cost of copying. Whether that time matters depends on how fast Chinese labs can catch up with their own research and chips.

FAQ

Here are short answers to common questions about distillation.

What is AI model distillation?

Distillation is training a smaller or newer model on the outputs of a stronger one. It is a common technique inside AI companies. It becomes adversarial when someone uses a rival's model without permission, breaking its terms of service.

Which companies did OpenAI accuse of distillation?

OpenAI has attributed a July campaign to individuals associated with Moonshot AI. It also says it briefed Congress in February on attempts by DeepSeek. Moonshot denies building its models through distillation.

Does distillation copy safety guardrails?

According to OpenAI, no. A distilled model can learn the capabilities of the original without the safeguards that govern how they are used. That is the main risk OpenAI's defenses target.

Sources

Portrait of Felix Laurent, author at thehype.news

Felix Laurent

News Writer at thehype. Felix is a journalist with experience in marketing. He covers artificial intelligence and emerging technologies, focusing on the ideas, innovations, and trends shaping the future of tech.

More articles by Felix Laurent →
On air · radio.thehype.news ↗ AI news radio — 24/7