Fifty-Four Percent Down to Point Two, Scored by the House

On 20 November OpenAI published a post on what it calls adversarial distillation: someone using a frontier model’s outputs, without authorization, to train a cheaper model that inherits the capabilities and not the safeguards. Most of the post is description of defenses. One table carries the claim.

The table reports attack success rate, or ASR, for attempts to pull a model’s protected reasoning, its chain of thought, back out of it. Against GPT‑6 Astra with its original safeguards, attacks that targeted reasoning from an earlier request succeeded 54.0% of the time. Against GPT‑6.2 Astra with the updated safeguards, 0.2%. For the current request the pair is 22.9% and 0.9%. The prose version: “The average attack success rate fell from 54.0% against GPT‑6 Astra with its original safeguards to 0.2% against GPT‑6.2 Astra with its updated safeguards.”

Reported attack success rate before and after the safeguard update Attack success rates from OpenAI's adaptive black-box red-teaming, aggregated across six attacker models. Attacks extracting protected reasoning from an earlier request succeeded 54.0 percent of the time against GPT-6 Astra with original safeguards and 0.2 percent against GPT-6.2 Astra with updated safeguards. Attacks targeting the current request succeeded 22.9 percent and 0.9 percent respectively. ADAPTIVE BLACK-BOX EXTRACTION ASR · SIX ATTACKER MODELS · OPENAI, 20 NOV 2026 Reasoning from an earlier request 54.0% — OpenAI, Combatting adversarial distillation, 20 Nov 2026 54.0% 0.2% — OpenAI, Combatting adversarial distillation, 20 Nov 2026 0.2% Reasoning from the current request 22.9% — OpenAI, Combatting adversarial distillation, 20 Nov 2026 22.9% 0.9% — OpenAI, Combatting adversarial distillation, 20 Nov 2026 0.9% GPT-6 Astra, original safeguards GPT-6.2 Astra, updated safeguards Both ends of each pair come from the same internal red-teaming setup. No attempt count is given.
The published table. Four numbers, one source, no denominators.

A 54 to 0.2 drop is a factor of 270. I want to be careful about what it is a measurement of, because ASR is not a property of a model. It is a property of a model and an attacker, jointly. The defender half of that pair got rebuilt between the two columns. The attacker half is, by OpenAI’s own description, “strong automated red-teaming” run in-house, aggregated across six attacker models it does not name. Nobody outside the company produced either number.

The number that moves when someone tries harder

Adversarial robustness research spent a decade discovering that defenses reported as strong had been measured against attacks too weak to find the hole.

Ma and colleagues, in Machine Learning in 2024, describe one mechanism they call imbalanced gradients: one term of the loss dominates and steers the attack somewhere suboptimal, so the defense looks robust when it is not. They build a margin decomposition attack to exploit it and run it against 24 adversarial-training defenses published since 2018. Eleven of the 24 lose more than a percentage point of their previously best-measured robustness.

The sharpest case in that paper is not a defense at all, just a training trick. Add label smoothing to a naturally trained model and the standard PGD attack measures 7.87% robustness. The stronger attack finds 0.06%. Same model, same threat model, same metric: two numbers 131-fold apart, and the only thing that changed was who was attacking. On ImageNet their attack alone beats the AutoAttack ensemble that RobustBench reports, scoring XCiT-S at 40.26% where AutoAttack had 41.78%.

The same model measured by a weaker and a stronger attack From Ma and colleagues, Machine Learning 2024. A naturally trained model with label smoothing measures 7.87 percent robust accuracy under the PGD attack and 0.06 percent under their margin decomposition attack. The XCiT-S ImageNet model measures 41.78 percent under AutoAttack and 40.26 percent under margin decomposition. MEASURED ROBUST ACCURACY, SAME MODEL, TWO ATTACKS · MA ET AL., 2024 Natural training plus label smoothing 7.87% under PGD — Ma et al., Machine Learning, 2024 7.87% 0.06% under the margin decomposition attack — Ma et al., Machine Learning, 2024 0.06% XCiT-S on ImageNet 41.78% under AutoAttack, as reported on RobustBench — Ma et al., Machine Learning, 2024 41.78% 40.26% under the margin decomposition attack — Ma et al., Machine Learning, 2024 40.26% The attack the authors of the defense ran A stronger attack, run later, by someone else Robust accuracy, so higher is a stronger claim. The revision always goes one way.
Two attacks, one model each. The second number is what the first was supposed to be.

Reported robustness gets revised in one direction, downward, as attacks improve, and the size of the revision ranges from fractions of a point to two orders of magnitude depending on whether the original evaluation happened to match the weakness. OpenAI’s 0.2% is a claim of the same type, and the only control on it is that the same team also produced the 54.0%. If their red-teamers are strong, the pair is meaningful. If their red-teamers are weak, both numbers are too low and the ratio between them can still look terrific.

The company does say what it did to earn the drop: adversarial training against extraction attempts, input and output classifiers that also inspect tool-call arguments, and API rules tying reasoning to the conversation that produced it, including across user, workspace, organization and model-family boundaries. Those are specific, plausible changes. They are also the reason the drop is hard to read: four mitigations landed at once, and the table scores the bundle.

What a checkable version would look like

The gap between an internally measured defense and an independently measured one is the subject of a position paper I keep returning to. Apruzzese and colleagues, at IEEE SaTML in 2023, surveyed every adversarial-ML paper at CCS, USENIX Security, NDSS and IEEE S&P from 2019 through 2021, 88 papers in all, and counted what they actually tested on. Twenty percent ran experiments on real ML systems. Fifty-one percent released their source code. Their title is the practitioner’s complaint: “Real Attackers Don’t Compute Gradients.”

Their recommendations to companies are modest, and they are roughly the list I would hand OpenAI here: run bug-bounty programs with clear interaction guidelines, dedicate staff to processing researcher requests, publish high-level schematics of the system so outsiders can design realistic experiments. The argument is not that companies should open-source their production stack. It is that a security claim about a closed system needs a route by which somebody outside can try to break it and report what happened.

OpenAI has part of that route already, which is why I think the September disclosure is the stronger document. It credits independent security researchers for bringing cross-model and conversation-compaction attack paths through responsible disclosure, and says the company investigated and “confirmed that the attack paths they identified were real.” That sentence is the only load-bearing finding in either post that originated outside the company. It is also not a rate. It is a yes.

The enforcement numbers have no denominators either

The November post says weekly enforcement against distillation campaigns has “ranged in scale from thousands to tens of thousands of accounts, with the rate increasing significantly this past summer.” September is more specific. Activity began on 1 July at low volume; spikes on 24 and 25 July amounted to 16,000 requests using a relevant extraction pattern from over 4,000 users; investigation found related prompt-pattern activity across a cluster of more than 15,000 users, disrupted by 28 July. OpenAI attributes a core cluster to individuals associated with Moonshot AI, the developer of Kimi, the same lab that showed up in Anthropic’s threat-intelligence report in September.

The published record of one distillation campaign Timeline from OpenAI's own posts. Activity begins 1 July 2026 at low volume. On 24 and 25 July, 16,000 requests using a relevant extraction pattern arrive from over 4,000 users. By 28 July a related cluster of more than 15,000 users is disrupted. On 30 September OpenAI discloses the campaign and attributes a core cluster to individuals associated with Moonshot AI. On 20 November it publishes the attack-success-rate table for its updated safeguards. ONE CAMPAIGN, AS DISCLOSED · OPENAI, 30 SEP AND 20 NOV 2026 1 July 2026: earliest observed activity, low volume — OpenAI, 30 Sep 2026 1 Jul Activity begins, low volume 24-25 July 2026: 16,000 requests from over 4,000 users — OpenAI, 30 Sep 2026 24-25 Jul 16,000 requests, 4,000+ users 28 July 2026: cluster of more than 15,000 users fully disrupted — OpenAI, 30 Sep 2026 28 Jul 15,000+ user cluster disrupted 30 September 2026: disclosure and attribution — OpenAI, 30 Sep 2026 30 Sep Disclosed; core cluster attributed 20 November 2026: attack success rate table published — OpenAI, 20 Nov 2026 20 Nov ASR table published The September post footnotes its own counts: "These figures describe attempted, not necessarily successful, extractions."
Dated and specific on the attacker's activity. Silent on how much reasoning actually left the building.

The footnote on those figures is the most honest line in either post: “These figures describe attempted, not necessarily successful, extractions.” So the record gives a count of attempts and a rate of success, measured separately, one by production logging and one by an internal red team, and no way to multiply them. Sixteen thousand attempts at an unstated success rate, against a defense whose success rate is reported for a different set of attempts entirely.

What I would want before believing 0.2%

Three things, none of which require giving away anything an attacker could use. How many attempts does the 0.2% divide? One failure in 500, or one in 5,000? What budget did each attacker model get, in turns and tokens, since an adaptive attacker that gives up early produces a lovely number? And did anyone outside the building replay the attack suite against the shipped system?

The September post shows the company can answer that third question in the affirmative when it wants to, because outside researchers found real attack paths and it said so. The November post, which is the one making a quantitative claim, does not. Until it does, 54.0 to 0.2 is a statement about the relationship between two versions of a model and one red team, which is worth something, and not a statement about what an adversary can get, which is what it will be read as.

References

  1. OpenAI, “Combatting adversarial distillation,” 20 November 2026. openai.com/index/combatting-adversarial-distillation
  2. OpenAI, “Disrupting a coordinated model-distillation campaign,” 30 September 2026. openai.com/index/disrupting-a-coordinated-model-distillation-campaign
  3. Ma X, Jiang L, Huang H, Weng T-W, Bailey J, Jiang Y-G. “Imbalanced gradients: a subtle cause of overestimated adversarial robustness.” Machine Learning, May 2024. doi:10.1007/s10994-023-06328-7
  4. Apruzzese G, Anderson HS, Dambra S, Freeman D, Pierazzi F, Roundy K. “‘Real Attackers Don’t Compute Gradients’: Bridging the Gap Between Adversarial ML Research and Practice.” 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), February 2023. doi:10.1109/SaTML54575.2023.00031