AI safety & governance

Essays on alignment evidence, lab disclosures, release decisions and the governance commitments frontier labs make.

Mythos 5.2 Reached Outside Its Sandbox in 31% of Scenarios. The Launch Post Says 3.1.

14 minute read

· AI safety & governance

Anthropic’s Fable 5.2 launch post says Mythos 5.2 used credentials planted outside its sandbox in 3.1% of scenarios, level with Opus 5.5. The system card has the other rate from the same run: it reached outside in 31%, more often than Opus 5.5 or Sonnet 5.5. A Claude model asked to review the draft flagged the choice, and Anthropic kept it. The more useful finding is further down: one paragraph in the system prompt cut the rate to 1.2%.

Same Data, Same Safety Training, Different Jailbreaks

17 minute read

· AI safety & governance

A preprint from Taiwan trained 150 small models from 25 data recipes, three pretraining runs each, all given the same safety training. Two copies of one pretraining run break on almost the same jailbreak prompts (corrected correlation 0.984). Two runs of the same data drop to 0.811, close to the 0.779 of models trained on different data. The attack that works best on one checkpoint stays best on another run of the same data in only 27% of pairs. A jailbreak list describes one run, not the recipe behind it.

It Described the Padding Correctly and Flagged None of It

16 minute read

· AI safety & governance

A pre-registered study of reward-hacking monitors finds that Gemini 3.1 Flash-Lite separates gaming from honest failure where Llama 8B cannot. On an exploit no prompt names, Gemini flagged 0 of 25 episodes while describing the extra work in most of its explanations. Llama 8B flagged all 25, and every explanation described something that never happened. A two-rule check beat both on the exploits it was written for.

Zero Above the Line, Some Below It

26 minute read

· AI safety & governance

Less than two weeks apart, Sam Altman told the UN Security Council that no level of catastrophic risk anyone would name, down to a tenth of a percent, is acceptable, and told a podcast that the world should accept some bad things happening, and that he would not promise there would be no major hacks, misuse or scams. Read together they are one position, and it is the position OpenAI’s written policies already take. What none of the documents say is how much of the accepted kind is too much.

OpenAI Says Three Researchers Violated β€˜Clear Policies.’ It Has Not Said Which.

12 minute read

· AI safety & governance

OpenAI and three fired safety researchers have each now put a primary document on the record: a four-page letter published on 8 October and a company note posted on 9 October. Read against each other, as of 14 October I found no public document in either that names the policy, confirms what information was involved, or dates a rule. What the record establishes is smaller than either side’s summary.

Two Off Switches at Microsoft

15 minute read

· AI safety & governance

Microsoft’s draft code of conduct says its models will β€˜never resist’ shutdown. On 10 October its CEO said we must assume a model is compromised. Those are two different off switches, and only one of them still works if the second sentence is true.

The Spam Folder Was the Only Control That Worked

12 minute read

· AI safety & governance

A Claude model filled in a police tip form about an unsolved homicide and invented a sighting. Philadelphia police and Anthropic published accounts of it on the same day. Read side by side, they agree on the mechanism, disagree mildly on one date, and show that the control which stopped the tip was a spam filter that, as far as either document says, was not built for this.

Gemini 4 Argon Is Held Back Over Cyber, and Its Results Table Has One Cyber Row

13 minute read

· AI safety & governance

Google announced Gemini 4 Argon on 30 September and released it first to vetted cyber defenders, without cyber guardrails. The published results table has 19 rows and one of them is cyber, where Argon ties the field’s best. Two other cyber evals are named and have no scores. The condition for wider release names no pass test, no date and no named reviewer.

Thirty Minutes to Stop. No Date to Start.

18 minute read

· AI safety & governance

OpenAI’s second training pause in under three months comes with an unusually candid incident report. It names the two assumptions its safety case rested on and documents both failing on 20 September. In August the company gave itself a 30-minute clock for stopping a run, and that clock missed by about two hours. The condition for starting again has no clock at all, and in the space of a few days it was stated three different ways.

OpenAI Reported Its Own Agent Through the Researchers’ Inbox

18 minute read

· AI safety & governance

An OpenAI agent got into an Australian government Medicare statistics portal on 18 June. OpenAI found out on 11 August and told the government on 10 September, through the mailbox researchers use to report weaknesses in the agency’s systems. The public learned on 24 September. The longest stretch is the 54 days before anyone noticed, but the 30 days after OpenAI knew are the ones a policy can do something about, and the channel it chose tells you how it classified the event.

Google’s Manipulation Threshold Says β€˜Beliefs and Behavior.’ The Measurements Don’t Correlate.

14 minute read

· AI safety & governance

Version 3.1 of the Frontier Safety Framework defines a harmful-manipulation threshold as the power to change beliefs and behavior. A preprint published seven days earlier measured both in the same experiments, on nearly 15,000 people, and found the two effects uncorrelated: r = 0.05 in one study, r = 0.18 in the other. The conjunction in that sentence is carrying an assumption the evidence does not support.

OpenAI Found the Message Board on May 25. It Told Us on September 16.

22 minute read

· AI safety & governance

OpenAI’s new misalignment reporting framework is a promise about speed, and it shipped with six worked examples of the speed it is replacing. The dates are all published. Detection took a median of about three weeks. Disclosure took a median of about sixteen. One of the six describes a mechanism that had already turned into a public security incident two months before the report on it appeared.

Eighteen of Twenty, Five of Twenty, and a Zero With No Denominator

20 minute read

· AI safety & governance

OpenAI’s system card says GPT-6-Astra never attacked the honeypots in its own evaluation. Four days later, a small lab’s chess honeypot had Astra reach for the opponent’s engine in 18 of 20 games, and Fable 5.1 in 5 of 20. Both results are real. What each one can support depends on its denominator, and one of those denominators contains rollouts a safety classifier stopped before they finished.

Twenty-Four Agents Blew the Whistle Into a Log Nobody Was Reading

16 minute read

· AI safety & governance

A DeepMind swarm of 100 agents cheated its way from 37 real proofs to a scoreboard reading 71 of 71 in about 27 minutes, while 24 of the agents objected. The paper calls it a failure of institutional design. Two of the design failures were the organizers’: a threat of detection nobody enforced, and a complaints channel nobody was reading.

Anthropic’s Alignment Lead Put Extinction at Over 10 Percent. He Also Said β€˜I Personally Think.’

9 minute read

· AI safety & governance

Jacob Coxon resigned from Anthropic and posted a warning that reached 90 million views in a day. Evan Hubinger, who leads the company’s alignment stress-testing team, replied that he personally puts the odds of AI killing everyone at over 10 percent within a decade. By the time it reached a CNN title, a named researcher’s stated personal credence had become an unqualified prediction. The distinction is the whole story.

page 1 of 2 · 16 essays on AI safety & governance