On 5 November METR published an independent assessment of how much AI is speeding up AI research at Anthropic. A day later Ajeya Cotra, who works at METR but was not on the project, wrote it up on her blog under the title “AI is dramatically accelerating AI research right now,” and it reached the Hacker News front page. Her reading is that Anthropic has already crossed the “dramatic acceleration” threshold in its own Responsible Scaling Policy, and that the report is “the most important update we have gotten about AI risk in the last couple of years.”
The report finds that since October 2025, benchmark scores for Anthropic’s models have been growing at “an estimated 1.9x the pre-October 2025 rate [95% confidence interval: 1.5–2.8x], implying that Anthropic has made roughly two years of historical progress over the last year.” METR’s staff then judge that “perhaps ≈80%” of the current acceleration is due to AI.
I read the whole public report, appendices included. The work is careful and unusually frank about its limits. But the headline is the least stable number in it, and the public can’t check the step from “faster” to “faster because of AI.”
What 1.9 measures
The yardstick is AECI, Anthropic’s internal version of the Epoch Capabilities Index. It combines many benchmark scores into one latent score per model, and METR uses Epoch’s public code to estimate it. To find acceleration, METR fits a line with one break in it, picks the best-fitting break date (7 October 2025 in the main specification), and reports the ratio of the slope after the break to the slope before. A ratio of 2 means two years of the old pace in one year.
The definition matches the policy’s. Anthropic’s RSP, version 3.4, effective 8 July 2026, says the threshold is met when it will “observe or expect double the rate of progress in AI aggregate capabilities compared to both the rate we’d expect and the fastest rate of extended progress we’ve observed in the absence of significant AI contributions to AI R&D,” and when “it is plausible that this doubling is substantially attributable to the automation of research and/or engineering.” A footnote defines “double the rate of progress” as “as much progress in one year as one would see in two years at baseline.”
The data include four internal models with codenames: Unreleased 2022, plus Models 2, 3 and 4, which finished training between July and October 2026. Those three recent ones are why METR’s 1.9x is higher than the 1.7x in Anthropic’s own Haiku 5.5 system card. METR says the inclusion of “Anthropic’s recent internal-only models” is “the most important difference.”
The curve under the curve
AECI assumes a model’s score on a benchmark is a logistic function of its latent ability. That is a claim about how difficulty is distributed across a benchmark’s tasks, including how many are broken or easy to guess. METR re-ran the analysis with five other link functions, and this table is where the robustness range of “1.1x and 2.9x” in the summary comes from.
METR is direct about why it kept the default: “The quality of fit is fairly similar across each of these link functions (except cauchit), and we do not have strong a priori reasons to prefer one. We continue to use logistic for our main results since it is the default link function used by Epoch and Anthropic.” That’s a reasonable convention, but a convention isn’t evidence for one shape over another. Out-of-sample R² for the logistic, probit, complementary log-log and log-log fits sits between 86.9% and 87.2%, and among those four the estimate ranges from 1.1x to 2.9x. The data can’t distinguish between “barely faster” and “nearly three times as fast.”
The other robustness checks are much tighter. Varying which snapshots count gives 1.9x to 2.2x, varying which benchmarks count gives 1.9x to 2.2x, and varying the regularizer gives 1.9x to 2.1x. So most of the analyst’s choices barely move the result. The one that does is the one with no principled answer.
Two more details push in opposite directions. Fitting the trend over all frontier developers instead of only Anthropic gives 1.4x. METR offers two readings of that: Anthropic was behind the frontier before late 2025, so its earlier slope was low and 1.9x overstates the change; or other labs’ early models were more tuned to benchmarks, so the 1.4x understates it. The second detail is saturation. Model 4 scores above 50% on every one of the 15 benchmarks it was tested on, while Opus 4.6, nine months earlier, scored under 50% on many. METR says this “makes us somewhat nervous about our ability to measure AECI now and in the future.” When it drops the three benchmarks with flat or concave fits, Model 4’s score moves from 178.6 to 179.7. The change is small, but the index is running out of tests that are hard for the newest model.
Twelve is not two
The number that made the rounds next to 1.9x was 12x. It comes from Slack polls in which staff react with emoji to estimate their speedup from the latest model. METR calls this “only very loose evidence,” since the responses “come from an unrepresentative subset of staff” and “reflect fairly low-effort consideration.” For the most capable internal model in October 2026, the average self-reported uplift was about 12x.
The RSP already warns against treating these as the same quantity. Its footnote says doubling the rate of progress “is not the same idea as ‘doubling researchers’ productivity,’ since doubling inputs does not necessarily double the rate of progress.” Cotra’s post turns 1.9x into a labor speedup with a short chain of arithmetic. I’ve laid it out below, because she only gets close to 12x at the last step.
She argues that if software progress is one input to “effective compute,” the compute-multiplier rate needs to grow more than 2x, “perhaps ~2.5-3x,” which she rounds to 2-3x. If labor and experiment compute contribute equally to finding those multipliers, labor uplift must be the square, “(~2-3x)^2 = ~4-9x.” And because “ideas are getting harder to find” and the human workforce isn’t growing any faster, the uplift needed today is higher still, “maybe the necessary labor uplift is ~10x.” She says this “lines up surprisingly well” with the 12x self-reports.
She says the chain is fragile. Her footnote says the range “omits uncertainty on the size of the contributions: for example if labor contributed ⅔, you would instead need to raise it to the power of 3/2.” By my arithmetic that gives 2.8x to 5.2x before the diminishing-returns adjustment, against 4x to 9x with the square. The match with 12x depends on an exponent nobody has measured for Anthropic.
Of the three steps, the diminishing-returns one has the best outside support. Bloom, Jones, Van Reenen and Webb found in the American Economic Review in 2020 that “research effort is rising substantially while research productivity is declining sharply,” and that “the number of researchers required today to achieve the famous doubling of computer chip density is more than 18 times larger than the number required in the early 1970s.” If that pattern holds inside a lab, a constant speedup in labor buys a falling rate of progress, and you need a rising speedup just to stay level.
The weaker link is the poll itself. METR’s own 2025 randomized trial is the best-known test of whether people can judge their own AI speedup. Sixteen experienced open-source developers did 246 tasks, each randomly assigned to allow or forbid AI tools.
Before they started, the developers forecast that AI would cut their completion time by 24%. Afterwards they estimated it had cut it by 20%. The trial found that “allowing AI actually increases completion time by 19%.” That was early-2025 tooling on open-source repositories, so I don’t read it as a claim that Anthropic researchers are slower. What it shows is that people’s sense of their own speedup can be wrong in sign, not just in size. METR links to that study in the report for exactly this reason, noting that self-report surveys “can overestimate true productivity impact.” I made the same point about OpenAI’s internal agent-spend figures in September: an input someone reports is not an output anyone measured.
The interviews were more measured than the polls. METR had Claude sort the interview notes by whether each researcher supported a 2x or greater acceleration in their area. Of eleven people: two yes, three leaning yes, four unclear, none leaning no, two no. METR calls that “mildly in favor.”
The attribution is in the redacted part
The 1.9x is the part of the report outsiders can mostly reconstruct. Whether the acceleration comes from AI is not. METR says it was “able to confidently determine that the acceleration was not due to an acceleration in the rate of growth of training compute,” and that “further details on this evidence were redacted.” For other causes, “there was not a consensus among Anthropic’s own staff and we did not find decisive evidence.” The ~80% figure is labeled an “overall subjective estimate.”
The report’s opening says that “Anthropic redacted a majority of the evidence we found most informative for these conclusions.” METR’s own table shows what made it into the public version: benchmark scores of internal models, yes; key intermediate measures, algorithmic progress estimates, measurements of R&D inputs, and bottlenecks and forecasts, no; evidence of AI’s contribution and staff interviews, partly. On the evaluation checklist METR fills in at the end, the item “The evaluator shared sufficient methodological details to allow for independent review of the results” is marked “No,” with the note “This was not possible due to redaction requirements.” The access was substantial: 2.5 full-time-equivalent staff, eleven weeks, about 15 interviews, and a full report of roughly 100 pages. But METR never had direct access to Anthropic systems, and Anthropic dropped a planned check of how its compute is split between capabilities and safety partway through.
Cotra, who agrees with the conclusion, says the same: “I wish much more of the underlying evidence was transparent so readers don’t have to rely on trust.” The report’s own authors put it more bluntly: “the public has to rely on METR’s judgment much more than would be ideal, given the weight of the questions answered here.”
Anthropic’s prospectus contains a risk factor that fits next to all this. In the S-1 it filed on 26 October, which I went through for its three bottom lines, the company writes: “As we increasingly leverage Claude to accelerate progress within Anthropic, our organization may lose visibility across certain processes in a way that may become difficult for any individual or group of humans to understand and control.” The same paragraph says its models “write a large and growing share of our code and increasingly perform our research.”
What I’d take from it
Most evaluations of lab claims amount to a press release and a benchmark chart. This one comes with a confidence interval, a robustness appendix that undercuts its own headline, and a list of the ways it could be wrong in each direction. I’d like every lab claim of this size to arrive in that form.
What the public part supports is narrower than “dramatic acceleration has been crossed.” Anthropic’s benchmark-index trend bent upward around October 2025. How much depends on a curve-shape assumption the fit cannot settle: 1.1x on one link function, 2.9x on another, 1.9x on the default, 1.4x if you compare against the whole industry. The index is running short of benchmarks that Model 4 can’t half-solve. And the claim that AI is the cause, which is the claim the RSP threshold and every compounding scenario depend on, rests on evidence that two organizations have seen and nobody else can.
The 12x poll can’t fill that gap. It agrees with 1.9x only through a squared exponent and a diminishing-returns guess, and the closest controlled test of self-reported speedup found the sign reversed.
Two things would change my view: the next data point, and what gets published with it. If the model after Model 4 lands where a 2x slope predicts, and its AECI is reported with the per-benchmark scores and a returns-to-tokens curve, the link-function question gets easier, because more points past the break narrow which shapes fit. The attribution question stays open until the intermediate measures on algorithmic progress are public in some form. METR has shown it can get that evidence. Whether anyone else sees it is up to Anthropic.
References
- METR (2026). Independent Assessment of AI R&D Acceleration at Anthropic in October 2026. METR, 5 November 2026; checklist updated 6 November 2026.
- Cotra, A. (2026). AI is dramatically accelerating AI research right now. Planned Obsolescence, 6 November 2026.
- Anthropic (2026). Responsible Scaling Policy, Version 3.4. Effective 8 July 2026.
- Anthropic, PBC (2026). Form S-1 Registration Statement. U.S. Securities and Exchange Commission, filed 26 October 2026. Risk Factors.
- Bloom, N., Jones, C. I., Van Reenen, J. and Webb, M. (2020). Are Ideas Getting Harder to Find?. American Economic Review, 110(4), pp. 1104-1144, April 2020.
- Becker, J., Rush, N., Barnes, E. and Rein, D. (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. arXiv:2507.09089, submitted 12 July 2025.
- Epoch AI. Epoch Capabilities Index and public estimation code. Accessed 7 November 2026.