Papalocal
Loading…
News
← All news
Technology

AI safety auditors find Anthropic's models improving twice as fast — and AI is driving it

A rare 11-week inside look at Anthropic found its AI models advancing at double last year's pace, with roughly 80% of that speedup attributed to AI tools accelerating the company's own research — and the most important evidence was redacted.

How does this story make you feel?

Anthropic's AI models are now improving roughly twice as fast as they were a year ago — and the main engine of that acceleration is AI itself. That is the central finding of an independent 11-week assessment published Thursday by METR, an AI safety research organization, following unprecedented access to the company's internal research data. The report is one of the clearest documented cases yet of AI materially speeding up its own development, and it arrives at a moment when the pace of that feedback loop is precisely what safety researchers say the public most needs to understand.

METR's team — equivalent to 2.5 full-time staff — worked inside Anthropic from August 12 to October 29, 2026, reviewing benchmark scores for unreleased models, training metrics, internal analyses, and conducting roughly 15 interviews with Anthropic researchers. Rather than accepting payment from Anthropic, the organization used Anthropic devices throughout the engagement for security purposes, The Decoder reported.

The headline number comes from the AECI, a composite metric bundling many benchmarks. Up until October 2025, Anthropic's models were adding roughly 15 AECI points annually. Since then, the rate has hit 29 — meaning the company achieved in roughly one year what previously took two, a jump The Decoder describes as roughly comparable to the leap from Claude 2.1 to Sonnet 4.5. METR is confident this acceleration is not simply the result of scaling up computing power, which has historically been the dominant driver of AI progress.

Now only bottlenecked on ideas & compute, no longer bottlenecked on code.— Anthropic employee (unnamed), quoted in METR report

METR estimates that approximately 80 percent of the current speedup traces back to AI being used in Anthropic's own research and development. Employees reported in informal Slack polls from October 2026 that the company's latest internal model made them roughly 12 times more productive on average — double what staff had reported just months earlier while using an earlier internal model called Mythos 5. Experiments that once took one to two days to set up are now described by employees as 'approximately free.' METR cautions that these self-assessments are 'very loose evidence,' likely unrepresentative and prone to overstating actual gains.

At present, the acceleration stems from AI supporting human researchers rather than operating on its own. METR also cautioned that "there are some signs that fully autonomous contributions will play a larger role soon," and the researchers said they found no clear bottlenecks that would slow the pace over the next 12 months.

The stakes of those findings are sharpened by Anthropic's own Responsible Scaling Policy, which flags a doubling of AI progress due to automated research as an early warning indicator for risks including recursive self-improvement and the sudden emergence of dangerous capabilities, according to The Decoder. METR considers it possible that Anthropic crossed that threshold as early as July 15, 2026. In a companion review of Anthropic's public risk report from August 2026 — in which the company stated the acceleration was 'less than a factor of 2' — METR found that the internal, unredacted version implied a value of 1.97, and that Anthropic itself called it 'plausible' that the threshold had been crossed. In METR's view, the public report downplayed both the degree of acceleration and the uncertainty surrounding it.

Three of Anthropic's newest models remain unreleased. Three models, given the working names Model 2, Model 3, and Model 4, finished training in July, August–September, and September–October 2026, respectively, according to METR. The researchers noted these are the codenames shared with them and that the models have different internal codenames. A fourth unreleased model dating to 2022, called Unreleased 2022, was also identified.

The report's most significant limitation is also its most visible: Anthropic held unilateral redaction rights over all non-public information in the final published version, and used them extensively. Of the roughly 100-page full report METR delivered to Anthropic, the publicly released version had most of its highest-rated evidence stripped out, among it forecasts for future models, breakdowns by training steps, and the bottleneck analysis. One chart in METR's published report carries the words 'FAKE DATA' in large letters where real figures were removed.

We submitted a ~100-page full report on our assessment to Anthropic. For the public version, Anthropic redacted a majority of the evidence we found most informative.— METR

Anthropic said its redactions were "aimed at removing confidential information that could accelerate other AI developers." METR noted that a large share of what was removed had "multiple plausible interpretations," leaving the public with limited ability to evaluate the findings on their own. The report added: "As a consequence, the public has to rely on METR's judgment much more than would be ideal, given the weight of the questions answered here."

The findings are not without internal dissent. One Anthropic researcher put AI's share of productivity gains at just 15 to 20 percent and expressed distrust of colleagues' self-reports. Another considered new hires the single most important factor in recent progress. When the analysis is broadened to include models from OpenAI, Google, and xAI, the industry-wide acceleration figure falls to about 1.4 — a result The Decoder attributes in part to Anthropic having trailed competitors in 2025, with a subsequent catch-up effect inflating its individual numbers.

On the question of near-term danger, METR aligns with Anthropic in rating the acute catastrophic risk from current models as "low." Even so, the organization said it cannot rule out another sharp acceleration over the next six months that could quickly push that risk rating higher. The assessment is also separate from a broader program Anthropic announced in October in which METR researchers are to receive 'employee-like' access to systems — a more expansive arrangement than what was granted for this audit.

The report lands as Anthropic is preparing to go public, and as CEO Dario Amodei has publicly called for AI labs to slow the pace of development. METR noted that Anthropic shared more information than was initially scoped for the project, 'surpassing our expectations' — but also that a small number of data requests were denied and that the pipeline of information was 'relatively narrow.'

Why it matters — If AI is genuinely doubling the pace at which it improves itself — and that feedback loop has no visible ceiling — the timeline to systems that can operate beyond meaningful human oversight may be compressing faster than safety frameworks, regulators, or the public currently assume.

⚠ Not yet confirmed

  • The true acceleration factor could be as low as 1.3 or as high as 3.
  • When models from OpenAI, Google, and xAI are factored in, the industry-wide acceleration figure drops to approximately 1.4, possibly due to a catch-up effect from Anthropic lagging competitors in 2025.
  • Anthropic is preparing to go public.

Sources differ on AI's share of Anthropic's productivity gains: Approximately 80% of the acceleration is due to AI, per METR's overall estimate (the-decoder.com) vs One Anthropic researcher put AI's share at just 15–20% and distrusted colleagues' self-reports; another cited new hires as the single most important factor (the-decoder.com)

Sources differ on Whether Anthropic's acceleration has crossed its own Responsible Scaling Policy warning threshold: Anthropic's public August 2026 risk report stated acceleration was 'less than a factor of 2' (the-decoder.com) vs METR found the internal unredacted version implied a value of 1.97; Anthropic called it 'plausible' the threshold was crossed; METR considers July 15 a possible crossing date (the-decoder.com)

Reported by theverge.com, metr.org, the-decoder.com

People in this story

Community

Be the first to comment.

Share this story
Text Email