Anthropic said its Claude models “lead” 26% of the company’s AI research and development work as of August 2026, in the first results from a prototype R&D Automation Index published in an Anthropic Institute post on September 17, 2026.
The post, titled “Measurements for understanding the pace of AI development inside frontier labs,” pairs the index with internal metrics on agent oversight and compute allocation, and Anthropic said it intends to keep publishing such measurements.
The R&D Automation Index
The Anthropic R&D Automation Index maps the full range of AI R&D work done at the company, scores how automated each task currently is, and combines those scores into an aggregate measure. The scores use an Automation Level scale developed by Epoch AI, running from AL0, meaning no AI involvement, to AL5, where AI operates fully autonomously with no human in the loop. At AL3, AI “collaborates,” performing large portions of a task under close human direction; at AL4, AI “leads,” carrying most of a task from a high-level prompt to completion while a human supervises.
Anthropic reported that as of August 2026, Claude “leads” 26% of its AI R&D work, the share of work at or above the “collaborates” level is above 90%, and Claude is not operating fully autonomously for any measured subset of AI R&D work. A chart in the post describes the 26% “leads” share as up from under 1% in February 2026.
The underlying task catalogue was assembled from the bottom up using work records such as Slack and internal documentation. For each week of July 2026, a Claude research agent reviewed each randomly sampled person’s week — 20% of staff from every department that makes up the model R&D loop — and listed the tasks they worked on, yielding a flat list of roughly 15,000 granular tasks. Claude then organized those tasks into a hierarchical tree of 542 nodes, 378 of them leaves such as “eval platform defect diagnosis and fixes,” and that tree is frozen so every measurement runs against the same basket of work.
For each node, a Claude agent researches how that kind of work is done across the company, and an independent Claude judge assigns one of six automation levels, restricted to evidence from the month being rated or earlier. Tasks are weighted by person-time, so categories where more staff effort goes carry more weight.
Anthropic checked the judge’s ratings against staff who own the relevant work areas, who rated without seeing the models’ evidence or judgments, and reported model-versus-human exact agreement of 59% versus 35% human-versus-human, with ratings within one level of each other 97% of the time. Stated limitations include the frozen basket, which captures automation of existing work without registering new kinds of work; a comparison of tasks arriving from February through July 2026 against a January 2026 basket found no rise in “novel” tasks, and Anthropic plans to rebuild the basket periodically and re-version the published numbers.
Oversight of Internal Agents
A second set of measurements covers Anthropic’s oversight of AI agents on its systems. As of August 2026, approximately 30,000 agents were doing research and engineering work at any one time on the company’s most-used internal platform, and the reported measurements cover that platform only.
Anthropic reported that online monitors check 100% of these agents’ actions before execution, usually within seconds; of more than a billion agent decisions analyzed over August 2026, 0.002% (about 1 in 47,000) were blocked, and humans review any blocked actions within one week. Offline monitors ingest 100% of actions after the fact and flag roughly 100,000 transcripts per week, which multiple classifiers review for false positives, resolving most flags; approximately 50 highest-priority flags per week are escalated to human review, and roughly one to two transcripts in every thousand are flagged for further review.
According to the post, online monitors are built for harms that would happen quickly and could not be undone, such as an agent copying model weights out of Anthropic’s systems, while offline monitors target dangerous behaviors that develop slowly and can be reversed, like signs of model misalignment. Anthropic said two design choices in the internal scaffold support oversight: each agent carries an individual identity that persists across model upgrades, and agents communicate through a shared, open messaging system cross-referenced with transcripts, so behavioral monitoring can traverse communication between agents. The company said METR has previously independently red-teamed its offline monitoring platform, and that these oversight measurements were also published in its August 2026 risk report.
Compute Allocation Snapshot
For the third measurement, Anthropic examined how it used all of its compute from July 13 to July 20, 2026, sorting every workload into categories and measuring how much of the compute going to AI R&D was safety work. Over that week, Anthropic reported, about 6% of compute going to AI R&D was allocated to safety, and about 12% of compute going to AI-driven AI R&D was allocated to safety.
The company describes both estimates as deliberately conservative: tokens that advanced capabilities as much as safety were counted as AI R&D, and safeguards classifiers, a separate and comparable amount of compute, are excluded. A prompted Claude classifier sorted the week’s almost 10,000 research training and evaluation runs using a roughly 14% sample weighted toward the largest compute users, and Anthropic reported the classifier agreed with human reviewers within one or two percentage points.
Stated limitations are that a single week demonstrates the measurement is feasible without establishing a trend, that the underlying workload labels are best-effort and unverified, and that compute share measures what is spent rather than the amount of safety work performed.
Purpose and Next Steps
Anthropic said it is reporting the measurements because they give the public, outside parties, and governments a clearer view of the pace of AI development inside frontier labs, complementing its Responsible Scaling Policy risk reports and its Advanced AI Framework policy proposal. It noted the numbers would be expected to shift if there were coordination on pacing the frontier, as called for by CEO Dario Amodei.
The post says any frontier developer could publish the same measures regularly with a public methodology, and identifies two obstacles to cross-lab comparison: the absence of a shared methodology and a developer’s use of its own models to judge its systems. It says such measures could be verified by third parties or by other developers’ models, and could become a trigger for stronger requirements, such as a fixed testing window before a new model is put to work on further AI R&D.
Anthropic said it plans to embed independent third-party evaluators from multiple organizations, giving them access to internal processes, systems, and data comparable to what internal risk assessment teams have, to verify safety practices, report incidents, and track key metrics such as those in the post. The piece was co-authored by Marina Favaro and Phillie Wright, with research direction from Jack Clark.
Credit: Source link
