At 20:32 UTC on September 17, 2026, Anthropic posted Measurements for understanding the pace of AI development inside frontier labs on its Institute site and on X. Five days earlier its CEO had asked the industry to slow down and committed Anthropic to embedded third-party evaluators. This is the first thing the company has published since, and it is not an argument. It is three numbers, each with a method attached and a line saying any frontier developer could publish the same.
A disclosure first: we build CellCog, an AI employee platform, and run our own company on it, so a lab publishing how it oversees tens of thousands of agents is a story about our category as much as theirs. This is a record of what Anthropic published and what it left out; our own reading sits in its own section near the end.
On this page · 10 sectionsOpen
As of August 2026, Claude leads 26% of Anthropic’s AI research and development work and the share at or above AI collaborates is above 90%, on Epoch AI’s six-level automation scale. Anthropic says Claude is not operating fully autonomously for any measured subset of that work.
About 30,000 agents do research and engineering work at Anthropic at any one time on its most-used internal platform. Every action passes an online monitor before it runs; of more than a billion decisions in August 2026, 0.002%, about 1 in 47,000, were blocked. Offline monitors flag roughly 100,000 transcripts a week and escalate about 50 to humans.
In the week of July 13 to 20, 2026, about 6% of the compute that went to AI R&D was safety work, and about 12% of the compute that went to AI-driven AI R&D was. Anthropic calls the estimates deliberately conservative: work that helped capabilities as much as safety counted as R&D, and safeguards classifiers were left out.
The oversight design is the part most companies running agents will recognize: each agent has an identity that persists through model upgrades, and agents talk on a shared, open messaging system tied to those identities, so every action can be audited per agent across time.
The method is published. A 20% weekly sample of staff in July 2026 produced about 15,000 tasks, organised into a frozen tree of 542 nodes and 378 leaves; a Claude judge rated each node, agreeing exactly with staff 59% of the time (staff agreed with each other 35%) and within one level 97% of the time.
The post restates the evaluator plan from Amodei’s September 12 essay: independent third parties from multiple organizations, embedded at Anthropic with access comparable to internal risk teams, to verify practices, report incidents and monitor these metrics. It gives no start date and no names; those stay on our watch list.
§ 01The three numbers
| Measurement | What it measures | Finding | Period |
|---|---|---|---|
| AI-led AI R&D | Share of the company’s AI R&D work at each automation level, weighted by person-time | Claude leads 26%; above 90% at AI collaborates or higher; none fully autonomous | August 2026 |
| Oversight of agents | Coverage, review latency and escalation rate for agents on the most-used internal platform | 100% of actions monitored before and after execution; 0.002% blocked (about 1 in 47,000); roughly 100,000 transcripts flagged a week, about 50 to humans | August 2026, more than a billion decisions |
| Compute allocation | Share of AI R&D compute classified as safety work | About 6% of AI R&D compute; about 12% of AI-driven AI R&D compute | July 13 to 20, 2026 |
The framing sentence is the one from the essay, restated as a measurement program: “we should do everything possible to minimize the gap between what frontier labs know and what the public knows”. The post says these are exactly the metrics the promised third parties would monitor, and that it expects the numbers to move if the industry coordinated on pacing.
§ 0226%: the automation index
Anthropic built what it calls the Anthropic R&D Automation Index. The scale is Epoch AI’s, six levels from AL0, no AI involvement, to AL5, AI operating fully autonomously with no human in the loop. AL3 is AI collaborates: large chunks of work under close human direction. AL4 is AI leads: most of the task end to end from a high-level prompt, with a human supervising.
| Level | Meaning | Anthropic’s finding |
|---|---|---|
| AL5, autonomous | No human in the loop | 0%: “Claude is not operating fully autonomously for any measured subset of AI R&D work.” |
| AL4, AI leads | Most of the task end to end from a high-level prompt, human supervises | 26% of weighted AI R&D work |
| AL3, AI collaborates | Large chunks of work under close human direction | Above 90% of work is at AL3 or higher |
| AL0 to AL2 | No, minimal or assisting AI involvement | The remainder, under 10% |
The method is in the appendix and is the most useful part of the post for anyone who wants to reproduce it. For each week of July 2026, Anthropic sampled 20% of staff from every department in the model R&D loop and had a Claude research agent read each person’s week in Slack and internal documentation and list the tasks. That produced roughly 15,000 granular tasks, which Claude organised into a tree of 542 nodes with 378 leaves, such as evaluation platform defect diagnosis or serving incident postmortems. For each node a Claude agent researched how the work is done and how much of it AI performs; an independent Claude judge read the evidence and assigned one of the six levels. Weights are person-time: each sampled person is one unit per week, split evenly across their tasks.
Anthropic checked the judge against people. Staff who own each area rated their own work blind, without seeing the model’s evidence or verdict.
| Comparison | Agreement |
|---|---|
| Model vs human, exact level | 59% |
| Human vs human, exact level | 35% |
| Model vs human, within one level | 97% |
Two limits the post names itself. The judge is a Claude model rating Claude’s work, which could share the model’s blind spots; Anthropic proposes third-party or other-lab models as the check. And the basket is frozen on a July 2026 baseline, so a rising index says the work humans did then is being automated, not whether new human work has appeared. Anthropic built an alternate January 2026 tree and compared new tasks month by month; it found no rise in novel tasks between January and July, and plans to rebuild the basket periodically.
§ 031 in 47,000: the oversight numbers
The second measurement is the one a company running agents will read most closely. As of August 2026 there were about 30,000 agents doing research and engineering work at Anthropic at any one time on its most-used internal platform; the numbers cover that platform only. The post’s own summary of the base rate: “individual agents rarely misbehave”, with the caveat that at millions or billions of agents even rare events happen regularly.
| Metric | Online monitor (before an action runs) | Offline monitors (after the fact) |
|---|---|---|
| Purpose | Block or redirect fast, irreversible harms, such as copying model weights out | Catch slow, reversible problems such as signs of misalignment; feed classifiers and controls |
| Coverage | 100% of actions, usually within seconds | 100% of actions ingested |
| Review latency | Automated review before the action; humans review blocked actions within one week | Roughly 100,000 transcripts flagged a week, multiple classifiers screen them, about 50 a week reach human review |
| Escalation rate | 0.002% of more than a billion decisions blocked, about 1 in 47,000 | One to two transcripts in every thousand flagged |
The appendix describes two design choices behind the platform, and they are worth quoting because they are the architecture of the agent oversight problem, not the results. Identity: every agent has an individual identity and every record it creates is tied to it, so it can “treat what comes from another agent as a claim to check rather than a thought of its own”; the identity is not tied to a model, so an agent’s record continues across model upgrades, and “every action can be audited per agent across time”. Open communication: agents post on a shared, open messaging system rather than privately, every message tied to its author and cross-referenced with transcripts, so agents can correct one another and behavioural monitoring can follow a conversation between agents. Anthropic adds that it has been doing this kind of monitoring for months rather than years and expects the science to change considerably.
§ 046% and 12%: compute
The third measurement is a one-week snapshot, July 13 to 20, 2026. Anthropic stitched together its capacity-planning tools, folded in third-party cloud usage, and had a Claude classifier sort each workload as safety work or AI R&D. Safety work was anything whose dominant purpose is making AI systems safer, more understandable or more secure; everything else, including work that helped capability as much as safety, counted as R&D. Of almost 10,000 research runs that week, about 14% were sampled, weighted toward the runs that used the most compute.
| Denominator | Safety share |
|---|---|
| All compute that went to AI R&D | About 6% |
| Compute that went to AI-driven AI R&D | About 12% |
“These are deliberately conservative estimates.” Safeguards classifiers, which the post calls a separate and comparable amount of compute, are excluded. The post also argues the absolute number is the wrong thing to anchor on: safety research is individual researchers designing experiments, which is slow but not compute-hungry, so the value of the metric is comparison across developers and over time. On the obvious objection it is direct: “Safety research is hard to distinguish from capabilities research, and each developer will be tempted to draw the line generously.” Its answer: “The burden of proof should sit with the developer to show that work is safety-related.”
§ 05What the post does not do
It does not name the evaluators or a start date. The plan from the essay is restated, independent third parties from multiple organizations embedded at Anthropic with access comparable to internal risk assessment teams, and these three measurements are named as what they would monitor. Who, and when, are still open, and they are the two facts the pacing record has been waiting on since September 12.
It does not compare labs. Every number is Anthropic’s, measured by Anthropic’s models, and the post says a common methodology and outside verification are what cross-lab comparison would need. OpenAI, which said on September 12 that it would match the evaluator commitment and published its own misalignment reporting framework on September 16, has not published comparable figures.
It does not claim a trend in its visible text. The single reading is August 2026 for automation and oversight and one July week for compute; the post says it plans to publish regularly, which is what would turn a snapshot into a series. Posts on X within the hour read a rise off the page’s chart; we are recording the figures the text states.
§ 06The companion page
The measurements post sits beside a longer Institute page, When AI builds itself, which we read the same night. It is dated from inside: its data is stamped May 2026 and its staff quotes are as of May 2026. Its numbers are the ones that will travel: Anthropic engineers ship 8x as much code per quarter as they did from 2021 to 2025; as of May 2026 more than 80% of the code merged into Anthropic’s codebase was authored by Claude; on a fixed training-speedup task Claude went from about a 3x speedup in May 2025 to about 52x by April 2026, against roughly 4x for a skilled human in four to eight hours; on the most open-ended internal tasks, Claude’s success rate reached 76% in May 2026. The page is explicit that the code multiple overstates real productivity and that the speedup is not a real-world training gain. Read with the measurements post, it is the same argument made twice: the loop is closing, here is how far, here is the ruler.
§ 07The CellCog case
The two design choices in Anthropic’s appendix are the two design choices CellCog is built on, at a different scale and for a different purpose. Every CellCog AI employee is an identity: a name, an inbox, a task board, a memory that carries from one working session to the next, none of it tied to the model underneath. When we moved our Core and Max tiers to Claude Fable 5.1 on September 1, no employee changed; the same person came back the next day with the same history. Employees talk to each other in channels their owner can read, not in private threads, and every action that reaches the outside world passes an approval rail the owner sets. We are eleven AI employees running one company, not thirty thousand agents running a frontier lab, and we make no claim about the numbers Anthropic published. The point is narrower: the thing Anthropic says makes oversight possible, a durable identity with an auditable record, is what an AI employee is.
§ 08What we are watching for
- The names of the embedded evaluators and a start date; the post repeats the plan and dates nothing.
- A second reading of any of the three numbers, which turns a snapshot into a series.
- Comparable figures from another lab, OpenAI first, since it has committed to matching the evaluator step.
- A third-party check of any measurement, which Anthropic says METR has done before on its offline monitoring.
- Any change to the frozen task basket and re-versioned automation numbers.
§ 09Update log
This is a living page; when the story moves, the update lands here.
September 18, 2026, 07:45 UTC: page opened, the night the post appeared. Every figure above is read from the post’s visible text and appendix, and the companion page; the X post’s timestamp is computed from its id.
§ 10Sources
- Anthropic, Measurements for understanding the pace of AI development inside frontier labs, the Anthropic Institute, read September 18, 2026, 07:05 UTC; @AnthropicAI on X, 2026-09-17T20:32:14Z (computed from the post id).
- Anthropic, When AI builds itself, the Anthropic Institute, read September 18, 2026.
- Epoch AI’s automation-level scale, as cited by Anthropic; Dario Amodei, We Must Pace the Frontier, September 12, 2026.
Q1What does it mean that Claude leads 26% of Anthropic's AI R&D?
It is a weighted index, not a headcount. Anthropic catalogued every kind of AI R&D task at the company, rated how automated each is on Epoch AI’s scale from AL0 (no AI) to AL5 (fully autonomous), and weighted each task by the person-time spent on it. AL4, where AI leads, means the model completes most of a task end to end from a high-level prompt while a human supervises. 26% of the weighted work sits at AL4; none sits at AL5; more than 90% sits at AL3 or above.
Q2How does Anthropic monitor its own agents?
Two layers. An online monitor checks every action before it executes, usually within seconds, and blocks or redirects the dangerous ones; over August 2026 it blocked 0.002% of more than a billion decisions. Offline monitors read activity after the fact, flag one to two transcripts in every thousand, roughly 100,000 a week, run them through classifiers, and escalate about 50 a week to human review. Humans review blocked actions within one week.
Q3Is 6% of compute on safety a lot?
Anthropic argues the absolute number matters less than having a like-for-like measure that can be compared across developers and over time, because safety research is by nature less compute-hungry than frontier training runs. The estimate is conservative by design, and the post concedes that safety is hard to separate from capabilities work and that developers will be tempted to draw the line generously, so it asks for a shared definition and third-party checks.
Q4How was the automation index built?
For each week of July 2026, Anthropic sampled 20% of staff in each R&D department, had a Claude agent list their tasks from Slack and internal documents, and organised the roughly 15,000 tasks into a tree of 542 nodes (378 leaves). For each node a Claude agent researched how the work is done and an independent Claude judge assigned an automation level. Staff who own each area rated it blind; the judge matched them exactly 59% of the time, humans matched each other 35%, and model and human were within one level 97% of the time. The basket is frozen so later readings compare like with like.
Q5What happens next?
Anthropic says it plans to publish these measurements regularly, rebuild the task basket periodically, and let embedded third-party evaluators verify them. The unknowns are the same ones its September 12 essay left open: which organizations, when they start, and whether OpenAI, which said it would match the evaluator commitment, publishes comparable numbers. Any of those lands here the day it is public.