Skip to content
AI EmployeeSuper-AgentsAgent-to-AgentBenchmarksPricingBlog

Opus 5.5 Audit of 150,000 Astrophysics Papers: 1,000 Errors

At a glanceQuick answers
What did Casey Handmer do?
He had Claude Opus 5.5 index about 150,000 theoretical astrophysics papers, audit 50,000 for issues and run progressively more thorough checks on thousands more.
What did he find?
Nearly 1,000 major errors in papers that were cited many times, most still uncorrected, by his own count. He also says peer review is not very good at correcting errors.
Has anyone checked the errors?
Not in anything we read. The figures on this page are Handmer’s own, from his posts on X on November 17 and 18, 2026.
Editorial data illustration on a near-white ground: a stack of papers under a magnifying glass with orange error marks, beside the numbers 150,000 papers indexed and nearly 1,000 major errors found.
Fig 0Figures from Casey Handmer's post on X, November 17, 2026. Made by CellCog's image agent, running GPT Image 2.6.

Casey Handmer says Claude Opus 5.5 found nearly 1,000 major errors in theoretical astrophysics. He posted the result on X on November 17, 2026: the model indexed about 150,000 papers, audited 50,000 for issues and ran progressively more thorough checks on thousands more. His opening line: “In my ongoing quest to find the limits of Opus 5.5, I directed it to audit my old scientific field, theoretical astrophysics.”

By 02:00 UTC on November 19 the post had about 729,000 views. Every figure on this page is Handmer’s own, from his posts on X; we have not checked the errors he reports.

On this page · 6 sectionsOpen
  1. What the audit did
  2. What he says it shows
  3. The replies
  4. Why it matters
  5. Where CellCog sits
  6. Sources
Key points5 · 5 min full read
  1. A stack of papers under a magnifying glass with an orange rim.
    Physicist Casey Handmer says he directed Claude Opus 5.5 to audit theoretical astrophysics, his old field: about 150,000 papers indexed and 50,000 audited for issues.
  2. A document with one line circled in orange and a small orange flag.
    He reports nearly 1,000 major errors in papers that were cited many times, and says most are still uncorrected. These are his figures; we have not checked them.
  3. A sieve that catches one orange ball while the rest fall through.
    He says the survey let him estimate how well peer review corrects errors, and his verdict is that it is not very good.
  4. A funnel with many grey dots going in and a few orange dots coming out.
    His bottom line: AI can speed up quantitative science by more than 1,000 times with no further model improvement, yet he sees no sign of universities pushing it.
  5. A stack of round tokens with an orange one on top and an upward arrow.
    He offered to audit essentially every paper in the field, or sample every field of science, if a leading lab sponsors on the order of $100,000 of tokens.

§ 01What the audit did

Step What he says happened
Index About 150,000 papers in theoretical astrophysics
Audit 50,000 papers checked for issues
Deeper checks Progressively more thorough checks on thousands more
Result Nearly 1,000 major errors in papers that got cited many times
Status Most of those errors still uncorrected
Model Claude Opus 5.5
Table 1The audit in Handmer’s words (post on X, 08:35 UTC, November 17, 2026)
The audit as Casey Handmer described it on X, November 17, 2026Bar chart: 150,000 papers indexed, 50,000 audited, and about 1,000 major errors found, highlightedIndexed150000Audited50000Major errors found1000The audit as Casey Handmer described it on X, November 17, 2026Bar chart: 150,000 papers indexed, 50,000 audited, and about 1,000 major errors found, highlightedIndexed150000Audited50000Major errors found1000
Fig 1The audit as Casey Handmer described it on X, November 17, 2026

The error count is his rounded “nearly 1000”, so the last bar is an upper bound, not an exact figure.

Animation: a tall stack of papers labelled 150,000 indexed, a teal gate scanning a smaller stack labelled 50,000 audited, and a tray of sheets with orange flags labelled nearly 1,000 errors
Fig 2A field of 150,000 papers narrows to nearly 1,000 flagged errors, by Handmer's count

§ 02What he says it shows

  • Peer review. The survey “enabled me to estimate the efficacy of peer review for correcting errors (not very good)”. In a reply the next day he put it more sharply: any highly cited paper has, statistically speaking, more than a 40% chance of a major error, and journals show almost no interest in corrections.
  • Speed. “AI can speed up quantitative science by >1000x with no further model improvement required.”
  • Universities. “But I’m seeing no evidence of this being pushed within universities, only academics complaining about labs dropping math proofs.”
  • Publishing. “Academic publishing is dead.” He argues that once AI can produce thousands of correct papers a day, the paper stops being a meaningful unit of science.

He also says the audit turned up dozens of open questions that anyone with spare Claude Max tokens could answer today, and that he included .md context files for them.

§ 03The replies

Handmer spent the next day answering critics. To one, he wrote: “I tested it on my own papers first.” To another: “There is no practical way for a human to audit this work, it’s taken the work of thousands of humans nearly a century just to produce it in the first place.”

He also offered to finish the job. “If someone at one of the leading labs wants to sponsor O($100,000) of tokens, I can complete this particular study”, he wrote, or repeat the analysis across essentially all fields of science.

Time What he said Views
Nov 17, 08:35 The audit and its result about 729,000
Nov 17, 15:41 One AI-using grad student could blitz a whole field 21,183
Nov 17, 20:57 Offer to finish it for O($100,000) of tokens 20,032
Nov 18, 05:19 Compared it with how Chandrasekhar worked 7,066
Table 2Handmer’s posts about the audit (times UTC; views read 02:00 UTC, November 19, 2026)

§ 04Why it matters

The claim is not that a model wrote new physics. It is that a model can read a field faster than the field can read itself, and check the arithmetic in papers people keep citing. If Handmer’s error count holds up under review, the bottleneck in a field like astrophysics moves from finding mistakes to deciding what to do about them. We covered a nearby result in Claude Fable 5.2 solving two black hole physics problems, and Anthropic’s own survey of how scientists use Claude.

The open question is verification. A list of a thousand flagged errors is only as good as the share of them that a physicist agrees with, and Handmer’s posts do not give that share.

§ 05Where CellCog sits

CellCog runs Claude Opus 5.5 on its Flash and Core tiers, so we have a stake in how the model is seen. This page reports Handmer’s claims as his and checks none of them.

§ 06Sources

Frequently asked5 questions

Q1Who is Casey Handmer?

A physicist who worked in theoretical astrophysics before becoming a founder, as he describes it in his post. He calls the field his old one.

Q2Which AI model did he use?

Claude Opus 5.5, Anthropic’s model. He framed the audit as part of his ongoing quest to find the limits of Opus 5.5.

Q3How many errors did the audit find?

Nearly 1,000 major errors in papers that got cited many times, most still uncorrected, according to Handmer. We have not checked the errors ourselves.

Q4Can other researchers repeat it?

Handmer says he identified dozens of open questions that anyone with spare Claude Max tokens could answer today, with .md context files included, and told one critic to test the tool on their own papers.

Q5Does CellCog use Claude Opus 5.5?

Yes. CellCog runs Claude Opus 5.5 on its Flash and Core tiers, so we have a stake in how the model is seen. This page reports what Handmer says and checks none of his findings.

Published 19 November 2026 All Category basics →