# Opus 5.5 Audit of 150,000 Astrophysics Papers: 1,000 Errors

> Casey Handmer had Claude Opus 5.5 audit about 150,000 astrophysics papers and says it found nearly 1,000 major errors, most still uncorrected.

- Author: Nitish Garg, Founder & CEO, CellCog
- Published: 2026-11-19
- Canonical (HTML): https://cellcog.ai/blog/casey-handmer-astrophysics-audit/
- Section: Insights / Category basics
- Publisher: CellCog (https://cellcog.ai), the AI employee platform. Blog index for agents: https://cellcog.ai/blog/llms.txt

## Key points

- Physicist Casey Handmer says he directed Claude Opus 5.5 to audit theoretical astrophysics, his old field: about 150,000 papers indexed and 50,000 audited for issues.
- He reports nearly 1,000 major errors in papers that were cited many times, and says most are still uncorrected. These are his figures; we have not checked them.
- He says the survey let him estimate how well peer review corrects errors, and his verdict is that it is not very good.
- His bottom line: AI can speed up quantitative science by more than 1,000 times with no further model improvement, yet he sees no sign of universities pushing it.
- He offered to audit essentially every paper in the field, or sample every field of science, if a leading lab sponsors on the order of $100,000 of tokens.

## At a glance

- **What did Casey Handmer do?** He had Claude Opus 5.5 index about 150,000 theoretical astrophysics papers, audit 50,000 for issues and run progressively more thorough checks on thousands more.
- **What did he find?** Nearly 1,000 major errors in papers that were cited many times, most still uncorrected, by his own count. He also says peer review is not very good at correcting errors.
- **Has anyone checked the errors?** Not in anything we read. The figures on this page are Handmer's own, from his posts on X on November 17 and 18, 2026.

**Casey Handmer says Claude Opus 5.5 found nearly 1,000 major errors in theoretical astrophysics.** He posted the result on X on November 17, 2026: the model indexed about 150,000 papers, audited 50,000 for issues and ran progressively more thorough checks on thousands more. His opening line: "In my ongoing quest to find the limits of Opus 5.5, I directed it to audit my old scientific field, theoretical astrophysics."

By 02:00 UTC on November 19 the post had about 729,000 views. Every figure on this page is Handmer's own, from his posts on X; we have not checked the errors he reports.

## What the audit did

*Table: The audit in Handmer's words (post on X, 08:35 UTC, November 17, 2026)*

| Step | What he says happened |
|---|---|
| Index | About 150,000 papers in theoretical astrophysics |
| Audit | 50,000 papers checked for issues |
| Deeper checks | Progressively more thorough checks on thousands more |
| Result | Nearly 1,000 major errors in papers that got cited many times |
| Status | Most of those errors still uncorrected |
| Model | Claude Opus 5.5 |

The error count is his rounded "nearly 1000", so the last bar is an upper bound, not an exact figure.

![Animation: a tall stack of papers labelled 150,000 indexed, a teal gate scanning a smaller stack labelled 50,000 audited, and a tray of sheets with orange flags labelled nearly 1,000 errors](https://cellcog.ai/blog/media/casey-handmer-astrophysics-audit/field-audited.svg)
*A field of 150,000 papers narrows to nearly 1,000 flagged errors, by Handmer's count*

## What he says it shows

- **Peer review.** The survey "enabled me to estimate the efficacy of peer review for correcting errors (not very good)". In a reply the next day he put it more sharply: any highly cited paper has, statistically speaking, more than a 40% chance of a major error, and journals show almost no interest in corrections.
- **Speed.** "AI can speed up quantitative science by >1000x with no further model improvement required."
- **Universities.** "But I'm seeing no evidence of this being pushed within universities, only academics complaining about labs dropping math proofs."
- **Publishing.** "Academic publishing is dead." He argues that once AI can produce thousands of correct papers a day, the paper stops being a meaningful unit of science.

He also says the audit turned up dozens of open questions that anyone with spare Claude Max tokens could answer today, and that he included .md context files for them.

## The replies

Handmer spent the next day answering critics. To one, he wrote: "I tested it on my own papers first." To another: "There is no practical way for a human to audit this work, it's taken the work of thousands of humans nearly a century just to produce it in the first place."

He also offered to finish the job. "If someone at one of the leading labs wants to sponsor O($100,000) of tokens, I can complete this particular study", he wrote, or repeat the analysis across essentially all fields of science.

*Table: Handmer's posts about the audit (times UTC; views read 02:00 UTC, November 19, 2026)*

| Time | What he said | Views |
|---|---|---|
| Nov 17, 08:35 | The audit and its result | about 729,000 |
| Nov 17, 15:41 | One AI-using grad student could blitz a whole field | 21,183 |
| Nov 17, 20:57 | Offer to finish it for O($100,000) of tokens | 20,032 |
| Nov 18, 05:19 | Compared it with how Chandrasekhar worked | 7,066 |

## Why it matters

The claim is not that a model wrote new physics. It is that a model can read a field faster than the field can read itself, and check the arithmetic in papers people keep citing. If Handmer's error count holds up under review, the bottleneck in a field like astrophysics moves from finding mistakes to deciding what to do about them. We covered a nearby result in [Claude Fable 5.2 solving two black hole physics problems](https://cellcog.ai/blog/claude-fable-5-2-black-hole-physics/), and Anthropic's own survey of [how scientists use Claude](https://cellcog.ai/blog/anthropic-how-scientists-use-claude/).

The open question is verification. A list of a thousand flagged errors is only as good as the share of them that a physicist agrees with, and Handmer's posts do not give that share.

## Where CellCog sits

CellCog runs Claude Opus 5.5 on its Flash and Core tiers, so we have a stake in how the model is seen. This page reports Handmer's claims as his and checks none of them.

## Sources

- Casey Handmer on X, [the audit](https://x.com/CJHandmer/status/2122609626588061816), 08:35 UTC, November 17, 2026
- Casey Handmer on X, [grad students and AI](https://x.com/CJHandmer/status/2122716758813643125), 15:41 UTC, November 17, 2026
- Casey Handmer on X, [the token sponsorship offer](https://x.com/CJHandmer/status/2122796358616891642), 20:57 UTC, November 17, 2026
- Casey Handmer on X, [Chandrasekhar](https://x.com/CJHandmer/status/2122922639274762671), 05:19 UTC, November 18, 2026
- Casey Handmer on X, [tested on his own papers](https://x.com/CJHandmer/status/2122924283915624927), 05:26 UTC, November 18, 2026
- Casey Handmer on X, [no practical way for a human](https://x.com/CJHandmer/status/2123097857498690001), 16:55 UTC, November 18, 2026
- Casey Handmer on X, [highly cited papers](https://x.com/CJHandmer/status/2123098978367762916), 17:00 UTC, November 18, 2026

## FAQ

**Who is Casey Handmer?**

A physicist who worked in theoretical astrophysics before becoming a founder, as he describes it in his post. He calls the field his old one.

**Which AI model did he use?**

Claude Opus 5.5, Anthropic's model. He framed the audit as part of his ongoing quest to find the limits of Opus 5.5.

**How many errors did the audit find?**

Nearly 1,000 major errors in papers that got cited many times, most still uncorrected, according to Handmer. We have not checked the errors ourselves.

**Can other researchers repeat it?**

Handmer says he identified dozens of open questions that anyone with spare Claude Max tokens could answer today, with .md context files included, and told one critic to test the tool on their own papers.

**Does CellCog use Claude Opus 5.5?**

Yes. CellCog runs Claude Opus 5.5 on its Flash and Core tiers, so we have a stake in how the model is seen. This page reports what Handmer says and checks none of his findings.

## Related

- [Claude Fable 5.2 Solves Two Black Hole Physics Problems](https://cellcog.ai/blog/claude-fable-5-2-black-hole-physics/index.md)
- [How Scientists Use Claude: 207,000 Sessions Studied](https://cellcog.ai/blog/anthropic-how-scientists-use-claude/index.md)
- [What Is an AI Employee? The 5-Part Test for a Standing AI Worker](https://cellcog.ai/blog/what-is-an-ai-employee/index.md)

## The AI employee for this read

[AI Research Assistant](https://cellcog.ai/ai-employees/ai-research-assistant): I built this page. For what it covers, hire a research assistant: it reads the primary sources and cites every line.

---

Markdown alternate of https://cellcog.ai/blog/casey-handmer-astrophysics-audit/. Try CellCog free, no credit card needed: https://cellcog.ai/signup
