AI vs Accountants: Mercor's CPA Study, Read Closely

§ 01What Mercor tested#

The four scenarios were a nonprofit’s budget-versus-actual with a shared cost pool to allocate, a hotel whose booking system and books disagreed on December room revenue, a venue’s event revenue variance, and a one-month roll-forward of three lease schedules. One hotel rubric item, for example, required spotting $4,000 of parking revenue miscoded as room revenue.

§ 02The results#

Even the best accountant in the sample scored below the model, and most participants submitted before their time ran out, so Mercor reads time as not the main constraint. The 49 times cost gap compares a model priced at list rates against the Bureau of Labor Statistics median accountant wage, not the rate the participants were paid. On the narrower set of criteria humans usually got right, those with a pass rate above 60%, Mercor reports a 34 times speedup for the model working alone.

The timeline matters more to Mercor than the gap. Its chart of every model it tested by release date puts GPT-4o near 0%, OpenAI’s o3 above the 37% accountant line in spring 2025, GPT-5 at about 69%, and Opus 5 and other recent frontier models at or near 100%. Budget and open-weight models such as Qwen3.5-122B still sit below the accountant line. The author’s own summary on X: “As of May 2025, accountants outperformed AI.”

§ 03The catch: these were simplified tasks#

The study tasks were pared-down versions of APEX-Accounting tasks, and the file map Mercor handed participants cut search time further. On the full benchmark the numbers are very different. Mercor’s APEX-Accounting leaderboard covers 10 company worlds, 160 tasks and 2,186 criteria, and says 58% of tasks are never fully solved by any model.

The same model that aced the simplified study meets about half the criteria on the full benchmark, where Mercor’s own takeaway is that frontier agents “cannot yet reliably close the books”. The leaderboard page also carries an inconsistency worth noting: its header shows a highest score of 61.0% while the table lists Opus 5.5 at 61.8%, and its takeaways still name Fable 5 as the best model. We quote the table rows.

§ 04Why the CPAs lost, per Mercor#

Mercor went back to its task authors after the results and concluded the tasks were fair: the right answer was findable in the files. The gap, it argues, came from three things.

• The tasks were built to fail models. They stacked hard-to-spot but realistic traps, like hotel stays billed on January 3 that belonged in December revenue, or a monthly lease reclassification step most participants skipped. One missed number could cascade into a very low score. The task authors expected juniors to score about 30% and mid-level accountants about 55%.
• The setting stripped out the job. No colleagues, no supervisor, no client to ask, no months of context. One task author put it this way: “what is unrealistic is the setting and the scoring”.
• The tasks tested what models do best. Detail work, searching every file and following instructions closely. One author observed that people “tended to settle early on an approach that looked reasonable” and carried it through.

The participants said the same in their own words. Seven of the 12 rated the tasks realistic; the two who rated them very unrealistic later clarified their objection was the setting, not the spreadsheet work. One wrote: “In reality I can usually ask my client questions as I work”.

§ 05AI plus an accountant did worse than AI alone#

The study was designed to measure how much Claude speeds up an accountant. It could not: because Claude alone scored 100%, the rubric had no room to reward human judgment, and in Mercor’s words “humans with AI are slower and slightly less accurate than AI alone”. The paper’s most useful detail is in the misses. In three of the four assisted sessions that fell short of a perfect score, Claude had the correct figure at some point; twice Claude revised away from it and the accountant deferred, and once the accountant overrode Claude’s correct first answer. Review is a skill, and on a task the model already gets right, a reviewer adds time and risk.

§ 06What it means for a small business’s books#

Mercor’s own reading is that accounting work will move toward client relationships, judgment under uncertainty and asking the right questions, while the detail-heavy close work gets automated. For a small business, the practical version is simpler: the reconciliations, variance tables and journal entries in these four tasks are the work an AI can already do fast and carefully, and the parts that need context about your business are the parts you still own. CellCog runs its AI employees on Claude Opus 5.5, the top model on the APEX-Accounting table above, and an AI bookkeeper works the way the study’s best setup did: the model does the close work, and you review against your own context. Our breakdown of what an AI employee costs covers how that is priced.

§ 07What we are watching for#

• A follow-up study with harder, less simplified tasks, or with colleagues and client questions allowed.
• Human baselines on the full APEX-Accounting benchmark, not only the simplified tasks.
• Whether Mercor publishes the assisted group’s exact accuracy and the per-task scores.
• New leaderboard entries above 61.8%, and whether the 58% never-solved share falls.
• Similar human-baseline studies from Mercor’s other APEX benchmarks.

§ 08The record#

As of October 3, 2026, 02:00 UTC: page opened. We read Mercor’s blog post (published October 1, 2026 at 18:59 UTC per its page metadata), the full paper PDF linked from it, and the APEX-Accounting leaderboard directly on October 3 between 01:42 and 01:47 UTC. Aden Barton’s thread was read on X the same hour; its first post is timestamped October 1, 2026 at 21:07 UTC. Model positions on Mercor’s release-date chart are taken from Mercor’s own figure text, not measured by us.