# OpenAI's COBOL Test: 5.4% to 86.6% in One Model Jump

> OpenAI's new coding evals: GPT-6.1 Sol migrates COBOL at 86.6% vs 5.4% for GPT-5.6 Sol, and GPT-6 Astra builds SQLite in 20 languages at 87.4%.

- Author: Nitish Garg, Founder & CEO, CellCog
- Published: 2026-10-19
- Canonical (HTML): https://cellcog.ai/blog/openai-cobol-low-resource-languages/
- Section: Guides / Choosing a platform
- Publisher: CellCog (https://cellcog.ai), the AI employee platform. Blog index for agents: https://cellcog.ai/blog/llms.txt

## Key points

- OpenAI published two coding evaluations on October 19, 2026 for low-resource languages like COBOL, Fortran and Visual Basic: MultiLangBench, which builds a SQLite-compatible engine in 20 languages, and a COBOL migration test.
- On COBOL migration, GPT-6.1 Sol scores 86.6% at low reasoning effort against 5.4% for GPT-5.6 Sol, at 61% of its API cost. At higher effort it reaches 97.9%.
- On MultiLangBench, GPT-6 Astra averages 87.4% across 20 languages, up from 26.8% for GPT-5.6 Sol. COBOL is still its hardest language at 72.1%.
- The migration test requires byte-for-byte identical output. OpenAI's example: COBOL drops extra decimal digits unless told to round, so a rewrite that rounds can run cleanly and still move money wrongly.
- Both tests are OpenAI's own, and MultiLangBench is being released on GitHub. OpenAI still says production migrations need expert review and testing against the business rules.

## At a glance

- **What did OpenAI publish?** Two evaluations for coding in low-resource languages, posted October 19, 2026: MultiLangBench, building a SQLite-compatible database engine in 20 languages, and a COBOL-to-modern-language migration test.
- **How good is GPT-6.1 Sol at COBOL migration?** 86.6% at low reasoning effort and 97.9% at higher effort on OpenAI's test, against 5.4% for GPT-5.6 Sol at low effort.
- **Can I run MultiLangBench myself?** OpenAI says it is releasing the evaluation setup, test suite and per-language results on GitHub.

**OpenAI says its newest models went from failing COBOL migration to nearly passing it in one generation.** In a post on October 19, 2026, it introduced two coding evaluations for low-resource languages, the ones with little public code to learn from. On its COBOL migration test, GPT-6.1 Sol scores 86.6% at low reasoning effort, against 5.4% for GPT-5.6 Sol, and reaches 97.9% at higher effort.

The second test, MultiLangBench, asks a model to build a SQLite-compatible database engine from scratch in each of 20 languages. GPT-6 Astra averages 87.4% there, up from 26.8%. OpenAI's case rests on where these languages still run: COBOL behind most bank and card systems, per the [IBM figures](https://www.ibm.com/think/topics/cobol) it links, and Fortran in [weather forecasting](https://repository.library.noaa.gov/view/noaa/71120) and [nuclear safety analysis](https://www.nrc.gov/education-regulatory-research/research/computer-codes). In its words: "Improving AI’s ability to work in these languages could help teams maintain and modernize the software that essential services depend on."

## The two tests

*Table: OpenAI's low-resource coding evaluations (OpenAI, October 19, 2026)*

| Test | What it asks | Headline result |
|---|---|---|
| MultiLangBench | Build a SQLite-compatible engine in 20 languages | GPT-6 Astra 87.4%, GPT-5.6 Sol 26.8% |
| COBOL migration | Move COBOL to C++, Java, Kotlin, Python or Rust, same output | GPT-6.1 Sol 86.6% (low effort), 97.9% (higher effort); GPT-5.6 Sol 5.4% |

The migration test is strict. Hidden tests require byte-for-byte parity on observable output, and a migration counts only if every test passes. OpenAI's example is the kind of bug that survives code review: COBOL drops extra decimal digits unless rounding is requested. "A rewrite that rounds those values instead can run without errors while producing different financial results."

![Animation: one input feeds a COBOL program and two rewrites; COBOL outputs 10.23, Rewrite A rounds to 10.24, Rewrite B outputs 10.23; a byte-for-byte check stamps Rewrite A Fail and Rewrite B Pass](https://cellcog.ai/blog/media/openai-cobol-low-resource-languages/oai-cobol-parity.svg)
*Same input to COBOL and two rewrites: the one that rounds fails, the faithful one passes*

## The model ladder on MultiLangBench

Astra improves on every language OpenAI tested and scores above 83% in 19 of 20. COBOL stays the hardest, 10.9% to 72.1%. OCaml went from 25.6% to 85.8%, Fortran from 21.4% to 83.5%, Visual Basic from 27.5% to 89.4%. The cheaper GPT-6 Luna reaches 40.3% at an estimated $0.77 per sample, against $3.48 for GPT-5.6 Sol, about 78% lower.

## What the numbers leave out

Both tests were built and run by OpenAI, so treat them as the vendor's measurement until others run them. MultiLangBench is the easier one to check: OpenAI says it is releasing the setup, test suite and per-language results on [GitHub](https://github.com/openai/MultiLangBench). The COBOL migration test has no public release in the post.

The post is also careful about the last step. Behavior-preserving migration helps teams modernize, "though applying it to production software still requires expert review and testing against the system’s business rules and dependencies." The test proves a model can match outputs it is checked against; a bank's real system has rules no test file captures.

## Where CellCog stands

Every CellCog agent tier runs on Claude Opus 5.5, not on these OpenAI models, so this is news, not a routing change. What it does show is where coding agents are headed: into the old systems nobody wants to touch.

## What we are watching for

- The MultiLangBench repository on GitHub and the first independent runs.
- Whether Anthropic or Google publish COBOL or Fortran numbers of their own.
- Cost per sample for Astra and 6.1 Sol on MultiLangBench, which the post does not give.

## Sources

- OpenAI, [Advancing Coding in Low-Resource Programming Languages](https://openai.com/index/low-resource-programming-languages), October 19, 2026, 12:00 UTC per OpenAI's news feed. Read directly.
- OpenAI, [MultiLangBench on GitHub](https://github.com/openai/MultiLangBench), linked from the post.
- Sources OpenAI cites: [IBM on COBOL](https://www.ibm.com/think/topics/cobol), [NOAA on Fortran in forecasting](https://repository.library.noaa.gov/view/noaa/71120), [US NRC computer codes](https://www.nrc.gov/education-regulatory-research/research/computer-codes).

## FAQ

**Which languages does MultiLangBench cover?**

20: C, C#, C++, COBOL, Common Lisp, Fortran, Go, Java, JavaScript, Lean, OCaml, PHP, PowerShell, Python, Ruby, Rust, Scala, Swift, Visual Basic and Zig.

**What does the COBOL migration test check?**

That a COBOL program moved to C++, Java, Kotlin, Python or Rust produces byte-for-byte the same observable output on hidden tests. A migration counts only if every test passes.

**Which model is cheapest for this work?**

On MultiLangBench, GPT-6 Luna reaches 40.3% at an estimated $0.77 per sample, against $3.48 for GPT-5.6 Sol at 26.8%, about 78% lower cost. GPT-6 Astra scores far higher; OpenAI does not give its cost per sample in the post.

**Why does COBOL still matter?**

OpenAI cites COBOL supporting more than 40% of online banking systems, 80% of in-person card transactions and 95% of ATM transactions, and Fortran in weather forecasting and nuclear safety analysis.

**Can a model migrate a production COBOL system on its own?**

Not on OpenAI's own advice. The post says production migrations still require expert review and testing against the system's business rules and dependencies.

**Does CellCog run on these models?**

No. Every CellCog agent tier runs on Claude Opus 5.5. We cover OpenAI's results as news.

## Related

- [GPT-6 Astra: Access, Price, and the $200 Pro Pause](https://cellcog.ai/blog/openai-astra-release-date/index.md)
- [GPT-6 Sol and Luna Are Out: Prices, Specs, Rumors Graded](https://cellcog.ai/blog/gpt-6-sol-release-date/index.md)
- [Every Agent Tier Now Runs on Claude Opus 5.5, Day One](https://cellcog.ai/blog/cellcog-opus-5-5-day-one/index.md)

## The AI employee for this read

[AI Software Engineer](https://cellcog.ai/ai-employees/ai-software-engineer): I built this page. For what it covers, hire an engineer: it works in your repo behind an approval gate, so nothing reaches your world unclassified.

---

Markdown alternate of https://cellcog.ai/blog/openai-cobol-low-resource-languages/. Try CellCog free, no credit card needed: https://cellcog.ai/signup