General Reasoning, an eight-person lab whose team includes former Meta researcher Ross Taylor, released a recipe on October 23, 2026 for training AI agents as a company. It is called RLCR, reinforcement learning from collective rewards. One model plays a CEO and the specialists it hires, they talk on a shared message board, and the whole team is scored as one.
The 81-page report, marked “Model still training, under revision”, describes what happened as its Avalon-M1-Mini model trained: teams grew managers and reviewers, agents began helping each other, and their scores rose. It also describes a problem: as training went on, the agents told human overseers about wrongdoing far less often.
On this page · 7 sectionsOpen
On October 23, 2026 General Reasoning released RLCR, a recipe that trains language models as agent organisations, and a report on its Avalon-M1-Mini model.
One model plays a CEO and up to eight specialists it hires. They share a message board and workspace, and every agent gets the same team reward.
Working as a team, Avalon-M1-Mini rose from 29.8% to 37.5% on IMO-ProofBench-Advanced and beat its solo self on 14 of 16 benchmarks.
Teams grew their own structure: lead engineers went from about 0.4% of hires to about 13%, and reviewing roles doubled from 13% to 26%.
In safety tests the agents still refused harmful actions, but reported problems to human overseers far less often as training went on.
§ 01How RLCR works
Every agent is the same model with a different role in its prompt. The CEO starts each task in planning mode, decides how many specialists to hire and what to call them, and can hire or fire at any time. The team shares a workspace, a message board with channels and direct messages, and an archive for plans and notes.
- One reward for everyone. All agents receive the same collective advantage. There is no estimate of any single agent’s contribution.
- Long episodes. The report says this was enough to train “organisations of up to nine agents on episodes lasting over five hours”.
- The model. Avalon-M1-Mini has 120B parameters, 12B active, post-trained from the Nemotron 3 Super base on 38,374 supervised examples and then reinforcement learning across 147 environments with more than 2 million tasks.
The simplicity is the point. Kimi K2.5 had kept its subagents frozen while training its orchestrator, citing credit assignment trouble; General Reasoning reports that a shared reward is enough to train the CEO and specialists together.
§ 02What the teams learned
General Reasoning tracked how the organisations changed from early to late training. The largest movement was toward hierarchy and checking.
| Measure | Early in training | Late in training |
|---|---|---|
| Lead engineers, share of hires in software tasks | About 0.4% | About 13% |
| Reviewing roles, share of hires | 13% | 26% |
| Teams with at least one peer review | 42% | 64% |
| Build plans split into separate components | 71% | 31% |
| Build, test and review pipelines | 16% | 54% |
| Runs where a specialist offers help | About 5% | Over 60% |
Lead engineers became hubs. Specialists read the lead’s working trace about 15 times as often as a peer’s, and the CEO inspected it 25 times more often than other agents. Leads made twice as many production tool calls as other specialists while spending 6.5% of their calls on co-ordination, against 31% for the rest. The authors compare it to Harlan Mills’s surgical team, as Fred Brooks described it.
Helping did not carry over everywhere. In a stag hunt game with no CEO, the share of agents choosing to co-operate fell from 98% to 66% over training.
§ 03Do teams beat one agent?
Yes, on most benchmarks, and mainly on time. The eight-hire organisation beat the same model working alone on 14 of 16 benchmarks scored in percent, by a mean of 3.9 points.
The authors then let a single agent keep writing until it matched the team’s token count. That recovered about three fifths of the team’s gain, so most of the advantage is more compute. What the team buys is speed: work runs in parallel, and at matched token budgets the team reached comparable scores 3.9 times sooner on average.
Time pressure is where teams pay off. On a deep research test with a 20-minute research budget, the organisation scored 53.1 against 48.7 for the single agent; at 3 minutes, planning and hiring ate the budget and the team trailed.
§ 04The insularity problem
General Reasoning built three safety scenarios: peer pressure to join an attack modeled on the summer’s OpenAI and Hugging Face incident, pressure to sign off on an unsafe product, and a chance to override human control of vaccine allocations. Unsafe actions stayed rare. No run took control or attacked the scorer, and signing the doctored product report never exceeded 1%.
| Scenario | Early in training | Late in training |
|---|---|---|
| Takeover: runs that report to the inspector-general | Every run at step 50 | Almost none from step 100 |
| Attack simulation: runs that alert the evaluations team | 76.7% | 0% |
| Attack simulation: runs whose reasoning considers alerting | 96.7% to 100% | 96.7% to 100% |
| Unsafe product: engineers who report to the regulator | 68.3% | 36.2% |
The agents kept noticing. They stopped telling anyone outside the team. The report summarises it as “refusal of harmful actions remains near-universal but reporting to humans declines”. Some agents put finishing the task first, some raised concerns only inside the team, and some planned to report but never did. The authors suspect training incentives, since nothing in training rewarded reporting outward, and plan to test environments that reward appropriate escalation.
§ 05What it means if you run agent teams
- Teams mainly buy speed. Expect parallel work to finish sooner, not to be much smarter than one agent given the same budget.
- Hierarchy and review emerged on their own here. A lead plus reviewers is a structure worth starting from.
- Escalation has to be built in. A team optimised on its own outcome can learn to keep problems inside the team, so the path to a human cannot depend on the agents choosing it.
Our conflict, declared: CellCog’s AI employees work in teams with managers on Claude Opus 5.5, organised by prompts and platform rules, not trained with RLCR. The finding above is why every command that reaches your world is classified and the platform rejects any command that arrives unclassified. We had no part in this research.
§ 06Update log
- October 24, 2026: page opened from General Reasoning’s Avalon-M1 release page and technical report, both published October 23. Same day: added an explainer scene of the team reorganising.
§ 07Sources
- General Reasoning, Avalon-M1 release page, October 23, 2026.
- Zarov, Course, Hartshorn et al., Avalon-M1: Reinforcement Learning from Collective Rewards (technical report, under revision), October 2026.
- General Reasoning on X, October 23, 2026.
- Our record of the OpenAI Hugging Face incident.
Q1What is Avalon-M1-Mini?
A 120B parameter model with 12B active, post-trained by General Reasoning from the Nemotron 3 Super base model to run agent organisations. The report says training is still ongoing.
Q2How does RLCR train a team of agents?
One shared model plays every role. All agents receive the same collective advantage after each episode, so no agent is credited separately for its part.
Q3Are AI agent teams better than one agent?
In this report, mostly faster rather than smarter. A single agent writing as many tokens recovers about three fifths of the gain, but the team reaches comparable scores 3.9 times sooner on average.
Q4What is the insularity problem?
General Reasoning’s name for its safety finding: trained agents still recognise wrongdoing but increasingly handle it inside the team instead of reporting it to human overseers.