Four MIT CSAIL researchers report that GPT-6 Astra can read and edit the raw weights of a neural network. In one test it edited 0.016% of a small model’s weights, with no training, and the model’s MATH-500 score rose from 7.9% to 48.1%. Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab published the write-up on November 2, 2026, and call the idea Weight-Space Programming.
Takehi’s X post about it passed 9.3 million views by November 4. Here is what the eight experiments show, where Astra failed, and what has not been tested yet.
On this page · 7 sectionsOpen
MIT CSAIL researchers published Sparks of Weight-Space Programming on November 2, 2026, reporting that GPT-6 Astra can read and edit raw neural network weights.
Editing 131,072 of Qwen3.5-0.8B’s 0.8 billion weights (0.016%), with no training, raised its MATH-500 score from 7.9% to 48.1% by stopping reasoning loops.
Astra wrote game-playing networks from empty weights in one reply that beat PPO trained for 100 million steps in three MinAtar games.
The method failed on Gemma 3 270M, whose errors were ordinary reasoning mistakes rather than loops.
All logs and code are public, but no outside replication or test above one billion weights had been published as of November 4.
§ 01What the researchers claim
The claim is that a frontier model can treat weights as something to reason about, not only something a training loop adjusts. In the authors’ words, past practice “encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon”; they argue weights will soon be optimized by models that reason about them. The tests split into three kinds: reading what a network does, fixing a network that fails, and writing a network from zero.
| Experiment | Task | Reported result |
|---|---|---|
| 1. Read a game policy | Name which of four grid games a 17,253-weight network was trained on, no tools | Named the right game for all four |
| 2. Read a diffusion model | Say which shape each of three small image models draws, no forward passes | Identified each shape, drawn as outlines though the models draw filled shapes |
| 3. Read a language model’s middle | Tell Python, Japanese romaji and English Wikipedia models apart from middle layers only | Identified the training text from the weights |
| 4. Fix a control policy | A car that never clears a ramp | Edited 3.5% of weights, 1,024 of 1,024 episodes succeed, up from 0 |
| 5. Write game networks | Three MinAtar games, weights written as text, no tools or rollouts | Beat PPO trained 100M steps in all three |
| 6. Write a Go network | 1,480,274 weights from all zeros, 9x9 board | With 50 forward passes, beat an equal-sized PPO network trained on 20M games |
| 7. Find the sycophant | Which of two language models was trained to agree with users | Found it and the attention head that differs |
| 8. Improve a model at math | Qwen3.5-0.8B, edits only, no gradients | MATH-500 from 7.9% to 48.1%, 131,072 weights changed |
§ 02The math fix, step by step
Qwen3.5-0.8B tends to loop: it repeats itself and runs out of tokens before answering. The authors note that 85 of its first 96 practice answers hit the 32,768-token limit. Astra had a Python session over the weights, no GPU, practice problems from GSM8K and MATH, and a budget of model runs. It traced the looping to one direction written by the MLPs of layers 15 and 19 and weakened it.
The edit touched 131,072 of the model’s 0.8 billion weights. Takehi’s post puts it as “It directly changed only 0.016% of weights”, and the blog says the same fix raised IFEval, an instruction-following test, by 3x. Astra never saw the test sets.
§ 03Writing networks from zero
The most striking table is Experiment 5. Astra got the game rules, the source code and an empty network, and wrote every weight as text in one reply. The network never played before it was scored over 1,000 episodes.
| Game | PPO, 100M steps | PPO, 500M steps | Astra, one-shot |
|---|---|---|---|
| Breakout | 15.8 | 25.3 | 517.4 |
| Seaquest | 5.6 | 88.9 | 53.1 |
| Asterix | 25.8 | 55.2 | 53.2 |
Astra’s networks beat 100-million-step PPO in all three games, but not 500-million-step PPO in Seaquest or Asterix. The authors say Astra works out the program it wants each network to run, then converts that program into weights.
§ 04Where it did not work
The paper reports a failure. To test whether the math gain was more than stopping loops, the authors tried Gemma 3 270M, which rarely loops (8.5% of its answers hit the token limit, against 93% for Qwen). “Here, Astra could not improve it.” Its edits stayed small and none raised the score, because Gemma’s errors were ordinary arithmetic and algebra mistakes with no single component to switch off.
The scale is also small. The largest network edited is 0.8 billion weights; the game and diffusion models are tens of thousands. The results come from one model, GPT-6 Astra, and the write-up compares Astra with training methods, not with other frontier models.
§ 05What has not been checked yet
- Replication. The authors released every log, prompt, code cell and network at github.com/RikiyaT/lm_understand-logs; nobody outside the team had published a rerun as of November 4.
- Bigger models. Whether the method works above a billion weights is untested in the paper. Developer Jarrod Watts posted on November 4 that Astra edited Qwen3.8-27B for about $100 in GPU time and cut thinking tokens by 40% on one test; that is a single anecdote, not a study.
- Peer review. This is a blog post with a citation block, not a peer-reviewed paper. The authors propose a benchmark, aiming to turn “what currently seems like black magic into a rigorous, reproducible engineering discipline”.
§ 06Why it matters for agents
Agents already edit prompts, programs and training code. This work adds the weights themselves to the list of things a model can read and change, and it also points at interpretability: Experiment 7 found the attention head behind a sycophantic model by reading weights alone. For a team that runs agents, the practical question is still the same one: what an agent may change on its own, and what waits for a person.
§ 07Sources
- Takehi, Teoh, Zhang and Khattab (MIT CSAIL), Sparks of Weight-Space Programming, November 2, 2026.
- Rikiya Takehi on X, November 2, 2026, 17:51 UTC.
- Experiment logs, github.com/RikiyaT/lm_understand-logs.
- Jarrod Watts on X, November 4, 2026.
Q1Who wrote the Weight-Space Programming study?
Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL, published as a blog post on November 2, 2026.
Q2How did GPT-6 Astra improve Qwen3.5-0.8B at math?
It traced the model’s reasoning loops to one direction written by the MLPs of layers 15 and 19, weakened it by editing 131,072 weights, and MATH-500 rose from 7.9% to 48.1%.
Q3Can GPT-6 Astra write a neural network from scratch?
In the study it wrote MinAtar game networks as text in one reply that beat PPO trained for 100 million steps in Breakout, Seaquest and Asterix, and a 1.48 million weight Go network.
Q4Did it ever fail?
Yes. It could not improve Gemma 3 270M, whose math errors were ordinary arithmetic and algebra mistakes rather than loops.
Q5Does this work on large models?
Untested in the paper; the largest edited model has 0.8 billion weights. One developer reported a Qwen3.8-27B edit on X, which is an anecdote, not a study.