🔍 Read the full analysis: AI Repairs AI: GPT-6 Astra Targets A Loop Through Weight Edits on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A November 2 blog post by four researchers affiliated with MIT CSAIL reports that GPT-6 Astra edited two layers of open-weight model Qwen3.5-0.8B, raising its MATH-500 score from 7.9% to 48.1%. The edit reduced, but did not eliminate, answers that hit the model’s token limit; the work is an early, non-peer-reviewed report and used extensive iterative testing.
Four researchers affiliated with MIT CSAIL report that OpenAI’s GPT-6 Astra improved the math benchmark performance of open-weight model Qwen3.5-0.8B by editing two of its layers, without gradient descent. In a blog post published November 2, they say the model’s MATH-500 score rose from 7.9% to 48.1%, while answers reaching the 32,768-token limit fell from 93.0% to 33.1%.
The work, titled Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights, describes GPT-6 Astra identifying an internal direction associated with repetitive answers and subtracting it from output columns in layers 15 and 19. The researchers say the change affected 131,072 weights, or 0.016% of the model’s 0.8 billion weights. The edited model’s published files show differences in the two layers’ down-projection weight tensors.
The authors’ logs describe an iterative process, rather than a single automated repair. Astra was given a Python environment to inspect and modify weights and received feedback from model evaluations. The highest-budget run involved 158 turns, 80 Python runs and 50 forward passes, over 20.3 hours. A forward pass could test up to 512 problems. The researchers say gradient descent, backward passes and fitting weights to a loss were prohibited, and that the final benchmark questions were not shown to Astra.
The reported gains varied by benchmark and test budget. On MATH-500, runs with 10 and 25 forward passes scored 12.3% and 14.1%, respectively, before the 50-pass run reached 48.1%. On GSM8K, the 50-pass result was 67.1%, compared with 13.8% for the base model; on IFEval, it was 62.3%, compared with 21.0%. The authors also report that 33.1% of edited-model MATH-500 answers still reached the token limit.
A New Route to Editing Open Models
The result points to a possible way of changing how an open-weight model behaves without running conventional fine-tuning. Rather than updating weights through gradient descent against a training objective, the method used a capable language model to inspect outputs, propose code changes and test their effects. If similar methods prove reliable, model developers could use them to address specific failures in models whose weights they can access.
For readers and organizations using AI systems, the distinction between repair and new capability matters. The researchers frame the intervention as reducing a known looping problem, not teaching Qwen new mathematical knowledge. Its benchmark improvements are substantial in the reported run, but the remaining token-limit rate shows that the behavior was not removed entirely. The results also suggest that evaluation budget matters: shorter runs delivered much smaller gains.
The approach applies only where weights can be inspected and modified. It does not show that a user can make the same changes to a closed model through an ordinary chat interface. Each weight edit also produces a modified model that would need separate testing before deployment. A change that helps on selected benchmarks could affect other behaviors in ways not covered by the reported evaluations.
Products Worth Considering
Apple Mac mini Desktop Computer with M4 chip, 10‑core CPU and GPU, 24GB Unified Memory, 512GB SSD, Gigabit Ethernet
Apple MacBook Air 13-inch with M5 chip, 16GB memory, 1TB SSD, and 12MP Center Stage camera (Midnight)
Apple MacBook Pro 14 2021 with M1 Max, 64GB RAM, 2TB SSD, Silver
As an affiliate, we earn on qualifying purchases.
The Model’s Known Looping Problem
Qwen3.5-0.8B is a small, 0.8-billion-parameter model. The source report says its model card warns that it can fall into thinking loops. On the researchers’ initial practice sample, 85 of 96 answers reached the 32,768-token ceiling, consistent with the issue they set out to address.
The blog post covers eight experiments involving the use of a language model to read or alter neural-network weights. The Qwen math test is the example that produced the strongest reported benchmark result. Another experiment did not succeed: the authors say Astra could not identify, from weights alone, which of two fine-tuned Gemma models had been trained to flatter users.
The report lists Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, with Takehi and Teoh marked as equal contributors. The authors are listed with MIT CSAIL. The post says the work is an early result; the source material says it had not been peer-reviewed or posted on arXiv as of publication. OpenAI is not listed as an author, and the source report says it found no OpenAI statement about the experiment.
“Fixing loops, however, is not the same as making a model better at math.”
— The researchers, in the blog post
Limits of the Reported Evidence
The findings come from a blog post and published logs, not a peer-reviewed paper, according to the source report. The material does not establish whether independent researchers have reproduced the change, whether the score gains hold across other sampling settings or model versions, or how the edit affects performance outside the listed evaluations.
The benchmark numbers also do not show that looping is solved: one-third of MATH-500 answers still hit the token ceiling after the largest-budget edit. The source provides results for a limited set of models and tests, and does not establish how often this approach would work on other failure modes. It is also not clear from the supplied material whether OpenAI reviewed, endorsed or otherwise participated in the experiment; the authors are not presented as OpenAI researchers.
Reproduction and Broader Testing
The next test is whether other researchers can reproduce the weight edit and its benchmark results using the published logs and model files. Further evaluations would need to check for regressions, measure how often loops remain under different settings and test whether similar edits help on models beyond Qwen3.5-0.8B.
The researchers’ post presents the work as early evidence that language models can reason about and alter neural-network weights. It does not announce a deployment or a broader release schedule. Until independent tests and wider evaluations are available, the reported improvement should be treated as a promising, narrow result rather than proof that AI systems can reliably repair models on their own.
Key Questions
What did GPT-6 Astra change?
According to the researchers’ blog post, Astra identified a direction linked to repetitive outputs and subtracted it from weight columns in layers 15 and 19 of Qwen3.5-0.8B. The change affected two down-projection tensors.
Did the edit eliminate the model’s loops?
No. The reported share of MATH-500 answers reaching the 32,768-token limit fell from 93.0% to 33.1%, but a substantial portion still hit that limit.
Was the model trained?
The researchers say they did not use gradient descent, backward passes or loss-based weight fitting. However, Astra used an iterative, feedback-driven process with code runs and model evaluations, so “without training” does not mean without testing or effort.
How established are the results?
The findings were reported in a blog post described by its authors as very early results. The supplied source says the work had not been peer-reviewed or posted on arXiv at the time of reporting, and does not describe independent replication.
Can the same method be used on any AI model?
The report does not establish that. This experiment involved an open-weight model whose internal weights could be inspected and changed. The authors also report mixed results across their experiments, so the method’s reliability on other models and problems remains uncertain.
Source: ThorstenMeyerAI.com
