The Daily Byte
Breaking
Welcome to The Daily Byte — real stories from across Pakistan.
Sunday, 22 November 2026 · Pakistan

The Daily Byte

Stories from Pakistan that deserve to be heard
Minor World

AI Model Rewrites Its Own Weights and Improves

By The Daily Byte ·22 Nov 2026 ·4 min read
Rows of server racks with network cabling in a data centre server room

A language model has edited its own weights and become measurably better at mathematics, without any further training. Researchers call the result self-rewriting: the model diagnoses its own failures, writes new values into its own weights, and tests the result against its unedited self.

The work is reported in a paper titled Language Models Can Rewrite Their Own Weights, posted on arXiv on 16 November and revised on 18 November. It comes from eleven researchers led by Jiajun Fan, using models from the Qwen3.5 family in sizes from two billion to 27 billion parameters. Only the largest showed the full behaviour.

How the model edited itself

Normally a language model is trained once and then frozen. Making it better takes another training run, or a prompt, or extra computation with every answer. The researchers gave a 27-billion-parameter model a different setup: access to its own weights, its graded answers on practice problems, and the ability to run itself, while blocking gradients, optimisers and network access inside the editing session.

The model began by finding a pattern in its own failures. Many of its wrong answers were not wrong mathematics: the model reached the right answer and then kept writing, repeating a sentence such as I will write the response until it ran out of its 16,384-token budget, so the answer was never counted. By comparing the hidden states of answers that finished with those that ran out of tokens, it built a direction separating the two and wrote a small low-rank edit into one layer of the network.

In the paper's first protocol, that edit lifted accuracy on the model's own practice problems from 22.4 percent to 29.6 percent. On a preregistered held-out test of 490 unseen problems, accuracy rose from 57.2 percent to 58.3 percent, a 1.1-point gain the authors report as statistically significant. The edit changed how the model ends its answers rather than giving it new mathematical ability.

The bigger gains came with a safety check

A second protocol added an external check: problems were set aside, and the model could only submit an edit after testing it against them. Across eight sessions it submitted three edits and withdrew five. Two of the submitted edits produced much larger held-out gains, of 4.0 and 6.6 points, both reported with very strong statistical significance.

The 6.6-point edit raised the time-step bias in all 48 linear-attention layers by 0.5, making the model less likely to run to the token limit. The edits also cut answers that exhausted the limit from 33.6 percent to 26.1 and 28.8 percent. Because rejected edits leave the model unchanged, accuracy across the sessions only rose.

Repairing a damaged copy of itself

The team also tested whether the model could repair a deliberately damaged copy of itself. They overwrote eight consecutive rows of one matrix with random noise two hundred times larger than the original values. The damaged copy solved none of its problems at first, producing broken text and repetition. The editing model was told only that some values had been corrupted, and it never read the undamaged weights.

It found the fault with a simple statistic, spotted one matrix as an extreme outlier, and proposed clipping weights larger than fifteen times the matrix median. After the repair, the damaged copy went from zero to 33 percent on 200 held-out maths problems. A repair found using only text maths was then tested on six image benchmarks the sessions had never seen, where the damaged copy rose from 7.3 percent to 70.2 percent accuracy, about 83 percent of the undamaged model's 84.3 percent. The repaired values were not copies of the originals. The model had restored function by constraining pathological magnitudes, not by reconstructing what was there before.

Only the largest model could do it

The ability appeared only at scale. The two-billion-parameter model never ran a forward pass of itself and submitted no edit. The four-billion model repeatedly introduced random changes that drove its accuracy to zero in its own tests. The nine-billion model submitted an edit that cut its held-out accuracy from 47.5 percent to zero, and failed to repair its damaged copy. Only the 27-billion model read its hidden states, tested edits against its unedited self, and kept only the edits the evidence supported.

There are limits. It is a preprint, not yet peer reviewed, and the gains are on a specific set of competition mathematics problems, not evidence of general capability growth. Each session also restarted from the base weights, so this was not a model improving itself again and again in a loop. The authors argue, though, that self-rewriting is a necessary step toward recursive self-improvement: a model that does not separate training from inference could keep upgrading itself through its life. That claim is now for other researchers to test.