ENGRAFT
Early-stage researchENGRAFT is an experimental technique, a few weeks old. What you read here are first measurements on one model, not a product.

Method

How ENGRAFT writes a fact into the table

Qwen3.8-Flash-Next reads 16 rows of a 320-million-row lookup table at every position. ENGRAFT finds new values for a small set of those rows by gradient descent, so that the model answers with the new fact, and ships them as an overlay the engine substitutes at read time.

The table

Some recent language models carry a large n-gram lookup table next to the transformer. DeepSeek calls the design Engram. Qwen3.8-Flash-Next (125B parameters, 6B active) has one, which llama.cpp calls the PLE table. Its rows are addressed by exact token n-grams and read before almost all of the model's computation.

How one position reads the tableThe last two and three tokens are hashed by 16 heads into 16 rows of a 320-million-row table; the rows are added to the model at block 1. An overlay rewrites some of those rows.1The last two and three tokens216 rows of 320 million3Added early in the modelDouglasQuail’sbigram · 8 rowstrigram · 8 rowsone position of“Douglas Quail’s profession is”hasha 14 × 8 slice · red rings: rows an overlay rewritesaddblock 47⋮block 1 + 16 rowsblock 0token embeddingthe rest of the model readswhat the rows added
1

Window

DouglasQuail’s

At each position the current token and its predecessors form a bigram and a trigram.

2

Hash

Each window is mixed with the model's own multipliers. Eight heads take the bigram hash, eight the trigram hash, each modulo its own share of the table.

3

Read

The 16 rows, 160 numbers each, stored in 4-bit blocks, are added to the model's hidden state at an early block.

4

Overlay

An overlay lists row indices and float32 values. The engine fork substitutes them at read time. The model file is never touched.

There is no free space to write into. A sample of about 3.3 million rows found none that was empty, so every edit overwrites something. The table carries real signal: zeroing all of it raises the loss on Italian text from 0.50 to 1.25 nats per token and changes the most likely token at 27% of positions. The addressing is documented bit for bit in docs/mechanism.md.

The descent, step by step

The model, the rest of the table and every weight are frozen. The only variables are table rows.

1

Usage corpus

Short sentences that use each fact in several forms: statement, question, cloze, paraphrase, chat turn. Three splits: training sentences for the descent, held-out sentences for the stopping rule, and test sentences frozen before the first descent and never used for any decision (3,209, 793 and 841 on Quail).

2

Rows

A subset of the rows the training sentences read, chosen within a row budget: 14,032 rows for the Quail corpus, out of 198,628 that its sentences touch.

3

Capture

One pass of the base model over the corpus records its targets and its expert routing.

4

Descend

A differentiable replica of the whole model runs the sentences in batches of 2,048 tokens. The loss rewards the answers and keeps every other position close to the base model.

5

Stop

When the success rate on the held-out sentences (not the test sentences) stops improving, not when the training loss flattens: the loss bottoms out long before the held-out rate does.

6

Measure

The overlay goes into the real engine with greedy decoding and is scored once on the test sentences; collateral damage is measured as KL to the base model on neutral text.

Why routing decides everything

Qwen3.8-Flash-Next is a mixture-of-experts model: every block sends each token to a few experts, a discrete choice. The first gradient probes through the engine were noise. Holding the routing fixed during the probe made them agree.

−0.2

Agreement between +ε and −ε gradient probes, routing free.

−0.999

The same probes with the routing held fixed.

96/97

Regressions that coincided with a routing change, in one analysis.

0.625→0.823

Exact answers when the routing is released during the descent; the other seed went from 0.668 to 0.838.

The descent therefore pins the routing to the base model's choices while the rows make their large moves, then releases it, so that the rest of the descent trains the computation the engine will actually run.

When facts compete

Facts about the same subject share rows by content: the subject's own n-grams are read by every sentence about it. When a fact failed, the overlay usually answered with a sibling fact of the same subject, and the fact with more training text won most of these contests.

37 failures that answered with another written fact, before weighting by mass

  • the fact with more training text won · 27
  • other cases · 10

Hash collisions were measured and refuted as a cause. Weighting each fact by its training mass helped: on one seed the per-fact success rate rose by +0.033 (95% interval +0.007 to +0.060); on the other the gain was about a third as large, with an interval that includes zero.

What it produces

Writing the facts is a gradient descent of hours on a single workstation, not an instant update. What comes out is one small file.

14,032

Table rows rewritten for the 100 Quail facts, out of 320 million.

~90 KB

Overlay per fact: 9,036,620 bytes for 100 facts.

In the public release

  • The method, and the descent and capture code
  • The measurement code: replica evaluation, damage, engine check, composition probes
  • The engine fork that applies overlays
  • The compiled Quail corpus, its row set and the overlays
  • The configuration of the measured run, and every result file behind the numbers

Not in it

  • Our optimized execution path. The public code runs the same descent.
  • The tooling that turns documents into a usage corpus. Its outputs, the corpus, the row set and the overlays, are published as data.

What came before

The first form of ENGRAFT, the surgical graft, wrote eight trigram rows of one trigger per fact. Seven of eight neutral facts took, but facts did not survive rephrasing, and hours per eight facts does not reach a corpus. It was replaced on 8 September 2026. The log records every decision and the measurement behind it.