Short answers about ENGRAFT

ENGRAFT (ENgram GRadient Routing-Aware Fact Transplant) is a method for writing new facts into the n-gram lookup table that some language models carry next to the transformer, such as the PLE table of Qwen3.8-Flash-Next or DeepSeek's Engram design. Only rows of the table are trained, by gradient descent; every model weight stays unchanged. The result is an overlay file that a llama.cpp fork applies at read time.

Not in the usual sense. The transformer's weights stay frozen and the model file is never modified. What is optimised, by gradient descent, is a selected set of rows of the lookup table: numbers the model reads, stored in a separate overlay file. Removing the file restores the model bit for bit. Whether that counts as a narrow kind of fine-tuning is a matter of words; what changes, and what does not, is exactly this.

RAG retrieves text and puts it in the prompt, where the model reads it as words. Here the lookup is part of the forward pass and returns vectors, so a written fact costs no context tokens and needs no retriever, index or embedding model at inference. We do not claim to beat retrieval at recall. How far RAG copes with new phrasings depends on its retriever; ENGRAFT covers, by construction, only the phrasings whose n-grams the usage corpus reads. Nor is storage cheaper: about 90 KB of overlay per fact. The two can be combined, and a head-to-head over the same corpus has not been run.

LoRA changes weights that every input goes through; ROME and MEMIT rewrite MLP weights with a closed-form update. In ENGRAFT the memory is explicit and addressed by token n-grams, only rows read by the corpus move, and every claim is checked on the real inference engine. A head-to-head comparison has not been run yet.

Models that carry an Engram-style n-gram table. Every measurement so far is on Qwen3.8-Flash-Next (125B parameters, 6B active). DeepSeek V4.1 Flash is the next target; its table differs and nothing has been measured on it.

Not in what we have measured. The Italian test set run against the Chinese overlay of the same facts gives exactly the base model's numbers (0.0048 exact answers against 0.0048), although the engine reads 922 overlay rows along the way. Measured on one pair of languages and one cell.

No ceiling appeared up to 300 facts on a corpus with about one fact per subject: first answer token at rank 1 was 0.804, 0.792 and 0.821 with 24, 100 and 300 facts. How many facts a single subject can hold is still an open problem.

A little, and measurably. With 100 facts the mean KL divergence from the base model on neutral text is 0.0131, about four times the noise of re-quantizing the same rows. It grows with the number of facts.

The overlay for 100 facts is 9,036,620 bytes, about 90 KB per fact. All our runs were made on a single machine with 128 GB of unified memory; writing the facts takes hours.

Not as working memory: a write is a gradient descent that takes hours, and facts about the same subject compete for the same rows, which is an agent's typical case. As periodic consolidation of what has stopped changing, it is plausible but untested.

The capture, descent, evaluation and damage code runs end to end against a synthetic model on any machine. Reproducing the measured runs needs the engine fork and the Qwen3.8-Flash-Next shards. The corpus, row set, overlays, configuration and every result file are in the repository.

A single human operator with Claude Code. The operator set the goals, approved each design step, ran the hardware and read every result; design, implementation, adversarial review and verification were done by separate model instances.