# ENGRAFT: facts written into a language model's n-gram table URL: https://engraft-engram.dev/ Updated: 2026-09-26 > ENGRAFT writes new facts into a language model's n-gram lookup table by gradient descent on its rows only. No weight changes; one removable overlay file. Open research · Qwen3.8-Flash-Next · llama.cpp # Facts written into a language model's n-gram table. ENGRAFT trains selected rows of the lookup table that some models carry next to the transformer, and nothing else. The transformer's weights stay frozen and the model file is never modified. The new facts ship as one small overlay file; remove it and the model is back, bit for bit. How it works (https://engraft-engram.dev/method/) Try the simulator (https://engraft-engram.dev/simulator/) Results (https://engraft-engram.dev/results/) Code and data ↗ (https://github.com/fulvian/engraft-ngram) ## Three numbers, and what limits each one All results (https://engraft-engram.dev/results/) 0.841 707 / 841 ### Exact answers, 100 facts about one invented world 0.841 0.005 - with the overlay - base model Limit: 841 test sentences, frozen before training and never used to stop it, but in the corpus's phrasings. Asked freely, one fact at a time: 35 of 98 with the overlay, 0 of 98 without. s0b/engine_results.json (https://github.com/fulvian/engraft-ngram/blob/b51ead5da6e25a132cdc0f74e3d01243e1562e14/data/quail/results/s0b/engine_results.json) 300 facts ### No capacity ceiling found so far 24 facts 0.804 100 facts 0.792 300 facts 0.821 Limit: A separate corpus of short facts, not Quail, and a different metric: the answer's first token ranked first, not the exact answer. One fact per subject; many facts about one subject is untested. capacity-curve/ (https://github.com/fulvian/engraft-ngram/tree/b51ead5da6e25a132cdc0f74e3d01243e1562e14/data/quail/results/capacity-curve) 0.0131 KL ### Collateral damage on neutral text, 100 facts KL divergence: how far the model's next-token probabilities move away from the base model's, averaged over text that mentions none of the facts. Zero means identical. 100 facts 0.0131 storage noise 0.0033 Limit: About 4× the noise of the table's own 4-bit storage, and it grows with the number of facts. s0b/damage_it_text.json (https://github.com/fulvian/engraft-ngram/blob/b51ead5da6e25a132cdc0f74e3d01243e1562e14/data/quail/results/s0b/damage_it_text.json) ## What the model does at every token The method (https://engraft-engram.dev/method/) What it is ## Memory addressed by tokens - A fact lives in the rows its n-grams hash to, and fires when a prompt contains those n-grams. - It sits inside the model's own forward pass: no context tokens, no retriever, no second model. - It travels as one file next to the model. The model files on disk are never modified. - The headline results are checked on the real inference engine, not only on a replica. What it is not ## Not general editing, not a RAG rival - It does not change what the model believes in general, only what some token windows read. - A sentence that shares no n-gram with the training corpus is not covered, by construction. - A fact written in Chinese left Italian questions exactly where the base model had them. Measured on one pair of languages and one cell. - Writing takes a gradient descent of hours, not a millisecond update. Where we are ## At the beginning ENGRAFT is young research. Everything on this site was measured on one model, Qwen3.8-Flash-Next. The main benchmark is Quail, a corpus of 100 invented facts; the capacity curve comes from a separate corpus of short facts. On test sentences shaped like the training corpus the overlay is right 84.1% of the time (707 of 841); asked freely in a chat, it answers 35 of 98 questions. Composition across different subjects, crowding, updates and comparisons with other methods are still unmeasured. We publish it this early so that it can be checked and challenged. The open problems (https://engraft-engram.dev/open-problems/) are the plan. How this was built ## One human, and Claude Code The code, the experiments and the write-ups were produced in Claude Code sessions driven by a single human operator, who set the goals, approved every design step, ran the hardware and read every result. Design, implementation, adversarial review and independent verification were done by separate model instances. You should know it before reading the numbers. We want the method checked, broken and improved. ## Releases Full log (https://engraft-engram.dev/log/) - v0.2.2 ### A working demo, and the first reader debts Positioning as token-addressed memory, a corrected engine demo, the row-sharing measurement, the first open problems raised by readers.21 Sep 2026Release ↗ (https://github.com/fulvian/engraft-ngram/releases/tag/v0.2.2) - v0.2.1 ### Three languages, and the mirror test The Quail corpus in English and Chinese next to Italian, and the Italian test set run against the Chinese overlay.21 Sep 2026Tag ↗ (https://github.com/fulvian/engraft-ngram/tree/v0.2.1) - v0.2 ### Usage-corpus descent, 100 facts The Quail corpus in Italian, one hundred facts written by a descent over a usage corpus, and a preliminary second language.19 Sep 2026Tag ↗ (https://github.com/fulvian/engraft-ngram/tree/v0.2) - v0.1.0 ### The surgical graft Eight neutral facts, each written into the trigram rows of one trigger. Seven of eight took, with zero interference between overlays.5 Sep 2026Release ↗ (https://github.com/fulvian/engraft-ngram/releases/tag/v0.1.0)