# ENGRAFT

> ENGRAFT (ENgram GRadient Routing-Aware Fact Transplant) writes new facts into the n-gram lookup table (Engram / PLE table) that some language models carry next to the transformer. Only selected table rows are trained, by gradient descent; the transformer's weights stay frozen and the model file is never modified. The result is an overlay file applied by a llama.cpp fork at read time, removable bit for bit. All measurements are on Qwen3.8-Flash-Next.

Status: early-stage, experimental research, a few weeks old; first measurements on one model, not a product.

Key facts, each with its limit:
- 100 facts about one invented world (Quail corpus): exact answer 0.841 (707 of 841) on a test set frozen before training and never used for stopping, in the real engine, base model 0.005. In free generation, one fact per question: 35 of 98 with the overlay, 0 of 98 without.
- Capacity: no ceiling up to 300 facts on a separate corpus, not Quail, with a different metric (first answer token at rank 1: 0.804 / 0.792 / 0.821 at 24 / 100 / 300 facts, about one fact per subject).
- Collateral damage at 100 facts: mean KL divergence from the base model 0.0131 on neutral text, about 4x the table's quantization noise.
- A fact did not cross languages: the Italian test set against the Chinese overlay reproduces the base model exactly. Measured on one pair of languages and one cell.
- Transformer weights frozen, not a replacement for retrieval: memory addressed by exact token n-grams.
- Built by one human operator (fulvian) with Claude Code.

## Pages

Each link is the Markdown version of the page; the HTML page is the same URL without `.md`, with a trailing slash.

- [Method](https://engraft-engram.dev/method.md): the table, the overlay, the usage-corpus descent, routing, cost
- [Results](https://engraft-engram.dev/results.md): every published number with its metric, file and limitation
- [Simulator](https://engraft-engram.dev/simulator.md): 98 chat questions and 841 test sentences, recorded answers with and without the overlay
- [Open problems](https://engraft-engram.dev/open-problems.md): measurements not yet run, with credit to the readers who proposed them
- [Log](https://engraft-engram.dev/log.md): releases and the decisions behind them
- [FAQ](https://engraft-engram.dev/faq.md): fine-tuning, RAG, LoRA, ROME/MEMIT, languages, capacity, damage
- [Paper](https://engraft-engram.dev/paper.md): the technical report, version 3 (26 Sep 2026); PDF at https://engraft-engram.dev/paper/engraft.pdf
- [About](https://engraft-engram.dev/about.md): authorship, names, licenses, citation, related work

## Source

- [Code and data](https://github.com/fulvian/engraft-ngram): corpus, overlays, configurations and result files. Code Apache-2.0; Quail corpus CC BY 4.0 on our own contributions; overlays fall under the Qwen Community License 1.0
- [Full text of this site](https://engraft-engram.dev/llms-full.txt)

## Optional

- [Engine fork](https://github.com/fulvian/llama.cpp/tree/fork-ple): llama.cpp with overlay support