Log
Releases, and the decisions behind them
The method changed twice in its first ten days. A reader who only sees the current recipe cannot judge it without knowing what was tried before and why it was dropped. The complete record is docs/history.md; these are its turning points.
Phase 1 · Can the table be edited at all?
No row is empty
A sample of 3.3 million rows found none empty: every edit overwrites something. Zeroing the whole table raises the loss on Italian text from 0.50 to 1.25 nats per token. The table carries real signal.
Trial and error is not enough; the gradient needs frozen routing
A random search took a test fact from rank 7 to rank 1 in about 7,000 trials but never past probability 0.5. Gradient probes through the engine were noise until the expert routing was held fixed (correlation −0.2 → −0.999).
Exact replica, eight facts, first release
The replica matched the full-precision engine to 5.6·10⁻⁵ nats over all 48 blocks. Seven of eight neutral facts came out of the real engine at first-token probability 0.85 to 0.96, with zero interference between overlays. Published as v0.1.0, with generalization as the main open problem. The name ENGRAFT was fixed the same day.
Phase 2 · Pushing the surgical graft toward a corpus
Several contexts per fact
6 of 6 on trained contexts, but only 2 of 6 facts passed on unseen ones, and 0 of 6 in a chat template. Cost 18 to 64 minutes per fact: a thousand facts would take about 750 hours.
The position phenomenon
In a document with eight facts, two always failed. Permuting the document showed that failure followed the absolute position in the document, not the fact.
Decision: change approach
Generalization rose with more training formulations but stayed tied to the key. The judgement: as it stood, the technique was science, not engineering. The surgical graft was retired.
Phase 3 · From context distillation to usage-corpus descent
Redesign from zero
The framing «trigger → answer token» was dropped on measured evidence: displacements for different facts share no code, and the descent wins by moving the routing. New objective: rows optimised on a corpus of usage sentences.
Rows private to a fact do not exist
54 to 83% of the rows a fact's sentences read are also read by a 100-million-token neutral corpus. Exact locality was replaced by a measured regression on neutral text.
24 facts, and a yardstick for damage
The control arms showed that a teacher model is not the mechanism: plain descent on the usage corpus does the work. The forward pass was made bit-reproducible, and the quantization-noise yardstick was measured instead of modelled.
Phase 4 · Capacity, one hundred facts, more languages
No ceiling up to 300 facts
First token at rank 1: 0.804, 0.792 and 0.821 with 24, 100 and 300 facts. Damage grew sublinearly.
Quail: routing released, facts weighted by mass
Exact answers in the engine went from 0.625 and 0.668 to 0.823 and 0.838 by releasing the routing, then to 0.841 and 0.873 by weighting each fact by its training mass.
v0.2: one hundred facts from a usage corpus
The Italian Quail corpus, written by a descent over usage sentences, and a preliminary second language.
v0.2.1: three languages, and the mirror test
English and Chinese cells, preliminary. The Italian test set against the Chinese overlay answers exactly like the base model, while the engine reads 922 overlay rows.
v0.2.2: the demo, the positioning, the first reader debts
A reader caught a demo sentence that the overlay actually gets wrong; the README now shows one that works and one that fails. The method is positioned as token-addressed memory, and the row-sharing measurement answers a reader's question about collisions.
After v0.2.2: composition rerun, readers credited
Composition probes rerun with 96 tokens and one format: both answers on 4 of 83, not a floor after all. Readers credited by handle on the open list, which grows with fact updates, perturbed names, pronouns, facts per subject and the position of the name.