How the Vector-Symbolic Clojure Got Here
A companion to A Vector-Symbolic Clojure, which describes the design as it is. This page records how it got that way: what broke, roughly in the order I found it, and what changed. The overview numbers its five traps by theme, so the numbers here differ. Two problems told here, the chimeras and the partial zero, belonged to an integer encoding that has since been replaced, so the overview no longer lists them. September–October 2026.
The starting point was the design of Tomkins-Flanagan and Kelly’s “Hey Pentti, We Did It!” (2024): list cells as bundles of their parts, binding by circular convolution, and thresholds “a little under 1 and a little over 0”. Each section below is one place where running real programs disagreed with it, or with my own first choices. The paper draft (§4) has the measurements, and scripts/failure_modes.clj rebuilds each failure next to the current code.
1. Bundled list cells
My first (fact 5) failed with “unable to resolve symbol: a”. Its list cells were the paper’s: a cell for (a . b) was just L⊗a + R⊗b. Recursion builds near-copies such as ((a . 5) . ()) and ((a . 4) . ()), 89% similar as bundles, so a read beat the wrong sibling by only 0.05. I did not isolate which lookup failed before replacing the cells. A rebuild on the current code shows the same message in 3 of 3 seeds at noise σ = 2, where cells with IDs give 120.
The change: every cell got a random pointer, shared by cells with equal contents (hash-consing). The margin rose to 0.66 and the noise tolerated by 2.8×.
2. Integers: from a table to clocks, and the chimeras
Integers started as powers Bⁿ of one vector, recognised by scanning the whole integer range on every cleanup. That scan dominated the run time. Following Kymn et al. (2025) and Hanley et al. (2025), the encoding moved to residues: one band of the vector per prime, read like six clocks. Cleanup became 72 checks instead of a scan.
The clocks brought chimeras: a sum of 11 and 12 matches 62 fake numbers exactly as well as its members. The readout over integers that showed this was replaced before the first commit, so the end-to-end failure no longer reproduces. The fix then: integers inside maps and sets sit behind their own pointers.
In October, running Hanley et al.’s own code showed that their product code has no chimeras: every coordinate mixes all six clocks, so two numbers agree only when they are equal. I had split the clocks into bands for a cheap readout, and the bands made the chimeras. The encoding moved to the product code. Its readout cannot read one clock at a time, so it keeps a memory of the numbers the interpreter has made, much like the very first table but holding only numbers in use, and reads anything else against all 1.6 million numbers by one Fourier transform. The pointers inside maps and sets stayed.
3. Map entries
On the first day, a benchmark task found (get {1 2 2 3} 2) returning 1: the entry 1 ↦ 2 was 1⊗2, which is also 2⊗1. The first fix stored every map value behind its own pointer, which made the pair directed at the cost of one extra memory row per value. In October a directed bind, k⊗Π(v), replaced it. Done as two operations, a bind and then a permutation, it made maps fail from noise σ = 2 instead of about 3, because the substrate’s noise compounds between operations. As one operation it matches the old robustness.
4. Zero in one dimension
B⁰ is the identity of convolution, a vector with all its weight in dimension 0. Damaging that one dimension, at a lesion of 0.5%, destroyed all arithmetic. In review I also saw that 5005 = 5·7·11·13 is 4/6 similar to zero on the banded clocks, so a threshold zero? would accept it (the product code later removed that overlap too).
The change: n = Bⁿ⊗Z with a random Z, so zero is spread over all dimensions; zero? and = compare identities. (+ 17 25) then came out right in 300 of 300 runs up to a 10% lesion.
5. Fixed thresholds
Every stored structure failed at noise σ ≈ 1.5–1.7, for every vector size, while the cleanup underneath survived σ ≈ 17. The interpreter recognised a value’s kind at cosine > 0.5 and a global at > 0.4, and those cutoffs do not grow with the dimension. A 4-list round trip worked 10 of 10 times at σ = 1 and 0 of 10 from σ = 2.
The change: decide by argmax, or relative to the chance level 1/√D. The same round trip now passes 10 of 10 up to σ = 3. One cutoff was missed and fixed later, in naming the primitive of a superposed call. The CEK machine’s own datapath still has fixed cutoffs: 0.5 in its PRIM instruction, 0.6 to decide a jump or a halt, and 0.4 as the floor under the nearest rule key, below which no rule matches.
6. Vectors as pointer chains, then as superpositions
Clojure vectors started as lists: chains of pointer cells. In October they became positional superpositions, ν(Σᵢ Πⁱ(V⊗xᵢ)), so that vectors with the same elements in the same places are similar and nth is one readout. Lists stayed pointer chains. Which of the two to use became the programmer’s choice.
The first version kept the vector’s length as a field inside it, under the same role that maps and sets use for their size. Reading the set #{[1 2] [1 3]} then pulled stored vectors in as members. Under a role of its own, any two vectors of one length were still 20% alike, which the superposition memory took as present. The length moved out of the vector, next to it in the memory.
7. Tuples of worlds
With vectors as superpositions, the dice and sprinkler demos built their joint worlds as vectors, and read back far from the exact joints (total variation 0.43–0.82) without any capacity error. A sum of vectors keeps each position’s distribution and loses which values went together: [1 1] + [2 2] is [1 2] + [2 1]. Joint worlds went back to lists.
8. Hopfield networks
The original paper suggests a modern Hopfield network as the cleanup memory. I tried it on unrelated items, on look-alikes and under damage: it ties the lookup table at best and loses on look-alikes, and a superposition needed a different memory altogether. The overview has the numbers.
Generated by Claude on Benjamin Schwerdtner’s behalf.
source: notebooks/history.clj