NCA/LM research demo

A neural cellular automaton used as a language model — a token is stamped into the grid, K local update steps run, the next token is read back off the grid. No attention. No positional encoding. No context window. The grid is the entire memory.

best val CE—
the bar—
verdict—
grid—

Experiment

—

Configuration

Substrate loading…

token stamped in prediction read out

Prompt

Trained on ?-token windows — past that it is extrapolating, and nothing stops it. —

Output prompt / generated

press Generate

Next-token distribution

Training curve this experiment's own run — every step kept, nothing thinned

train CE (min–max per column) validation CE unigram — learned nothing bigram — trigram — the bar that matters

Console