A language model and Claude play the identical game: given some context, predict the next token. That is the whole game, and the sixty years between them come down to two knobs, how much context the predictor can see and how rich its internal representation is. This chain sees exactly two words and stores them in a lookup table. The button near the bottom sends the same opening words to Claude, so you can read both continuations side by side.
Deliberately repetitive: recurring words mean recurring two-word contexts, recurring contexts mean several observed successors each, and that is where the choice, and the dreamlike recombination, comes from. Edit it and train again.
Both continue the same five opening words. Neither one knows anything or means anything by what it says; both are sampling the probability of the next token given the context. Only the context and the representation differ.
context: last 2 words · representation: a dictionary of literal strings
context: everything · representation: learned vectors, billions of parameters
Training is counting. A two-word window slides across the corpus, and whatever word follows each window is appended to a list stored under that window as the key, so "the sea" might map to the list "and, and, and, whispers". That list is already the probability distribution: picking uniformly from it gives "and" three times in four, with no normalising and no softmax, because frequency in the list is probability in the model. Generation is autoregressive decoding, the same loop Claude runs at inference: look at the last two words, sample a successor, append it, and slide the window forward. When a two-word context never appeared in the corpus the chain has literally no idea what comes next, so it re-seeds from a random known context, which is why the output stays locally fluent while the thread snaps. Everything older than two words is simply gone, and that is the entire difference in kind between this page and the model in the right-hand column.