Two inputs, four hidden units, one output, and every matrix multiply and derivative written by hand. XOR is the problem that killed neural networks in 1969 and revived them in 1986: no straight line can separate it, so the network has to bend the space itself. Watch the loss curve sit on a plateau while the hidden layer figures out what to represent, then fall off a cliff when it does. The picture on the right is what the network currently believes about every point in the square.
The flat stretch at the start is real and it is the whole story: the network is not stuck, it is searching for a hidden representation that makes XOR linearly separable. Once it finds one, the loss collapses.
2 → 4 → 1 · sigmoid
outputs 0outputs 1 · corners are the four training points
The network holds seventeen numbers: a two-by-four weight matrix and four biases for the hidden layer, then a four-by-one matrix and one bias for the output, all initialised randomly. A forward pass multiplies the input by the first matrix, adds the biases, squashes each result with a sigmoid, and repeats for the output layer, which is exactly the matrix arithmetic written out longhand in the source of this page. Backpropagation then applies the chain rule backwards: the output error times the sigmoid's derivative gives the output gradient, that gradient flows back through the second weight matrix to blame each hidden unit in proportion to how much it contributed, and each weight moves a small step against its own gradient. Training runs in short bursts inside animation frames, so the browser stays responsive and you can watch the plateau break in real time. XOR needs the hidden layer because it is not linearly separable, and the heatmap shows what that means: the network learns to fold the input square until a single line can cut the two classes apart.