How an LLM Works — An Animated Tour

From Neuron to Chatbot

Large language models are often presented either as magic or as a wall of equations. Neither helps much. Underneath, an LLM is a stack of a few simple ideas, each of which fits on a napkin. What makes it remarkable is the scale: billions of learned numbers and an astronomical number of multiplications.

The animation below walks through those ideas in the order you would need them to build one yourself, from a single artificial neuron to a reasoning model that uses tools. Press play, pause whenever you like, and use the chapter menu to jump around.

The idea came from watching the animation Addy Osmani created based on his article How modern browsers work. If a browser can be explained that way, so can a language model.

The Animation

Keyboard: space to play or pause, arrow keys to skip five seconds. On a phone, full screen in landscape works best.

What It Covers

1. The next-token trick: everything an LLM does comes down to predicting the next token.
2. An artificial neuron: inputs, weights, a bias and a nonlinear activation (ReLU and GELU).
3. Neural networks: layers of neurons, and why each layer is one matrix multiplication.
4. Training: cross-entropy loss, backpropagation and gradient descent on a real loss curve.
5. Ways to learn: supervised, unsupervised, self-supervised and reinforcement learning.
6. Tokenization: Byte-Pair Encoding and token IDs.
7. Embeddings: meaning as geometry, and rotary position encoding (RoPE).
8. Self-attention: queries, keys, values, softmax and the causal mask.
9. The transformer block: multi-head attention, feed-forward layers, normalization and the residual stream.
10. Mixture of Experts: a router sends each token to a few specialised sub-networks.
11. Generation: logits, temperature, sampling and the KV cache.
12. Generative AI: discriminative vs generative models, and the family of text, image, audio and video generators.
13. Making an assistant: pre-training, supervised fine-tuning, RLHF and RL on verifiable rewards.
14. Scaling: scaling laws and test-time compute for reasoning models.
15. Context, tools & agents: the context window, tool calls, the agent loop, and why models hallucinate.
16. Recap: the whole machine in one loop.

Honest Simplifications

Every number on screen is actually computed: the neuron’s weighted sum, the softmax probabilities, the gradient descent steps, the k-means clusters and the router’s gate weights. To keep things readable, a few inputs are staged:

• The token IDs are illustrative, not taken from a specific tokenizer.
• The attention scores are hand-picked so that “it” clearly attends to “cat”. A real model learns them.
• The embedding map squashes thousands of dimensions into two.
• The loss landscape has one weight. Real models descend in billions of dimensions at once.

How It Is Built

Nothing here is a video. The animation is drawn live on an HTML <canvas> by a single dependency-free script, and every frame is a pure function of time. That makes scrubbing, pausing and resizing exact, and the whole thing weighs about 80 kilobytes of unminified JavaScript. The colours are read from this site’s own design tokens, so the animation always matches the page around it.

Embedding it is one element and one script:

<link rel="stylesheet" href="css/llm-explainer.css" />
<figure data-llm-explainer></figure>
<script src="js/llm-explainer.js" defer></script>

Written and animated together with Claude, a large language model explaining how large language models work.