How to Bake a Language Model

All of LLM training in one sitting, with a minimum of math and a maximum of cheek

AUTHOR'S NOTE

This is a guide for people who want to know what's happening inside the machine, but don't want three semesters of linear algebra for it. Each step gets one analogy, one line of math (really just one), and one pointer where to go if it hooks you.

The order is the same as in actual production: from text through the machine to the finished assistant.

1. TOKENIZATION — CHOPPING TEXT INTO LEGO

The model doesn't read letters or words. It reads tokens: pieces of text created by an algorithm walking through a giant pile of text and gradually gluing the most frequent pairs of characters into bigger chunks. The frequent word "the" is one token. The word "Vrchlabí" can easily be four, because the machine has to assemble it from the bricks it had at hand.

Analogy: it's Lego. You have a limited set of bricks (typically 30 to 150 thousand kinds) and any text can be built out of them. That's why the model can "read" a word it has never seen — it just builds it from pieces.

Math: text → a sequence of integers $(t_1, t_2, \dots, t_n)$ , each from the interval $[0, V)$ , where V is the vocabulary size. GPT-2 has V = 50,257.

Where to go next: Sennrich, Haddow & Birch (2016), "Neural Machine Translation of Rare Words with Subword Units" — the original BPE paper. And then try typing a few sentences into https://platform.openai.com/tokenizer, because once you see how many tokens Czech diacritics eat, you'll understand why models speak English more cheaply.

2. EMBEDDINGS — GIVING EVERY BRICK COORDINATES

The number 4,731 means nothing by itself. So we assign every token a long list of numbers (a vector), and during training those vectors shift so that tokens with similar meanings end up close to each other. "King" and "queen" land next to each other. "King" and "screwdriver" don't.

Analogy: every word gets GPS coordinates in a world that has, say, 4,096 dimensions instead of two. Nobody enters those coordinates by hand. The word moves in on its own, according to the company it keeps in texts.

Math: $E \in \mathbb{R}^{V \times d}$ , row $E[t]$ is the vector of token t. For Llama 2 70B, d = 8,192.

Where to go next: Mikolov et al. (2013), "Efficient Estimation of Word Representations in Vector Space" — word2vec, the famous $\text{king} - \text{man} + \text{woman} \approx \text{queen}$ .

3. ATTENTION, EVERY TOKEN LOOKS AROUND THE ROOM

Here's the trick that won the transformer the game. Every token looks at all the tokens before it and asks: "Which of you matters to me right now?" The pronoun "she" turns toward the nearest female name. The verb "paid" turns toward the who and the what.

Analogy: a party. Every guest wears a tag with what they offer (key), and carries in their head what they're looking for (query). For every pair you compute how well they match, and each guest then mixes a cocktail out of what the best-matching ones are saying (values). The word "attention" means exactly that: whom I pay attention to and in what proportion.

Math (the one line worth memorizing):

\operatorname{Attention}(Q, K, V) = \operatorname{softmax}\!\left( \frac{Q K^{\mathsf{T}}}{\sqrt{d_k}} \right) \cdot V

The division by $\sqrt{d_k}$ is there so that with large vectors the dot products don't get huge and the softmax doesn't lock onto a single guest. A detail, but a detail without which it doesn't train.

And because one look around the room isn't enough (one notices grammar, another meaning, a third what's inside the quotation marks), it's done several times in parallel: multi-head. Several guests at one party, each noticing something different.

Where to go next: Vaswani et al. (2017), "Attention Is All You Need". But first read Jay Alammar, "The Illustrated Transformer" — it has the pictures the original paper is missing.

4. THE RESIDUAL STREAM, OR THE HIGHWAY EVERY LAYER ONLY ADDS A NOTE TO

The model has dozens of layers (Llama 70B has 80). If each layer overwrote its input, after thirty layers nothing would remain of the original word. So they do it smarter: the input gets copied forward unchanged and the layer only adds its note to it.

Analogy: a shared document passing through 80 editors. Nobody deletes, each one just adds a comment in the margin. Anyone who wants can still read the original text even after eighty rounds.

Math: $x_{l+1} = x_l + F(x_l)$ . That's all. The simplicity is a feature: the gradient always has a straight path back, because the derivative of x with respect to x is 1.

After attention, each layer also has a small feed-forward network (MLP), which works on each token separately... something like "now that you know whom you paid attention to, think it over". Most of the model's parameters sit right here.

Where to go next: He et al. (2015), "Deep Residual Learning for Image Recognition" — this is where transformers inherited it from. And from your folder, Xie et al. (DeepSeek, 2026), "mHC: Manifold-Constrained Hyper-Connections" — what happens when someone tries to widen that highway into more lanes and finds that stability falls apart until they put the "derivative equal to one" back.

5. THE OBJECTIVE, NOT "GUESS THE WORD" BUT "SPREAD THE BETS"

Now the important thing that popular explanations most often get wrong. We don't train the model to guess the next word. We train it to say, for every word in the vocabulary, the probability that it comes next. After "Good" it puts a high weight on "morning", a decent one on "evening", a small one on "idea" and a microscopic one on "screwdriver".

Analogy: a bookmaker. We don't want him to pick the winner. We want him to post odds on all the horses, and we punish him according to how low a price he gave the one that actually finished first. A bookmaker who gives 99% to one horse ends up worse in the long run than one who honestly spreads the bets where the real uncertainty is.

Math: for the correct next token $t^{*}$ , the loss is

L = -\log p(t^{*})

Summed over billions of positions in text. If the model gave the correct token p = 0.5, it pays 0.69. If it gave it 0.01, it pays 4.6. If it gave it 0.99, it pays 0.01. This is called cross-entropy and it's the only thing the model "wants" during pretraining.

From this follows something kalomaze describes nicely: a model with low loss must be able to represent uncertainty. It has to know where text is predictable (code, math) and where it is loose (poetry, chat). It's not memorization. It's a learned map of where language is solid and where it's liquid.

Where to go next: kalomaze, "Understanding Transformers... (beyond the Math)" (2025) — especially the chapter on the output layer. And Karpathy, "Let's build GPT: from scratch, in code, spelled out" (YouTube, 2023) — two hours after which you'll write the whole thing in 300 lines of Python.

6. GRADIENT DESCENT, DESCENDING A MOUNTAIN BLINDFOLDED

We have a loss. We have billions of numbers (parameters) that affect it. How do we set them? For each number we compute how much the loss would change if we nudged it a little (that's the gradient), and we nudge it in the opposite direction. Many times. On the order of millions of steps.

Analogy: you're standing on a hill in fog and want to get down. You see nothing, but you feel the slope under your feet. You take a step downhill. Feel again. Step. There are 70 billion variables, but the principle stays the same — feel the slope, step, feel the slope.

Math: $\theta \leftarrow \theta - \eta\, \nabla_{\theta} L$ . The letter $\eta$ is the step size (learning rate). Too big a step and you jump over the valley. Too small and you'll be there till Christmas.

Where to go next: 3Blue1Brown, the "Neural networks" series (YouTube) — the chapters on gradient descent and backpropagation have the best visualizations I know.

7. SCALING, OR THE RECIPE THAT HOLDS SURPRISINGLY PRECISELY

How much data, how many parameters, how much compute? It turned out this isn't fortune-telling. The loss falls with model size and data volume along smooth curves, from which you can predict how good a model you haven't launched yet will be.

Analogy: you bake bread and discover that the flour-to-water ratio works the same for a loaf and for an industrial oven. That's not a given — for most things in the world it isn't like that.

Math: Chinchilla (2022) gives as the compute-optimal ratio roughly 20 training tokens per parameter. A model with 70 billion parameters wants ~1.4 trillion tokens. (Today's models train on multiples of that ratio, because it's cheaper to have a smaller model that is then cheap to run.)

Where to go next: Hoffmann et al. (2022), "Training Compute-Optimal Large Language Models". And Kaplan et al. (2020), "Scaling Laws for Neural Language Models" — the older version that kicked the whole thing off.

8. POST-TRAINING, TURNING THE INTERNET PARROT INTO AN ASSISTANT

After pretraining, the model is a text completer. Give it a question and it may happily generate ten more questions, because that's what forums look like. To make an assistant out of it, three steps follow:

SFT (supervised fine-tuning). You show it a few tens of thousands of examples of "question → good answer". It learns the format. Still the same cross-entropy, just on hand-written data.

Preferences (RLHF / DPO). Humans compare two answers and say which is better. From this the model learns what is liked, not just what is probable. RLHF builds a special "reward model" for this and uses reinforcement learning; DPO shortcuts it and learns directly from the pairs.

RL on reasoning (2024–25). The model gets tasks with a verifiable answer (math, code), is allowed to think, and gets a reward only for a correct result. No humans. This is where reasoning models like o1 and DeepSeek-R1 came from.

Analogy: pretraining is reading a library. SFT is an internship where they show you how to write an email. Preferences are a boss who keeps returning your emails with the note "not this". RL on reasoning is an exam where nobody cares how you got there, only whether the result is right.

Math (DPO, one line): increase log p(preferred) − log p(rejected) relative to the original model. Formally

L_{\mathrm{DPO}} = -\log \sigma\!\left( \beta \left[ \log \frac{\pi(y_w \mid x)}{\pi_{\mathrm{ref}}(y_w \mid x)} - \log \frac{\pi(y_l \mid x)}{\pi_{\mathrm{ref}}(y_l \mid x)} \right] \right)

where $y_w$ is the winning and $y_l$ the losing answer.

Where to go next: Ouyang et al. (2022), "Training language models to follow instructions with human feedback" (InstructGPT). Rafailov et al. (2023), "Direct Preference Optimization". DeepSeek-AI (2025), "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning".

9. SAMPLING, AND TEMPERATURE ISN'T RANDOMNESS

The finished model gives you a distribution at every position. You have to pull something out of it. "Temperature" is the parameter that sharpens or flattens that distribution before you draw from it.

Analogy: you have a wheel of fortune with slots of different sizes. Temperature 1 = the wheel exactly as the model drew it. Low temperature = you enlarge the big slot even more, the small ones almost disappear. High temperature = all the slots approach the same size. Then you spin. No randomness gets "added" anywhere; only the shape of the wheel changes.

Math: $p_i \propto \exp(z_i / T)$ . As $T \to 0$ , the highest $z_i$ always wins (greedy). At $T = 1$ it's exactly what the model trained.

Where to go next: Holtzman et al. (2019), "The Curious Case of Neural Text Degeneration" — why greedy decoding sends even an excellent model into loops, and where nucleus sampling comes from.

THE WHOLE RECIPE ON ONE INDEX CARD

text → tokens → vectors → [attention + MLP, add to the stream] × $N$ layers → odds on every token → cross-entropy → gradient → repeat a trillion times → SFT → preferences → RL on results → done, set the temperature.

None of this is hard on its own. What's hard is that it only works once there's an unbelievable amount of it — and that nobody knows exactly why this particular mix. Most of the parts are there because they proved themselves, not because anyone derived them. Which is good news: it means a lot of them can still be improved.