One scroll, start to end

001

Before anything can learn, agree on what a thing that learns even is.

Model

A model is a rule with adjustable numbers in it.

Take the taxi fare: fare = rate × kilometres + base. The shape of the rule is fixed — multiply, then add — but the rate and the base are numbers you can change. Set the dials to 18 and 40 and the rule prices any ride in the city: a 12-kilometre trip comes out at 18 × 12 + 40 = ₹256. Nothing in that formula knows anything; it is arithmetic waiting for its two numbers to be chosen well.

The split matters more than the formula. The rule's shape is chosen by a person — someone decided fares should be a multiply and an add, not a square root or a lookup table. The numbers are found by learning, which is what the rest of this road is about. Keep the two jobs separate in your head: humans pick shapes, procedures set numbers, and the word model covers the pair.

That is all the word ever means, from a two-number fare formula to a chatbot. The networks behind modern assistants are rules of exactly this kind carrying hundreds of billions of adjustable numbers, their shape still drawn by human hands. Hold onto this picture and nothing ahead will be mystical: bigger models are the same idea with more adjustable numbers, not a different kind of thing.

The honest limit is that a model can only ever say what its shape allows. The fare rule draws a straight line through the city, so if real taxis add a night surcharge after 11pm or a waiting charge in traffic, no setting of two dials will ever fit — the error is built into the shape, not the numbers. Choosing a shape too simple for the world is a mistake no amount of learning can repair. Lesson 1 walks you through building this exact fare rule yourself.

And this one you can do, not just read — Lesson 1 on the panel walks you through training it yourself.

002

A model is a rule with adjustable numbers — now look closely at one of those numbers, because everything ahead is about turning them.

Parameter

A parameter is one of those adjustable numbers — a dial.

The fare formula has two: the rate per kilometre and the base fare. Each dial moves the answer in its own way, and the way is measurable. Nudge the base up by 1 and every fare rises by exactly ₹1, whatever the distance. Nudge the rate up by 1 and a 12-kilometre ride rises by ₹12 while a 3-kilometre hop rises by ₹3 — the same turn of a different dial bends the answers differently. That per-dial fingerprint is what everything ahead will exploit.

Counting dials is how the field sizes its models. The taxi rule has two. A small image recogniser might carry a few million. The networks behind modern chatbots hold hundreds of billions — yet every one of them is still just a number waiting to be set, stored as an ordinary value in memory. A model with 70 billion parameters at two bytes each is roughly 140 gigabytes of numbers on disk, and nothing else.

Training, whatever machinery surrounds it, is only ever this: finding the setting of the dials that makes the rule's answers match the world. Two dials can be tuned by hand in an afternoon of guessing and checking. Billions cannot, which is why the next few stops build the tools — a score for wrongness, a direction to turn, a stride — that let the tuning run itself.

One caution before the counting becomes a habit. Parameter count is the field's favourite headline number and it oversells easily: more dials mean more capacity, not more sense, and a bloated model can lose to a smaller one trained on better data. And in a big network no single dial means anything a person could name — the rate dial's honest legibility is a luxury of small models, gone long before you reach billions.

And this one you can do, not just read — Lesson 1 on the panel walks you through training it yourself.

003

The dials sit at some setting; whatever the rule says at that setting, before anyone checks it, has a name.

Prediction

A prediction is what the rule says before you check the answer.

Feed 12 kilometres into the fare formula with the dials at 18 and 40 and it commits to ₹256 — on the record, before you look at the receipt. In the field's notation this is ŷ = f(x; w): x is the input, the 12 kilometres; w is the current setting of the dials; f is the rule's shape; and ŷ, said y-hat, is whatever comes out. The hat marks it as a guess. The bare y, the receipt's ₹301, belongs to reality.

The model speaks first; reality replies second, and the order matters more than it looks. A guess made after seeing the answer proves nothing — anyone can predict yesterday's rain. A guess made before is testable, and testable is what makes learning possible at all. India's monsoon forecast works the same way: the IMD publishes its seasonal number in April, months before the first cloud, precisely so the monsoon itself can grade it.

Every score in the field rests on this small ceremony: predict first, then check. It is also where projects quietly cheat. If a scrap of the answer leaks into the input — training a fare model on rides whose recorded tips already encode the total, or testing a medical model on scans it saw during training — the predictions look brilliant and mean nothing. Data leakage is the polite name, and it is one of the commonest ways a promising result dissolves.

One more honesty. A prediction carries no built-in confidence. The rule answers ₹256 with the same flat certainty whether its dials are well-tuned or freshly random; nothing in ŷ says trust me this much. The output is only what the current setting produces — its quality lives entirely in how the dials were set, and setting them is the story the next stops begin.

And this one you can do, not just read — Lesson 1 on the panel walks you through training it yourself.

004

A prediction is only a guess on the record — next, the single number that says how badly it missed.

Loss

Loss is one number saying how wrong the model currently is.

The rule predicted ₹256; the receipt says ₹301 — a miss of 45. A loss function turns that miss into a score, and the commonest choice squares it: loss = (guess − truth)², here (256 − 301)² = 2025. Squaring does two useful things at once: it makes overshooting and undershooting count the same, and it makes large errors count far more than small ones — a miss of 90 costs four times a miss of 45, not twice.

Across a whole day of rides the misses are gathered into one number, usually the average: L = (1/N) · Σ (ŷ − y)², the squared misses summed over all N receipts and divided by N. Zero means every fare was predicted perfectly; bigger means worse. One number now stands for the model's entire performance on everything it has seen, and that number has a name you will meet at every stop from here on.

One number sounds like a poor summary of many mistakes, and it is — but the poverty is the point. A single score gives the model a single thing to make smaller, and make-this-number-smaller turns learning from a vague ambition into a concrete task a machine can run. Every training run you will ever see is, underneath, the same sentence: adjust the dials until L stops falling.

Choose the loss badly, though, and the model will faithfully optimise the wrong thing. Score a hospital triage model on plain accuracy for a disease one patient in a hundred has, and predicting healthy for everyone scores 99 per cent while catching nobody. The model did exactly what it was told; the telling was wrong. The loss is the one place your intentions enter the machinery, and it deserves the care that implies.

And this one you can do, not just read — Lesson 1 on the panel walks you through training it yourself.

005

Loss handed you one number for how wrong you are — it never said what to do about it.

Gradient

A gradient says which way to turn each dial to make loss smaller.

What loss leaves open, the gradient answers: for every dial, one number saying how the loss moves if you nudge that dial up. In symbols it is ∂L/∂w — the slope of the loss with respect to that one dial, all the others held still. Positive means turning the dial up makes things worse; negative means up is downhill. In the taxi model that is two numbers, one for the rate dial and one for the base.

The numbers are concrete. With squared loss, the slope for the base dial works out to 2 × (guess − truth), and for the rate dial to 2 × (guess − truth) × kilometres. On the 12-kilometre ride the model priced at ₹256 against a receipt of ₹301, the miss is −45: slope −90 for the base, −1080 for the rate. Both negative, so both dials should turn up — and the rate's slope is twelve times steeper, because on this ride the rate matters twelve times as much.

The wonder is the price. Knowing the right nudge for a billion dials sounds like a billion experiments — nudge one, re-measure the loss, put it back, repeat. It is not. Calculus, run backwards through the rule by an algorithm called backpropagation, hands over the entire list of slopes for roughly the cost of one extra prediction. That bargain, more than any piece of hardware, is why training enormous models is possible at all.

The honest limit is locality. A slope is only true where you stand: it describes the terrain under your feet, for nudges small enough that the ground does not curve away. It says nothing about the landscape two valleys over, and following it never guarantees you are heading towards the best setting that exists — only towards somewhere lower than here. Every step in training is taken on this modest, strictly local advice.

And this one you can do, not just read — Lesson 1 on the panel walks you through training it yourself.

006

The gradient tells you which way is downhill for every dial — all that remains is to take the step, and keep taking it.

Gradient descent

Turn every dial a little downhill, over and over. That is training.

Taking it is one line: w ← w − η·∂L/∂w. Read it as a recipe. For each dial w, look up its slope ∂L/∂w, multiply by a small stride η, and subtract — subtract, because the slope points uphill and you want down. Then predict again, measure again, and do it again. That loop, applied to every dial at once, round after round, is the whole of training.

On the taxi model it looks like this: predict a batch of fares, see the misses, nudge the rate and base dials against their slopes, and go again. Start the dials anywhere — 0 and 0, 50 and 500 — and after a few hundred rounds they settle near 18 and 40, the city's fare chart recovered from nothing but receipts. The panel beside this text is running exactly this: watch the amber guess line swing towards the paper-white truth as the loop turns.

It is worth pausing on how humble this is. The algorithm at the bottom of every headline system is a ball rolling downhill in small steps — sketched by Cauchy in 1847, more than a century before anyone dreamed of learning machines. Everything since, from deeper models to faster chips, is scaffolding around this one move. Nothing smarter has been needed.

The guarantee is thinner than it feels. Each step promises only somewhere lower than here — not the lowest place there is. The walk can stall on a plateau where every slope is nearly zero, or settle into a valley that is merely local. That the loop nonetheless finds excellent settings in enormous models is an empirical fact the field leans on daily and still cannot fully explain.

And this one you can do, not just read — Lesson 1 on the panel walks you through training it yourself.

007

Gradient descent says turn the dials a little — how little turns out to decide whether training crawls or explodes.

Learning rate

How far you turn the dials each step — too small crawls, too big explodes.

That how-little has a symbol: it is the η in w ← w − η·∂L/∂w, the one number multiplying every slope before the dials move. The gradient gives the direction; η sets the stride. Take the taxi ride where the rate dial's slope came out at −1080. With η = 0.0001 the dial turns up by about 0.1 — a cautious correction. With η = 0.01 it lurches by 10.8, sailing far past the true rate of 18 in a single step.

Too small and training crawls, thousands of steps to cross a shallow valley. Too large and every step overshoots the bottom, the loss bouncing higher each round until the numbers blow up entirely — the panel beside this text will show you both failures if you drag the stride to either end. Between the two lies a band where descent is quick and steady, and finding that band is the job.

No formula gives the right value — it depends on the terrain — so the craft is honest trial: try one stride, watch the loss, drop it tenfold, watch again. Practitioners' opening guesses are small fractions like 0.001 or 0.0003, and this dial is the first thing they reach for when a run goes wrong. Among all the settings a human still picks by hand, this one decides the most.

The idea's limit is its bluntness: one stride for every dial, when a real landscape is steep in some directions and nearly flat in others. A stride safe for the steepest dial crawls along the flattest. Modern optimisers soften this by adapting the stride per dial as training runs, and schedules shrink it over time — but they decorate the choice rather than remove it. Someone still picks the number, and the number still decides whether the walk arrives.

And this one you can do, not just read — Lesson 1 on the panel walks you through training it yourself.

008

Loss gives every setting of the dials a height — put the heights together and training's terrain comes into view.

Loss landscape

Every setting of the dials has a height; training walks downhill on that terrain.

For the taxi model the terrain is drawable. Rate along one axis, base along the other, and above every pair a height: L(rate, base) = (1/N) · Σ (rate·km + base − fare)², the average squared miss over all receipts if you froze the dials right there. Compute that height everywhere and a smooth bowl appears, and its lowest point sits at 18 and 40. The panel beside this text draws the bowl and lets you drop a walker anywhere on it.

Every idea so far takes its place on this one picture. A setting of the dials is a point on the ground. The loss is the altitude at that point. The gradient is the slope underfoot, pointing up the steepest way. The learning rate is the length of the stride. Gradient descent is the walk itself — read the slope, step against it, repeat until the ground flattens.

Training never sees the map. Charting the whole surface even for two dials means recomputing the loss at thousands of settings; for a billion dials it is beyond every computer that will ever exist. The walker only ever feels the slope at the point where it stands and steps downhill — which is why the whole apparatus of gradients and strides needed to exist at all.

Be careful with the picture, though. For two dials the terrain is a clean bowl; for a billion it is a landscape in a billion dimensions — ridges, plateaus, uncounted valleys — and the tidy 3-D plots you will see everywhere are two-dial slices through it, chosen to be lookable. Hill-walking intuition misleads out there: in very high dimensions, truly closed valleys that trap the walker are rarer than the pictures suggest, and vast flat plateaus are commoner. The enduring surprise of deep learning is that walking downhill through such terrain works anyway.

And this one you can do, not just read — Lesson 1 on the panel walks you through training it yourself.

009

Everything so far assumed somebody wrote down the true fares — learning from written-down answers is its own regime, with its own cost.

Supervised learning

Learning from examples where somebody already wrote the answer down.

The regime works like studying with an answer key. You gather examples where somebody already wrote the truth down — this email is spam, this scan shows a tumour, this 8-kilometre ride cost ₹176 — into pairs (x, y), input beside answer. Then the loop you already own runs unchanged: predict ŷ = f(x; w), score the miss against the written y, follow the gradient downhill. The only new ingredient is that the y column exists at all, and that someone had to fill it in.

That someone is the regime's defining cost. ImageNet, the dataset behind the deep-learning boom of the 2010s, holds about fourteen million photographs, every one labelled by human beings — years of crowdsourced clicking. A medical imaging project pays radiologists by the scan. The labelled data is very often the most expensive line in the whole project, dearer than the computers — which is exactly the cost self-supervision was invented to escape.

Where answers come cheap, though, it remains the sharpest tool there is. Every UPI transaction is eventually confirmed or disputed; every delivered parcel gets an actual delivery time; every flagged email gets a verdict from the person who flagged it. When the world writes the y column for you as a by-product of ordinary life, supervised learning gives a clear target, a clean score, and every stop on this road so far applies without modification.

The honest limit lives in the key itself. The model learns the answers as written, not the truth: if labellers disagree, tire, or share a blind spot, those flaws are pressed into the dials with perfect fidelity. A tumour dataset labelled under one hospital's conventions carries that hospital's habits to every hospital the model later serves. Supervised learning is only ever as good as its answer key, and the key was written by people.

010

Supervised learning leaned on an answer key at every step — now take the key away and see what can still be learned.

Unsupervised learning

Finding structure in data nobody labelled.

Something survives the removal: structure. Ten thousand UPI customers with no labels at all still fall into groups — night shoppers, bargain hunters, one-time visitors — because their transaction patterns bunch together whether or not anyone names the bunches. Unsupervised learning is the family of methods that find such groups on their own, and more broadly any pattern the data carries by itself: clusters, outliers, directions along which the examples vary most.

The classic method, k-means, is four moves repeated. Choose how many groups you want, say k = 4. Place four centre points anywhere among the customers. Assign every customer to its nearest centre. Move each centre to the average position of the customers assigned to it. Repeat the last two moves until nothing shifts. No answer key is consulted at any point — the data sorts itself around wherever the centres settle.

The honest difficulty: with no labels there is no score, so there is no clean way to be told you got it wrong. Run k-means with k = 4 and you get four tidy groups; run it with k = 6 and you get six, equally tidy. Two clusterings of the same customers can both look plausible, and no loss function will arbitrate between them. The judge is usefulness — did the campaign built on these segments actually work? — and usefulness is judged by a human.

Held against supervised learning the trade is plain: no answer key to pay for, but no key to be graded by either. That looser grip is why unsupervised methods usually serve as explorers rather than deciders — surfacing customer segments for a marketer to name, flagging the one transaction in a million that sits far from every cluster for a fraud analyst to inspect. The pattern-finding is automatic; the meaning is still assigned by people.

011

Supervised learning's labels are expensive; here is how modern AI gets answer keys for free, out of the data itself.

Self-supervised learning

Hiding part of the data and making the model guess it — where modern AI gets its scale.

The trick is to hide part of the data and make the model guess it. Cover the next word of a sentence, a patch of a photograph, the next second of audio — then the uncovered part becomes x, the covered part becomes y, and you have supervised learning's pairs without paying anyone. 'She hailed a taxi to the ___' is an input; 'station' is its label; and the label cost nothing, because the sentence carried it all along.

Mechanically nothing else changes. The model puts a probability on every possible next word, and the penalty is −log p(the word that actually came next): guess the truth likely and the penalty is near zero, guess it unlikely and the penalty soars. Then gradients, then descent, exactly as before. The panel beside this text is playing the game live, masking words and scoring the guesses.

This is where modern AI gets its scale. Nobody labelled the web; the web labels itself, one hidden next-word at a time. Supervised projects were still paying people to label thousands of examples while language models trained on trillions of words — recent open models report training sets around fifteen trillion tokens. The answer key is free and there is effectively unlimited homework, and that arithmetic, more than any single invention, is the engine of the current era. Lesson 2 has you play the masking game yourself.

The bill arrives elsewhere. A model trained to continue the internet learns the internet as it is — its errors, prejudices, and confident nonsense come free with the labels, pressed into the dials at the same trillion-word scale as everything else. Predicting the likeliest next word is not the same as saying true or helpful things, which is why every deployed assistant needs further training on top before anyone should trust it. The labels are free, but not free of what the labellers — all of us — wrote.

And this one you can do, not just read — Lesson 2 on the panel walks you through training it yourself.

012

Training turns the dials; but almost every prediction a model ever makes happens after the turning stops.

Inference

Using a trained model. No dials move; it just answers.

Once the turning stops, the model enters the phase it will spend nearly all its life in. The numbers are frozen; an input flows through them to an output, ŷ = f(x; w) with w fixed, and nothing flows back — no loss is computed, no gradient, no update. When you ask a chatbot a question or your camera locks onto a face, no learning is happening. The taxi rule at this stage prices a 12-kilometre ride at 18 × 12 + 40 = ₹256 in a microsecond, and could do so a billion times without changing.

The economics split along the same line. Training is the enormous one-time cost — weeks of machines, bills that for frontier models run to tens of millions of dollars. Inference is the small cost paid on every single use, a fraction of a rupee per chatbot reply. But small multiplies: at a hundred million users asking daily, the small cost is the one that dominates the bill, which is why so much engineering effort goes into making frozen models cheaper to run.

Frozen also buys a quiet virtue: repeatability. Given the same input and the same dials, the rule returns the same output — any variety you see in a chatbot's replies is deliberate dice added at the output, not learning happening underneath. It means a deployed model can be audited once and trusted to be, number for number, the same machine on the millionth query as on the first.

The limit is staleness. Frozen dials cannot learn from their own mistakes: a fare model trained before the city revised its rates keeps quoting the old chart, fluently and confidently, until someone retrains it. The world moves and the numbers do not. Every deployed model is a photograph of its training data, and like a photograph it ages — the only cure is to go back and turn the dials again.

013

The maths track opens with the shape all data takes on its way into a model: a plain list of numbers.

Vector

A list of numbers describing one thing.

A song might be [tempo, loudness, danceability, year]. A customer might be [age, visits per month, average spend]. An auto-rickshaw trip might be [distance in km, minutes in traffic, hour of day]. The order is fixed and the meaning lives in the positions: slot one always means the same thing across every song or every trip. Once written this way, the thing becomes a point in space, with one axis per number — four numbers, four axes. You cannot picture four dimensions, but the arithmetic never asks you to.

The payoff is that geometry starts to mean something: things whose vectors sit close together are similar things. Distance is computed exactly as on a map, just with more axes: d = √((a₁−b₁)² + (a₂−b₂)² + …) — subtract matching entries, square each gap, add them up, take the root. Two trips of [12 km, 40 min, 9 am] and [11 km, 44 min, 9 am] land close together; the 2 am airport run lands far away, and a fare model can treat it differently for that reason alone.

Almost everything a modern model does — search, recommendation, analogy — is some form of measuring distances in that space. Modern systems push this to vectors nobody designs by hand: inside a large language model, every word arrives as a list of thousands of numbers, often 4,096 of them, and no engineer chose what any axis means. Training arranged the space so that useful neighbours ended up nearby, which is all the model needs.

The honest catch is units. A gap of one kilometre and a gap of one minute both contribute a 1 to the sum, though they are not remotely the same size of difference; measure the trip in metres instead and the distance is suddenly dominated by it. Closeness in vector space is only as meaningful as the scaling you chose, which is why practitioners rescale every axis before trusting any distance — and why an unscaled vector can quietly rank an absurd neighbour first.

014

One vector describes one thing; a grid of them, it turns out, is the field's favourite machine.

Matrix

A grid of numbers — usually a stack of vectors, or a machine that transforms them.

It earns its keep in two roles. The first is storage: stack vectors as rows and a whole dataset becomes one object — ten thousand customers, each [age, visits per month, average spend], is a 10,000 × 3 grid you can feed to a model in one go. The second role is stranger and more important: a matrix is a machine that transforms vectors. Numbers in, numbers out, one multiplication — a grid three wide and two tall takes any three-number vector and returns a two-number one.

The entries are the recipe. Each output number is a weighted mix of the inputs: out₁ = w₁₁·in₁ + w₁₂·in₂ + w₁₃·in₃, and the second row makes out₂ the same way with its own weights. Write y = Wx and that is the whole machine — W holds the mixing weights, x is what walks in, y is what walks out. Change one entry and you have changed what the machine does, which is exactly what turning a dial means here.

Every layer of every neural network is, at its core, a matrix in this second role. A 7-billion-parameter language model is, for the most part, a few hundred such grids, many of them thousands of numbers on a side, applied one after another. That is why graphics chips, built to multiply grids of numbers for video games, turned out to be the hardware the whole field was waiting for — the games industry had spent decades perfecting exactly this machine.

The honest limit: a matrix can only stretch, rotate, and mix. It is linear — double the input and the output exactly doubles — so it cannot bend, threshold, or say only-if. Worse, stacking helps not at all: ten matrices multiplied together collapse into one matrix, no more expressive than where you started. Networks only become interesting because a small nonlinear step sits between the grids. The matrix is the muscle of deep learning, but on its own it is a one-trick muscle.

015

A matrix earns its keep the moment you multiply with it — one operation, most of AI's electricity bill.

Matrix multiplication

The single operation that eats most of the world's AI electricity.

To multiply two matrices, take each row of the first against each column of the second: multiply the paired numbers, add them up, and the little sum becomes one cell of the result. Row [2, 1, 3] against column [4, 0, 5] gives 2·4 + 1·0 + 3·5 = 23, and 23 sits in one cell. That is the entire operation — multiplies and adds arranged in a grid, with no cell's answer depending on any other cell's.

That independence is the whole story. Because every cell can be computed at the same instant, the work spreads perfectly across thousands of simple processors — which is exactly what graphics chips, built to render video games, already were. A modern AI chip performs on the order of a thousand trillion multiply-adds per second, and it reaches that figure only because nothing in this operation ever waits on anything else.

The bill adds up fast. Producing one token from a language model costs roughly two multiply-adds per parameter, so a 70-billion-parameter model spends about 140 billion operations on every single word it writes — and training replays that, forwards and backwards, over trillions of words. Nearly everything a modern model does, in training and in use, is this one operation, which is why it eats most of the world's AI electricity.

One honest correction to the folklore: the arithmetic is not always the bottleneck. The multiplies are so fast and so parallel that chips regularly stall waiting for the numbers themselves to arrive from memory — during chat-style generation, moving the weights, not multiplying them, is often what sets the speed. Buying more raw multiply power then buys you nothing. The operation is simple; feeding it is the engineering problem.

016

Matrix multiplication was built from a smaller move — two vectors, one number, a measure of agreement worth meeting on its own.

Dot product

One number saying how much two vectors point the same way.

Every cell of a matrix multiplication was secretly one of these. The dot product takes two vectors and returns one number: multiply matching entries, add everything up — a·b = a₁b₁ + a₂b₂ + …. Big and positive means agreement; near zero means unrelated; negative means opposed. Score your film tastes [0.9 thriller, 0.1 romance, 0.8 cricket documentary] against a film's profile [0.8, 0.2, 0.7]: 0.72 + 0.02 + 0.56 = 1.30. A romance with profile [0.1, 0.9, 0.0] scores 0.18. The first film wins the recommendation.

This one operation quietly runs the modern internet. 'People like you also bought' is a dot product between your vector and other shoppers'. Attention inside a transformer scores every pair of words with — a dot product. The panel beside this text is running exactly this: drag either vector and watch the one number rise as they turn to agree, and sink below zero as they come to oppose.

Geometry explains the behaviour: a·b = |a|·|b|·cos θ — the two lengths multiplied, times how aligned the directions are. That cos θ is 1 for parallel vectors, 0 at right angles, −1 for exact opposites, which is why the same formula reads naturally as a similarity meter. Direction carries the meaning; the dot product measures it while also, quietly, weighing size.

And that weighing is the honest catch. A long vector scores high against nearly everything, agreement or not, the way a loud voice dominates a room. Practitioners therefore often divide the score by both lengths — cosine similarity — keeping only the alignment. Attention layers make a version of the same repair, dividing their scores by the square root of the vector width, because raw dot products balloon as vectors get wider, and the balloon drowns every subtler comparison.

017

Behind every gradient you have turned dials by sits one idea from calculus, and it fits in a sentence.

Derivative

How fast one number changes when you nudge another.

A derivative answers one question: if I nudge this number, how fast does that number move? Your speedometer is a derivative — how fast position changes as time changes. Nothing more exotic is involved. Written down, it is the ratio of two nudges: nudge the input by a tiny ε, see how far the output moves, divide — dL/dw ≈ (L(w+ε) − L(w)) / ε — then let the nudge shrink toward zero so the answer describes this exact point rather than a neighbourhood.

Take a metered taxi: fare = 50 + 15·km. Nudge the distance by one kilometre and the fare moves by ₹15, always — d(fare)/d(km) = 15, a straight line's slope. Curves are more interesting. For the squared loss from earlier on the road, loss = (guess − truth)², the derivative is 2·(guess − truth): overshoot the truth by 3 and the slope is 6, undershoot by 3 and it is −6. The sign says which way is downhill; the size says how urgently.

Training cares about one specific version of the question: if I nudge this dial, how does the loss move? Answer that for every dial and you know exactly which way to turn each one. That answer, collected across all dials, is the gradient — for a large language model, billions of these ratios, all computed exactly, at every single update, by the chain rule waiting at the next stop.

The honest limit is locality. A derivative is the slope right here, trustworthy only for small nudges; it promises nothing about the terrain a big step away, which is exactly how an over-eager learning rate leaps clean past the valley it was descending into. And a slope of zero only means flat — the bottom of a valley, the top of a ridge, and a wide dull plateau all read identically as 0, and the derivative alone cannot tell you which one you are standing on.

018

A derivative handles one step; a model is a chain of steps, and nudges must be traced through all of them.

Chain rule

How to trace a nudge through a long chain of steps. The engine of all training.

Gear A turns gear B turns gear C: if you know each link's ratio, you multiply them to get the whole chain's ratio. If one turn of A gives three turns of B, and one turn of B gives half a turn of C, then one turn of A gives 3 × 0.5 = 1.5 turns of C. That multiplication is the entire rule. In symbols: if y depends on u and u depends on x, then dy/dx = (dy/du)·(du/dx) — the outer step's slope times the inner step's slope.

A deep network is a long chain — layer feeding layer feeding loss. To learn how the loss responds to one deep-buried weight, multiply the local slopes along the path between them: ∂L/∂w = ∂L/∂out · ∂out/∂w. If the loss climbs 6 for each unit the output rises, and the output rises 0.5 for each unit the weight rises, the weight's gradient is 3. The chain rule is how the blame for a wrong answer travels backwards through every link, and backpropagation is nothing but this rule applied with good bookkeeping.

The panel beside this text is running exactly this: three linked stages, each with its own local slope, and a nudge you can push in at one end. Watch the nudge get scaled by each link in turn — the product of the ratios is precisely the number training would use to turn that dial, and it changes the moment any single link's slope does.

The honest trouble is that long products are unstable. A hundred layers whose links each scale the nudge by 0.9 pass on 0.9¹⁰⁰ ≈ 0.00003 of it — the signal vanishes before reaching the early layers; links of 1.1 instead compound to around 13,000, and the update explodes. Much of modern network design — residual connections, normalisation layers, careful initial dial settings — is engineering with one purpose: keeping this product of a hundred slopes close enough to 1 that blame survives the journey backwards.

019

Models rarely deal in yes or no — they deal in how sure, and how sure needs rules of its own.

Probability

A number from 0 to 1 for how much you believe something.

A probability is belief, written as a number between 0 and 1. Rain tomorrow at 0.8 means: on all the days that looked like today, it rained on about eight in ten of them. The scale's two ends are the only certainties — 0 for impossible, 1 for guaranteed — and everything honest lives strictly between. When the monsoon forecast says an 80% chance of rain over Mumbai, it is making a claim you can audit: gather a hundred such forecasts and count.

Two rules do most of the work. Beliefs about outcomes that cannot both happen must add up: if rain is 0.8, no-rain must be 0.2, because together they exhaust the possibilities and total belief is always exactly 1. And beliefs about independent events multiply: two fair coin flips at 0.5 each make both-heads 0.5 × 0.5 = 0.25. Multiply-when-independent is the rule models lean on hardest — and the one most often abused, since real events are rarely as independent as the arithmetic assumes.

The mark of a good model is not confidence but calibration: of all the times it says 0.8, the thing should actually happen about 80% of the time. A model that says 0.99 and is wrong every tenth time is lying to you with numbers. Weather services track this deliberately — reliability curves comparing stated chances against observed frequencies — and a well-run forecast survives the audit.

One honest gap remains even then: a probability records how sure, never why, nor on how much evidence. A 0.5 born of total ignorance and a 0.5 born of years of careful study print identically, and nothing in the number warns you which one you are holding. Models inherit this silence — a language model's confident 0.9 on a fact it has barely seen looks exactly like its 0.9 on one it has seen a million times.

020

One probability rates one outcome — a model must spread its belief across every outcome at once, and that spread has laws.

Distribution

Belief spread across every possible answer, adding to one.

A distribution is belief spread across every possible answer at once, adding up to exactly 1. Not 'the next letter is e' but 'e: 40%, a: 20%, space: 15%…' — a full spread of opinion. The fixed budget is the law that gives it teeth: because the total is pinned at 1, raising belief anywhere must lower it somewhere else. Confidence is never free; it is always taken from the other outcomes.

Language models output nothing else. Every token a chatbot writes was drawn from a distribution over its whole vocabulary — tens of thousands of entries, each assigned its slice of belief, computed fresh at that position. After 'the batsman lofts it for', the spread might put 0.55 on 'six', 0.30 on 'four', and scatter the remaining 0.15 across everything else the model has ever seen. The confidence you sense in its prose is a spread of numbers you are not shown.

Writing is then sampling: draw from the spread and commit. Draw greedily — always the biggest slice — and the prose turns repetitive and stiff; draw faithfully and an unlucky 2% outcome occasionally lands on the page. Every creativity control a chatbot exposes is a knob that reshapes this spread before the draw, sharpening it toward the favourite or flattening it toward the long tail.

The honest limit: a spread can be perfectly well-formed and wrong everywhere. Summing to 1 is bookkeeping, not truth — it is the model's belief, not the world's. And the spread only covers outcomes on its list: anything outside the vocabulary receives exactly zero, forever, no matter what evidence arrives. A distribution is opinion in its most disciplined format. Disciplined opinion is still opinion.

021

A distribution lays your belief out in full — the next question is how much surprise it still leaves you carrying.

Entropy

How surprised you should expect to be. Low entropy means confident.

A fair coin carries maximum surprise for two options — you truly cannot do better than guessing. A coin that lands heads 99% of the time carries almost none. Entropy makes that precise. One outcome's surprise is log₂(1/p) bits — the rarer the event, the bigger the number — and entropy is the surprise you should expect on average: H = Σ p · log₂(1/p), each outcome's surprise weighted by how often it actually arrives.

Run the numbers. The fair coin: two outcomes at 0.5, each worth log₂(2) = 1 bit of surprise, expected value 1 bit — the maximum for two options. The 99% coin: 0.99 · log₂(1/0.99) + 0.01 · log₂(1/0.01) ≈ 0.08 bits, almost nothing, because the common outcome barely surprises and the shocking one barely ever happens. A fair six-sided die sits at log₂ 6 ≈ 2.58 bits. More options, spread more evenly, means more expected surprise.

For a learner, entropy is a floor. If the data itself is noisy, no model — however large — can average less surprise than the noise supplies. Your lesson-2 model walks its loss down to exactly this floor and stops, because stopping there is correct: what remains is not failure to learn but genuine uncertainty in the world, and no amount of further training buys it back.

Two honest cautions. Low entropy means confident, never correct — a model certain of the wrong answer has beautifully low entropy and is still wrong. And the floor itself is usually unknowable for real data: nobody holds the true distribution of English sentences, so nobody can compute language's actual entropy — only estimate it, using models whose own imperfections inflate the estimate. The floor is real, but you learn where it is mostly by failing to get below it.

022

Entropy set the floor of unavoidable surprise — cross-entropy measures how far above that floor a model's beliefs actually sit.

Cross-entropy

The loss used whenever a model picks from a list of options.

The measuring is simple: when the truth arrives, look up the probability the model had given to what actually happened, and charge it by how small that number is. The bill is −log p(truth) — a light fee for 0.9, a punishing one for 0.01, an unpayable one for a flat zero, since the log of zero runs off to infinity. Confidently wrong is the most expensive way to be, and the charge grows without bound as the model's belief in the truth shrinks.

This is the loss behind nearly every model that picks from options: every next-word guess a chatbot makes is billed this way, one token at a time, and the falling loss in the field's famous training charts is cross-entropy. The average bill splits cleanly in two: the entropy floor the data itself supplies, plus the extra the model pays for believing the wrong spread. Training can only shrink the second part; the first belongs to the world.

The panel beside this text is running exactly this: reshape the model's spread, let the truth land, and watch the bill. Pile belief on the right answers and the average charge slides toward the floor; get cocky about a wrong one and a single arrival of the truth costs more than a hundred humble mistakes. Lesson 2 walks you through training a real model against this exact bill.

The honest limit: the bill reads only one line — the probability of the exact truth. There is no credit for near-misses. Put 0.4 on 'couch' when the answer was 'sofa' and you are charged as though you had put 0.4 on 'helicopter'; the loss itself cannot see that one mistake is reasonable and the other absurd. Models do learn that sofas and couches behave alike, but they learn it from patterns in the data, never from any mercy in the scoring.

And this one you can do, not just read — Lesson 2 on the panel walks you through training it yourself.

023

Probabilities multiply, and multiplied probabilities shrink toward nothing — practitioners have one standard trick for keeping them alive.

Working in logs

Why practitioners take logarithms of everything: multiplication becomes addition, and tiny numbers stop vanishing.

Multiply the probabilities of a thousand-word sentence together and the result is so small no computer can hold it — a thousand words at 0.05 each multiply to 10 to the minus 1,301, while ordinary floating point gives up near 10 to the minus 308. The product rounds to zero and every comparison dies. Take logarithms first and the same computation becomes a sum of ordinary-sized numbers: log(a·b) = log a + log b, so multiplication becomes addition, and nothing vanishes.

The same sentence in log space: each word contributes ln 0.05 ≈ −3.0, and the thousand of them sum to −3,000 — a perfectly ordinary number a computer holds without complaint. Comparing two candidate sentences is now comparing two sums, and the sums preserve exactly the ordering the products would have had, because log never reorders: whatever was bigger before logging is bigger after. The panel beside this text is running exactly this — watch the raw product starve to zero while its logged twin strolls along.

This is why practitioners live in log space. The loss a language model reports is a logged probability; likelihoods in papers arrive pre-logged; even progress charts are drawn on log axes so a tenfold change looks the same size everywhere. When a formula is wrapped in a log, it is rarely deep theory — it is plumbing, keeping small numbers alive.

The plumbing has one honest cost: logged numbers defeat linear intuition. A loss falling from 2.3 to 2.0 looks like a modest trim, but undoing the log — e^0.3 ≈ 1.35 — reveals the model assigning 35% more probability to every token, compounding across every word of every answer. Small gaps on a logged chart are large gaps in the world. Read log-space numbers as if they were plain ones and you will underrate every improvement you see.

024

A model is a rule with blanks for numbers — the first honest question is which measurements of the world get to fill them.

Feature

One measurable thing you feed the model.

Fill the blanks badly and no amount of training rescues the rule. A feature is one measurable thing you feed the model: the distance of a taxi ride, the hour of the booking, the age of the account, whether it rained. Each becomes one number in the input, one x waiting for its weight — fare = w1·km + w2·hour + b is a model with exactly two features, and every feature you add gives the rule one more blank the world is allowed to fill.

Choosing those measurements well used to be most of the job, and on tabular business data it still is. Practitioners spend their days composing features rather than models: turning two raw columns into a ratio, a date into a day-of-week, a skewed price into its logarithm — you did precisely that at the last stop, working in logs. A fraud model given ten well-chosen columns routinely beats a grander model fed a hundred careless ones.

Deep learning's real revolution was making feature design itself learnable. Give the network raw pixels or raw text and let the early layers invent the measurements: that is what the sliding filters of 1989 actually were — learned features, edge and texture detectors nobody hand-wrote. The early layers of a modern vision network still discover the same kinds of measurement, only now there are millions of them, tuned by gradient descent instead of by a researcher's intuition.

The honest catch is that a feature records what was convenient to measure, not what caused anything. A pincode is easy to log, and in most Indian cities it quietly encodes income, language and community; feed it to a loan model and the model will happily use it. The rule with blanks cannot tell a cause from a correlate. Feature choice is therefore where bias enters first — before any training begins, someone decided which measurements of the world count.

025

Supervised learning runs on examples with the answer written down — time to look hard at who writes it, and at what cost.

Label

The right answer, written down by someone, for the model to be scored against.

Someone, in every case, is a person — or a process a person set up. A label is the right answer, written down by someone, for the model to be scored against: 'Spam.' 'Cat.' '₹176.' Mechanically it is the truth term in the loss — in loss = (guess − truth)², the label is the truth, the fixed number the model's guess gets pulled towards. Supervised learning is impossible without labels, and nothing about them is free.

ImageNet, the dataset behind the 2012 breakthrough, holds about fourteen million labelled images and took years and tens of thousands of human hours before a single model trained on it — much of the work done by crowdworkers paid a few cents per judgement. When people say data is the real asset, labelled data is usually what they mean, because the answers are the expensive part.

Who writes them decides what they cost and what they are worth. Users label for free without noticing — every message you mark as spam trains a filter, every 'was this ride okay?' tap after a trip is a label. Annotation offices from Hyderabad to Nairobi label for a few rupees per item. And a radiologist marking tumours on scans costs hundreds of times more per label, which is one reason medical datasets stay small.

The limit to hold onto: a label is an opinion committed to ink. Ask three annotators whether a comment is abusive and they will disagree on the borderline cases, which are exactly the cases that matter. Even ImageNet's carefully checked labels turn out to be wrong roughly six percent of the time. A model can never be more right than the answers it was scored against — up close, the written-down truth is somebody's best guess.

026

You have features and labels now — but before training touches them, hide some, or you will never know if it learned.

Train and test split

Hide some data from training, or you will only ever measure memorisation.

The mechanism is a single act of self-denial. Shuffle your 10,000 taxi receipts, deal 8,000 into a training pile and lock the other 2,000 in a drawer before the model exists. Train only on the first pile. Then, once, measure on the drawer. This is the difference between an exam made of questions from the textbook and an exam the student has never seen — the panel beside this text is dealing exactly this split.

Skip it and your scores measure memorisation, not learning. A model with millions of dials has room to store its examples the way a student stores past papers, and it will ace anything it has effectively stored. Every real result you have ever seen quoted was — or should have been — measured on held-out data; a training-set score is not a small overstatement, it is a different quantity wearing the same name.

The split must be honest in two quiet ways. First, split before any preprocessing: compute your scaling, your vocabulary, your feature choices on the training pile alone, or information seeps across — Data leakage, a few stops ahead, is a whole catalogue of these seeps. Second, random dealing lies when the data has time in it. To predict next week's fares, train on the past and test on the future; shuffling lets Tuesday's receipts help explain Monday's, a rehearsal reality will never grant.

One limit even a clean split cannot fix: the drawer only certifies performance on data like the data you had. And eighty-twenty is a convention, not a law — with millions of rows a one-percent test slice is plenty, while with three hundred rows no split is comfortable and you resort to fancier rationing. The held-out exam proves the student learned the syllabus. It says nothing about a question from a different book.

027

The test set proves learning happened — but choosing settings needs its own held-out slice, or the proof gets spent along the way.

Validation set

A third slice, used for choosing settings, so the test set stays honest.

The mechanism is rationing. Cut the data three ways, not two. Train on the first slice. Use the second — validation — to make every choice training cannot make for itself: try learning rates of 0.001, 0.01 and 0.1, train one model with each, keep the one with the best validation score. Choose the model size the same way, and the moment to stop training. Touch the third slice — test — exactly once, at the very end, to report the number.

Why the ceremony? Because every choice made by peeking at a dataset leaks information from it. Each 'try a setting, keep the better score' is a tiny act of fitting — not of the model's weights, but of your decisions — to that particular slice. Tune on your test set a hundred times and you have quietly trained on it; the final number becomes flattery, not measurement. The validation set exists to absorb that damage so the test set stays clean.

The failure has a public record. Kaggle competitions score entries on a public leaderboard — in effect a shared validation set — and a hidden private one. In contest after contest, teams that climbed the public board through hundreds of submissions drop sharply when the private scores unseal: they had tuned themselves to the slice they could see. The same slow staleness reaches research, where a benchmark reused by a whole field for a decade gets gently overfit by the community itself.

The honest limit is that validation spends data you would rather train on. With three hundred rows, one fixed slice is too noisy to trust, so you rotate: k-fold cross-validation splits the data into, say, five parts, validates on each part in turn while training on the other four, and averages the five scores. More honest, five times the compute. There is no free slice; there is only choosing what to pay.

028

The train and test split exists because of one treacherous failure: a model that aces its training data and folds on anything new.

Overfitting

Learning the noise in your examples instead of the pattern behind them.

The failure has a signature you can watch for. As training runs, the error on training data falls and keeps falling; the error on held-out data falls with it, then turns and climbs. A taxi-fare model that is off by ₹3 on its own receipts and by ₹40 on fresh ones has not learned the fare chart — it has learned those receipts. Overfitting is learning the noise in your examples instead of the pattern behind them.

The student who memorises past papers word-for-word aces every past paper and fails the real exam — that student is an overfit model. Memorisation is cheap for a model with many dials: given enough capacity it can store the answers rather than compress them into a rule, and nothing in the training loss objects, because the training loss only ever sees the past papers.

You saw the seed of it in lesson 1: the receipts carry noise, so a curve threading through every single point perfectly would be learning the drivers' detours, not the fare chart. Fitting the data better made the model worse. That inversion is the whole point of the held-out drawer — it is the only place where the difference between threading and understanding shows up as a number.

One honest complication: the tidy story — more flexibility, more overfitting — is not a law. The largest modern networks can fit their training data perfectly, noise included, and still generalise well, a behaviour called double descent that surprised the field itself. Treat overfitting as a real and constant danger, watch the held-out error, and distrust anyone who tells you the relationship between model size and failure is simple.

029

Overfitting is a model that cares too much about its examples — meet the equal and opposite failure.

Underfitting

A model too simple to capture what is actually going on.

The opposite failure is a model too simple to capture what is actually going on. A straight line forced through data that genuinely curves will be wrong everywhere, forever, no matter how long you train it — the error is built into the shape of the rule, not the settings of its dials. Delhi's fare structure genuinely has a flag-fall, a per-kilometre rate and a night surcharge; force fare = w·km through it and every midnight ride is mispriced by design.

The two failures announce themselves differently, and that difference is your diagnostic. Overfitting: training error low, held-out error high — the gap is the tell. Underfitting: both errors high and stuck together, because the model cannot even do well on the data it was shown. The remedies point opposite ways too — an underfit model wants more capacity, richer features or longer training, while an overfit one wants restraint.

The pair of failures — too rigid, too impressionable — bracket the whole craft. Most practical model-building is walking the ridge between them, with the validation score as your altimeter: add flexibility while the validation error keeps falling, stop the moment it turns. Nearly every technique still ahead on this road is a way of walking that ridge with more grace than bare trial and error.

The honest caution: both-errors-high does not always mean the model is too simple. Sometimes the features simply do not contain the answer — no model, however flexible, predicts a surge multiplier from distance alone, because the information is not in the input. Capacity cannot conjure information. Before reaching for a bigger model, ask whether a bigger model could even in principle know; often the fix is a better measurement, not a deeper network.

030

Overfitting named the failure; now name the success it steals — doing well on data you have never seen.

Generalisation

Doing well on things you have never seen. The only success that counts.

It has a measurement, not just a mood: the gap between the training score and the held-out score. A fare model off by ₹4 on its own receipts and ₹5 on fresh ones is generalising; off by ₹4 and ₹40, it is not. Generalisation is doing well on inputs you have never seen, and it is the only success that counts — a model perfect on its training data and lost on new data has learned nothing worth having.

It is also the honest miracle of the field. Nothing guarantees that patterns from yesterday's data extend to tomorrow's inputs; there is no theorem of the universe that says the world keeps its habits. That it so often works — when data is plentiful, models are regularised, and the new inputs resemble the old — is why any of this is an industry rather than a curiosity.

The fine print is that word 'resemble'. The standard assumption is that new data is drawn from the same distribution as the training data — same city, same era, same kind of customer. Generalisation, as machine learning actually delivers it, means doing well on new examples of the sort you sampled, not on a world that has since changed its habits.

Which names the limit: distribution shift. A monsoon-onset model trained on 1990–2020 rainfall meets a climate drifting away from its training years. Demand forecasts trained on 2019 behaviour collapsed within weeks in March 2020. No held-out score warned them, because the held-out data came from the old world too. A model generalises to the world you sampled — hold the applause until you know which world it is being asked about.

031

Underfitting and overfitting are not two separate accidents — they are the two ends of a single trade with proper names.

Bias and variance

Two ways to be wrong: too rigid, or too jumpy.

Bias is the error of a model too rigid to bend where the truth bends — a straight line forced through a curve misses the same way every time, on every fresh sample, predictably. Variance is the error of a model too impressionable — retrain it on a slightly different sample of receipts and it tells a wholly different story. One failure is stubbornness; the other is jumpiness. Underfitting is what high bias looks like from outside; overfitting is what high variance looks like.

For squared error the decomposition is exact: expected error = bias² + variance + noise. Bias² is how far your average model, taken over many resamples, sits from the truth; variance is how much individual models scatter around that average; noise is the part no model can remove — the drivers' detours are in the receipts forever. The panel beside this text runs the experiment live: fresh samples, refitted curves, the rigid fit missing identically each time while the flexible one thrashes.

The uncomfortable law is the trade. Making a model flexible enough to cut bias usually raises variance, and taming variance usually adds bias. You cannot zero both; you can only choose where to stand. Nearly every technique still ahead — regularisation, forests of voting trees, dropout — is one more way of buying a better position in this exchange, a better spot on the ridge between stubbornness and jumpiness.

Two honest footnotes. The decomposition is exact only for squared error; for other losses it is a guide, not an identity. And the textbook trade is not destiny: more data cuts variance without adding bias, which is the cleanest purchase in the whole field, and very large networks trained on very large datasets have repeatedly lowered both at once. The trade is real at fixed data; it is not a wall.

032

Learning rate set one step size for every dial — but a feature in kilometres and a feature in rupees demand different steps.

Scaling your inputs

Put features on comparable scales, or training crawls along a canyon floor.

Suppose one feature is trip distance in kilometres — single digits — and another is annual income in rupees — six or seven of them. The loss landscape those features build is a long thin canyon: the income direction is thousands of times steeper than the distance direction, so a step gentle enough for the steep walls crawls hopelessly along the floor, while any step bold enough for the floor ricochets off the walls.

The fix is one line of arithmetic per feature: x' = (x − μ) / σ, where μ is the feature's mean over the training data and σ its spread, the standard deviation. Subtracting μ centres the feature on zero; dividing by σ makes its typical variation one unit. Kilometres and rupees both come out as small numbers hovering around zero — the same data restated in comparable units, with no information added or lost.

That restatement reshapes the canyon into something much closer to a round bowl, and in a round bowl one learning rate serves every direction: gradient descent stops ricocheting and heads straight for the bottom. It costs one line of code, and forgetting it is one of the commonest reasons a perfectly good model refuses to train — the loss just sits there, drifting, while the optimiser inches along the canyon floor.

Two cautions keep it honest. Compute μ and σ on the training slice only, then apply them unchanged to validation and test — computing them on everything lets the test data whisper into training, which is exactly the crime the next stop, Data leakage, prosecutes. And scaling is a gradient-descent problem, not a universal law: decision trees split on thresholds and could not care less what units a feature arrives in.

033

The train and test split only protects you if the answer stays out of the inputs — and it leaks in more ways than you would guess.

Data leakage

When the answer sneaks into the inputs and your results become a lie.

The leaks are rarely as obvious as pasting the label into a column. A hospital model 'predicting' a disease from records that include the treatment already prescribed for it. A churn model fed a field that only gets filled in after the customer leaves. Feature scaling computed on all the data, test slice included, before splitting. Receipts from the same customer dealt into both piles. In every case the model looks brilliant, because it is quietly reading the answer sheet.

Time is the classic carrier. Shuffle time-stamped data and Tuesday's fraud helps explain Monday's; a model scored at 99.9% in that rehearsal walks into production, where the future has not happened yet, and folds. The mechanism is always the same: some input carries information that will not exist at prediction time, and the model — an optimiser with no context and no scruples — uses it, because using it lowers the loss.

It is the most expensive class of bug in applied machine learning, because nothing crashes. The score comes out wonderful, the demo dazzles, the system ships — and then the leaked column does not exist yet, and the model has nothing. The discipline is simple to state and easy to break: everything, preprocessing included, must be decided on training data alone, then merely applied to the rest.

Even discipline has limits, because leakage can hide in the data itself. Chest X-ray models trained to spot pneumonia turned out to be partly reading which hospital a scan came from — portable machines, used on sicker patients, leave visible traces — so the models passed their held-out tests and stumbled at the next hospital. The real defence is suspicion: when a score looks too good, it usually is, and the first suspect is a leak.

034

Labels let you score a model — but when 99 labels in 100 say the same thing, the score starts flattering.

Class imbalance

When 99% of examples are one class, 99% accuracy means nothing.

Suppose one transaction in a hundred is fraud. A model that shrugs and says 'not fraud' to everything scores accuracy = correct / total = 99/100 = 99% while catching zero fraudsters — the score is real and the model is useless. Class imbalance means the labels are lopsided enough that accuracy stops measuring anything you actually care about; the majority class sets a floor, and every score must be read against that floor, not against zero.

The honest scorecard splits by class. Of the actual frauds, how many did the model catch? Of the transactions it flagged, how many were really fraud? Those two numbers can be dreadful while overall accuracy glitters, because a hundred-to-one majority drowns the minority in any average. Score each class separately instead of averaging them away, and the shrugging model's true performance — none of the frauds caught — is exposed in one line.

The fixes attack from both sides. Reweight the loss so a missed fraud costs, say, fifty times a false alarm — the model is then paid to care about the rare class. Or resample, so fraud appears far more often in each training batch than in life. The rare class is usually the whole point — fraud, tumours, factory defects — which is exactly why this failure hides in plain sight.

The fixes carry their own bill. A model trained on rebalanced or reweighted data no longer believes fraud is rare, so the probabilities it reports come out inflated and must be recalibrated before anyone treats them as odds. And every extra fraudster caught is bought with false alarms — each one a genuine UPI payment frozen at the worst possible moment. Choosing that exchange rate is a business decision wearing a maths costume; no metric chooses it for you.

035

Generalisation improves with more varied examples — and there is a way to manufacture variety from the data you already own.

Data augmentation

Making more training data by changing what you have in ways that keep the answer true.

Flip a photo of a cat and it is still a cat. Rotate it a few degrees, crop it, dim the lighting — still a cat, and each version is a new training example that cost nothing. Data augmentation manufactures more data by changing what you have in ways that leave the label true, and it has been standard practice since 2012: the networks that cracked ImageNet trained on random crops and mirror-flips, multiplying 1.2 million photographs into effectively endless variants.

The mechanism matters: the transforms are drawn fresh on every pass, so the model never sees exactly the same pixels twice. Each epoch, every cat arrives newly cropped, newly flipped, newly brightened. What you are really doing is telling the model which differences do not matter — teaching invariances by example rather than by design. 'A mirrored cat is a cat' becomes something the network absorbs from thousands of mirrored cats, without anyone writing the rule down.

The craft is knowing which changes keep the answer true. Flip a cat, fine; flip a handwritten 6 and you have made a 9 with the wrong label, which is worse than no new data at all. Rotate a chest X-ray upside down and you are teaching anatomy that does not exist. Every augmentation is a claim about the world — this transformation preserves the truth — and a wrong claim is trained in as thoroughly as a right one.

The limit to remember: augmentation manufactures variety, not information. No amount of flipping conjures a dog breed you never photographed, a monsoon pattern outside your years of records, an accent missing from your audio. It stretches the neighbourhood of what you already have; it cannot annex what you do not. When a dataset is missing a slice of the world, the honest fix is still the expensive one — go out and collect it.

036

You hold the whole training loop — loss, gradient, descent — so point it at the smallest model that genuinely learns: a straight line.

Linear regression

Fit a straight line. The smallest model that genuinely learns.

The line is two dials and nothing else: fare = w·km + b — multiply each feature by a weight w, add a starting value b, out comes a number. The taxi meter is exactly this machine, roughly 18 per kilometre times distance plus a base fare of 40, and training it is the downhill walk you already hold: turn the two dials until predicted fares match the receipts.

Run one step by hand. The dials start at w = 15, b = 30. A receipt says a 10 km ride cost ₹238; the line guesses 15×10 + 30 = ₹180, so the error is −58 and the loss is (guess − truth)² = 3,364. The gradient tells each dial its share of the blame: ∂L/∂w = 2·(guess − truth)·km = −1,160, and ∂L/∂b = 2·(guess − truth) = −116. Update with w ← w − η·∂L/∂w, where η is a small learning rate, and both dials rise toward the true tariff. The panel beside this text is running exactly this loop.

It earns its place twice. Practically, it is still everywhere — pricing, forecasting, drug dosing — because when the truth is roughly linear nothing beats it, and everything reads plainly: each weight says what one unit of that feature costs, in rupees or millimetres of mercury. Pedagogically, it is the smallest model that genuinely learns, and every network ahead on this road is this same multiply-and-add with a bend attached.

The honest limit is the shape itself. Real fares bend — night surcharges, waiting time, surge pricing — and a straight line fitted to a curved truth is confidently wrong everywhere at once, not just at the edges. Worse, its legibility lends it false authority: a clean per-kilometre figure looks like an explanation even when the leftover errors are shouting that the world is not a line. Check them before you trust it.

And this one you can do, not just read — Lesson 1 on the panel walks you through training it yourself.

037

A straight line predicts any number at all — but 'spam or not' wants an answer between 0 and 1, a probability.

Logistic regression

Bend a line into a probability, and you can classify.

So bend the line's output through an S-curve. Keep the score s = w·x + b exactly as before, then squash it: p = 1 / (1 + e^(−s)). A large positive score slides toward 1, a large negative one toward 0, and s = 0 lands at exactly 0.5 — undecided. The line has not changed; it has merely started speaking probability. Above the boundary: probably spam. Far above: almost certainly.

A spam filter makes it concrete. Features: how many exclamation marks, whether the sender is in your contacts, whether the word lottery appears. Suppose the weights give a mail a score of s = 2.2; then p = 1/(1 + e^(−2.2)) ≈ 0.90 — nine in ten. Training turns the same dials downhill, but on a loss built for probabilities: cross-entropy, loss = −log p assigned to the true answer, which punishes a confident wrong call brutally and a hedged one gently. The panel beside this text lets you watch the probabilities re-shade as the boundary moves.

It remains the workhorse of applied machine learning: fast, stable, and legible, since each weight says plainly which features push toward yes and by how much. When a bank must explain a declined loan — as regulators increasingly demand — this is very often the model behind the explanation, precisely because its reasons can be read straight off the dials.

Two honest limits. The boundary it draws through feature space is still straight, so patterns where the classes interleave — spammers who mimic your contacts — are beyond it without hand-built features. And the 0.90 is only as honest as the training data: a filter trained on last year's spam will announce nine-in-ten confidence about tricks it has never seen. A probability is an output, not a promise.

038

Features already place every example as a point in space — and once things have positions, nearness alone can predict.

Nearest neighbours

Predict whatever the most similar examples did. No training at all.

Store everything; compute nothing until asked. To predict for a new example, measure its distance to every stored one — d = √((a₁−b₁)² + (a₂−b₂)² + …), the straight-line distance across all the feature axes — take the k closest, and answer whatever most of them did, or their average if the answer is a number. That is the whole algorithm. There is no training at all.

Price a flat in Pune with it. Features: 900 square feet, 2 km from the station, second floor. The five nearest sold flats in your records went for 52, 55, 58, 60 and 63 lakh; average them and the answer is about ₹57.6 lakh, delivered without a single dial being turned. A new ride is guessed from the rides it most resembles; a patient is triaged like the patients whose vitals sit closest.

The price of skipping training is paid at answer time. Every prediction searches the whole memory, so it slows as the data grows — with ten million records, each query does ten million distance sums — the opposite economics of everything else on this road, where training is slow and answering is instant. Clever spatial indexes soften the search, but only up to a point.

And similar quietly means close in your features. Leave area in square feet beside distance in kilometres and the big numbers drown the small: every neighbour ends up chosen by floor area alone. Irrelevant features poison it the same way, adding noise to every distance. Scale the axes and prune the features, though, and it is the fastest honest baseline in the field: a fancy model that cannot beat looking things up is not earning its keep.

039

Every model so far turned dials downhill — this one just learns which yes/no questions sort the labels cleanest.

Decision tree

A flowchart of yes/no questions, learned from the data.

At each fork the tree auditions every question it could ask — every feature, every threshold — and keeps the one that splits the labels most cleanly: is the ride longer than 12 km? Is the hour past midnight? Then it asks the same of each half, and again of each quarter, growing the flowchart until the leaves are nearly pure — almost all one label.

Cleanest has a number. A common one is Gini impurity, G = 1 − p² − q², where p and q are the fractions of the two labels in a group — 0 for a pure leaf, 0.5 for a 50/50 muddle. Take 100 rides of which 40 ended in fare disputes: G = 0.48. Split on longer than 12 km, and suppose the long side holds 30 disputes among 40 rides, the short side 10 among 60: the weighted impurity falls to about 0.32. The tree tries every candidate split, computes exactly this, and keeps the biggest drop. The panel beside this text grows one fork at a time so you can watch it choose.

No dials, no gradients, no downhill walk — a genuinely different way to learn. And the finished model reads as a list of rules: if distance is over 12 km and the hour is past midnight, expect a dispute. Auditors and doctors rightly love this; a triage tree fits on one printed page and can be argued with line by line.

The catch is appetite. A tree grown deep enough will happily carve a private leaf for every quirk in the data — one leaf per memorised receipt — making it one of the easiest models in the field to overfit. It is also twitchy: change a handful of examples and the top question can flip, rebuilding the entire flowchart below it. The standard cures, capping depth or demanding a minimum of examples per leaf, blunt the tree — and set up the better idea that comes next.

040

A decision tree grown deep memorises its data — the cure is not pruning one tree but growing hundreds and letting them vote.

Random forest

Hundreds of shallow trees voting beat one deep tree.

One deep tree memorises its data — pure variance, in the language you now have. So grow hundreds of trees and make each one different on purpose. Each tree trains on a bootstrap sample — n examples drawn from your n with replacement, so about 63% appear and the rest sit out — and at every fork it may consider only a random handful of the features, commonly √p of them, four out of sixteen. Then all the trees vote.

Each tree is individually jumpy, but their errors, grown from different data and different questions, point in different directions and largely cancel. That is the arithmetic of averaging: the scatter of a mean of m independent judges shrinks roughly as 1/m. Predicting whether a waitlisted train ticket will clear, a forest of 500 trees might have 410 vote yes — and that 0.82 vote share doubles as a usable confidence. The panel beside this text shows a single tree's ragged boundary melting into the forest's smooth one.

This is the wisdom of crowds made mechanical: averaging many diverse, imperfect judges beats one flexible expert, provided their mistakes disagree — which is precisely what the deliberate randomness buys. Forests need little tuning, shrug off overfitting, and for two decades have been the sensible first serious model on almost any business dataset. It rarely tops a leaderboard; it almost never embarrasses you.

Two things are traded away. The single tree's legibility is gone — five hundred flowcharts voting explain nothing plainly, and you are back to reading feature-importance scores instead of rules. And a forest can only average what its trees have seen: ask it about a 60 km fare when the data stops at 30 km and it returns a 30 km answer, flatly refusing to extrapolate. Sometimes that caution is a virtue. It is still a limit.

041

Trees again — but where the forest grew them blind to each other, gradient thinking says each new tree should study the last ones' mistakes.

Gradient boosting

Each new tree fixes the mistakes of the ones before. Still wins on tabular data.

Boosting grows trees in sequence, not in parallel. Train a small tree, often just a few forks deep. Look at where it is still wrong. Train the next tree not on the labels but on those leftover mistakes, then add a fraction of its correction to the running total: F ← F + η·tree, where η is a small learning rate, perhaps 0.1. Repeat for hundreds of rounds. Each tree is weak on its own; the model is the sum of small corrections, each aimed at what remained.

The name is earned literally. For squared-error loss the leftover mistake is just the residual, truth − guess; in general each new tree is fitted to the negative gradient of the loss with respect to the current predictions — the direction that lowers the loss fastest. The whole sequence is gradient descent performed with trees instead of dial-turns. Concretely: the running model guesses ₹220 for a fare that was ₹260, the residual is 40, the next tree learns to add about 40 for rides like it, and with η = 0.1 the model banks ₹4 of that correction, leaving the rest for later trees.

Its modern forms — XGBoost, LightGBM — still win most tabular-data competitions, a standing reminder that on spreadsheet-shaped problems, from credit scoring to crop-yield tables, a neural network is not the default answer. A tuned boosted model on a few hundred thousand rows routinely matches or beats one, trains in minutes on a laptop, and costs almost nothing to serve.

The honesty: because every tree studies the previous errors, boosting will eventually study the noise — let it run unchecked and it fits the very quirks a forest's independent trees would have averaged away. It needs a validation set watching for the moment held-out error turns upward, and it is far more sensitive to its dials — tree depth, η, number of rounds — than the forest that needed almost none.

042

Logistic regression draws a boundary that separates the classes — but of all the boundaries that separate them, which deserves your trust?

Support vector machines

Find the boundary with the widest possible gap around it.

Many boundaries can separate two classes; a support vector machine demands the one with the widest empty corridor around it — the margin between the classes at its fattest. Only the points pressing against that corridor matter: move any other example and the boundary does not budge. Those few pressing points are the support vectors, and the finished model is stored as just them.

The geometry compresses into one clean demand. The boundary is w·x + b = 0, and the corridor's width works out to 2/‖w‖ — the shorter the weight vector, the wider the margin. So training says: minimise ‖w‖ subject to y·(w·x + b) ≥ 1 for every example, where y is +1 or −1 for the two classes — each point on its correct side, clear of the corridor. The fatness of a margin has become a number you can optimise.

The wide margin is a bet on generalisation: new examples land near old ones, and a boundary with room to spare misclassifies fewer of them. A trick with dot products — the kernel — swaps x·z for a function like e^(−‖x−z‖²), which behaves as if the data had been lifted into a vastly higher-dimensional space, letting the same machine draw curved boundaries without ever building that space. Through the 1990s and 2000s this made SVMs the field's reigning champion — the benchmark on tasks like handwritten digits that neural networks had to beat to come back.

The limit is scale. Kernel methods compare examples pairwise, so a million training examples implies a trillion-entry table of similarities looming behind the maths; past a few hundred thousand examples, training turns painful in a way a network's does not. That, more than accuracy, is why deep learning displaced it — and why SVMs still thrive on small, clean, expensive datasets such as medical assays.

043

Unsupervised learning promised structure without labels — here is the workhorse that actually finds the groups.

Clustering

Grouping things nobody labelled, by how close together they sit.

The workhorse is k-means, and it is almost embarrassingly simple. Guess k centre points. Assign every example to its nearest centre. Move each centre to the mean of its members. Repeat until nothing moves. Every loop can only shrink the total squared distance from examples to their centres — Σ‖x − centre‖² — so the shuffle always settles, usually within a few dozen rounds.

Run it on a food-delivery app's customers, two features each: average order hour and monthly spend. With k = 3, the centres drift for a dozen iterations and stop on three crowds — late-night bargain orders, weekday-lunch office regulars, weekend family splurges. Nobody labelled anyone; the groups fell out of nearness alone, and now they can be named, studied, and messaged differently. Customers became night owls and bargain hunters without a single label in the data.

As unsupervised learning warned, there is no answer key. You must choose k, and the data will obligingly split into five clusters whether or not five kinds of thing exist. k-means also has tastes of its own: it favours round, similarly sized blobs, and a different random start can settle on a different answer — so practitioners run it several times and keep the tightest result.

Two groupings can both look plausible, and the only judge is whether the groups mean something to a human — do the weekend splurgers actually behave differently when you treat them differently? Used with that humility, clustering is often your first real look inside a new dataset: cheap, fast, and honest about being a suggestion rather than a verdict.

044

Vectors gave every example hundreds of axes — unsupervised learning's other trick is noticing how few of them are really in use.

Dimensionality reduction

Squash many features into a few that keep most of the differences.

A dataset with two hundred features per example lives in a two-hundred-axis space, but the examples rarely use the room — many features rise and fall together, saying nearly the same thing. Dimensionality reduction finds the few directions along which the data actually varies and re-describes every example by its position along just those, discarding axes that carry almost nothing.

The classic method, PCA, chooses those directions by pure variance. The first component is the single direction of maximum spread; the second captures the most of what remains, at right angles to the first; and so on down the list. Each example's new coordinates are dot products — z = v·x, the example projected onto each chosen direction v — so the whole transformation is the multiply-and-add you already own. Often a dozen directions hold most of what two hundred features were saying.

Take a season of T20 batting data: forty-odd columns per batsman — strike rate by phase, dot-ball percentage, boundary rates against pace and spin. Project onto the first two components and the players spread across one readable plot, anchors drifting to a side and finishers to another — structure no forty-column spreadsheet would ever show you. The gains are compression, speed, and the ability to plot the unplottable.

The price comes in two parts. The new axes are blends of the originals, with no names of their own — component one is a stew of strike rate and dot balls, and explaining it takes work. And variance is not importance: PCA keeps the directions where the data spreads widest, which are not always the directions that matter for your prediction, so a small but crucial signal can sit in the discarded axes. It is also strictly linear; curved structure needs other tools.

045

Watching for overfitting is one defence — the stronger one is charging the model a price for complexity up front.

Regularisation

Penalise complicated answers so the model stops chasing noise.

Regularisation changes the question training asks. Instead of what dials make the loss smallest, it asks what dials make the loss small while staying small themselves. Mechanically, a penalty joins the loss: total = loss + λ·Σw², where Σw² is the sum of the squared weights and λ is the price you set on complexity. Every extreme dial setting must now pay for itself in genuinely better predictions.

The effect is a taste for boring explanations. A curve threading through every noisy receipt needs wild coefficients; the penalty makes wildness expensive, so the model keeps only the structure the data solidly supports. In the fare model, a weight of 90 on day-of-week that exists only to absorb one strange Tuesday now costs 90² = 8,100 in penalty — unable to earn that back in loss, it shrinks toward zero, while the honest per-kilometre weight, which cuts the loss on every receipt, keeps its size.

The penalty's shape matters. Squaring the weights — called L2, or ridge — shrinks everything smoothly. Summing absolute values instead, λ·Σ|w| — L1, or lasso — pushes weak weights to exactly zero, so the model performs its own feature selection and hands you a shorter list. Deep learning's weight decay is this same L2 idea under another name, applied to millions of dials at once.

It is the field's standing answer to overfitting, but λ is one more dial — yours rather than the model's, tuned on the validation set. Set it too high and the penalty smothers real structure as readily as noise: the model underfits, blandly wrong everywhere. And the tax falls unevenly unless features share a scale — a distance in metres needs a weight a thousand times larger than one in kilometres, and the penalty punishes it for that accident. Scale first; then charge for complexity.

046

Linear regression multiplied, added, and stopped; the artificial neuron does exactly that, then bends the result — and everything changes.

Artificial neuron

Multiply the inputs by weights, add them up, bend the result.

In symbols the whole unit is out = f(w1·x1 + w2·x2 + … + b). Each input x is multiplied by its own weight w, everything is summed along with a bias b, and the total is passed through a bending function f — commonly: keep it if positive, zero it otherwise. That is the entire artificial neuron: the taxi-fare formula with a kink. The multiply-add half is exactly what you trained by hand at the start of this road; the bend is the only new ingredient.

Give it real numbers. A neuron judging whether a ride looks suspicious might read distance 12 km and hours-past-midnight 2, weight them at 0.3 and 1.5, and carry a bias of −5. The sum is 0.3·12 + 1.5·2 − 5 = 1.6 — positive, so it passes 1.6 forward. A daytime 4 km hop scores 0.3·4 + 0 − 5 = −3.8, and the bend flattens that to zero: the neuron stays silent. The bias is doing the thresholding — evidence must beat five points before this unit speaks at all.

The bend is what makes owning many of these worthwhile. Chains of pure multiply-adds collapse into a single multiply-add, so straight-line units stacked a hundred deep can draw nothing one of them could not. Bent units are different: stacks of them can draw shapes no straight line can, which is the discovery the next several stops unpack piece by piece.

A caution about the name. The design was inspired by brain cells, but a real neuron is a living cell with thousands of synapses, chemical signalling, and timing behaviour that no weighted sum captures — the artificial version borrows the poetry, not the biology. And one unit alone is feeble: a single crease in an otherwise flat opinion. Its power is entirely a property of crowds.

047

Inside every neuron sit two kinds of dial, and before you stack thousands of neurons they are worth telling apart.

Weights and biases

The weights say what matters; the bias says where to start.

You have met both dials already, unnamed. In the taxi fare, the 18 rupees per kilometre is a weight — it says how much distance matters. The 40-rupee base fare is a bias — charged before the wheels turn at all. A neuron's weights tilt its opinion toward the inputs that matter; its bias sets where the opinion starts before any evidence arrives. Every neuron on this road, and every neuron in the largest model ever trained, carries exactly these two kinds of number and no others.

Why give them separate names? Because they do different work. Look at the sum w1·x1 + w2·x2 + b: set every input x to zero and the weights vanish from the answer entirely, but b remains. Weights compare — no weight can move the answer without an input to multiply. A bias moves it unconditionally, which lets a neuron set its own threshold for switching on: a large negative bias means the unit demands strong evidence before it fires at all.

The panel beside this text is running exactly this. Drag a weight and the amber line tilts — steeper means that input matters more, and the pivot stays put. Drag the bias and the whole line slides up or down without changing its slant. Two dials, two visibly different jobs: one changes what the neuron listens to, the other changes how much persuading it needs.

When someone says a model has seven billion parameters, this is what they are counting: weights and biases, nothing else. One honest deflation, though. The split is bookkeeping, not deep mathematics — a bias is just a weight attached to an imaginary input permanently fixed at 1, and libraries often store it that way. Useful names for two habits of the same kind of dial, not two different substances.

048

The neuron's bend looked like a footnote; it is actually the entire reason stacking layers buys anything.

Activation function

The bend. Without it, a hundred layers collapse into one straight line.

Here is the collapse, done in full. Two straight layers in a row compute w2·(w1·x + b1) + b2, and multiplying out the brackets gives (w2·w1)·x + (w2·b1 + b2) — one combined weight, one combined bias. A single layer could have produced exactly that. Repeat the argument and a hundred linear layers equal one: stacking is pointless in the strict mathematical sense, the whole tower folding flat like a paper fan.

Put a bend between the layers and the algebra can no longer fold. Each layer now carves its own crease into the input space before handing the result on, and the next layer creases the already-creased picture. Depth starts to buy expressiveness: boundaries stop being single straight cuts and become folded, cornered shapes. The bend looked like a footnote on the neuron; it is the entire reason a stack is worth more than its widest layer.

The field has tried many bends. The sigmoid, 1/(1 + e^−x), squashes any input into the range zero to one. Tanh centres the same squash around zero. ReLU, max(0, x), keeps positives and zeroes negatives. GELU, a smoothed relative of ReLU, sits inside today's transformers. Any of them stops the collapse; which one you choose decides how easily the error signal flows backwards, and that choice is the next stop's whole story.

One deflation before moving on. The bend makes rich shapes possible, never inevitable — a bent network still learns nothing if training goes badly. And a badly chosen bend actively hurts: the sigmoid's flat tails throttle the error signal so severely that early deep networks barely trained at all, which is a large part of why the winning bend, ReLU, is almost embarrassingly simple — and why its simplicity is precisely what won.

049

Of all the bends an activation function could take, the one that won the field is the crudest on offer.

ReLU

Keep positives, zero the negatives. Absurdly simple, and it won.

ReLU is one line of arithmetic: relu(x) = max(0, x) — if the number is positive, keep it; if negative, output zero. That is the whole bend: a hinge, flat on one side, slope exactly one on the other. It is barely a curve at all, and it beat every elegant S-shaped activation the field had spent decades preferring.

It won on gradients. Take the slope: one wherever the input is positive, zero wherever it is negative. The smooth older bends flatten at both ends, and a flat region means a near-zero gradient — blame arriving from later layers gets multiplied by almost nothing and dies on its way through. ReLU's slope on the positive side is exactly one, so blame passes through undiminished wherever the unit is awake. Cheapness helps too: max is a single comparison, while a sigmoid must compute an exponential every time.

The proof arrived in 2012 with AlexNet, the network that ignited deep learning. Its authors reported that with ReLU it reached a training-error benchmark roughly six times faster than the same network using tanh — and a six-fold speedup changes what a research group can afford to attempt. ReLU became the default bend for a decade of vision models largely on that showing.

The catch: a neuron pushed permanently negative outputs zero forever, gradient and all — a dead dial no further training revives, and a carelessly large learning rate can kill a sizeable fraction of a layer this way. Patches exist; leaky ReLU passes a small slope through on the negative side so a buried unit can still crawl back. And the crown has moved: modern transformers mostly use GELU, a smoothed hinge. ReLU won the decade, not the argument.

050

A network's final layer hands you raw scores, but a distribution demands numbers that sum to one — something must convert.

Softmax

Turn any list of scores into probabilities that sum to one.

The conversion is two moves: p = e^s / Σ e^s — raise e to each raw score s, then divide each result by the sum of them all. The exponential forces every number positive, whatever the score's sign; the division forces the total to land at exactly one. Every entry becomes a share, the biggest score takes the largest share, and the order of the scores is preserved. What went in as an arbitrary list comes out as a distribution.

Work it through. A classifier's last layer says 3.1 for cat, 1.2 for dog, −0.5 for car. Raise e to each: about 22.2, 3.3, and 0.6, summing to roughly 26.1. Divide, and the shares come out near 0.85, 0.13, and 0.02. Notice what the exponential did on the way: a 1.9-point lead in raw score became an 85-to-13 lead in probability. Softmax exaggerates, and that editorial push is usually what you want from a decision-maker. Lesson 2 has you build this conversion yourself.

Every word a chatbot has ever written passed through a softmax first: the model's raw scores over its whole vocabulary — tens of thousands of them — converted into the spread of belief it samples its next word from. Dividing the scores by a temperature before the softmax is the sharpening dial: low temperature exaggerates the leader further, high temperature flattens the field and lets long shots through.

The honest limit: the output looks like confidence and often is not. A network can put 99 percent on the wrong answer, especially on inputs unlike its training data — and the sum-to-one format forces it to spread full belief regardless. Show the cat-dog-car classifier a photo of an auto-rickshaw and it must still hand out shares totalling one. Softmax guarantees the shape of a belief, never its wisdom.

And this one you can do, not just read — Lesson 2 on the panel walks you through training it yourself.

051

One neuron forms one opinion; give a whole row of them the same evidence and you get a panel.

Layer

A row of neurons all looking at the same inputs.

A layer is a row of neurons all reading the same inputs, each with its own weights — so each forms its own opinion about the same evidence. One might fire on long rides, another on late-night ones, a third on airport pickups. Nobody assigns those roles; each neuron simply owns a different set of dials, so the same facts strike each one differently.

Stack every neuron's weights as the rows of one grid W and the whole layer becomes a single line: y = f(W·x + b). One matrix multiply computes every opinion at once, which is why graphics cards — machines built to multiply matrices — became the furnaces of deep learning. The counting is easy too: a layer of 128 neurons reading a 784-pixel digit image carries 128×784 = 100,352 weights plus 128 biases, every one of them trainable.

The layer's output is that panel of opinions, handed to the next layer as its evidence. This is the move that builds meaning: panels judging panels. In a trained image network the early layers tend to respond to edges and simple textures, and later layers to whole objects — which is why the standard story says early layers find edges and late layers find faces.

That story is tidier than the truth, and honesty demands the footnote. Individual neurons are often polysemantic — one unit firing for cat ears, car grilles, and a particular shade of green — and a concept usually lives smeared across many units rather than in any single one. The panel's verdict is real; clean one-neuron-one-meaning labels mostly are not, and reading them out remains an open research problem.

052

Panels judging panels: stack layers, keep the bends, and you have the original deep network.

Multilayer network

Stack bent layers and you can draw any shape at all.

Stack bent layers and you get a multilayer network — the plain, original deep model. Feed evidence in at one end; each layer re-describes it; read the answer off the other end. Written out for three layers it is out = f(W3·f(W2·f(W1·x + b1) + b2) + b3): matrix multiply, bend, repeat, with every W and b a trainable dial and the bends keeping the tower from folding flat.

A real and famous specimen: a digit reader for postal codes. Take the 784 pixels of a scanned digit as input, pass them through a hidden layer of 128 bent neurons, then an output layer of 10 scores — one per digit — finished with a softmax. That is roughly 101,000 parameters, small enough to train on a laptop in minutes, and it reads better than 98 percent of the standard MNIST test digits.

Its power has a blunt statement: with enough bent pieces you can approximate any function whatsoever — the next stops make that claim precise. But the catch was never expressiveness. The catch was training the stack: knowing which of a hundred thousand dials to blame for a wrong answer. That is why backpropagation, not depth itself, was the breakthrough.

The honest limit of the plain stack: it treats its input as an unstructured list of numbers. Shift that digit two pixels sideways and every weight meets unfamiliar evidence — the network has no built-in idea that nearby pixels are related or that a shape is the same shape anywhere on the page. Architectures that bake such structure in, convolutions and transformers among them, exist precisely because the all-purpose stack wastes its dials relearning the obvious.

053

Stack layers and you can draw any shape — but first, the tiny puzzle that stalled the field for a decade.

The XOR wall

The problem one layer cannot solve — and the reason the field stalled for a decade.

XOR is a rule a child follows daily: the staircase light with a switch at each end. Flip either switch and the light changes, so the light is on when exactly one switch is up. Plot the four cases — off-off and on-on give darkness, off-on and on-off give light — and try to separate lit from unlit with one straight line. You cannot: the two lit cases sit diagonal to each other, and a single line has no way to cut them free. The panel beside this text lets you try; every straight line you drag will strand at least one corner.

A single layer can only draw straight boundaries, so this toy defeated it — and in 1969 Minsky and Papert made the limit famous, helping freeze neural-network research for roughly a decade. The perceptron had been launched a decade earlier on extravagant promises of machines that would see, talk, and learn, so the credibility crash was proportional to the hype. Four data points, one page of geometry, and a field's funding went with them.

The bitter joke is how small the fix is. Give the network one hidden layer of two neurons: the first computes step(x1 + x2 − 0.5), firing when at least one switch is up; the second computes step(x1 + x2 − 1.5), firing only when both are. The output neuron takes the first opinion minus the second — on when at least one but not both. Two creases where one line failed. The wall was never about networks, only about networks one layer deep.

One correction to the folklore, because this course does not lie. The freeze had more causes than a single book: computers of the era were feeble, and nobody yet had a practical way to train hidden layers even had they wanted to. Minsky and Papert were right about single layers; the decade was lost as much for want of backpropagation as for want of morale.

054

The multilayer network just claimed it can draw any shape at all — a boast that big deserves checking.

Why depth works

Enough bent pieces can approximate any function you like.

Here is why the boast holds. Each ReLU neuron contributes one crease — flat, then sloped, hinged where it switches on. Add enough creased pieces, each with its own position and slope, and you can trace any curve the way short straight segments trace a circle: crudely with a few, as closely as you like with more.

The recipe is a sum of hinges: f(x) ≈ a1·relu(x − c1) + a2·relu(x − c2) + …, where each c says where a hinge switches on and each a says how sharply the slope changes there. The panel beside this text is running exactly this — hinges added one at a time, the amber sum bending closer to the paper-white target curve with every crease. Three hinges give a caricature; thirty give a fit you must squint to fault.

The universal approximation theorem, proved around 1989, made this exact: one hidden layer, wide enough, can approximate any reasonable function. Read the fine print, though. It says such dial settings exist, not that gradient descent will find them, and it says nothing about how many examples you would need to pin them down. Existence is not a recipe; training still has to do the finding, and the theorem offers training no help at all.

And 'wide enough' can mean absurdly wide: matching a genuinely wiggly function with a single layer can demand astronomically many hinges. Depth earns its keep here — a deep network folds space and then folds the folds, reusing each crease many times over, so it builds the same shapes with far fewer dials. The theorem justifies the enterprise; it flatters shallow networks far more than practice ever has.

055

The chain rule traces a nudge through any chain of steps, and a network is exactly such a chain.

Backpropagation

Send the blame backwards through the network with the chain rule.

The forward pass makes a guess. Comparing with truth gives an error. Backpropagation carries that error backwards, layer by layer, using the chain rule to compute how much each individual weight contributed to the miss. For one weight deep in the stack the arithmetic is a product of local slopes — ∂L/∂w = ∂L/∂h3 · ∂h3/∂h2 · ∂h2/∂h1 · ∂h1/∂w — each factor asking one layer how strongly it passed the nudge along.

The reason this matters is cost. The naive alternative is to nudge each dial, re-run the network, and watch the loss: for a seven-billion-parameter model that is seven billion forward passes to earn a single training step. Backprop gets every gradient at once — one forward pass, one backward pass, and the backward pass costs about the same as the forward. All seven billion answers for roughly twice the price of one guess.

It is worth deflating properly: backprop is not the learning. It is fast bookkeeping for gradients. The learning is still the same downhill walk you did on the taxi fare — backprop just makes the 'which way is downhill' question affordable for billions of dials at once. Nothing new is decided in the backward pass; it only reports.

The honest limit lives in that product of slopes. Chain thirty factors together and if most sit below one, their product shrinks toward nothing — the vanishing gradient that starved early deep networks, and a large part of why ReLU's slope of exactly one mattered. And the brain almost certainly does not do backpropagation; no one has found neurons running their errors backwards. The algorithm is an engineering triumph wearing a borrowed biological name.

056

A layer can only eat vectors — so words, products, and people must first be turned into vectors worth eating.

Embedding

Turning a word, a user, or an image into a vector of learned numbers.

An embedding turns a thing — a word, a product, a user — into a learned vector, placed so that similar things land near each other. 'King' and 'queen' end up close; 'king' and 'teaspoon' end up far. The machinery is humbler than the effect: a lookup table. One row of numbers per item, and 'embed token 4,072' means nothing more than 'fetch row 4,072'. Those rows are ordinary weights, nudged by backpropagation like every other dial in the network.

The tables are large. GPT-2's vocabulary ran to 50,257 tokens, each embedded in 768 numbers — about 38.6 million dials spent purely on the dictionary, before a single layer of judgement. Nobody designs the coordinates; they emerge from training, because rows that help predictions drift toward useful positions and rows that do not get pushed elsewhere. Recommenders play the same trick with no words at all: films and viewers embedded in one shared space, where nearness is a prediction that you will press play.

And the space carries structure nobody explicitly asked for — directions that behave like meaning, famously king − man + woman landing near queen. The arithmetic is literal: subtract the hundreds of numbers for man from king, add woman, and hunt for the nearest row. Whole families of relationships — gender, tense, capital cities — show up as roughly parallel arrows across the space. The map is learned, and the map is the asset.

Two honest footnotes. The arithmetic party trick is oversold: the famous examples are curated, the standard test quietly forbids returning the word you started from, and off the demo reel the analogies fail often. Worse, the map inherits its data's prejudices — occupation vectors drift gendered, so nurse sits nearer woman than surgeon does. The asset and the liability are the same object: whatever the text contained, the geometry now contains.

057

Gradient descent said 'over and over' — with a million examples, you need words for one step and one full lap.

Epoch and batch

One pass through the data, taken in handfuls.

Two words of training vocabulary, and they are really just division. A batch is the handful of examples you average a gradient over before turning the dials once. An epoch is one full pass through the entire training set, taken batch by batch. Steps per epoch = N / B, dataset size over batch size: a million examples in batches of a hundred is ten thousand dial-turns to the epoch. Thirty epochs, a routine run, is three hundred thousand turns. The vocabulary exists because 'over and over' at this scale needs bookkeeping.

Why handfuls at all? A million examples will not fit through the hardware at once, and one example at a time wastes chips built to do thousands of multiplications together. A GPU is happiest chewing a few hundred examples in parallel, so the batch is sized to fill it. Averaging helps too: one taxi ride's gradient is one noisy opinion, but a hundred rides averaged point somewhere trustworthy. Batch size is a real dial with real consequences, and the epoch becomes the clock of training: runs are measured in epochs the way voyages are measured in days.

Make it concrete with the fare model. A million Mumbai taxi receipts, each a distance and a fare. Batch one: rides 1 to 100, average their gradients, turn the dials once. Batch two: the next hundred. Ten thousand batches later every receipt has been consulted exactly once — epoch one is done — and the procedure starts again from a reshuffled deck. The reshuffle matters: identical batches in an identical order every epoch would let the model learn the order itself, which is worth nothing.

The unit misleads if you trust it alone. 'Trained for fifty epochs' says nothing without the batch size, and the largest language models break the vocabulary entirely: their datasets run to trillions of words, so training often ends before a single full pass — epoch zero point seven. Nor are bigger batches simply better: they smooth the gradient but spend more computation per turn, and very large batches tend to settle at dials that generalise slightly worse. The handful is a compromise you tune, not a nuisance you eliminate.

058

Taking the data in small handfuls sounded like a compromise; the noise it adds turns out to be a gift.

Stochastic gradient descent

Use a small random handful each step. Noisier, and far faster.

The honest gradient asks every training example for its opinion before each single step — a full census, a million receipts consulted, one dial-turn. Stochastic gradient descent polls instead: grab a small random batch, compute its gradient, step immediately. In symbols the update is unchanged, w ← w − η·g, but g is now averaged over a random hundred examples rather than all million. The direction is noisy but roughly right — a poll's average error shrinks with the square root of its sample size — and you take ten thousand steps in the time the census takes one.

The trade is spectacularly lopsided. A batch of a hundred costs one ten-thousandth of the census, and its direction typically points within a few degrees of the true downhill. The fare model shows why: any random hundred Mumbai receipts contain the same basic truth — longer rides cost more — so their averaged gradient points broadly where all million would. You do not need to interview the whole city to learn which way the street slopes.

The surprise, discovered in practice, is that the noise is not just tolerable but useful. A jittering walker does not settle into every narrow crevice of the loss landscape; it rattles out of sharp, fussy minima and comes to rest in broad valleys — and dial settings in broad valleys generalise better, because a test example that lands slightly off-target still scores well. The compromise turned out to be a feature.

Honesty about the gift: the jitter never stops. Near the bottom, where the true gradient is almost zero, the noise dominates, and the walker mills about the minimum instead of settling on it — which is precisely why the learning-rate schedules ahead shrink the steps late in a run. And the tidy story that noise reliably finds flat, generalising valleys is a working picture, not a theorem; researchers can construct cases where it fails. SGD is the field's workhorse because it is cheap and it works, not because it is fully understood.

059

Stochastic steps jitter, stall on plateaus, zig-zag in canyons — a ball that keeps some speed does none of these.

Momentum

Let the ball keep some speed so it rolls through flat patches.

Momentum keeps a running velocity. Each step, blend the fresh gradient into the direction you were already moving — mostly old direction, a little new signal — and step along the blend: v ← β·v + g, then w ← w − η·v. The dial β is the ball's heaviness, typically 0.9, meaning the velocity is nine parts memory to one part news. The walker becomes a heavy ball: gradients that keep pointing the same way build up speed, and gradients that keep flipping sign cancel out.

The arithmetic of the build-up deserves one look. If the gradient along some direction holds steady at 0.2, the velocity climbs towards 0.2/(1 − 0.9) = 2 — ten times the single-step push. If the gradient flips between +3 and −3 across a canyon, the blend averages towards zero. Same formula, opposite fates: persistent signals are amplified tenfold, oscillating ones are silenced.

That fixes two chronic ailments at once. In a narrow canyon — a loss landscape shaped like a riverbed, steep across, gentle along — plain SGD ricochets wall to wall while creeping along the floor; momentum averages the zig against the zag and moves down the canyon's length. On near-flat plateaus, where gradients are whispers, the stored speed carries the ball across. And under stochastic descent's batch noise, the velocity doubles as a moving average that smooths the poll's jitter for free. Almost no serious training run goes without it.

The cost of heaviness is exactly what physics suggests: a ball with speed cannot stop on a coin. Momentum overshoots minima and swings back, and with β pushed towards 0.99 the swings can grow into slow oscillations a lighter ball would never suffer. It also keeps one extra number per dial — the velocity — which on a billion-parameter model is a billion extra numbers held in memory. Speed is bought, not free.

060

Momentum gave the ball speed, but every dial still shares one step size — and dials do not all want the same one.

Adam and friends

Give every dial its own step size, tuned as you go.

Adam gives every dial its own step size, adjusted continuously. For each parameter it keeps two running averages: m ← β1·m + (1−β1)·g, which way its gradients have been pointing — momentum under another name — and v ← β2·v + (1−β2)·g², how large they have typically been. The update divides one by the other: w ← w − η·m/(√v + ε). A dial whose gradients are reliably tiny sees a tiny √v, so the division makes its turns bolder; one that swings violently gets reined in. The ε is merely a guard against dividing by zero.

Why per-dial step sizes matter: dials in a real network live at wildly different volumes. In a language model, the embedding row for a rare word — 'Secunderabad', seen once in a million tokens — receives gradients thousands of times smaller than the row for 'the', which fires in every sentence. One shared step size must choose between starving the rare word and destabilising the common one. Adam's division levels them: every dial moves at a roughly comparable pace however loud or quiet its gradient runs.

Since its 2014 debut Adam has been the field's default: forgiving of a roughly-chosen learning rate, and the first thing anyone reaches for. Nearly every large language model you have heard of was trained with Adam or its close variant AdamW, which repairs a subtle interaction with regularisation's weight penalty. The 'and friends' covers a family — AdaGrad, RMSProp — built on the same move of dividing by the gradient's typical size.

The bill is memory — those two running averages mean two extra stored numbers per parameter, tripling the optimiser's footprint. For a seven-billion-dial model with its bookkeeping kept in 32 bits, that is roughly 56 extra gigabytes of notes about the dials, rivalling the dials themselves. And default does not mean best: on many image tasks, plain SGD with momentum, tuned patiently, still lands at dials that generalise slightly better. Adam's promise is not the finest answer; it is a good answer without a fight.

061

You picked one learning rate and held it fixed for the whole run; the best runs change their mind midway.

Learning-rate schedules

Start bold, finish careful.

So a schedule changes the learning rate as training runs: start bold, finish careful. Big early steps cross the loss landscape's open country fast; small late steps settle precisely onto the valley floor instead of stepping back and forth across it. This is the direct cure for stochastic descent's late-stage milling — the batch noise never quietens on its own, so the steps must. The panel beside this text is running exactly this: the same descent under a fixed rate and a decaying one, and only one of them comes to rest.

The most common shape has a formula. Cosine decay sets η = η_max · ½(1 + cos(π·t/T)): the rate glides from full strength at step t = 0 down to zero at the final step T, following half a cosine wave — gentle at both ends, fastest in the middle. Many runs also open with a warmup: a few hundred steps of linearly ramping the rate up from zero, gentle steps while the dials are still strangers to the data and one wild early gradient could fling them somewhere foolish.

The idea borrows its oldest name from metallurgy — annealing, cooling metal slowly so its structure settles well. It matters more than it sounds: the same model, data, and optimiser can land at noticeably different loss under different schedules, and large language model runs treat the schedule as load-bearing, typically warming up over the first fraction of a percent of steps and decaying to around a tenth of the peak rate. A run that looks broken is sometimes just a run that was never allowed to slow down.

The honest catch is that a schedule must know the future. Cosine decay contains T, the total number of steps, decided before the run begins — stop early and you are stranded mid-glide at a rate too hot to have settled; extend the run and the schedule has already cooled to nothing. It also adds more settings to choose — peak rate, warmup length, final rate — to a process already crowded with choices, as the stop on hyperparameters is about to make official.

062

Before the first gradient ever flows through your stacked layers, every dial must be set to something — and something is a choice.

Initialisation

Where the dials start decides whether training ever gets going.

You cannot start every dial at zero. Neurons in a layer with identical weights receive identical blame and turn identically forever — a panel of clones can never become a panel of specialists — so the dials must start random, purely to break the symmetry. But random at the wrong size is fatal too. Each neuron sums hundreds of weighted inputs, and that sum grows with both the weights' size and their count: too large and signals swell layer by layer into garbage, too small and they fade to silence before reaching the output.

The fix is randomness at a calculated scale — set each weight's typical size from how many inputs its neuron listens to, so signals keep roughly the same strength across every layer. The recipe named for Kaiming He, built for ReLU layers, draws each weight with typical size √(2/n), where n is the neuron's input count: a neuron listening to 512 inputs gets weights around 0.06. The arithmetic is tuned so the spread of the weighted sum comes out level — what one layer passes on is, on average, no louder and no softer than what it received.

The recipes carry their inventors' names — Xavier initialisation for the older squashing activations, He initialisation for ReLU — and their arrival in the early 2010s is part of why deep networks abruptly went from nearly untrainable to routine. The failure they prevent compounds quickly: signals scaled by even 1.2 per layer grow sixfold across ten layers and nearly forty-fold across twenty. A choice made before the first example is ever seen decides whether the deepest layers ever hear anything worth learning from.

The limit is that initialisation only buys the opening. It sets the dials somewhere trainable; it cannot rescue a bad architecture or a wild learning rate, and modern furniture — the normalisation layers just ahead, skip connections — has made networks far more forgiving of a sloppy start, so the recipes matter less than they did a decade ago. But less is not nothing: stack fifty plain layers with naively sized random weights and training still dies before the first epoch ends.

063

Backprop multiplies link after link on the journey back, and long chains of multiplication rarely stay polite.

Vanishing and exploding gradients

Blame that fades to nothing, or blows up, on its way back through deep networks.

Backprop multiplies the chain's link ratios together, and a deep network is a long chain. If each layer passes blame back scaled by, say, a half, then thirty layers deep the blame has been halved thirty times: 0.5^30 is about one billionth. The early layers hear nothing and never learn. Tip the ratios above one instead and the same compounding runs the other way — 1.5^30 is roughly 190,000 — and the gradient explodes into uselessness, flinging dials to garbage in a single step. The panel beside this text lets you set the per-layer ratio and watch both fates unfold.

The old squashing activations made vanishing almost mandatory. A sigmoid's slope is at most 0.25, so every sigmoid layer multiplies the passing blame by a quarter at best — ten layers in, the gradient is down by a factor of a million before the weights even join the product. This arithmetic is why depth stalled for years after backprop was known: the recipe worked, but only for shallow stacks. Nobody's network was broken. Everybody's chain was too long.

The whole 2010s toolkit reads as a campaign against this one multiplication. ReLU keeps slopes at exactly one for every active neuron, so its links multiply blame by one, not a quarter. He initialisation sizes the weights so the product starts level. Normalisation layers, next on the road, re-centre the numbers mid-chain. Skip connections let blame bypass layers entirely, giving every gradient a short path home. Deep networks were never hard to build. They were hard to reach.

The campaign manages the disease rather than curing it. Exploding gradients are still handled with a blunt instrument — clipping, which caps any gradient above a chosen size and throws the excess away. And any design that applies the same layer over and over — the old recurrent networks read a sentence by reusing one layer per word — multiplies the same ratio hundreds of times, and vanished over long sentences despite every remedy; that stubbornness is partly why the field later moved to architectures with short paths between everything. The multiplication always wins eventually. The craft is keeping the chain short enough that it wins slowly.

064

Gradients vanish or explode when the numbers inside the network run wild — so re-centre those numbers, layer by layer.

Normalisation layers

Re-centre the numbers inside the network so training stays stable.

A normalisation layer stands between layers and re-centres what flows through. The recipe: take the batch's numbers at this point in the network, subtract their average μ and divide by their spread σ, so x̂ = (x − μ)/σ lands with average zero and spread one; then let two learned dials, γ and β, set whatever new scale and shift the network prefers: y = γ·x̂ + β. However far the previous layer's output has drifted, the next layer receives numbers in a familiar range — and the learned dials mean nothing is forced, only offered.

The drift it cures is real and compounding. Suppose the numbers leaving some middle layer have crept up to an average of 40 with a spread of 12 as the earlier dials moved during training. Every layer downstream must keep re-adapting to that moving target, and gradients scale with the numbers they pass through — which is exactly how the vanishing-and-exploding trouble gets started. Normalisation resets the target every batch: average zero, spread one, always.

The effect, when batch normalisation arrived in 2015, was blunt: deeper networks trained faster, tolerated bolder learning rates, and stopped being fragile about initialisation. The honest footnote is that the field still argues about exactly why it works — the original story, that it tames 'internal covariate shift', has been challenged repeatedly, with experiments showing the benefit survives even when that shift is deliberately reintroduced. A rare thing: a technique adopted universally on results, with the explanation still in committee.

Its dependence on the batch is also its weakness. The statistics μ and σ come from whichever examples happen to share the batch, so an example's output depends on its batch-mates — training and inference behave differently, and at batch size one the recipe collapses entirely. Transformers use a sibling, layer normalisation, which computes the same μ and σ within a single example, across its own numbers, no batch required; it is part of how they stay steady dozens of blocks deep.

065

Regularisation punished complicated answers with a penalty term; here is a blunter instrument — sabotage the network while it learns.

Dropout

Switch off random neurons while training so no single one becomes indispensable.

During training, before each step, switch off a random fraction of the neurons — commonly half — as if they were never there. Next step, a different random half. No neuron can count on any particular colleague existing, so none can become indispensable; the network is forced to spread every pattern across many redundant paths. The procedure is two lines: for each neuron, flip a coin with keep-probability p, zero the losers, and scale the survivors up by 1/p so the layer's overall output keeps its usual strength.

It is regularisation by sabotage, and it works for the same reason a team that trains with random members absent copes well with injuries — a cricket side that has practised without its opening bowler does not collapse when he pulls a hamstring. The combinatorics are absurd in the best way: a layer of 4,096 neurons with half dropped has more possible thinned versions than there are atoms in the universe, and training wanders through a fresh one every single step.

At inference everything switches back on, and the trained network behaves like an average over the countless thinned networks it briefly was — an ensemble of millions of models, bought for the price of one. Hinton's lab introduced it in 2012, it helped power AlexNet's celebrated ImageNet win that same year, and it costs almost nothing: no new dials, no extra memory, a handful of coin flips per step.

It has aged into a specialist rather than a staple. Dropout shines when the model is large relative to its data and memorisation is the main threat; the largest language models, trained for barely one pass over oceans of text, have little time to memorise anything, and often set dropout to zero. Set it too high anywhere and you are not regularising but starving — a network sabotaged harder than it can compensate for simply learns slowly and badly. The dial turns both ways.

066

Your validation set has been watching the whole run; at some point it starts telling you to stop.

Early stopping

Stop when the held-out score turns, not when the training score does.

Watch two scores as training runs: loss on the training data, and loss on the validation set. The training score falls for as long as you care to keep going — memorising noise still counts as progress there. The validation score falls, flattens, then turns upward. That turn is overfitting beginning, live. Early stopping means quitting at the turn and keeping the dials from that best moment; the panel beside this text runs the whole drama, both curves and the turn.

The procedure is exact. Every so often — say once per epoch — measure validation loss. If it beats the best seen so far, save the dials. If it fails to improve for a set number of consecutive checks, the patience, stop and restore the last saved dials. Concretely: validation loss falls from 0.82 to 0.61 by epoch twelve, scrapes down to 0.60 at fifteen, then reads 0.62, 0.63, 0.65. With patience three, the run halts at epoch eighteen and ships the epoch-fifteen dials.

It is the cheapest regularisation there is: no penalty term, no new dials, just the discipline to stop and a saved copy of the dials from the right step. The patience exists because validation curves wobble — the validation set is a finite sample, so its loss jitters even when nothing real has changed. One blip is noise; five in a row is a verdict.

Two honest cautions. First, every peek at the validation set spends a little of its independence: use it to choose the stopping step, the learning rate, and six other settings, and the 'held-out' score has quietly been optimised for — which is why a final, untouched test set exists at all. Second, the tidy fall-flatten-turn silhouette is not guaranteed; some large runs flatten for ages and then improve again, so a patience set too short can kill a run that was merely pausing. The turn is evidence, not prophecy.

067

The learning rate was a number you chose, not one the model learned — that difference has a name and a large family.

Hyperparameters

The settings you choose, as opposed to the numbers the model learns.

The model learns its parameters; you choose its hyperparameters. Learning rate, batch size, number of layers, neurons per layer, dropout fraction, patience, warmup length, when to stop — the settings that shape a training run rather than being shaped by it. They are the oven's temperature and timer, as opposed to the cake. Every stop on this stretch of road has quietly added one or two to the pile, and by now the pile is a dozen deep.

The awkward truth is that no gradient exists for them — you cannot walk downhill on 'how many layers', because changing it rebuilds the model rather than nudging it. So they are searched: pick settings, train, compare validation scores, pick again. The procedure is nested: an outer loop proposes settings, an inner loop runs a complete training under them, and the validation set judges. Try learning rates of 0.1, 0.01, 0.001 and 0.0001 across three batch sizes and you have committed to twelve full training runs before touching any other dial.

Since every trial is an entire training run, the search is expensive, and a large share of practical deep-learning skill is knowing which of these outer dials to try turning first — learning rate nearly always, then batch size and model size, with the rest touched only once the big ones are settled. One finding worth keeping: sampling settings at random beats marching through a tidy grid, because a grid spends its budget re-testing values of dials that turn out not to matter.

The honest limit shows up in published numbers. When one method beats another by a whisker, the difference is often tuning budget rather than the idea — the winner was simply searched harder. And at the largest scales the search collapses entirely: nobody trains a frontier model a dozen times to tune it, so its hyperparameters are extrapolated from small, cheap runs and a good deal of institutional folklore, with no guarantee the extrapolation holds.

068

Overfitting, a wild learning rate, a bug — every training disease writes its signature on one falling line.

Reading a loss curve

The single most useful diagnostic skill in the whole field.

Plot loss against training steps and you get the field's ECG. A healthy run falls steeply, then eases toward a floor. A curve that explodes upward says the learning rate is too high; one that flatlines from the start says too low — or a bug upstream of the model entirely. Violent jaggedness suggests batches too small or steps too bold. The panel beside this text deals out these silhouettes live; learn their shapes and you can diagnose a run at a glance, before consulting a single other number.

The curve even tells you where it should begin. A model guessing blindly among ten classes should start near loss = ln 10 ≈ 2.3, because cross-entropy charges −ln p and blind guessing gives every class p = 0.1. A first reading far above that means broken inputs or wild initialisation; suspiciously below means the data is leaking answers. Experienced practitioners check this one number before anything else — a smoke alarm that costs nothing.

The essential version carries two lines, training and validation loss together. While both fall, the model is learning. When training keeps falling and validation turns upward, you are watching overfitting happen in real time — memorisation, live on screen, the very turn that early stopping waits for. Practitioners read these silhouettes the way doctors read heartbeats, and the first question asked of any ailing run is: show me the curve.

The ECG has blind spots. Loss is a proxy, and a falling proxy does not guarantee that the thing you care about improves — a fare model can polish its average error while staying reliably wrong about airport surcharges, and a classifier's loss can improve while its accuracy sits still. A beautiful curve can also hide a beautiful mistake: if validation examples leaked into training, both lines fall gorgeously and mean nothing. The curve is where diagnosis starts, never where it ends.

069

Working in logs rescued you from numbers too small to store; now ask how few bits a number really needs.

Mixed precision

Use smaller numbers to train faster, without losing the answer.

Every dial is stored in bits, and the standard 32 bits per number is more precision than training usually needs. Mixed precision stores most numbers in 16 bits instead: half the memory, and modern chips multiply the short numbers several times faster — an NVIDIA A100's specialised 16-bit units deliver roughly sixteen times the raw multiply rate of its plain 32-bit path. The model's answer does not need micrometre engraving on every dial; a seven-billion-dial model slims from 28 gigabytes of weights to 14, and the halved traffic speeds everything that moves numbers around.

The danger is the one working in logs already taught you: small numbers vanishing. Gradients are often tiny, and fp16 cannot represent ordinary numbers below about 6×10^-5 — beneath that floor, a tiny gradient rounds to exactly zero, blame silently deleted. The panel beside this text shows how much of a gradient's population falls off that cliff as the bits shrink.

The standard cure has two parts. First, loss scaling: multiply the loss by a large constant, say 1024, before backprop — every gradient in the chain inherits the factor, lifting the tiny ones clear of the floor — then divide it back out before turning the dials. Second, keep one full-precision 32-bit master copy of the dials, so millions of small updates accumulate without rounding away; the fast 16-bit working copy is refreshed from it each step. Done right, you train nearly twice as fast and land on the same answer.

The same answer, mostly. Runs at the largest scales sometimes hit sudden loss spikes traceable to precision, which is why the field drifted to bfloat16 — a different 16-bit format that keeps fp32's range by spending its bits on exponent rather than detail, making overflow nearly impossible and loss scaling unnecessary. And the appetite does not stop at 16: training in 8-bit formats is live research, with real accuracy losses when pushed carelessly. Every bit removed coarsens the grid the numbers must snap to. The bargain has a floor.

070

An epoch over real data can take hours or days — and a machine that dies at hour forty takes every step with it.

Checkpoints

Save the dials often; training runs die.

A checkpoint is all the dials, written to disk. Training a serious model runs for days on machines that fail — a power cut, a full disk, a rented GPU reclaimed mid-epoch. Save every so often and a crash costs you hours; save never and it costs you the run. A forty-hour run that checkpoints hourly can lose at most one hour to any disaster. It is Ctrl+S for a weeks-long walk downhill.

A resumable checkpoint holds more than the dials. To continue a run rather than restart it, you also save the optimiser's memory — Adam's two running averages per parameter — plus the step count, so the learning-rate schedule resumes mid-glide, and even the data shuffler's position. Resume from dials alone and the optimiser wakes with amnesia: workable, but a measurably different run. The sizes are blunt: a seven-billion-dial model is 14 gigabytes in 16-bit weights alone, and nearer 80 with full training state — which is why saving is scheduled, not constant.

Checkpoints are more than insurance. The file you get when you 'download a model' is a checkpoint — training's entire output is the dials. Saving along the way also lets you rewind to the step where the validation score looked best, which is how early stopping keeps its promise, or branch one run into several experiments from a common ancestor. The dials are the asset, and checkpoints are how the asset exists at all. Lesson 2 walks you through saving and reloading one yourself.

The honest footnote is that the dials are necessary but not sufficient. A checkpoint without the code that defines the architecture, and the exact recipe that turned raw data into numbers, is a piano roll without the piano — many a lovingly saved file has been orphaned by a refactor. Frequency is a real trade too: writing tens of gigabytes stalls the run and fills disks at industrial speed, so teams keep a rolling few recent saves plus occasional permanent ones, deleting the rest. Insurance has premiums.

And this one you can do, not just read — Lesson 2 on the panel walks you through training it yourself.

071

A matrix is a grid of numbers — and every photograph you have ever taken is exactly that, nothing more.

Images as numbers

A photograph is a grid of brightness values, nothing more.

Zoom into any photograph far enough and it dissolves into a grid of tiny squares, each holding one brightness number — 0 for black, 255 for full white. The range is 0 to 255 because each value lives in one byte: eight bits, 2^8 = 256 possible levels. A colour photo is three such grids stacked, one each for red, green, and blue, so a single pixel is a triple like (210, 165, 90) — the warm brown of a glass of chai.

The scale is worth pausing on. A twelve-megapixel snap is a grid roughly 4,000 pixels wide by 3,000 tall, times three colours: thirty-six million numbers, and that is all a machine ever sees. Photograph a railway ticket and the printed text is nothing but rows of values near 0 against paper near 255; every system that reads that ticket starts from those numbers and nothing else. Position is just row and column — pixel (1200, 340) is a location in a matrix, exactly the object you already know.

This is liberating and humbling at once. Liberating, because a grid of numbers is something a model can eat — all of computer vision starts here, and everything ahead on this road is arithmetic on this grid. Humbling, because nothing in those numbers says cat. The same cat shifted two pixels left is a completely different matrix; a cloud crossing the sun changes every number at once while changing nothing you would call the picture. Closing that gap between numbers and meaning is the entire job ahead.

One honesty about the grid itself: it is already an interpretation. Your phone's sensor measured light, then software denoised, sharpened, and tone-mapped before any number reached the file — two phones photographing the same chai stall produce two different matrices of the same scene. And 256 levels quantise smooth light into steps, which is why deep shadows in cheap photos band into visible stripes. The numbers are all a machine gets, but they were never a neutral record of the world.

072

Now that an image is numbers, a layer could read it — but wiring every neuron to every pixel wastes millions of dials.

Convolution

Slide a small filter across the image and look for one pattern everywhere.

The fix is to stop wiring and start sliding. Take a tiny grid of weights — three pixels by three, say — and move it across the whole image, at each stop multiplying it against the nine pixels beneath and summing to one number: out(i, j) = Σ k(a, b) · img(i+a, j+b), each kernel weight times the pixel under it, all added up. The result runs high where the patch matches the pattern and low where it does not: one small pattern-detector, applied everywhere, like pressing a stencil against every part of a wall.

The economy is the point. Wire one neuron to every pixel of even a modest 224 by 224 colour photo and it needs 150,528 dials; the sliding filter needs nine, or twenty-seven counting all three colours, reused at every position. Its output is not one number but a map: a grid the size of the image where each cell scores how strongly the pattern lives at that spot. You have met this operation without the name — the blur tool in every photo editor is a convolution whose nine weights are all one-ninth, averaging each pixel with its neighbours.

And the reuse buys something a plain layer never had: a cat's ear is a cat's ear wherever it appears, because the same detector visited every spot. Shift the cat two pixels left — the disaster of the previous stop — and the ear's high score simply moves two cells left on the map. The assumption that patterns repeat across an image is built into the wiring, not learned from data, which is exactly why convolutional layers learn from far fewer photos than fully wired ones.

The honest cost is near-sightedness. A three-by-three window relates a pixel only to its immediate neighbours; the engine of a train and its last coach are invisible to each other. Only stacking layer upon layer slowly widens what one output can see. And the built-in assumption misleads when position itself is the meaning — in a chest X-ray, the same shadow means different things on the left and right side, and pure convolution is structurally indifferent to which is which.

073

Convolution slides something small across the image — time to look at what, exactly, is doing the looking.

Filters and kernels

The small grid of weights that does the looking — learned, not designed.

Write one out and it stops being mysterious. Take the kernel with rows (1, 0, −1), (1, 0, −1), (1, 0, −1) and slide it: over a patch bright on the left (values near 200) and dark on the right (near 50), the sum is 3·200 − 3·50 = 450, a loud vote; over a flat wall of 120s the positives and negatives cancel to exactly zero. That arrangement is a vertical edge detector. Others light up on horizontal edges, a patch of green against brown, a corner — the kernel's numbers are dials like any others, and the arrangement is the pattern it seeks.

For decades people designed these by hand. Blur is a kernel of nine equal ninths; sharpen is a centre weight pulled high with negative neighbours; emboss, and every filter in a photo app, belongs to the same family. The Sobel edge kernel, published in 1968, still runs today in factory machine-vision lines checking bottles and welds. Hand-built kernels were a craft, and a good one.

The break with that history is that nobody designs these any more. A kernel's weights start random, and training turns them downhill like every other dial — the network invents its own detectors, edge-finders and texture-spotters emerging simply because they lower the loss. Open a trained vision network's first layer and you find, unprompted, the very edge and colour-blob filters engineers once crafted by hand. Nothing told the network that edges matter; edges earned their place.

That famous picture flatters, though. First-layer kernels are readable because they touch raw pixels; from the second layer on, kernels slide over maps of other kernels' outputs, and naming what they detect becomes guesswork. And a real network learns them in bulk — sixty-four kernels in a typical first layer, hundreds per layer further in — many of them messy, some near-duplicates, some doing nothing measurable at all. The tidy story of one kernel, one nameable pattern is true mostly at the bottom of the stack.

074

Convolution reports where every pattern fired, pixel by pixel; often all you need to keep is that it fired.

Pooling

Shrink the picture, keep the evidence.

The procedure could not be simpler. Take each small neighbourhood of a filter's response map — two by two, say — and keep only the strongest value: out = max(a, b, c, d), so the four scores 3, 9, 4, 7 become just 9. The map halves in each direction, three-quarters of the numbers are discarded, and the loud votes survive the shrinking. The filter said the pattern fired near here; pooling keeps the fired and lets go of the exact pixel.

Do this between layers and the shrinking compounds. Five poolings take a 224-wide map down to 7, which means a detector in a late layer, still looking through a modest three-by-three window, is effectively surveying a third of the original photograph. Pooling is how near-sighted filters end up with a wide-angle view — each halving doubles how much of the world fits in the next layer's window.

What you buy is tolerance. After pooling, a feature shifted a pixel or two lands in the same cell, so the network stops caring exactly where the ear sat — and cat never depended on that anyway. Whether a photo contains a cricket ball does not change when the ball drifts three pixels across the frame; pooling makes the network agree.

What you pay is precision, because fine position is gone, and some tasks want it back. Tracing a tumour's boundary in a scan, reading small print on a cheque, counting grains — these need pixel-level answers, and networks built for them must laboriously rebuild the resolution pooling threw away, or avoid pooling entirely. Many modern designs replace max-pooling with strided convolutions, letting the network learn its own way of shrinking — but shrink they still do, because the trade itself is sound: keep the evidence, spend the coordinates.

075

Learned kernels do the looking and pooling keeps the evidence — stack them, and parts start assembling into things.

Convolutional networks

Edges become shapes become objects, layer by layer.

Stacked, the pieces become a hierarchy. The first layer's filters find edges and blobs of colour. The next layer slides over those maps, not the photo, finding arrangements of edges — corners, curves, textures. Layer by layer the detectors grow more abstract: edges into ears, ears and whiskers into a face. Recognition as assembly from parts, learned end to end — and no one tells layer three to look for ears; detectors for ears emerge because, downstream, they lower the loss on saying cat.

The design is old. Yann LeCun's 1989 cheque-reading network was already exactly this — convolutions, pooling, a handful of layers — and by the late 1990s its descendants were reading millions of handwritten cheques and postal codes. What the idea lacked for twenty years was not correctness but scale: enough labelled images to learn from, and enough compute to learn with.

In 2012 both arrived. AlexNet — eight learned layers, 60 million dials, trained for about a week on two gaming GPUs — entered ImageNet, a contest over 1.2 million labelled photographs, and cut the best error rate roughly in half, from 26 per cent to 15. Fields do not usually change direction in a year; this one did. Within two, nearly every serious vision system was a convolutional network, and the same bet — old architecture, new scale — became the template for the decade.

The honest caveat is what these networks actually learned. Test them carefully and they lean on texture more than shape: render a cat's outline filled with elephant-skin texture and a standard ImageNet network confidently says elephant. They also stumble on the familiar in unfamiliar places — a cow on a beach gets stranger answers than a cow in a field. Assembly from parts is real, but the parts are statistical conveniences, not the concepts you would have chosen.

076

Deeper stacks of layers should see more — but you already watched blame vanish on its long walk back through depth.

Residual connections

Let the signal skip layers, and suddenly you can train very deep networks.

The residual connection is a wire that skips a layer: the layer's output is added onto its own input, y = x + F(x), so the layer only needs to learn the difference — the residual — rather than re-describe everything. If it has nothing useful to add, it can learn F(x) = 0, weights near zero, and the signal passes through untouched. Contrast the plain stack, where doing nothing means learning to copy the input exactly through a pile of multiplications — a surprisingly hard thing to learn.

Now walk the blame backwards. Differentiate y = x + F(x) with respect to x and you get 1 plus the layer's own contribution. That standing 1 is the motorway: gradient flows straight through the addition, undiminished, however deep the stack — instead of being multiplied by dozens of possibly-small factors on the long walk back. The vanishing you watched earlier is not so much cured as bypassed.

The evidence was immediate. Before this wire, stacking much past twenty layers made networks worse — a 56-layer plain network scored worse than its 20-layer sibling even on its own training data, so the problem was optimisation, not overfitting. In 2015 ResNet trained 152 layers and won ImageNet at 3.6 per cent error — below the roughly 5 per cent a careful human manages on that test — and the trick spread everywhere. The transformer block keeps its own signal steady with exactly the same wire.

What the wire does not buy is understanding. It makes depth trainable, not automatically useful: much of a very deep residual network turns out to be gentle refinement, whole layers can be deleted after training with only mild damage, and pushing past a thousand layers returns almost nothing. Depth stopped being the obstacle in 2015. It quietly stopped being the answer soon after — the gains moved to width, to data, and, on the road ahead, to attention.

077

Convolutional networks learned to recognise pictures; the stranger feat is conjuring them, starting from a distribution of pure noise.

Diffusion models

Learn to remove noise, then start from pure noise and remove it all.

Take a photograph and add a little noise, then a little more, until only static remains. Now train a network to reverse one step: shown a noisy image, estimate the noise so it can be subtracted. In symbols, the trainer builds noisy = image + σ·ε, where ε is random static and σ says how much was poured in; the network outputs a guess ε̂, and the loss is (ε̂ − ε)² — squared error against the true noise, which the trainer knows exactly, because it added it. That is the whole training task, practised at every level of ruin from nearly-clean to pure static.

Generation runs the film backwards. Start from pure static and denoise, step by step — typically twenty to fifty steps, each subtracting a fraction of the predicted noise — until a picture that never existed emerges. Ask for an auto-rickshaw in monsoon rain and the early steps commit only to smears, a dark mass low, a grey wash above; the middle steps find wheels and a canopy; the last steps sharpen rain streaks and lettering. Big decisions early, details late.

Why not generate in one leap? Because make an image is an impossible jump, while remove a little noise is a small, learnable one — a hard problem paid off in easy instalments. This is the engine inside the image generators that startled everyone from 2022 onwards; Stable Diffusion runs on roughly a billion dials, trained against about two billion captioned images scraped from the web.

The costs are honest and specific. One sits at the far end: dozens of denoising steps per picture, each a full pass through the network, so an image costs tens of times what a classification does — much research since has gone into collapsing fifty steps towards one. The other sits at the start: the model can only pull pictures out of the distribution it swallowed, so the web's biases and clichés arrive intact, and anything needing exactness — the five fingers of a hand, the lettering on a shopfront — long came out subtly, confidently wrong.

078

A feature measured one thing at one moment — but text, speech, and prices arrive in order, and the order carries the meaning.

Sequence data

Data where order carries meaning — text, audio, prices, DNA.

A sequence is x₁, x₂, … up to x_T — and both things that make it awkward are visible in the notation: the length T is different every time, and the subscripts matter. Shuffle a photograph's rows and you get a damaged photograph; shuffle a sentence's words and you get no sentence at all. The values alone are not the data. The order is half the information, and sometimes nearly all of it.

It is also most of what the world sends you. Text, speech, music, share prices, DNA — and a monsoon: a season's rainfall is around 120 daily readings, and the same July total of 300 millimetres means a comfortable month spread across twenty wet days or a flood delivered in two. Sort a patient's heart-rate trace into ascending order and you have destroyed the diagnosis while keeping every single number.

That property broke the models that came before. A network that eats fixed-size inputs has no honest way to eat as much past as matters: pad short inputs and you feed it noise, chop long ones and you throw away the past, and either way each position gets its own dials, so word three and word four are strangers. The struggle to give machines a working memory is the whole story of the decade before transformers — recurrence, gating, and attention are all answers to this one awkwardness, and they are the next stretch of road.

One honest correction to the slogan: order carries meaning, but not always much of it. A bag-of-words spam filter ignores order entirely — it just counts which words appear — and still catches most spam, because free, winner, and urgent betray the message in any arrangement. Calling data sequential is a modelling choice with a price tag. Pay for order-aware machinery when order pays you back; a shuffled sentence is no sentence, but a shuffled shopping basket is the same shopping.

079

Before any model can read a sequence of text, the text must become pieces — and the pieces are rarely whole words.

Tokenisation

Chopping text into the pieces a model actually sees. Rarely whole words.

The chopping follows a fixed menu, and the menu is built by counting. Start with single characters. Scan mountains of text for the most frequent adjacent pair — t next to h, say — and merge it into one new piece; then count again, merge again, tens of thousands of times. The result is a vocabulary of perhaps 50,000 to 100,000 pieces in which common words earn their own token while rarer ones split into fragments, so unbelievable may arrive as un, believ, able. The model never sees letters or words — only the id numbers of these pieces.

In English the pieces average about four characters, three-quarters of a word, so an everyday sentence runs eight to ten tokens. But the menu reflects whatever text it was counted from, which is mostly English web pages. Hindi written in Devanagari recurs far less in that pile, so its words splinter into many small fragments — the same sentence can cost two or three times the tokens it costs in English. You are billed by the token, and the tokeniser decides how many tokens your words become; the meter runs faster for some languages than others.

The menu quietly shapes everything downstream. It is why models miscount the letters in strawberry — the word arrives as a couple of opaque chunks, not a string of letters the model could examine. It is why digits behave oddly: 1234 might be a single piece while 1235 is two, which is part of why arithmetic wobbles. Lesson 2 walks you through chopping text yourself and watching exactly where words shatter.

And the menu is frozen. Built once, before training, it never learns; new slang, new product names, unusual spellings and code all splinter into fragments the model must laboriously reassemble. A tokeniser tuned on one kind of text quietly taxes every kind it under-sampled — a small, unglamorous component making decisions the glamorous model downstream can never see, let alone undo.

And this one you can do, not just read — Lesson 2 on the panel walks you through training it yourself.

080

Your multilayer network eats inputs of one fixed size, and a sentence refuses to be one — unless you feed it one step at a time.

Recurrent networks

Read one step at a time and carry a memory forward.

Feed the sentence one token at a time, and let the network carry a running summary — a vector it hands forward to itself. At each step the update is h_t = f(W·x_t + U·h_{t−1}): the same weight matrices W and U combine the new token with the summary so far, a squashing function f keeps the result in range, and out comes the updated summary. Because those dials are shared across every step, one small network reads a sentence of any length — the fixed-size problem dissolved by walking.

Everything the network knows about the past must live in that one vector, rewritten at every step. Predicting the next word of the train from Chennai to, the summary at the final step must still hold train and Chennai — packed in with everything since — to favour Bengaluru over breakfast. Get the carrying right and the network genuinely uses context; that was the breakthrough, and for years this was the field's working answer to sequences.

Its flaws were structural. The summary is re-squeezed at every step, so early words fade — by token forty, token one is a rumour. And training unrolls the sequence into one long chain, a hundred-word sentence becoming in effect a hundred-layer network, down which blame famously vanishes on the long walk back to the first word.

And the hardware hates it. Step ten cannot begin until step nine finishes, so the chips built for doing everything at once sit waiting, one small matrix multiply at a time — a recurrent network with a million dials can be slower to train than a parallel network with a hundred million. Fading memory, vanishing blame, and an unfillable queue: remember all three, because the next stops on this road are, in order, the repairs.

081

A recurrent network's carried memory fades just as gradients vanish — unless the network can learn what to keep and what to drop.

LSTM and gating

Learned gates that decide what to remember and what to drop.

The repair is to stop overwriting and start deciding. An LSTM fits the carried memory with learned gates — small valves, each a number between 0 and 1, computed fresh at every step from the current input and the last summary. The cell update reads c_t = f·c_{t−1} + i·ĉ: the forget gate f scales how much of the old memory survives, the input gate i scales how much of the new candidate ĉ is written in, and an output gate separately decides how much of the cell to reveal. The network learns when to hold on: keep the subject of the sentence, let the adjectives go.

The arithmetic is the cure. When the forget gate sits near 1, a value rides the cell from step to step barely touched — a highway through time — and blame flows back along the same road without shrinking at every step, taming the vanishing that cripples plain recurrence. In the trains that left Howrah this morning were late, the cell can carry the plural of trains across five intervening words and still choose were over was; hundreds of steps of such holding are routine where plain recurrence managed dozens.

Invented in 1997 by Hochreiter and Schmidhuber, the design waited years for its moment and then had a long one: for two decades, gated recurrence simply was serious sequence modelling. By 2016 Google Translate ran on stacks of LSTMs, and the next-word suggestions above your phone keyboard were an LSTM working in your pocket — reading your half-typed sentence step by step, gating what to keep.

What the gates never fixed was the queue. One step after another, however clever the valves — step ten still waits for step nine, and the parallel hardware still idles. The gates also multiply the machinery: four sets of weights where plain recurrence had one, all protecting a memory that remains a single vector of perhaps a thousand numbers. Hold that thought about the single vector; it is about to become the whole problem.

082

One recurrent network can read; put a second beside it to write, and translation becomes a machine.

Encoder–decoder

Read the whole input, then write the whole output.

Bolt two recurrent networks together and give them different jobs. The encoder reads the whole input — an English sentence, say — updating its carried memory token by token, and keeps only the final vector. The decoder starts from that vector and writes the output one token at a time, and here is the loop that matters: each token it emits is fed back in as context for choosing the next, until it produces a special stop token. Read everything, then say everything: translation as pack-and-unpack.

The read-first discipline is not pedantry. Languages disagree about order: where is the station comes out in Hindi as station where is — the question word in the middle, the verb at the end. A machine translating word by word as it reads would have to emit the Hindi ending before reaching the English words that determine it. Packing the whole sentence first lets the decoder unpack it in a different order entirely, which is most of what translation is.

This design made neural machine translation real around 2014, replacing decades of hand-built phrase tables and rules with one machine trained end to end — and within two years it was serving Google Translate. More lastingly, it drew the shape almost everything since has kept: an encoder that understands, a decoder that generates. You will meet the pair again, together and separately, for the rest of this road.

Two honest weaknesses. The decoder eats its own words: one early wrong token becomes context for everything after it, and errors compound with no way back — pick the wrong opening word and the whole sentence bends around the mistake. And watch the handover closely: the entire meaning of the input, however long, must pass through that one final vector. That pinch has a name, and a stop of its own, next.

083

The encoder hands the decoder one vector meant to hold a whole sentence — look hard at that handover.

The bottleneck problem

Squeezing a whole sentence into one vector loses the sentence.

Put numbers on the handover. Whatever the encoder read, the decoder receives one fixed-size vector — a few hundred numbers, say 512 — whether the input was three words or three hundred. A short sentence fits comfortably. A long one is a suitcase packed until it will not close: each new clause pressed in smudges what is already there, and by the end of a paragraph the details of the opening have been squeezed out.

The failure showed up on the graphs. When researchers in 2014 plotted translation quality against sentence length, quality slid steadily as sentences grew — precisely because the container did not. Try it on an errand: two sleeper tickets, Pune to Goa, the fourteenth, lower berth for my mother, window side if possible, and a veg meal. Ten details, one vector; a system with this architecture reliably delivers the tickets and loses the meal.

Notice the kind of mistake this is — not a bad dial setting but a bad architecture, a flaw no amount of downhill turning can fix, because no setting of the dials makes a fixed container hold unbounded content. Nor is a bigger vector a cure: doubling 512 to 1,024 moves the wall back a few clauses and the slide resumes, since sentences can always grow and the suitcase cannot.

Be fair to the single vector, though: the failure is specific. For judging whether a review is angry, one summary is plenty — squeezing is exactly what a verdict needs, and half of practical machine learning happily lives on such compressions. The break comes when generation must reproduce the details of a long input from a summary that no longer contains them. The cure is not a bigger vector. It is letting the decoder look back at the whole input while it writes — and it has a name: attention.

084

If one bottleneck vector cannot hold a sentence, stop squeezing — let the output look back at every input word directly.

Attention

Let every position look directly at every other and decide what matters.

So keep every input word's vector on the table, and at each step of the output let the model glance across all of them and decide, with learned weights, which ones matter right now. Nothing is carried forward, nothing fades with distance — direct lines, all pairs, at once. The bottleneck asked one vector to hold a whole sentence; attention refuses to summarise until the moment of use, and builds a different summary for every position that asks.

The glance is arithmetic you already own. Every pair of positions gets a score from a dot product between learned vectors — large when the two point the same way, near zero when they are unrelated. Softmax turns each row of scores into weights that sum to one, and the output is a weighted blend of the inputs: out = Σ wᵢ·xᵢ, where wᵢ = e^(sᵢ)/Σ e^(sⱼ) and sᵢ is this position's score against input i. Because the vectors are learned, what counts as relevant is trained, not written in.

In 'the animal did not cross the street because it was too tired', the word 'it' reaches straight back to 'animal' across seven words — asking, matching, collecting. Attention was born in translation for exactly this reason: rendering that sentence into Hindi, the model needs the pronoun's owner at the moment of choosing the verb form, and a direct line beats a fading memory of seven words ago every time.

Two honest cautions. All pairs means the work grows with the square of the sentence — a bill that shadows everything from here on. And the maps of attention weights, often printed as proof the model 'looked at' the right word, are weaker evidence than they appear: a weight says where the blend came from, not why, and models with strange-looking attention patterns can answer just as well.

085

Attention lets every position decide what matters; here is the machinery of deciding — a question, a set of labels, and contents.

Query, key, value

Ask a question, match it against labels, collect the contents.

Each token's vector is pushed through three learned matrices, and the three outputs play three different roles. q = W_q·x is the query, the question this position is asking. k = W_k·x is the key, the label a position advertises to the world. v = W_v·x is the value, the actual content it will hand over if chosen. Three small matrices, each a grid of trained dials, each turning the same token into a different instrument.

The procedure is fixed once the vectors exist. Score the query against every key with dot products, divide by √d — the square root of the vector length, which stops long vectors producing wild scores — soften the results with softmax into weights, then blend the values in proportion: out = Σ wᵢ·vᵢ. In production models the pieces are modest: queries and keys of 64 or 128 numbers each, matched and mixed millions of times per sentence.

The three-way split is what makes attention learnable rather than a fixed lookup: what a word asks for, what it advertises, and what it hands over are different jobs, each with its own dials to train. A library runs on the same separation — your question, the titles on the spines, the contents of the books — and it would not work if those three were forced to be the same thing. 'Bank' can advertise itself as a noun to grammar-hungry queries while handing over its money-or-river content to whichever position collects it.

Do not expect the roles to stay tidy. The names query, key and value describe the wiring, not a contract; training bends the three matrices to whatever lowers the loss, and inspecting a real head rarely shows a clean question meeting a clean label. The library story is a good map of the mechanism and a poor map of what any particular trained head is actually doing.

086

Queries, keys, and values have to come from somewhere — the decisive move is letting the sentence supply all three itself.

Self-attention

The sentence attending to itself, which is how context gets built.

Every token issues a query, and the same tokens supply the keys and values — the sentence interrogating itself. Each word asks the rest of its own sentence for what it needs and comes back enriched: the vector for 'bank' after blending with 'river' is no longer the 'bank' that would have blended with 'loan'. The panel beside this text is running exactly this — one sentence, every token asking at once, the score table filling in.

The mechanics are one tidy sweep. For n tokens, build n queries, n keys and n values; score every query against every key to fill an n×n table; softmax each row; blend the values. A twelve-word sentence makes 144 scores. Nothing here happens in order — position one and position twelve compute their answers at the same instant, which is what lets the whole thing run in parallel on a GPU.

This is how context stops being carried and starts being computed. No memory trundles along the sequence, the way the bottleneck's did; meaning is assembled fresh at every position from direct lines to everywhere else. Move 'river' further from 'bank' and nothing decays — word two and word two thousand are the same single dot product away. That flatness with distance is the clean break from everything recurrent that came before.

The cost is written into the mechanism. Every token scoring every other is a quadratic bill: double the sequence and the table quadruples. The twelve-word sentence cost 144 scores; a 1,000-token page costs a million; a 100,000-token book costs ten billion. Taming that bill shadows everything that comes after — much of modern engineering is the art of not paying it in full.

087

One pass of self-attention is one way of looking, and a sentence needs several — grammar, reference, tone — watched at once.

Multi-head attention

Several attention patterns at once, each watching for something different.

So the transformer runs attention several times in parallel — eight, sixteen, ninety-six copies called heads, each with its own query, key and value matrices, each free to learn a different habit of looking. One head tracks who did what to whom, one watches neighbouring words, one pairs opening and closing brackets. Their gathered results are stitched together and passed through one final learned matrix, so the block hands on a single vector per token, informed by every way of looking at once.

The trick is that many heads cost no more than one. A model whose vectors are 768 numbers wide does not run twelve heads of width 768; it splits the width, giving each head a 64-number slice in which to build its queries, keys and values. Twelve narrow attentions in parallel do roughly the arithmetic of one wide one — the heads are not extra spend, they are the same spend divided into specialists.

Nobody assigns the heads their jobs; the specialities emerge from training, because a division of labour lowers the loss. Researchers peering inside trained models find heads tracking grammar and reference — in 'the bowler Bumrah, who had limped off earlier, returned', one head links 'who' to 'Bumrah' while another holds the sentence's main verb open — alongside heads doing nothing legible at all.

That illegibility is the honest caveat. Some heads can be deleted from a trained model with barely any loss, which says the division of labour is partly redundancy, and the tidy stories — the grammar head, the reference head — describe the exceptions we can read, not the rule. The design is a hedge, not a plan: many cheap, narrow spotlights turn out to see more than one wide, expensive one, even when nobody can say what each is watching.

088

Self-attention would score a shuffled sentence exactly the same — somewhere, order has to be put back in.

Positional encoding

Attention has no sense of order, so order has to be added back in.

Run self-attention on a shuffled sentence and every score comes out identical — dot products between the same set of vectors, in any order, are the same dot products. The mechanism treats its input as a bag of tokens, blind to who sits where. So order is injected at the door: each position gets a vector of its own, a signature for first, second, seventeenth, added to its token's embedding before any attention runs. 'Bank' at position three and 'bank' at position nine now arrive as different vectors, and attention can tell them apart.

The original transformer built the signatures from sine waves: position pos gets sin(pos/10000^(2i/d)) and the matching cosine in its i-th pair of slots — fast-turning waves in the early slots, slow ones later, like a clock with many hands. Read all the hands together and every position has a unique signature, while nearby positions get similar ones, which is exactly what grammar needs.

The stakes are not decorative. 'The truck hit the taxi near Dadar' and 'the taxi hit the truck near Dadar' are the same bag of eight tokens, and an insurance claim turns entirely on which one was written. Without positional signatures the model literally cannot hold the two apart; with them, word order becomes something attention can score on, like everything else.

Later models let training invent the signatures, or fold relative distance straight into the attention scores instead. None of it extrapolates for free: a model trained on positions up to a few thousand meets position fifty thousand as a stranger, which is one reason stretching context windows takes real retrofitting. Keep the humility of it: word order, which a recurrent network got free by construction, had to be bolted back on. That is the price of ripping out recurrence — and everyone pays it gladly.

089

Multi-head attention, plus the skip connections that saved deep vision networks — assemble them and you hold the modern model's whole blueprint.

The transformer block

Attention, then a small network, twice per layer, with skips. That is the whole thing.

The block is a short recipe. First, multi-head attention: every token gathers context from every other. Second, a small feed-forward network applied to each token alone: digest what was gathered. Each of the two steps is wrapped in a residual skip and a normalisation, so information and gradients can flow past it untouched if need be — the same trick that saved deep vision networks, doing the same job here. Stack the block dozens of times. That is the machine; there is no other secret ingredient.

The proportions are worth seeing once with real numbers. In GPT-3 the vectors are 12,288 numbers wide, the feed-forward step expands each to four times that width and back, and the block repeats 96 times — 175 billion dials in total, most of them living in those unglamorous feed-forward layers, not in attention. Attention decides where to look; the feed-forward layers hold most of what the model knows.

The block's quiet superpower is parallelism. Within a layer, no token waits for the previous one to finish — every position computes at once, so training spreads across thousands of chips. Attention made the model better; parallelism made it affordable. Recurrent networks were not so much out-thought as out-scheduled, and both halves were needed for what followed.

One honest limit is baked into the uniformity. Every token, from 'the' to the decisive figure in a contract, flows through every dial of every block and pays exactly the same compute; the architecture has no way to shrug at easy tokens or linger on hard ones. That rigidity is a standing invitation, and the field keeps accepting it — mixture-of-experts models, further down this road, are one answer.

090

The vision road resumes: with the transformer block in hand, try cutting an image into patches and treating them like words.

Vision transformers

Cut the image into patches and treat them like words.

The recipe is almost rude in its simplicity. Cut a 224 by 224 photograph into patches of sixteen by sixteen pixels — 196 of them — flatten each patch into a vector of 768 numbers, add a positional signal so the model knows which patch came from where, and feed the lot to a standard transformer as if the patches were words in a sentence. No filter slides anywhere. Attention lets any patch consult any other directly, in one hop — a patch of sky asking a patch of wing what it belongs to, with no waiting for stacked layers to widen the view.

The surprise is what it throws away. Convolution's built-in wisdom — nearby pixels matter, patterns repeat across the image — is simply not assumed; beyond the positional tag, the transformer starts knowing nothing about space at all. Everything a convolutional network gets free in its wiring, a vision transformer must rediscover from photographs. On modest datasets it loses to CNNs for exactly that reason: with only ImageNet's million-odd images to learn from, the free wisdom wins.

With enough images, it pulls ahead. The 2020 paper that introduced the design pretrained on a private collection of about 300 million labelled photographs before beating the best convolutional networks on ImageNet — the field's uncomfortable lesson, again, that learned structure beats designed structure once data is plentiful, and that the assumptions which save you data are also the ceiling on what you can learn.

The honest bill is quadratic. Attention compares every patch with every other: 196 patches make about 38,000 pairings; double the image's side and the patches quadruple, so the pairings grow sixteen-fold. High-resolution work — medical scans, satellite tiles — strains this, which is why hybrid and windowed-attention variants exist, quietly re-importing a little of convolution's locality. The pendulum between designed structure and learned structure swings; it has not stopped.

091

Self-attention lets every token see every other — including the future, which ruins the exam when the future is the answer.

Causal masking

Hide the future so the model must predict it rather than read it.

During training the whole sentence sits in the machine at once — including the very words the model is supposed to predict. Left alone, self-attention would let position five read position six, and the exam is void. The causal mask blanks out the future: before the scores are softened into weights, every score from a position to a later one is forced to minus infinity, so softmax hands it e^(−∞) = 0 — a weight of exactly nothing. Every token may consult its past and nothing else.

Laid over the score table, the mask is a triangle of blocked cells above the diagonal, and the panel beside this text shows it at work. That triangle is what turns a transformer into an honestly trained language model: every position becomes its own next-token exercise, all scored in one parallel pass. A fifty-word sentence yields fifty predictions — position ten predicting word eleven from words one to ten, position thirty predicting word thirty-one — none of them cheating.

The failure mode proves the point. Remove the mask and training loss collapses towards zero almost immediately while the model learns nothing usable: 'the 12951 Rajdhani departs Mumbai at 17:00' is trivially completed when '17:00' is already in view. It is copying answers straight off the page, and a perfect score on a copied exam predicts nothing about the real test, where the future genuinely is not there.

The mask has a price. A causally trained model reads everything one-eyed — even at answering time, when the full document sits in front of it, each token's representation ignores what follows. Models trained without the mask, like BERT, see both directions at once and were long the better choice for pure understanding tasks. Generation demanded the blindfold, and the field decided generation was worth it.

092

Under the causal mask, past tokens never change — so why recompute their keys and values for every new token?

KV cache

Remember past keys and values so each new token is cheap.

Token twelve's key and value are the same whether you are predicting token thirteen or token two hundred — the mask guarantees the past never changes, so computing them again is pure waste. Keep them. The cache stores every past position's keys and values; each new token computes only its own query, key and value, appends the pair, and attends over everything stored. The panel beside this text shows the cache growing one column at a time as tokens arrive.

The saved work is easy to price. Without the cache, generating token n means re-encoding all n tokens from the top, so a 1,000-token answer costs 1+2+…+1,000 — about half a million token-passes. With it, each step does one token's worth of new work plus one sweep over the stored past. It is also why the first token of a reply takes noticeably longer than the rest: the whole prompt is being read and cached, and everything after rides on that.

The trade is compute for memory, and the memory is not small. Each token stores a key and a value in every layer: in a 7-billion-parameter model of the Llama 2 shape — 32 layers, vectors 4,096 wide, two bytes a number — that is roughly half a megabyte per token, so a 4,000-token conversation holds about two gigabytes of cache beside the fourteen the weights already occupy.

In a long chat, then, it is the cache, not the weights, that fills the chip — a long context window is largely this memory, priced. The honest caveat is that caching buys speed and nothing else: the model attends over stored keys exactly as it would over fresh ones, so a cached conversation is not remembered any better, and once the window's edge arrives, cached or not, the oldest tokens are simply gone.

093

You can draw the transformer block from memory — now meet the almost insultingly simple objective every large model is trained on.

Next-token prediction

The entire training objective: guess the next piece of text.

Take a transformer, feed it text, and ask one question at every position: what comes next? 'The taxi pulled up to the ___' — the model spreads belief over its whole vocabulary, fifty-odd thousand tokens wide, and the true next token scores it. That is the entire training objective of every large language model. No grammar module, no fact module, no list of skills — one exercise, repeated trillions of times. The panel beside this text is playing it live.

The score is cross-entropy, which here collapses to something small: loss = −log p(truth), the negative log of whatever probability the model gave the token that actually came next. Put 90% on 'kerb' and the loss is near zero; put 1% on it and the loss is steep; be confidently wrong and it is steeper still. Averaged over an enormous corpus, this one number is the whole report card, and gradient descent turns every dial to pull it down. Lesson 2 has you run this loop yourself.

Its poverty is the point. Predicting the next word well forces you to absorb grammar, because grammar changes what comes next; facts, because 'the capital of Karnataka is' has one good continuation; style, arithmetic, even a rough sense of the people doing the writing. Nobody programmed those skills in; they are what the loss landscape demanded on the way down. The objective is narrow — what it takes to do well at it is not.

Hold on to what the objective is not. The model is graded on matching what text tends to say next, not on whether it is true, kind or useful — those words appear nowhere in −log p(truth). A model can score beautifully while fluently continuing nonsense, and much of the rest of this road is about the gap between predicting text and meaning it.

And this one you can do, not just read — Lesson 2 on the panel walks you through training it yourself.

094

Next-token prediction is self-supervised — the answer key is free — which means the homework can be the whole internet.

Pretraining

Months of next-token prediction over an enormous pile of text.

Pretraining is that free homework run at a scale hard to hold in your head: months of next-token prediction over trillions of tokens of web pages, books and code, on thousands of chips turning billions of dials downhill together. Self-supervision is what makes the pile usable — the text is its own answer key, so nobody labels anything, and the only real preparation is janitorial: scraping, deduplicating, filtering out the worst of the web before it is fed in.

The numbers deserve saying plainly. Meta's Llama 3 models were pretrained on about 15 trillion tokens — hundreds of times the text a person could read in a lifetime — and frontier runs burn compute priced in the tens of millions of dollars, drawing megawatts. Underneath, nothing new is happening: the same loop you already know, w ← w − η·∂L/∂w, every dial nudged against its own gradient, just repeated at planetary scale.

The output is not a product but a raw material: a base model that continues text and does little else you would want. Ask it a question and it may reply with three more questions, because online, questions often follow questions. Every chatbot you have used began as one of these. The pretrained model holds nearly all the knowledge; everything that comes after — the instruction-following, the manners — is comparatively brief finishing school.

Two honest limits. The model absorbs the pile as it is, so the web's biases, myths and boilerplate are pressed into the dials alongside its knowledge — filtering helps and does not finish the job. And the pile itself is not bottomless: high-quality public text is being consumed faster than the internet writes it, which is why labs now chase licensed archives, transcripts and synthetic data. The homework was free; more homework is getting expensive.

095

Pretraining costs millions, so labs asked the accountant's question: what exactly does more data, more dials, more compute buy?

Scaling laws

More data, more parameters, more compute — and loss falls predictably.

Train a family of models, varying only size, data and compute, and plot the loss. It falls along an astonishingly clean line — a power law holding across many orders of magnitude, from models that fit on a laptop to models that fill a data centre. The panel beside this text draws it: straight on log-log axes, which is a power law's signature. So before committing millions to a giant run, a lab trains small models for pocket change and reads the big one's loss off the chart.

Written down, the law says loss ≈ A/N^α + B/D^β + E: one falling term for parameter count N, one for dataset size D, and a floor E that no scale removes — the irreducible noise of language itself. The exponents are small, which is the sobering part: each equal step down in loss costs roughly ten times more compute than the last. Progress is predictable, and predictably expensive.

That predictability is why anyone dared build the giants: the result was forecast before the money was spent. It also rewrote the recipe. The 2022 Chinchilla work showed the big models of the day were fed far too little data for their size — a 70-billion-parameter model trained on 1.4 trillion tokens, roughly twenty tokens per parameter, beat much larger rivals — and the field pivoted from ever-bigger to better-fed.

Mind what the chart measures. Scaling laws forecast loss — average next-token error — not abilities, and the mapping from slightly lower loss to what a model can actually do appears nowhere on the graph. A lab can know its next model's loss to two decimal places and still be surprised by what it does, which is exactly where the road goes next.

096

Scaling laws promise the loss falls smoothly — but the abilities riding on that loss do not always arrive smoothly.

Emergent abilities

Skills nobody trained for, appearing once a model is large enough.

Scale a model up and most skills improve smoothly, tracking the falling loss. But some arrive like a switch: smaller models fail at multi-step arithmetic or obscure translation essentially completely, and then, past some size, the ability is simply there. In the GPT-3 experiments, three-digit addition sat near zero across the smaller models of the family and leapt in the largest. Nobody trained arithmetic in on purpose; it surfaced.

The mechanics of the surprise are worth a moment. Loss is an average over every token everywhere, and a rare skill can be improving underneath — better digits, better carrying — while moving that average almost nothing. If the test only counts fully correct answers, all the hidden progress registers as zero until the last piece lands, and then the score jumps from nothing to something at once.

That is also the counter-argument. Emergence is contested as well as celebrated: careful re-analysis showed some famous jumps are artefacts of scoring, all-or-nothing metrics hiding gradual progress underneath, and when partial credit is given — marks for each correct digit — several cliffs flatten back into slopes. Some abilities really were climbing smoothly all along; we were measuring them with a switch.

The unsettling part survives the debate. Whether the underlying progress is smooth or stepped, nobody can yet list, in advance, what the next larger model will and will not be able to do; the scaling chart names its loss and stays silent about its skills. For anyone planning around these systems — a hospital, a bank, a regulator — that forecasting gap, not the philosophy, is the practical fact of emergence.

097

Softmax hands the model a full spread of belief over next tokens; something still has to pick one.

Temperature and sampling

How the model chooses among possible next words, and why it varies.

The picking rule matters more than it looks. Always take the top token — greedy decoding — and the prose goes rigid and loops, the same safe word after the same safe word. Draw in proportion to belief and it breathes. Temperature is the dial between them: every score is divided by T before softmax, p = e^(s/T)/Σ e^(s/T), so low T sharpens the spread towards the favourite and high T flattens it towards a gamble.

Watch the dial work on real numbers. Say the model's beliefs for the next word after 'the auto driver asked for' are 60% 'fifty', 30% 'a', 10% 'directions'. At temperature 1 you sample exactly those odds. At 0.5 the spread sharpens to roughly 78%, 20% and 2% — the favourite pulls away. Push towards zero and 'fifty' becomes certain; push towards 2 and the three drift towards a coin toss among them. Same beliefs, different courage.

In practice the distribution is also trimmed before drawing: top-k sampling keeps only the k likeliest tokens, nucleus sampling keeps the smallest set covering, say, 90% of the probability — both exist to stop the long tail of thousands of barely-possible tokens from occasionally injecting gibberish. This machinery is why the same question can get different answers on different days: chance is deliberately left in, then fenced.

The trade is real. Higher temperature buys variety and ideas at the price of more nonsense; temperature zero buys repeatability, which is what you want for code and arithmetic, not brainstorming. But do not read the dial as a truth knob: temperature zero makes the model consistent, not correct — it will repeat its favourite mistake with perfect reliability. Lesson 2 puts this dial under your fingers.

And this one you can do, not just read — Lesson 2 on the panel walks you through training it yourself.

098

The KV cache remembers every token so far — and that remembering is exactly what a model pays for.

Context window

How much the model can hold in mind at once, and why it costs so much.

Everything the model is currently attending over — your question, the conversation so far, the document you pasted — lives in that cache, and the context window is simply its size limit: how much the model can hold in mind at once. Anything that scrolls out has, for the model, never happened; there is no fainter, older memory behind it. Inside the window, attention lets each new token consult every stored one, and the panel beside this text shows the window filling and the bill climbing together.

The bill has two parts, both growing with length. Memory: the cache stores keys and values for every token — half a megabyte or so per token in a mid-sized model, gigabytes for a long conversation. Work: each new token attends over everything stored, so token 100,000 does a thousand times the looking-back of token 100. That is why long context is both a headline feature and a real cost: windows stretched from about 2,000 tokens in GPT-3 to a million in Gemini 1.5 within a few years, at serious engineering expense.

A million tokens is roughly seven or eight full-length novels, pasted whole. The practical shift is real: a lawyer can drop an entire case file, a developer an entire codebase, and ask questions with nothing summarised away. The window, not the model's knowledge, is what that work rides on — the file was never trained into the dials; it sits in the cache, being attended over.

But a model that can hold a whole book is not guaranteed to use it well. Measured carefully, retrieval is strongest for facts near the window's start and end and sags in the middle — the lost-in-the-middle effect — and attention spread over more can mean attention paid to less. A wide window is capacity, not comprehension; the marketing routinely blurs the two.

099

Sampling picks the plausible next word — plausible, note, not true.

Hallucination

A model trained to sound right will sound right even when it is wrong.

A language model is trained to continue text plausibly — that is the entire objective, all the way down. Truth was never the target; sounding right was, because sounding right is what next-token prediction rewards. Hallucination is what it looks like when those two things part ways: the machine doing exactly what it was built to do, in circumstances where that is not what you wanted.

The mechanism needs no malfunction. Softmax must spread belief over the vocabulary and sum to one whether the model knows the answer or not, and sampling will pick something plausible either way. Ask for a case citation and the model generates one token at a time, exactly as it generates everything: 'Sharma v. State of Maharashtra, 2014' — perfectly formatted, confidently delivered, and possibly assembled from the statistics of how citations look rather than from any case that exists.

This has left the lab. In 2023, lawyers in a New York case were sanctioned after filing a brief whose precedents ChatGPT had invented — six cases, complete with quotes and docket numbers, none real. The style is the trap: false statements arrive in the same fluent, confident voice as true ones, because the voice is what was optimised. Fluency is evidence of training, not of truth.

The honest limits cut both ways. Retrieval, grounding and careful tuning genuinely reduce hallucination; nothing yet eliminates it, and vendors' cures are routinely oversold. And the word itself misleads — it suggests a glitch in an otherwise truthful machine, when there is no truthful machine underneath. Treat a model as a brilliant colleague who never says 'I'm not sure': useful precisely when you check what matters.

100

A transformer runs every token through every dial; at billions of dials, a question becomes unavoidable — must it?

Mixture of experts

Only wake the part of the network you need for this token.

The answer the field settled on is no — most of the model can sleep. A mixture of experts replaces each block's feed-forward network — where most of the dials live — with many parallel copies, the experts, plus a tiny learned router. The router reads each token's vector, scores every expert, and sends the token to only the top one or two: out = Σ gᵢ·Eᵢ(x), where the gate gᵢ is zero for every expert not chosen. For any given token, most of the model does no work at all.

The effect is to split parameter count from compute cost. Mixtral 8x7B carries eight experts per layer and routes each token to two: about 47 billion parameters stored, roughly 13 billion doing work on any token. A model can hold vast knowledge while each token pays for a small slice of it, which is why many frontier models are quietly built this way — DeepSeek's V3 stores 671 billion parameters and activates about 37 billion per token.

The routing is learned like everything else, and nobody tells the experts what to specialise in. Peer inside and the division of labour is real but rarely tidy — experts drift towards punctuation, code, particular languages — decided by whatever split lowered the loss, not by any human taxonomy of subjects. It is less a panel of professors than a sorting habit that happened to work.

The catches are concrete. Routers must be pushed — with an added balancing loss — to spread the work, or a few favourite experts absorb all the traffic while the rest never learn. And sleeping is not free: every expert must sit in memory ready to be woken, so a mixture of experts saves compute, not RAM — Mixtral's 47 billion parameters all need a home even while most of them doze through each token.

101

Inference froze the dials. So how do you change what a frozen model does?

Prompting

Steering a finished model with words alone. No dials move.

You change what it reads. At inference the frozen network computes one fixed function: given everything in the context window, a probability for each possible next token. The dials set the function; the words you type are its input. So prompting is choosing the context the model continues from — and since it learned from a world of documents, the framing you set selects the kind of document it writes. No dials move. The same weights, fed different openings, produce a legal brief, a recipe, or a poem.

That is the whole mechanism behind 'you are an expert lawyer' improving legal answers. The phrase did not unlock a hidden lawyer module; it moved the continuation into a region of the model's experience where careful legal prose lives, because in the training text those words sat near careful legal prose. Formally you are steering p(next token | context) by editing the condition — the only part of the equation you are allowed to touch.

A bank running a UPI support bot uses exactly one frozen model with a written preamble: answer in two short sentences, reply in the language the customer used, and if money left the account but did not arrive, give the auto-reversal timeline before anything else. Change that preamble and tomorrow the same model behaves like a different employee — no retraining, no GPUs, a deployment that costs one edit to a text file.

Prompting is context engineering, not programming — which is both its power and its unreliability. An instruction in the prompt is not a rule; it is one more piece of context, competing with everything else in the window. Reword a request slightly and the answer can swing; paste in a document containing 'ignore your instructions' and the model may weigh those words too, which is the whole problem of prompt injection. You have steered the model. You have not commanded it.

102

Prompting sets the scene with framing; sometimes the clearest framing is a few worked examples.

Few-shot examples

Show the pattern in the prompt instead of training it in.

Sometimes the clearest framing is not a description of the task but the task already done, three times. Show the model three customer reviews labelled positive or negative, then a fourth with the label missing, and it continues the pattern — because the most probable continuation of a list of worked examples is another worked example. No dials move; the pattern lives entirely in the prompt. The panel beside this text is running exactly this: add or remove examples and watch the model's guess for the last item change.

This was GPT-3's headline surprise in 2020: at 175 billion parameters, a model trained only to predict the next token behaves as if it learns from the examples in front of it. The trade named it in-context learning, and nobody explicitly built it — it emerged, because so much of training text consists of patterns, lists, tables and question-answer pairs that a good predictor of text must become a good continuer of patterns.

It is the cheapest specialisation there is. A shop sorting UPI complaint messages — 'wrong number', 'failed but debited', 'fraud' — can skip training entirely: paste five labelled complaints into the prompt and classify the sixth, at the cost of a few hundred extra tokens per call. Going from zero examples to a handful routinely lifts accuracy by double-digit percentage points on tasks like this, which is why 'add examples' is the first fix any practitioner reaches for.

The learning is perishable. It vanishes the moment the prompt does, which is exactly what separates it from fine-tuning: nothing was written into the dials, so nothing persists. It is also touchy — reorder the examples or change their formatting and accuracy can shift by several points — and every example spends context window that must be paid for on every single call, forever. Few-shot rents the skill; fine-tuning buys it.

103

Pretraining gave the model the world's general habits; your problem has specifics the world never wrote down.

Fine-tuning

Take a trained model and keep training it on your own data.

So you keep training. Fine-tuning is the same loop as pretraining — forward pass, loss, w ← w − η·∂L/∂w — run briefly and gently on your own examples instead of the world's. Nothing about the machinery changes; what changes is the data and the caution. The learning rate η is set small, perhaps a tenth of pretraining's, and the run lasts hours rather than months, because you are adjusting an educated model, not raising one from zero. The general skills stay; the specifics of your domain settle in on top.

The economics are the point. The pretrained model cost millions; your fine-tune costs an evening on a rented GPU. A hospital with 20,000 discharge summaries, or a startup with 50,000 Hinglish customer-support chats, can push a general model through two or three passes over that data for a few thousand rupees of compute and get a specialist that speaks their vocabulary — drug names, ticket codes, the way their customers actually type. It is how a modest team gets a specialist model without a lab's budget.

Reading a loss curve matters more here than anywhere, because your dataset is tiny by the model's standards. Watch training loss and validation loss together: with 50,000 examples a large model can start memorising within a single epoch, training loss still falling while validation loss turns upward — the overfitting signature you already know. Practitioners stop early, keep η small, and treat the validation curve as the only honest witness.

The honest failure is catastrophic forgetting. The dials that hold your specifics are the same dials that hold the world's general habits, and every update overwrites a little of both. Tune too hard and the model gains your jargon while losing grammar, general knowledge, or the safety behaviour someone else trained in. Gentle is not a courtesy; it is the whole discipline.

104

Fine-tuning works at all because of a quiet fact worth naming on its own: learning carries.

Transfer learning

Most of what a model learned on other data is still useful on yours.

Learning carries because the early layers of a network learn the world, not the task. A vision network trained on a million ordinary photographs has already learned edges, textures, and shapes; a language model trained on the open web has already learned grammar, names, and how arguments hang together. Those are not photograph skills or web skills — they are world skills, and your problem lives in the same world. Transfer learning is the fact that makes fine-tuning worth doing: most of what a model learned on other data still applies to yours.

The procedure is almost embarrassingly simple. Where training from scratch initialises every weight at random, transfer initialises them at the pretrained values, then fine-tunes. That single change of starting point is the whole trick — the walk downhill begins most of the way down. A scanner reading chest X-rays for tuberculosis can start from a network pretrained on ImageNet's 1.2 million labelled photos and reach useful accuracy with a few thousand labelled scans, where training from zero would need millions it will never have.

It bends the economics of the whole field. Whoever holds the mountain of data trains once; everyone downstream adapts for pocket change, which is why a pretrained model is now the standard starting point and training from zero the exception, reserved for labs holding both the mountain and the electricity bill. One expensive climb, thousands of cheap descents.

The quiet condition: the old world must resemble yours enough for its habits to help rather than mislead. Photograph habits transfer poorly to radar traces or protein structures, and the failure has a name — negative transfer — where the inherited start is worse than a random one. The inheritance is also indiscriminate: a model pretrained on the web carries the web's biases into your clinic or your courtroom, and nobody chose them on the way in.

105

Fine-tuning still turns billions of dials; the matrix multiplications underneath suggest a far smaller patch.

LoRA and adapters

Train a tiny patch instead of the whole model. Fits on a free GPU.

Look at where fine-tuning's cost actually sits: every layer is built on weight matrices, and updating a matrix means storing gradients and optimiser state for each of its entries. LoRA's bet is that the update itself is simple. Freeze the big matrix W and learn two thin ones beside it, B and A, so the layer computes W·x + B·A·x — the product B·A is the patch, and only B and A train. If W is 4096 by 4096, it holds about 16.8 million numbers; with rank r = 8, the thin pair holds 65,536 — under half a percent.

The trade's word for the bet is low-rank: the change a fine-tune makes tends not to need the matrix's full freedom, so a rank-8 patch captures most of what a full update would learn. Across a 7-billion-parameter model the trainable dials shrink to a few million, and with the base weights frozen the whole job fits on a free Colab GPU with 16 gigabytes of memory. Someone with no budget specialises, overnight, a model that cost millions to pretrain.

Because the base never changes, adapters swap like lenses. One frozen model, plus a patch of a few tens of megabytes for legal drafting, another for Hindi medical triage, another for your company's tone — loaded per request, or merged into the weights when you settle. A hosting service can serve hundreds of customers' fine-tunes from a single copy of the base model, which full fine-tuning could never afford.

The limit is the bet itself. When the change you need is not simple — a new language from scratch, deep new knowledge, a capability the base never had — a rank-8 patch cannot hold it, and quality quietly caps below a full fine-tune. LoRA adjusts a model's manner and vocabulary beautifully; it does not add a new floor to the building.

106

Mixed precision shrank the numbers to train faster; now shrink them further, to run on hardware you actually own.

Quantisation

Store the weights in fewer bits so the model fits on the hardware you have.

The mechanism is a ruler. Pick a scale s for a group of weights, then store each weight as its nearest whole step: q = round(w/s), reconstructed later as w ≈ s·q. In 8-bit, q may take 256 values; in 4-bit, just 16. The dials stop being finely machined and become notched — each snapped to the nearest allowed value — and the snap error is tolerable precisely because mixed precision already proved the network survives coarse numbers.

The prize is reach. A 7-billion-parameter model stored in fp16 needs about 14 gigabytes — a data-centre card. In 8-bit it is 7; in 4-bit, around 4, which fits beside your browser on an ordinary laptop, and a 3-billion model quantised the same way fits in a phone. Halve the bits and you halve the memory — and since generating text is mostly the work of hauling weights from memory, halving them speeds it up too.

This is what makes local AI real rather than a demo. The llama.cpp ecosystem runs 4-bit models on laptops with no GPU at all, and the loss is startlingly small — a well-quantised 4-bit model typically scores within a point or two of the original on standard benchmarks. The running surprise of recent years is how far you can push before quality crumbles.

It does crumble, and unevenly. Benchmarks average over easy ground, and the damage pools in the corners: rare languages, long chains of arithmetic, precise recall of names and numbers. A handful of outlier weights matter far more than the rest, and naive rounding of those can wreck a model — modern schemes measure which weights are sensitive and spend their few bits protecting them. Below 4 bits, even the tricks strain.

107

Prompting showed that the model answers from its context — so put the facts you need directly into it.

Retrieval-augmented generation

Look the facts up and put them in the prompt, instead of training them in.

So put them there yourself, at the moment of the question. Retrieval-augmented generation looks the facts up instead of hoping the model memorised them: your question is embedded, the closest passages from your own documents are fetched, and they are pasted into the prompt above the question. The model answers from evidence sitting right there in its context, not from distant training — the same trick as prompting, with the framing assembled by machinery instead of by hand.

The pipeline is three honest steps. Split your documents into passages and embed each one, once, into a vector. At question time, embed the query the same way, score every passage by dot product against it, and keep the top few — five passages of two hundred words costs about a thousand tokens of context. Then generate, with an instruction to answer from the passages and cite them, so the answer arrives with receipts.

Ask a railway helpdesk bot 'train cancelled by railways, ticket booked online — do I get my money back?' and retrieval fetches the refund rule: for trains the railways cancel, the fare returns automatically to the account that paid. The model's job shrinks from knowing Indian Railways policy to reading three paragraphs of it aloud, accurately — a far easier task, and one you can audit clause by clause.

This attacks two weaknesses at once: knowledge that went stale the day pretraining ended, and private material the model never saw. But the ceiling moves rather than vanishes. Ask something the retrieval step failed to fetch — wrong words, missing document, stale index — and the fluent voice is back on its own, answering anyway. A RAG system is only as truthful as its worst retrieval, and a citation can look crisp while pointing at a passage the model half-ignored.

108

Retrieval-augmented generation said 'fetch the closest passages' as if that were easy — here is the machinery that makes it fast.

Vector search

Finding the passage that means the same thing, not the one with the same words.

The machinery starts from what you already hold: embed every passage as a vector, once, and store the lot. A query arrives, is embedded the same way, and each stored vector is scored against it with a dot product — q·d, large when the two directions agree. Return the highest scorers. That is the whole idea, and it is why 'how do I get my money back' finds the refund policy though the two share barely a word: they were near neighbours in meaning-space before the query ever arrived. The panel beside this text is running exactly this search.

The engineering problem is scale. A passage embedding might hold 768 numbers; a billion of them is roughly three terabytes to scan for every single query, one dot product at a time — far too slow. Vector databases build geometric indexes instead: structures like HNSW link each vector to its near neighbours, so a search hops greedily from an entry point towards the query, touching a few thousand vectors instead of a billion, and answers in milliseconds.

The price is the word approximate. A hop-based search can miss the true closest match — indexes are tuned to find it perhaps 95 to 99 times in a hundred, and pushing recall higher costs speed and memory. Approximate, fast, and close enough: that trade runs semantic search, recommendation systems, and every RAG pipeline in production, where a near-best passage in twenty milliseconds beats the perfect one in twenty seconds.

The deeper limit sits in the vectors themselves. Nearest in embedding-space means nearest by the embedding model's notion of meaning, learned from general text. It can rank 'cancel my ticket' close to 'book my ticket' — same vocabulary, opposite intent — and it is blind to what your domain considers decisive, like the difference between two drug dosages. The index faithfully returns the closest vectors; whether closest means most relevant is a promise the embedding may not keep.

109

Transfer learning moves knowledge between problems; it also moves between models — from a giant to a student you can afford.

Distillation

Train a small model to copy a big one.

The move is teacher and student. Run the giant on your inputs and record not just its answers but its full distribution over answers — the soft spread of belief. 'Almost certainly cat, faintly fox' teaches far more than the bare label 'cat', because the teacher's doubts encode how the options relate. The student, a much smaller network, trains to match that distribution: its loss is the cross-entropy between its own probabilities and the teacher's, rather than against hard labels.

One dial makes the doubts legible: temperature. Divide the scores before the softmax — p = e^(z/T) / Σ e^(z/T) — and a T above 1 flattens the distribution, raising the faint options into view so the student can see them. Hinton's group named the method in 2015, and the phrasing stuck: the small model is not shown what the big one concluded so much as how it weighs the alternatives.

It is transfer learning between models rather than tasks: a giant's expensive experience compressed into something cheap to run. DistilBERT is the textbook number — 40 percent fewer parameters than BERT, around 60 percent faster, keeping roughly 97 percent of its benchmark performance — and the same recipe is how flagship abilities reach phones and free tiers. Small distilled models routinely embarrass models several times their size, because they inherited judgement they could never have earned from their data alone.

The ceiling is the teacher. A student trained to copy inherits the teacher's mistakes with the same fidelity as its skills — biases, blind spots, and confident errors all transfer — and it cannot rise above the source. The copy is lossy too: the further you shrink, the more the subtle capabilities go first, reasoning before recall. Distillation compresses judgement; it does not create any.

110

Gradient descent always had a right answer to walk towards; now take the answer key away entirely.

Reinforcement learning

No answer key. Just consequences, and a score you want to raise.

Now nothing marks your homework. The agent acts, the world responds, and a score arrives — often late. Nobody says which move was right; the agent must work out, from consequences alone, which of its choices deserved the credit. The loop is bare: observe the situation, choose an action, collect a reward, repeat — and the goal is not to be correct at any step, but to make the total reward over time as large as possible.

The maths says exactly that. The agent maximises the return G = r₁ + γ·r₂ + γ²·r₃ + …, the sum of rewards to come, with each future reward shrunk by a discount γ just under 1 — say 0.99 — so that sooner counts a little more than later. There is no loss shrinking towards a truth, because there is no truth on file; gradient descent still runs underneath, but what it climbs is expected score.

It is how you train anything that acts: game players, robot arms, recommendation loops. AlphaGo is the landmark — trained largely by playing itself millions of times, rewarded only +1 for a win and −1 for a loss, it beat Lee Sedol in 2016 with moves no human had taught it. The panel beside this text runs the same idea in miniature: an agent blundering through a small world, its behaviour visibly firming as reward accumulates.

Its curse is exactly its freedom. With rewards sparse and delayed, figuring out which action mattered — the credit assignment problem — is brutally hard, and the standard answers are data-hungry: agents commonly need millions of tries to learn what a person gets in dozens. RL remains the most temperamental branch of the field; the same code, reseeded, can master a task on Monday and fail it flat on Tuesday.

111

Reinforcement learning runs on a score you want to raise — time to look hard at who writes that score.

Reward

The number the agent is trying to maximise — and the thing it will exploit.

The score is a function someone wrote: r(situation, action), a number handed to the agent at each step, and the agent's entire purpose is to make the sum of those numbers large. That sentence is the most dangerous line of the whole design. The agent will not pursue what you meant. It will pursue what you wrote — and every gap between the two is an open invitation.

The canon example is a boat. In 2016, OpenAI trained an agent on the racing game CoastRunners, rewarding it with the game's own points. The points came from hitting targets along the course, so the agent learned to spin in a lagoon collecting respawning power-ups — on fire, colliding with walls, never finishing the race — while the score climbed past what honest racing earned. The reward was satisfied. The race was not.

Nothing about this is exotic. Reward a food-delivery dispatcher for average delivery time and it learns to refuse far-away orders; reward a call-centre model for calls resolved per hour and it learns to resolve them by hanging up. In each case the number was a proxy for a purpose, and the optimiser found the gap — faster and more shamelessly than any employee gaming a bonus scheme, because it has no idea a purpose exists.

Every reward is a wish made to a very literal genie, and specifying wishes well is a discipline in itself: shaping terms, penalties, human oversight of what the score misses. The honest admission is that no fix is final. A reward is always smaller than the intention it stands for, so somewhere in every design the gap survives — the craft is making it too expensive to find.

112

An agent chasing reward meets a gambler's dilemma before it meets anything else.

Explore or exploit

Take the known-good option, or gamble on finding better.

Before it can chase reward, the agent must decide where reward even lives. You know a restaurant that is reliably good; there is a new one across the street. Eat where you know and you may never find better; keep trying new places and you eat badly often. Every reinforcement-learning agent faces this at every step: exploit the best action found so far, or explore for a better one — and the score it is maximising gives no direct instruction about which.

Neither pure strategy works. Pure exploitation locks in the first decent habit forever; pure exploration never cashes in what it learns. The standard compromise is almost insultingly simple — ε-greedy: with a small probability ε, say 0.1, act at random; otherwise take the best-known action. Start ε high and decay it, and you get the practical shape of a well-lived learning curve: explore hard early, settle down late. The panel beside this text runs a row of slot machines under exactly this rule — change ε and watch what the agent discovers, and what it forfeits.

The tension is fundamental, not a nuisance, and it prices real decisions. An ad system must burn some impressions on unproven ads to learn their worth; a streaming service must show you some titles it is unsure about; adaptive clinical trials shift patients towards the treatment doing better while still dosing some on the alternative, precisely because certainty is not free. The cost of exploring is paid in real clicks, real evenings, real patients.

The honest limit is that ε-greedy explores stupidly — its gambles are uniformly random, as likely to retry a known disaster as to probe a promising unknown, and cleverer schemes that explore where uncertainty is highest cost more machinery. And in the world, exploration is not always allowed to be cheap: a hospital cannot try treatments the way an agent tries slot machines. Some arms of the bandit are not yours to pull.

113

Reinforcement learning keeps saying 'the agent chooses' — the policy is the rule doing the choosing.

Policy

The rule the agent follows for choosing what to do next.

Situation in, action out: the policy is that mapping, written down. In a tiny world it can be a lookup table — this square, move left. In a serious one it is a network, and the standard form is a distribution: π(a|s), the probability of taking action a in situation s. A network playing Breakout takes the screen's pixels in and puts a spread over joystick moves out. Which makes it dials, and dials can be trained.

That reframes the whole problem. Reinforcement learning is not memorising right answers, because there are none — it is turning the policy's dials so the behaviour they produce collects more reward. Gradient descent survives the move; what changes is what the gradient points at. Not a smaller error against a label, but a bigger score from the world.

The classic update makes the logic explicit. After an episode, nudge every dial in the direction that makes the actions you actually took more probable, scaled by how well things went: w ← w + η·G·∂log π(a|s)/∂w. When the return G was high, the moves that produced it become more likely; when G was negative, the same arithmetic pushes them away. No teacher marks any individual action — the score of the whole episode stands in for all of them.

That substitution is also the weakness. One score smeared over hundreds of actions is a noisy signal: brilliant moves inside a losing game get punished, blunders inside a winning one get reinforced, and only averaging over many episodes washes the injustice out — one reason RL eats so much experience. And a trained policy is a network's reflexes, not a rulebook: it cannot easily tell you why it swerved, only that swerving used to score.

114

You can fine-tune on examples and chase a reward; combine the two, and a text-predictor learns manners.

Learning from human preference

People rank two answers; the model learns which kind to give.

The combination works in three moves. First, collect judgements: show people pairs of the model's answers to the same prompt and ask which is better — no formula for 'helpful' required, only a choice. Second, train a reward model, a separate network that takes an answer and outputs a score, tuned so the preferred answer of each pair scores higher. Third, fine-tune the original model with reinforcement learning against that learned score, nudging it towards the kind of answer people picked.

The reward model's training has a clean form: given scores r_A and r_B for two answers, the probability that a person prefers A is modelled as p = e^(r_A) / (e^(r_A) + e^(r_B)), and the network learns to make its scores match the recorded choices. The tuning stage adds one crucial leash — a penalty for drifting too far from the original model — without which the policy happily degenerates into whatever gibberish the reward model overrates.

This is the step that turned raw text-predictors into usable assistants. InstructGPT, in 2022, showed that a model tuned this way on a few tens of thousands of human comparisons was preferred by users over a raw model a hundred times its size — and the difference in manner between a base model and a chatbot you can actually talk to is mostly this.

Its limit is inherited from its source. The model learns what evaluators prefer, judged in minutes, and that includes their blind spots: confident answers over hedged ones, agreeable answers over corrections, plausible fluency over checkable truth. The polished manner is real, but answers that please are not always answers that are right — RLHF optimises for the reader's approval, and approval is one more proxy.

115

RLHF teaches a model what people prefer — the harder question is whether that is what anyone intended.

Alignment

Making a capable model actually do what was intended.

Intended is the harder word. Alignment is the gap between capable and intended: a model can be superb at its objective while the objective itself points slightly away from what anyone wanted. RLHF closes part of the gap — the model learns what people prefer — but preference is a proxy, judged in minutes by evaluators who cannot check everything, and a proxy pursued hard enough drifts from the purpose it stood for. You met this as the reward's literal genie; alignment is the same problem at the scale of everything the model does.

The drift has a measurable face: sycophancy. A model can learn to be agreeable, confident, and flattering, because those score well with human judges, without becoming more truthful — tell it your wrong answer first and its agreement rate climbs. Nobody trained deceit in; the training simply rewarded approval, and approval parts company with accuracy exactly where the user cannot tell the difference, which is where help was needed most.

The stakes scale with capability and autonomy. A misaligned spam filter wastes minutes; a misaligned model screening loan applications or triaging patients pursues its objective quietly, at speed, and at scale, and its errors compound before anyone reads a log. The working response is layered rather than solved: better preference data, models critiquing models, red teams paid to break behaviour, interpretability work trying to read intentions off the dials — each a partial patch, none a proof.

Nobody has a finished answer. Alignment is an open research problem, and treating it as solved is itself one of the failure modes — a system certified as aligned invites exactly the trust that makes its remaining misalignment expensive. The honest position is narrower: aligned, so far, on the behaviours we knew to test, and silent everywhere else.

116

The reward is what you wrote, not what you meant — and optimisers are relentless about the difference.

Reward hacking

Any measure that becomes a target stops being a good measure.

Relentless is the right word, because the optimiser searches the rule as written, with no picture of the purpose behind it. Reward hacking is the agent's oldest move: satisfy the number while defeating its point. The spinning boat from Reward is the canon example — score climbing, race never finished — and it has siblings everywhere: simulated walkers rewarded for covering distance have learned to grow tall and fall over. Metres gained, walking betrayed.

Goodhart's law said it first about people — any measure that becomes a target stops being a good measure — and an optimiser finds the holes faster and more shamelessly than any bureaucrat, because it never shares the designer's unstated assumptions about what obviously was not meant. The panel beside this text lets you play the designer: set a reward for a small agent and watch it discover the reading you did not intend.

The lesson outgrows RL, because the mechanism needs only a proxy and pressure. Exam scores stand in for understanding, so coaching optimises the scores; engagement stands in for value, so feeds optimise outrage; average call-handling time stands in for service, so agents learn to hang up. Whenever anything is trained against a proxy — machine, employee, or institution — expect the proxy to be gamed, in proportion to how hard it is pushed.

The trap in the lesson is thinking a better-written reward escapes it. Every patch narrows one hole and defines a fresh boundary to probe; whoever fixed the boat's score would meet the next exploit, not the end of exploits. What actually helps is defence in depth — several measures at once, human spot-checks, penalties for weirdness — and the humility to treat a rising score as a claim to investigate, never as proof the intention was met.

117

The validation set gives you a score — but which score? The wrong one will hide your worst failures inside a flattering number.

Choosing a metric

Accuracy, precision, recall — and why the wrong one hides your failure.

Accuracy is the obvious score: accuracy = correct guesses ÷ total guesses. And it can lie. A screening model that says 'healthy' to everyone is 99% accurate when the illness touches one person in a hundred — you met this trap with class imbalance. The flattering number hides the only failures that matter. So practitioners split the one question into three, and the split is where the honesty lives.

Precision asks: of all the alarms you raised, how many were real? precision = true alarms ÷ all alarms raised. Recall asks: of all the real cases out there, how many did you catch? recall = cases caught ÷ all real cases. Run the numbers on a screen of 10,000 people where 100 are actually ill. The model flags 120; 80 of those are right. Precision is 80/120, about 0.67. Recall is 80/100, 0.80 — it missed a fifth of the sick. Accuracy? The 9,860 healthy people it correctly cleared push it to 99.4%. Same model, three verdicts.

The choice between them is a value judgement wearing a maths costume. A spam filter should prize precision — losing a real email to the junk folder hurts more than letting one spam through. A cancer screen should prize recall — a false alarm costs a second test, a miss costs a life. No metric is simply correct; there is only the cost of each kind of mistake, and you must decide whose pain counts. Choose the metric before you look at the scores, because afterwards you will choose the one that flatters.

Two warnings. Combined scores like F1 = 2·precision·recall ÷ (precision + recall) fold the trade-off back into one convenient number — and hide exactly the judgement you just made. And the previous stop follows you here: any metric you optimise hard enough stops measuring what you meant. A team rewarded purely for recall will simply flag everyone; reward hacking is not only a model's vice. The metric is an instrument reading, not the thing itself.

118

One metric grades one slice of behaviour; before shipping you need the whole verdict — does this thing actually work?

Evaluation

Deciding whether it works, before your users decide for you.

Evaluation is the dress rehearsal before your users become the test set. You already hold the machinery — held-out data, an honestly chosen metric — and evaluation is running it deliberately, as a procedure. Slice the validation set: typical cases, rare ones, the ugly inputs people actually type. Score each slice separately, because an average is where failures go to hide. Then compare against the boring baseline you claim to beat — yesterday's model, or a simple rule of thumb.

Concreteness helps. A model reading UPI transactions for fraud might score 98% overall and still be blind to one slice — say, first-time payees at night, where the fraud actually lives. A slice-by-slice table reads 98, 97, 99, 61 — and the 61 is the launch decision. Evaluation is also about the floor, not just the average: what is the worst thing this model does, and how often? One catastrophic slice can sink a product that aces the mean.

For models that write open-ended text this gets genuinely hard: there is no single right answer to score against. So people grade samples by hand against a rubric, or ask one model to judge another's answers — cheaper, and it inherits the judge's own blind spots. Public benchmarks let you compare models, but they decay: their questions leak onto the internet and into training data, so a model can ace an exam it has effectively already seen.

That is the honest limit of the whole enterprise: an evaluation is a photograph of behaviour on the inputs you thought to try. Users will type things you did not think to try, in spellings you did not imagine, with intentions you did not model. High scores age; scepticism does not. The best teams treat evaluation as a living suite — every failure found in the wild becomes a new test case, so the rehearsal keeps up with the play.

119

Inference froze the dials; now someone is waiting on the other end, and the wait has two very different measurements.

Latency and throughput

How fast one answer arrives, versus how many you can serve at once.

Latency is how long one user waits for one answer; throughput is how many answers you produce per second. They are not the same dial, and improving one often worsens the other. A bus moves more people per hour than a taxi — better throughput — but the taxi gets one person there sooner. Serving a model is choosing, constantly, between the bus and the taxi.

The two are tied together by a queue. In a steady system, requests in flight = arrival rate × latency — so a service that takes 200 ms per answer and holds 50 requests in flight is clearing 250 answers a second. Push arrivals past what the hardware clears and the queue grows without bound; latency is then no longer your model's speed but the length of the line in front of it. Most 'slow model' complaints are actually queue complaints.

Users feel latency in their bodies: past about a tenth of a second an interface stops feeling instant, and past a few seconds they leave. It is why chatbots stream their answers word by word — the full reply takes just as long, but the first word arrives in under a second, and the wait stops feeling like a wait. Two numbers matter to a streaming chatbot: time to first token, and tokens per second after it. Perceived latency is a design material, not just a number.

The number that misleads is the average. Report a mean latency of 300 ms and you can still be delivering two full seconds to the slowest one per cent — the p99 — and at a million requests a day that is ten thousand miserable waits. Tail latency is where batching pauses, memory housekeeping and unusually long prompts hide. Measure the percentiles, and remember that every trick that fattens throughput tends to fatten the tail too.

120

Every answer occupies expensive hardware for the length of its latency — which means every answer has a price.

What it costs to run

Working out the price of an answer before you offer it to a million people.

An answer has a price because it occupies expensive hardware for a slice of time. A chip is rented by the hour; each request meters off its share of that hour. The arithmetic is short: cost per answer ≈ hourly rent ÷ answers per hour. Rent a serving GPU for roughly ₹250 an hour and clear 10,000 answers in that hour, and each answer costs about 2.5 paise. Clear only 500 — poor batching, long prompts — and it costs 50 paise. Same chip, twenty times the price.

This is why model APIs bill like a taxi's fare chart — a rate per token read and a rate per token written, with writing priced several times higher because generation is the slow part. For mid-sized models the rates sit in the region of tens to a few hundred rupees per million tokens. A 500-token answer at ₹250 per million output tokens is about 12 paise before you count the prompt — and a long context costs real money precisely because the model must read every token of it, every single time.

You already know the split from training against inference: training is the enormous one-time cost, inference the small cost paid on every use — and at scale the small one wins. Multiply 12 paise per answer by a million users making five requests a day and you are spending ₹6 lakh daily; the fraction of a rupee has become the business. Much of the field's craft — quantisation's fewer bits, LoRA's tiny patches, distillation's smaller copies — is really engineering aimed at shrinking this one number.

The per-answer figure can still mislead. It assumes the hardware is busy; a GPU rented round the clock to serve traffic that only surges at lunchtime bills you for the idle hours too, so the real price of an answer depends on how flat your demand is. And a price of zero on the user's screen never means zero — it means someone upstream is paying, usually at a loss, usually temporarily. Work out the fare before you offer the ride to a million people.

121

Serving one request at a time leaves the hardware nearly idle — the chips would rather eat a crowd, and latency pays for it.

Batching and serving

Serve many requests at once, because the hardware prefers it.

The chips that run models are built to multiply big grids of numbers, and they have a strange property: pushing thirty-two requests through the dials together costs barely more time than pushing one. The reason is that generating a token is mostly a memory errand. Every weight must be fetched from the chip's memory to its arithmetic units, and the fetching, not the multiplying, sets the pace. Fetch once, and the same weights can serve one request or a crowd.

Put numbers on it. A 7-billion-parameter model in fp16 is about 14 GB of weights. A serving chip streams memory at roughly 2 TB a second, so one full read of the dials takes around 7 milliseconds — and that read happens for every generated token. Serve one user and you get one token per read; serve a batch of thirty-two and you get thirty-two tokens for nearly the same 7 ms. A single request leaves most of the machine idle; the batch is how you stop paying for silence.

The catch is that a batch must wait to fill, and waiting is latency — the bus again, holding at the stop until enough passengers board. Serving is the art of that trade: how long to hold, how large a batch, at what point a fuller machine makes every individual user slower. Modern language-model servers refine the trick with continuous batching — sliding requests in and out mid-flight as answers finish at different lengths, so a short question never waits for a long essay to end.

Batching flatters throughput and quietly taxes the tail. The user who arrives just after a batch departs waits for the next one; under bursty traffic, batches leave half-empty and the economics sag. And the crowd shares one pool of memory — every request's context claims its slice, so big batches and long conversations fight over the same gigabytes. The hardware prefers a crowd; the person at the back of it may not.

122

Your evaluation passed on the world as it was — and the world has already moved on.

Drift and monitoring

The world changes after you ship; the model does not.

Your model froze the day training stopped; the world did not. Prices rise, slang mutates, fraudsters study your defences and adapt, the taxi company revises its rate per km — and the dials still encode the old world. Incoming data slowly stops resembling what the model learned from, and its answers quietly decay. The failure has two flavours: sometimes the inputs shift, and sometimes the inputs look the same but the right answers change underneath them — a fare model's odometer readings are unchanged, but the tariff is not.

Monitoring is the countermeasure, and it is a procedure, not a vibe. Log every input and every prediction. Compare this week's input distributions against training's — average fare, share of night rides, vocabulary of incoming messages — and raise an alarm when a histogram moves. Track live outcomes wherever the world grants them: the fraud that gets reported, the fare the customer actually paid. And keep replaying your frozen evaluation set on schedule — a fixed exam passed by a fixed model should score the same forever, so when live traffic scores differently, the gap is drift, measured.

The clearest demonstration came in 2020, when lockdowns broke nearly every deployed demand forecast at once — no training set contained a spring in which traffic simply vanished. Smaller versions happen constantly: a monsoon pattern shifts, UPI adds a new payment flow, a phone camera update changes every image a vision model receives. Nothing errored. The models kept answering with full confidence, which is the dangerous part — confidence is not recalibrated by a world the model cannot see.

Monitoring has an honest gap of its own: the truth arrives late. You learn a loan went bad months after the model approved it, so outcome dashboards trail the damage. Input-drift alarms fire earlier but cannot say whether accuracy actually fell — inputs can shift harmlessly. Shipping a model is therefore not an ending. You have signed up to watch, and to retrain, on a schedule the world sets rather than the one you planned.

123

Checkpoints preserve the dials; reproducibility preserves everything that produced them — code, data, and every roll of the dice.

Reproducibility

Seeds, versions, and data snapshots — or your result was luck.

Run the same training twice and you can get two different models: the dials start at random values, the data arrives in shuffled order, dropout flips random coins on every step. Reproducibility is pinning all of it — the random seed that determines every roll, the exact version of the code, the exact snapshot of the data, the versions of every library underneath. A checkpoint preserves where a run ended; this preserves the ability to make the run happen again.

Even the seed is not the whole story. Chips add floating-point numbers in whichever order finishes fastest, and floating-point addition is not associative — (a + b) + c can differ from a + (b + c) in the last digit. Those last digits feed gradients, gradients move dials, and over a million steps two identically seeded launches can still drift apart. Full determinism can be forced, at a speed cost; most teams settle for statistical reproducibility — same recipe, same score to within noise.

Here is why the pedantry pays. Your new trick lifts validation accuracy from 91.2% to 91.6% — but retrain the old model with a different seed and it lands anywhere between 90.9% and 91.7%. The improvement is inside the dice. The honest procedure is to run several seeds of both recipes and compare the spreads, not the single best numbers. Without that, a good number proves nothing: maybe your change helped, maybe the dice were kind.

Machine learning has its own replication problem — published results that nobody, sometimes not even the authors, could produce again, because a data file changed or a dependency silently updated. The habit that prevents it is unglamorous, like keeping receipts: log the seed, tag the code, freeze the data, record the library versions, alongside every checkpoint you save. It is also the entire difference between an experiment and an anecdote.

124

A convolutional network can name what a photograph shows; a machine moving through the world needs that from raw sensors, continuously.

Perception

Turning raw sensor readings into a usable picture of what is out there.

The demand is not one answer to one image but a picture of the world, refreshed many times a second. From a stream of raw readings — pixels from cameras, echoes from radar, laser returns from LiDAR — perception builds a running description of the scene: what is out there, where it is, and what it is doing. The convolutional machinery underneath is the same; the job wrapped around it is new.

The procedure has two beats. Within each frame: find the objects, name them, place them. Across frames: match this frame's detections to last frame's objects, so a pedestrian is not rediscovered thirty times a second but tracked as one person with a position, a velocity, and a short predicted future. A car's perception stack runs this whole cycle at roughly 30 hertz — around 33 milliseconds per pass — because the street will not wait for a slower answer.

Put it to work on an Indian junction and the difficulty shows itself. A camera frame holds an auto-rickshaw half-hidden behind a bus, a cyclist cutting a diagonal, a cow that no benchmark dataset centred and lit. Benchmark photos are framed, lit, and clean; sensor streams have glare, monsoon rain on the lens, motion blur, and objects overlapping objects. The gap between a benchmark and a windscreen is the whole difficulty.

And every mistake flows downstream. A machine cannot decide well about a world it has misread: a missed cyclist never enters the braking calculation, a phantom obstacle triggers a phantom swerve. That is why perception is the load-bearing floor of everything built above it — planning, control, and safety all inherit its errors — and why the honest measure of a perception system is not its average frame but its worst one.

125

Convolutional filters find a pattern anywhere in an image — and the first pattern anyone paid to find was handwriting.

Reading text from images

Finding where the writing is, then working out what it says. The oldest paying job in computer vision.

LeCun's 1989 network read handwritten digits on real cheques for real banks, which makes reading text from images the oldest paying work in computer vision. The job splits cleanly in two. Finding the writing is vision: convolutional filters sweep the image and propose boxes around words, exactly as they find any pattern anywhere. Reading it is sequence prediction: within each box, pixels become characters, one after another, in order.

The reading half works like this. The network slides across the word image and, for each thin vertical slice, outputs a probability for every character plus a blank. A decoding step then collapses the stream — repeated letters merge, blanks vanish — so slices reading 'TT-A-XX-II' become 'TAXI'. Because the model is trained on whole words, it also absorbs the habits of language: an ill-formed scrawl after a 'q' is read as 'u', because English says so.

It runs your life more than you notice. Passport gates read the machine-readable zone; number-plate cameras read registrations at motorway speed; UPI apps read the code on a shop's card; a receipt app reads the taxi bill from lesson 1 — ₹247, distance 12.4 km — and files it. Modern systems clear printed English at well above 99 percent character accuracy, which is why the paperwork of whole industries has quietly become photographs.

The honest failure is confident misreading. A '1' and a '7' in an unfamiliar hand, a faded '₹500' on a crumpled receipt, a curved shop sign in low light — the model still outputs its best guess, with no built-in way of saying 'illegible'. On a cheque amount, a wrong digit delivered with high confidence is worse than a refusal, which is why serious pipelines route low-confidence reads back to a human.

126

Convolutional networks say a dog is somewhere in the picture; a car braking for a pedestrian needs to know exactly where.

Object detection

Not just what is in the picture, but exactly where, with a box around it.

The network reads the image once and outputs a set of boxes, each carrying three things: a class, a confidence, and four coordinates — centre x, centre y, width, height. Underneath, the convolutional layers do their usual pattern-finding; the new part is extra outputs trained to predict those coordinates alongside the label. The loss simply adds the pieces: how wrong the class was, plus how far the box edges sit from the hand-drawn truth.

Two small mechanisms tidy the output. Box quality is scored by overlap: IoU = area of overlap ÷ area of union, so a perfect box scores 1 and a wild one near 0. And because the network proposes many boxes around each object, non-maximum suppression keeps only the most confident box and deletes its heavily overlapping rivals — one pedestrian, one box.

Speed is what made detection matter. Early systems cut out thousands of candidate regions and classified each in turn, taking many seconds per image; the breakthrough generation — YOLO ran at 45 frames per second in 2016 — predicted every box in a single pass. That is the difference between analysing a photograph and watching a street, and it is what lets a car braking for a pedestrian know exactly where, in time to matter.

The honest catch is the confidence threshold. Every box comes with a score, and someone must choose the cut-off. Set it low and you drown in false boxes — the car brakes for shadows. Set it high and you miss the child behind the parked car. There is no threshold that gives you both, only a dial that trades one failure for the other, tuned to which mistake costs more.

127

A detection box around a pedestrian is still mostly road — some decisions need the answer pixel by pixel.

Segmentation

Labelling every single pixel — road, person, sky — instead of the picture as a whole.

A box is a rough sack. Segmentation empties it and labels every single pixel: this one is person, this one kerb, this one sky. The output is not a list of boxes but a second image, painted in classes, its boundaries following the true outlines of things — the gap under a lorry, the arm raised away from the body, the exact curve of the kerb.

The mechanism is classification multiplied. The network first shrinks the image through convolutional layers to grasp context — what neighbourhood each pixel lives in — then upsamples back to full size, with skip connections carrying the fine edges across so boundaries stay sharp. Each output pixel gets its own probability over classes and its own cross-entropy loss, the same loss as ordinary classification, just run half a million times per image on a 1024 by 512 frame.

That precision is what the serious work needs: the exact edge of a tumour on a scan, the exact drivable surface ahead of a car, the exact silhouette that lets your phone lift you cleanly out of a background. A box around a tumour is useless to a surgeon; the boundary is the answer. And the idea comes in two strengths — semantic segmentation paints classes, while instance segmentation also separates this person from that one, so a crowd becomes countable.

The price is paid upstream. Every training pixel needs a truth to learn from, and hand-labelling one street scene — tracing every car, railing, and leaf against the sky — can take a person over an hour. That makes segmentation datasets some of the most expensive in the field, and it is why labs lean on synthetic scenes and coarser labels, accepting a little wrongness to afford any labels at all.

128

Perception has painted the scene, but a camera image is flat — and the world stubbornly is not.

Judging distance

Working out how far away things are, from flat images or from time-of-flight sensors.

Machines recover the missing dimension in two ways. Measure it: fire light at the scene and time the reflection, as LiDAR does. Distance = speed of light × time ÷ 2 — the division because the pulse travels out and back — so an echo returning after 200 nanoseconds means the surface is 30 metres away. The clock doing this must resolve billionths of a second, which is much of what you are paying for in a LiDAR unit.

Or infer it, the way your own eyes do. Two cameras a hand's width apart see slightly shifted views, and depth = focal length × baseline ÷ disparity: the nearer the object, the larger its shift between the two images, so measuring the shift yields the distance. One camera alone can lean on learned cues instead — things shrink with distance, textures compress, far hills go hazy — the habits of perspective absorbed from millions of photographs.

A network trained on those one-eyed cues can guess a full depth map from a single photograph, which is how a mid-range phone blurs the background of a portrait without any special hardware: it estimates depth per pixel, then softens whatever it judges far. Remarkable — and it remains a guess, an inference about how scenes usually behave rather than a measurement of this one.

The failures follow from that. Mirrors and puddles show a scene at the wrong depth; a printed photograph of a road held up to the lens contains every learned cue and no actual road. That is why machines that must never be wrong about distance tend to measure as well as infer — LiDAR and radar to anchor the truth, learned depth to fill in the detail between the beams.

129

Distance says how far things are; the machine also needs to know which way they — and it — are facing.

Pose and orientation

Which way something is facing, and how its parts are arranged. What a drone needs to know about itself.

Pose is position plus orientation — six numbers, three for place, three for tilt — and a drone lives on them. Its gyroscopes report rate of turn and its accelerometers feel every push, hundreds of times a second. The estimate is kept by integration: angle ← angle + rate × dt, the current tilt updated by however fast it was turning over the last slice of time, with dt perhaps a five-hundredth of a second. The propellers correct against that running estimate before you could notice the wobble.

Integration alone drifts — every reading carries a speck of error, and adding hundreds of readings a second adds hundreds of specks. So the estimate is anchored: the accelerometer knows which way gravity pulls, which pins the tilt over the long run even while the gyroscope handles the fast changes; a compass or a camera can pin the heading the same way. Fast sensor for the moment, steady sensor for the truth.

Point the same idea at bodies and ordinary video becomes measurement. A network marks where each shoulder, elbow, and knee sits in every frame — seventeen keypoints is a common recipe — and suddenly physiotherapy runs without sensors, a bowler's action is analysed frame by frame, and animation is captured without suits. The skeleton was in the video all along; pose estimation just reads it out.

The catch is compounding. Instruments drift, each small error feeds the next estimate, and a cheap gyroscope left uncorrected can wander by several degrees within a minute — enough, over a long flight, to put a drone somewhere it firmly believes it is not. So pose must keep being corrected against landmarks the machine can actually see: the horizon, a wall, GPS overhead. Dead reckoning is a memory, and memories need checking against the world.

130

Pose locates you on a map you already have — but a robot's first assignment is usually somewhere unmapped.

Mapping while moving

Building a map of somewhere you have never been, while working out where in it you are.

To know where you are, you need a map; to build a map, you need to know where you are. SLAM refuses to choose. The machine picks out landmarks — a corner, a doorway, a lamp post — estimates its own motion between sightings, and grows the map while pinning its pose inside it, each estimate correcting the other.

The engine is prediction against observation. From its motion estimate, the machine predicts where a known landmark should appear; it then measures where the landmark actually appears; the difference corrects both at once — the pose a little, the landmark's mapped position a little, each in proportion to how uncertain it was. It is probability's familiar update, run continuously over hundreds of landmarks. Even so, small errors accumulate as it moves, and the map slowly warps.

Then the machine recognises somewhere it has been before — the same doorway, seen again after a long loop of corridor — and that single recognition, called loop closure, snaps the whole map straight: the accumulated warp is spread backwards along the path and cancelled. This is what a robot vacuum does to your flat and an AR headset does to your living room, and the idea was born of places with no GPS and no map at all: mines, seabeds, other planets.

The honest weakness is sameness. A featureless corridor offers no landmarks to pin against; a warehouse of identical aisles offers landmarks that all look alike, and a false loop closure — recognising the wrong aisle as a revisit — bends a good map worse than drift ever did. Moving furniture and passing people corrupt landmarks too. SLAM trusts the world to stay put and look distinctive, and the world only mostly obliges.

131

Perception from one sensor is a single witness — the camera, the radar, and the gyroscope all testify, and they disagree.

Combining sensors

Camera, radar, and inertial readings all disagree slightly. Fusion decides who to believe.

Every sensor has a blind spot. Cameras are rich but helpless in fog and darkness; radar punches through weather but sees coarse blobs; inertial sensors feel every motion and slowly drift. Fusion treats each reading as a belief with an uncertainty attached, then combines them the way probability says to: trust the confident witness more, and let agreement sharpen the estimate.

The arithmetic is a weighted average, weights set by trust. If radar puts the car ahead at 42 metres give or take 3, and the camera says 38 give or take 1, the fused estimate lands near 38.4 — close to the tighter witness, nudged by the looser one — and its uncertainty is smaller than either alone. That last part is the quiet magic: two mediocre sensors that agree beat one good sensor standing alone. The panel beside this text is running exactly this — two noisy witnesses, one sharpened belief.

The classic machinery for this, the Kalman filter, predates the field's glamour — it helped navigate Apollo to the moon. It alternates two moves: predict where the state should be from physics, then correct with the newest measurements, weighting by uncertainty every cycle. Sixty years on, some version of it runs in essentially every drone, car, and phone that knows where it is.

The modern argument is about redundancy. Some carmakers insist cameras alone suffice, since humans drive on eyes; others pay for radar and LiDAR precisely because when one sense fails, the disagreement itself is the warning. And fusion carries fine print of its own: the averaging assumes the errors are independent. Sensors that share a blind spot — two cameras in the same fog — agree confidently and wrongly. Fusion is an engineering answer to an honesty question.

132

Perception describes the world and a policy chooses the response — control is where the choice finally reaches the motors.

Control policies

Going from what the machine sees to what the motors actually do.

Perception delivers the state of the world, a policy picks the action, and control turns that choice into voltages and torques — then the world moves, perception reads it afresh, and the loop runs again. The thermostat is the ancestral form: measure the error, push against it, repeat. Everything since is refinement of that one move.

The workhorse refinement is PID: command = Kp·error + Ki·∫error dt + Kd·d(error)/dt. The first term pushes in proportion to how wrong you are right now. The second accumulates lingering error, so a steady shortfall — a drone forever 20 centimetres low — eventually earns extra thrust. The third watches how fast the error is changing and brakes against overshoot before it happens. Three dials, tuned by hand or by learning, running perhaps 400 times a second on a hovering drone.

What makes control hard is that acting changes what you see next. A wrong label on a photo just sits there; a wrong torque tilts the drone, which distorts the next reading, which invites a worse correction — errors compound around the loop. Overcorrect and the whole system oscillates like a new driver's steering, sawing left and right past the lane it wants.

Decades of control theory exist because 'push against the error' hides that much subtlety — delays in sensing, motors that saturate, physics that changes when the payload does. Learned policies now sit beside the classical ones: a network can absorb quirks no engineer modelled, but it inherits the same loop and the same danger, and it is far harder to prove stable. The maths of the thermostat still earns its keep where lives depend on the loop.

133

Reinforcement learning needs a million mistakes, and a real machine pays for each one — so make the mistakes in software.

Simulation to reality

Train a million times in a simulator, because crashing a real drone costs money.

A real drone pays for every mistake in carbon fibre; a simulated one pays nothing at all. So build the world in software: a physics simulator where the policy can crash ten thousand times an hour, run faster than real time, in parallel copies, for the price of electricity. The recipe is plain — model the world as equations stepped forward in small slices of time, let reinforcement learning run its usual loop of act, score, adjust inside them, and when the policy is good, carry the finished dials across to the real machine, unchanged.

The trick that makes the crossing survivable is randomisation. Vary the simulator's friction, masses, lighting, and delays on every run — this floor slippery, that one grippy, this motor lagging 10 milliseconds, that one 30 — so the policy cannot overfit one tidy physics and must learn habits that hold across many. Reality, with luck, is one of them.

It is how a famous robot hand learned to twirl a cube: OpenAI's Dactyl practised in simulation for the equivalent of many years of manipulation — impossible in any lab — across thousands of randomised versions of physics, then did the trick in the flesh on hardware that had never rehearsed it. The real hand met a world it had, in a statistical sense, already lived many versions of.

The bargain has costs. Randomisation buys robustness by demanding caution — a policy trained for a thousand possible physics moves conservatively in the one that is actual — and some things simulators simply model badly: the crumple of cloth, the smear of rain on a lens, the behaviour of pedestrians. What the simulator leaves out, the policy never met, and that remainder has a name of its own: the reality gap.

134

Generalisation meant doing well on data you never saw — and reality is the harshest held-out set a policy will ever face.

The reality gap

Everything that works in the simulator and fails in the rain.

The reality gap is everything the simulator left out: glare, rain on the lens, a tyre going soft, dust on a sensor, pedestrians doing what no physics engine predicts. A policy that mastered the simulation meets these as inputs from a world it never trained on — and you know this failure by name. It overfit the simulator, and reality is the held-out set.

The panel beside this text is running exactly this: a policy scoring highly in its clean training world, then fed the same world with the dials of reality turned — noise on the sensors, friction it never felt — and you can watch the score fall as the gap widens. The mechanism is distribution shift, stated plainly: the model's error grows with the distance between the world it trained on and the world it is asked about, and no amount of skill inside the old world buys that back.

The gap narrows and never closes. Randomised training helps; fine-tuning on real data helps; better simulators help — yet every simulator is itself a model, and models simplify. Somebody chose what to leave out, and the leavings are exactly where the policy is blind. A car validated under Californian sun still has to meet a Mumbai monsoon, standing water and all.

This is the honest reason polished demo videos run years ahead of shipped products. The demo lives close to the simulator's world — good weather, rehearsed routes, a safety driver's quiet interventions off camera — and the product has to live in the rain, every day, for everyone. When the two look identical on screen, the gap between them is exactly the part you cannot see.

135

Attention learned to align written sequences; speech is a sequence dissolved into pressure waves, accents and idling lorries included.

Speech recognition

Turning a pressure wave into words, with all the accents and background noise intact.

A microphone hands the model thousands of pressure readings per second — 16,000 samples a second is standard — no letters, no word boundaries, just wave. The first move is compression into a spectrogram: a map of how much energy sits at each frequency, sliced every 10 milliseconds, turning one second of sound into about a hundred frames of texture. That is what the network actually reads.

From there it is sequence-to-sequence at its rawest. An encoder digests the spectrogram frames; a decoder writes out characters or word-pieces; and attention aligns stretches of sound to pieces of text, discovering for itself where one word dissolves into the next. Nobody tells it where 'next' ends and 'train' begins in a station announcement — the alignment is learned from pairs of audio and transcript, nothing more.

For decades this was a hand-built pipeline — sound units, pronunciation dictionaries, a separate language model — maintained by specialists. End-to-end learned models swept all of it away once enough transcribed audio existed: OpenAI's Whisper trained on about 680,000 hours of it. Quality is scored as word error rate, errors per hundred words, and on clean English speech the best systems sit in the low single digits — roughly human.

The honest residue is unevenness. A system trained mostly on some accents transcribes those accents best, and the error rate you get depends on how much the training data sounded like you. The same model that nails a London newsreader can stumble on Indian English spoken over a ceiling fan and street traffic — not because the speech is unclear, but because its training diet said so. The average hides who pays.

136

Recognition turned voice into text; going back means reinventing everything it threw away — pitch, pace, and breath.

Speech synthesis

Going the other way — text into a voice that does not sound like a robot.

Text radically underspecifies speech. 'Fine.' can be contentment, surrender, or a warning, and nothing on the page says which — so synthesis is generation, not lookup. The model predicts sound step by step the way a language model predicts words, having learned pitch, pace, and breath from thousands of hours of human recordings, and it must invent everything recognition threw away.

The usual recipe runs in two stages. First, text becomes a spectrogram — the same frequency-over-time map recognition reads, now written instead of read — carrying the melody and timing of the sentence. Then a vocoder turns that map into an actual waveform, tens of thousands of samples a second, each conditioned on the sound so far. Split this way, one network learns how the sentence should go, the other how a human throat actually sounds.

The robotic voice died when synthesis became learned rather than assembled. Older systems glued together snippets of a voice actor's recorded syllables, and you could hear every seam; new ones generate the waveform whole, hesitations and intakes of breath included. A voice can now be cloned from a few seconds of audio. The same machinery reads to the blind, returns speech to people losing theirs to disease, and gives apps in every Indian language a voice no studio ever recorded.

And it phones your grandmother pretending to be you. Cloning from seconds of audio is precisely what voice fraud needed, and banks that once used 'my voice is my password' have been retiring it. There is a quieter limit too: the model produces sound, not understanding — it will read '₹1,50,000' or a doctor's name with perfect confidence and wrong stress, fluent in the voice while knowing nothing of the meaning.

137

Self-supervision made data label itself; a robot goes further — it acts, and lets the consequences do the labelling.

Embodied learning

A machine that learns by acting in the world and living with the consequences.

Self-supervision taught language models by hiding the next word. A robot's version is stronger: act, and let the world do the grading. Predict what the gripper will feel before it closes, what the camera will show after the push — then compare with what actually happens. Every action is an experiment, and prediction error is a loss no human had to label.

The loss is the familiar one: loss = (predicted outcome − actual outcome)², squared error between what the robot expected and what physics delivered. The difference is where truth comes from — not a labeller's annotation but the world itself, arriving a second after the action. Each grasp both advances the task and buys a gradient, so practice and data collection become the same activity, which is exactly what made self-supervision powerful the first time round.

The bottleneck is that reality runs at one second per second. Language models feasted on an internet the world had already written; there is no internet of touch, torque, and consequence — every robot must gather its own experience, in real time, with real wear on real parts. Google once ran an arm farm — over a dozen robot arms grasping around the clock for months — to collect what a language model would call a rounding error of data. Robot experience is the field's scarcest resource.

The honest limits are wear and danger. Exploration that costs nothing in language — a bad next-word guess — is broken crockery, stripped gears, or worse in a kitchen, so a learning robot must be curious and careful at once, which caps how boldly it can experiment. And experience is stubbornly personal: data gathered on one arm transfers imperfectly to another with different joints and different friction, so the scarce resource does not even pool cleanly.

138

Latency was money when a chatbot answered; on a drone in flight, it turns into physics.

Fast enough to matter

A drone deciding in 20 milliseconds is a different problem to a model answering in two seconds.

A drone flying at 20 metres per second covers 40 centimetres while a 20-millisecond decision is being made. Perception, choice, and command must all fit inside the control loop's beat, every beat — at 50 hertz that is a 20-millisecond budget, perhaps 10 for perception, 5 for the policy, the rest for the command and a margin. A late answer is not a slow answer; it is an answer about a world that no longer exists.

The deadline redesigns everything. There is no round trip to a datacentre — 50 to 100 milliseconds of travel can spend the whole budget before any thinking happens — so the model must live on the machine, on a few watts of onboard computer. This is why the tricks you met earlier matter most here: distillation to make a small model carry a big one's skill, quantisation to squeeze the weights into fewer bits, architectures sized to the hardware that must run them.

And sized to the worst case, not the average. A model that usually answers in 12 milliseconds but occasionally takes 40 is not a fast model with rare slow moments; it is a crash with good statistics, because the one late frame may be the one with the wire fence in it. Real-time engineering budgets for the 99th percentile, while most benchmarks quote the mean — read the fine print accordingly.

The honest trade is capability. The onboard model is smaller, blunter, and less knowing than the one in the datacentre — a price paid so that its answers arrive about the world as it still is. A chatbot may be slow gracefully; a drone may not, and much of physical-world AI is the discipline of choosing what to give up so the deadline is never missed.

139

Generalisation means projecting yesterday's patterns onto tomorrow's people — which is precisely the danger when yesterday was unfair.

Bias in data and models

A model trained on the past will repeat the past, including its unfairness.

A model trained on the past learns the past — including its unfairness, laundered into arithmetic. The mechanism is nothing exotic: gradient descent minimises average loss on the records it is given, so if the records encode skewed decisions, the lowest-loss dials are the ones that reproduce the skew. A hiring model trained on years of skewed decisions will reproduce them, at scale, with a straight face. Amazon built exactly this: a CV-screening model trained on a decade of its own hiring, which learnt to penalise the word 'women's' — as in 'women's chess club captain' — and was scrapped in 2018.

Deleting the sensitive column does not help. Gender, caste, and religion leak through proxies — pincode, surname, college, employment gaps — and a network's whole job is finding correlated signals, so it quietly reconstructs what you removed. Worse, fairness resists a clean fix: when two groups have different base rates in the data, a 2016 result showed you mathematically cannot make a model both calibrated (a score of 0.8 meaning 80% for everyone) and equal in its error rates across groups. You must choose which unfairness to keep, and the choice is a value judgement, not a calculation.

The trap is the objectivity costume: 'the algorithm decided' sounds neutral, but the algorithm distilled the data, and the data recorded the people. A loan model that approves fewer applicants from one neighbourhood is not inventing fresh discrimination; it is compressing old discrimination into a number and applying it faster and more uniformly than any prejudiced officer could. The uniformity is the danger — a biased human is at least inconsistent, but a biased model makes the same skewed call every time, millions of times.

Fixing it is genuinely hard because history is the only training data anyone has. You can reweight examples, audit error rates group by group, or constrain the training objective, and each helps at a measurable cost in accuracy; none of them manufactures the fair history that was never recorded. The panel beside this text is running exactly this: train on skewed records, watch the skew reappear in the decisions, reweight, and watch the trade-off move rather than vanish. Honest deployment means measuring the skew and owning the choice — not announcing that the machine has no opinions.

140

You can now build a multilayer network yourself — and still not answer the simplest question about it: why did it say that?

Interpretability

Working out why the model said that. Genuinely hard, and unsolved.

Open a trained network and you find millions of dials, set by gradient descent, none of them labelled. You can read every number — every weight is right there on disk — and still not know why the model refused this loan or flagged that scan. The reasons are not hidden; they are smeared. A single decision passes through every layer, and the 'why' lives in the joint behaviour of the whole stack, not in any dial you can point at. Interpretability is the work of recovering reasons from the weights: which inputs mattered, what pattern an internal neuron actually tracks, where in the stack a decision was made.

The simplest tool is the gradient you already know, pointed at the input instead of the dials: ∂output/∂input asks, for each pixel or word, how much the answer would move if that part of the input nudged. Run it on a chest X-ray classifier and you get a heat-map of the pixels that pushed towards 'pneumonia'. It sometimes works. It also fails a basic sanity check: a 2018 study randomised a network's weights and several popular saliency methods produced nearly the same maps — the explanation was tracking edges in the image, not the model's reasoning.

Deeper work opens the network itself. Researchers have found individual features with real meaning — curve detectors in vision models, neurons that fire on a single concept, small circuits inside language models that implement copying — and can sometimes edit them and watch behaviour change accordingly. Progress is real but small: these are verified accounts of tiny fragments, and nobody has a full account of any frontier model, systems now measured in hundreds of billions of parameters.

Meanwhile these systems already help decide loans, bail, and diagnoses. When one is wrong, 'why' is a question the person on the receiving end deserves answered, and today it mostly cannot be. Be equally wary of the opposite failure: tools that always produce an explanation. A plausible-looking story generated after the fact is worse than silence, because it lets everyone stop asking. The honest state of the art is a small set of trusted fragments and a lot of confident-sounding guesswork — and telling those two apart is itself unsolved.

141

Overfitting looked like a scoring problem — until you remember what memorised training data can contain.

Privacy and memorisation

Models can repeat back training data, including data that was never meant to be public.

You met overfitting as memorising examples instead of patterns, and it looked like a scoring problem — validation loss creeping upward. Do it at scale and the memorised examples come back out. In 2021, researchers extracted hundreds of verbatim training sequences from GPT-2 just by prompting it well and keeping the outputs the model found suspiciously easy to predict — names, addresses, and contact details among them. Rare strings are the most vulnerable: a phone number that appears once in the data, a medical detail from a forum post, a private email that leaked into a scrape. With nothing to generalise from, the model stores the string itself.

The harm is quiet and personal: something posted for one small audience resurfaces from a machine serving millions. A phone number left on a 2014 classified ad, scraped along with the rest of the web, can surface in a chatbot's answer a decade later. And there is no clean deletion — you cannot reach into the dials and remove one person, because the string is not a row you can drop but a pattern spread across weights, so honouring a removal request properly means retraining, at a cost of millions of dollars per run.

The main defence has an equation. Differential privacy trains with a modified update: clip each example's gradient to a fixed length, add Gaussian noise, then step — w ← w − η·(clip(g) + noise). The clipping caps how loudly any one person's data can speak; the noise drowns whatever whisper remains; a budget called ε tracks the total leakage across training. It provably limits memorisation, which is a rare and valuable kind of guarantee.

The proof costs accuracy, and the cost is real: the noise that hides one person's data also blurs the rare patterns a model most needs, so strongly private training can lose several points on hard tasks. That is why differential privacy is cited far more often than it is used — deployed at scale by a handful of firms for telemetry, and almost never for training large language models. For now the honest summary is short: the data went in, some of it can come out, and nobody can cleanly take it back.

142

Generalisation promised good answers on unseen inputs; some inputs are engineered, pixel by pixel, to break that promise.

Adversarial examples

Small, invisible changes to an input can flip a confident answer.

Take a photo a model classifies correctly and confidently, and add a perturbation too small for any human to see. The recipe is gradient descent run in reverse: training nudged the dials downhill on loss, and the attack nudges the input uphill instead — x′ = x + ε·sign(∂L/∂x), where ε sets the invisible step size and the gradient says which direction hurts the model most. The confident answer flips. In the famous 2014 example, a panda becomes, with high confidence, a gibbon, at ε = 0.007 — about two brightness steps out of 255, invisible in practice.

This is generalisation's fine print. The training data covered natural photographs, which occupy a thin sliver of all possible pixel grids; the model's decision boundaries are only pinned down near that sliver, and an attacker steers straight into the unpinned space beside a real image. Doing well on natural unseen inputs says nothing about inputs built to deceive. The attacks even transfer: a perturbation computed against one model often fools another trained separately, so the attacker does not need your weights to start.

The attacks survive the real world. In 2018, researchers showed that a few carefully placed stickers make a vision system misread a stop sign as a speed-limit sign from a moving car. Spam and fraud have lived this forever: every filter deployed teaches the other side what to write next, and a UPI-fraud model that blocks one message template will meet a reworded one within days. Wherever a model faces an adversary — fraud, spam, moderation, driving — assume one is coming.

A decade of published defences has been broken almost as fast as it appeared, often within months, by the same evaluation done more carefully. The one that survives, adversarial training — generating attacks during training and learning to resist them — costs several points of clean accuracy and only hardens the model against the attack styles it practised on. So treat robustness claims the way this course treats every claim: as a measurement with a scope, not a property. 'Robust' without 'against what, at what ε' is marketing.

143

Hallucination is one entry on a longer list — and honesty means writing the whole list down.

What models cannot do

The honest list, kept up to date, of where these systems fail.

The honest list, today: models state falsehoods in a confident voice, as you saw with hallucination; they know nothing after their training data ends; they are fragile away from the data they saw; they cannot reliably say 'I don't know'; and their fluency is not understanding — a wrong answer arrives in the same polished prose as a right one. Each entry traces back to mechanics you have already met: a next-word predictor has no truth dial, a frozen checkpoint has no today, and generalisation was only ever a promise about data resembling the training set.

The entries compound. In 2023, a New York lawyer filed a court brief written with ChatGPT that cited six cases; the cases did not exist, and when he asked the model to confirm they were real, it confidently did. Three list items fired at once — hallucination, no reliable 'I don't know', and fluency mistaken for understanding — and the check he ran was itself a question to the thing that had erred. The correct check was a database lookup that would have cost him minutes.

Maintaining the list is a procedure, not a mood. For any task you care about, write down concrete cases with known answers, including cases dated after the training cutoff and cases just outside the model's comfortable distribution; measure, record the date, and re-measure when the model changes. A claim about what models cannot do is an empirical claim with an expiry date, and it should be stored like one.

The list is a moving target, which is the hard part. Items fall off it — tasks confidently declared impossible have fallen within a year or two — and new failure modes arrive with new capabilities. Both errors of honesty are live: sellers who claim their model has no limits, and sceptics reciting limits it no longer has, sometimes quoting measurements three model generations old. Keeping the list current is the discipline, and it is this course's own standing risk: some sentence on this page will age badly. Check the date; re-run the test.

144

Pretraining consumed a sizeable fraction of everything ever written — someone wrote all of it, and almost nobody was asked.

Provenance and consent

Where training data came from, and whether anyone agreed to it.

Pretraining ate an enormous pile of text and images — and every piece of it was made by a person. Books, code, forum answers, journalism, art, scraped because it was reachable, not because anyone agreed. The scale makes the point concrete: recent open models train on roughly fifteen trillion tokens, and the LAION-5B image set behind popular diffusion models holds about 5.8 billion image-text pairs, gathered by crawler. The model's fluency is a compression of that labour, and almost none of the people compressed were asked, credited, or paid.

The collection mechanism explains the consent gap. Crawlers like Common Crawl follow links and archive whatever loads, respecting robots.txt — a 1994 convention written to manage search engines, which indexed pages and sent readers back to them. Training is a different bargain: the model absorbs the writing and the reader never arrives. The tool everyone used to signal consent was answering a question nobody had asked yet, and opting out today does nothing about the copies already inside deployed weights.

The reckoning is happening in court. The New York Times sued OpenAI and Microsoft in December 2023, showing prompts that made GPT-4 reproduce its paywalled articles nearly verbatim; Getty Images sued Stability AI over millions of scraped photographs; authors' and artists' suits run in parallel. The outcomes will shape what future models may train on. And the scrapes were indiscriminate: one artist found her own private medical photographs, taken by her doctor, sitting inside the LAION dataset.

Underneath the law sits a plainer question — 'publicly visible' has never meant 'free to take'. Be honest in both directions, though: courts may yet rule much of this lawful, licensing deals are now being signed, and a settlement that pays large publishers does nothing for the forum poster whose patient answers taught the model the most. Consent, credit, and compensation are not solved problems; they are open questions the field built on top of, and there is no clean side to stand on.

145

Inference produces an answer; everything that follows depends on whether that answer is a suggestion or an act.

How much rope to give it

The difference between a system that suggests and one that acts on its own.

The same model can be wired up two ways. It drafts an email and waits — a suggestion. It sends the email, executes the trade, steers the car — an action. Nothing about the model changed; what changed is what its output is connected to. Autonomy is that wiring decision, and it is made by people, not learned by models. The technical shape is plain: a suggestion system's output lands in front of human eyes; an acting system's output lands on an interface with side effects — a payment rail, a motor, a send button.

The right length of rope follows a two-question procedure. For each action the system could take, ask: how costly is a mistake, and can it be undone? Score those honestly and the wiring mostly writes itself. A drafted reply costs nothing and reverses freely — full autonomy is fine. A UPI autopay debit is bounded and refundable within rules — automation with limits and receipts. A medical dose or a market order is expensive and irreversible — a human stays between output and effect.

The canonical lesson is Knight Capital, 2012: a trading firm deployed software that began sending unintended orders at machine speed, and in forty-five minutes lost about 440 million dollars — several times its annual profit — before humans could pull the plug. Nothing had to be intelligent for this; the system was simply wired to act, and acting compounds faster than people can notice. Reversibility judged at design time is often optimism: each trade was individually reversible, but not four million of them.

And connections are cheap to add, so the rope tends to lengthen quietly — a system installed to suggest becomes, one integration at a time, a system that acts, with no single moment where anyone decided that. The two-question test also flatters itself: costs are judged per action, but autonomous systems act in volume, and a mistake that is trivial once is an incident at ten thousand repetitions. Rate limits, spending caps, and kill switches are not accessories; they are the difference between a bad output and a bad day.

146

You have just weighed how much rope to give a system — the self-driving industry wrote that question down as a ladder.

Levels of self-driving

The precise ladder from cruise control to no steering wheel, and where the hard part sits.

The ladder runs from 0 to 5. Level 1 assists with speed or steering; level 2 handles both while you watch; level 3 drives itself but may summon you back; level 4 needs nobody, inside a fenced zone; level 5 needs nobody, anywhere. Each rung transfers a slice of the driving — and of the responsibility — and the panel beside this text walks the same ladder, showing who holds which part of the task at every rung.

The hard part sits in the middle. Level 3 asks a human who has been idle for an hour to retake control within seconds — handing back the rope at exactly the worst moment. Studies of takeover put the cost of regaining situational awareness at several seconds even for an alert driver, and at 100 kilometres per hour, five seconds is nearly 140 metres of road. Many builders skip the rung entirely and aim straight for 4.

Level 4's fence is not a compromise; it is the design. Waymo's cars carry paying passengers with nobody in the driver's seat — but only inside mapped, rehearsed zones such as Phoenix, chosen for wide roads and forgiving weather. The fence sits where the reality gap is narrowest. Outside it, the same car is not slightly worse; it is out of scope, which is a precise engineering statement, not a hedge.

Marketing blurs the rungs, and the blur is the danger. A level-2 system with a confident name invites level-4 trust: hands drift from the wheel, eyes drift to a phone, and the human slice of the driving quietly goes unstaffed while the badge implies otherwise. That gap has already cost lives. The ladder's real lesson is that the question is never only what the machine can do — it is who is responsible this second, and whether they know it.

147

Once you decide how much rope a system gets, you must decide exactly where a human hand stays on it.

Keeping a person in the loop

Where a human must approve before anything happens, and why that line is drawn.

A human-in-the-loop design puts a person between the model's output and the consequence: the system proposes, a human approves, and only then does anything happen. Where to draw that line follows from the two questions you just used to set the rope — the loop belongs wherever a mistake is expensive and cannot be taken back: a diagnosis, a dismissal, a weapon, a large sum of money. Everywhere else, forcing approval buys nothing but delay, and worse, it spends the reviewer's attention where it is not needed.

The catch is that a person in the loop is not automatically a check. Run the numbers on a screening desk: a model flags 300 cases a day and is right about 297 of them. The reviewer sees a wall of correct suggestions, learns that agreeing is nearly always right, and by week three is averaging a few seconds per case — approving, not reviewing. Automation bias has a long record in aviation and medicine: trained pilots and radiologists, told by a machine that all is well, miss what they would otherwise have caught.

The loop is real only if the human has the time, the information, and the standing to say no. Time: an approval queue sized so scrutiny is possible — thirty considered decisions a day, not three hundred reflexes. Information: the model's confidence and the evidence behind the flag, not a bare yes or no. Standing: a reviewer whose overrides are welcomed rather than held against them; if every 'no' triggers paperwork and every 'yes' is free, the incentive gradient does the deciding. The panel beside this text lets you feel it: approve a stream of suggestions and watch your own error rate climb as the pace rises.

So treat 'a human approves every decision' as a claim to be audited, not a comfort. Measure the override rate: a reviewer who has not said no in a thousand cases is either supervising a perfect model or has become a rubber stamp, and perfect models are not on offer. The honest design decides where people genuinely add judgement, gives them few enough cases to exercise it, and automates the rest openly — rather than staffing a ceremony that exists so someone is available to blame.

148

The limits list covered models failing; the harder harms arrive when a model works exactly as designed, for the wrong person.

Deliberate misuse

Fraud, impersonation, and manufactured evidence — the harms that come from it working, not failing.

Some harms come from models failing. These come from models working: fluent scam messages at industrial scale, cloned voices asking relatives for money, fabricated evidence of things that never happened. Nothing needs to go wrong for the harm to happen — a language model that writes persuasive prose writes persuasive fraud, and a voice model that clones a voice from a few seconds of audio does not check whose voice it is. The capability and the abuse are the same feature, pointed differently.

The uncomfortable asymmetry is economic: generation is cheap and verification is expensive. A convincing phishing message costs a fraction of a paisa in compute and seconds to produce; checking it properly costs a human minutes, and scam economics only needs a tiny hit rate — send a million messages for a few hundred rupees, and one victim in a hundred thousand pays for the whole campaign. One person can now produce what once needed a team, in fluent English or Hindi or any of a dozen Indian languages that scam operations previously wrote badly.

India is living a concrete version: 'digital arrest' scams, where callers posing as police or customs officials — sometimes on video, sometimes with cloned voices — keep victims on the line for hours and walk them through transferring their savings. Government cybercrime figures put the losses in the hundreds of crores for 2024 alone, and the victims include retired professors and doctors, not the credulous stereotype. The scripts are old confidence tricks; what the models changed is the fluency, the personalisation, and the price of running thousands of attempts in parallel.

Defences exist — provenance standards, detection models, bank-side friction on large transfers, and plain scepticism drilled as habit: hang up, call back on a number you found yourself. But the arms race is real, and pretending otherwise teaches nobody anything. Detection models are classifiers, which means they generalise imperfectly and an adversary probes exactly where they fail; each defence deployed becomes training feedback for the other side. The stable protections are the ones that do not depend on spotting the fake: verified channels, delays on irreversible transfers, and family rules agreed before the phone rings.

149

Diffusion pulls convincing images out of pure noise — misuse begins when the image is of you.

Deepfakes and synthetic media

When seeing is no longer believing, and what that costs a society.

Diffusion showed you images conjured from pure noise; the same machinery now conjures a photograph of an event that never happened, or a cloned voice saying words never said, in minutes, for nearly nothing. Fabricating convincing evidence used to take skill and money — studio time, editors, forgers. That cost has collapsed to a prompt and a consumer GPU, and the cost of checking has not collapsed with it. The panel beside this text runs the honest version of the demonstration: the same denoising steps you watched build a landscape will build a face that belongs to no one — or to someone.

The damage runs in both directions. Fabrications get believed: non-consensual imagery, overwhelmingly of women, is already the most common deepfake harm by a wide margin, and cloned voices are already used in fraud — in one 2024 case, an employee in Hong Kong transferred about 25 million dollars after a video call with what looked and sounded like his company's senior officers, every one of them synthetic. India met its own version when a fabricated video of a film actress spread faster than any correction could chase it.

And real recordings get denied — once anything can be faked, 'that's fake' becomes a defence against genuine evidence. Researchers call it the liar's dividend: the mere existence of deepfakes pays out to anyone caught on tape. This corrodes quietly. Courts, elections, and newsrooms all run on the assumption that a recording settles at least something; when it settles nothing, disputes fall back to whoever shouts loudest or already holds power.

Provenance standards and watermarks help at the margin. C2PA content credentials cryptographically sign a photo at the camera and record every edit, which works — for media that opts in, until the signature is stripped by the oldest attack there is: photographing the screen. Detection models exist and lag the generators they chase. So the honest position is narrow: verify provenance where it exists, distrust urgency, and accept that shared trust in recordings, once spent, does not come back cheap.

150

What models cannot do bounds whose work they can — and that boundary is moving through real livelihoods.

Work and displacement

Which jobs change, which disappear, and who carries the cost.

Models rarely take a job whole; they take tasks. A job is a bundle — a translator drafts, checks terminology, negotiates tone, manages clients — and automation slices the bundle, not the person. What models cannot do bounds the change, but the bound moves. The arithmetic is blunt: if machine drafts make each worker twice as productive at half of the bundle, the same demand needs fewer people, and 'the model did not take the job' is small comfort to the one person now doing what two did.

Drafting, translating, transcribing, routine coding: work that was someone's living is already partly machine work. Transcription that billed by the audio-hour now competes with software that does it in minutes for a few rupees; stock illustration competes with generation; and India's IT-services industry, which employs over five million people, is watching the routine end of coding and support — precisely the rungs where careers used to start — become the models' strongest suit. The entry-level rung matters most: automate it away and the ladder loses its bottom.

Every previous technology wave eventually created more work than it destroyed — 'eventually' being the load-bearing word. Weavers displaced by power looms were not retrained into mill engineers; a generation absorbed the loss while the statistics improved. The gains go to owners of the models and the firms that deploy them; the costs land on particular people, in particular places, in the middle of their working lives. Averages recover; individuals often do not.

Retraining is real but slower and harder than the word suggests — a forty-five-year-old medical transcriptionist does not become a prompt engineer by attending a webinar. And honesty cuts both ways here too: confident predictions of mass unemployment have a long record of being wrong, and confident denials that this wave differs have no record yet either way. The defensible claim is narrower than both: tasks are moving now, the movement is uneven, and who carries the cost is a choice societies make, not a property of the models.

151

Scaling laws made capability a thing money buys predictably — so it matters enormously who has the money.

Who owns the compute

Training frontier models costs more than most countries will spend, and that concentrates power.

Scaling laws turned capability into a purchasing decision: more compute, more data, more parameters, predictably less loss. The predictability is the point — an organisation can budget for capability the way it budgets for a factory. A frontier training run now costs on the order of hundreds of millions of dollars, consumed as tens of thousands of specialised accelerators running for months, each chip costing as much as a car, drawing the electricity of a small town while they run. So the most capable models are trained by the few organisations that can pay, and by no one else.

The supply chain narrows at every layer. One firm dominates the design of training chips; one Taiwanese fabricator manufactures nearly all of the leading-edge silicon; a single Dutch company builds the lithography machines the fabricator needs. Export controls already treat these chips as strategic goods. This is a shape computing has not had before: the web ran on commodity servers anyone could buy, but frontier training runs on hardware whose entire global supply is spoken for years ahead.

That concentrates decisions, not just money: what the models refuse, whose languages they serve well, what they are optimised for — settled inside a few companies, mostly in one country. A model trained mostly on English and priced in dollars makes quiet choices about Hindi, Tamil, and Bengali speakers that no one in those languages voted on. Concentration is a harm on its own terms, before any model gets anything wrong.

The honest counterweights are real. Open-weight releases push against the lock-in, and smaller models close some gaps — a distilled model running on one server now does what needed a frontier model two years ago, so yesterday's capability diffuses even while today's stays concentrated. But diffusion follows the frontier at a distance; whoever trains at the edge decides first what exists. The oversold version of this stop is a permanent monopoly; the accurate version is a lead of a year or two, held by whoever owns the compute — which is quite enough to matter.

152

Interpretability is unsolved and autonomy keeps growing, so you must learn to supervise what you cannot read.

Watching a system you cannot read

How you supervise something whose reasoning nobody can inspect.

Interpretability being unsolved leaves one option: supervise the behaviour, since you cannot read the reasoning. You watch what it does, because what it thinks is not on offer. This is not defeat; it is how we already handle every complex system we cannot inspect from the inside — pilots, markets, other people. The difference is that a deployed model acts at machine speed and volume, so the watching has to be engineered, not just assigned to someone.

In practice, oversight is five habits. Evaluation suites before deployment: hundreds of known cases with known answers, re-run on every new version. Logging: every input and output kept, so any incident can be replayed. Sampling: a fixed slice of live decisions — say one in a hundred — pulled for human audit whether or not anything looks wrong. Red-teaming: people paid to make the system fail on purpose, before someone unpaid does. Tripwires: hard thresholds that pull the system back automatically — a fraud model whose block rate doubles in an hour gets frozen first and investigated second.

A payments firm running fraud models lives this daily: the model blocks transactions in milliseconds, so a bad update means a thousand wrongly frozen accounts before anyone's phone rings. The tripwire — block rate outside historical bounds pauses the rollout — costs almost nothing and holds no opinion about why the model changed. It does not need one. Behavioural oversight works precisely because it makes no claim about the reasoning; it simply refuses to let unexplained change act at scale.

The gap is one generalisation already taught you: tests only cover situations somebody thought to test, and a system can pass every evaluation and still fail on the input nobody imagined. Red-teaming narrows the gap; nothing closes it. So oversight has to scale with autonomy — the more a system acts, the more watching it deserves — and today the watching lags the acting almost everywhere, because evaluations cost money and shipping earns it. That gap is where the incidents live. This road ends here on purpose: not with a solved problem, but with a defensible way to stand next to one.