Every concept, walkable from the start

A model is a rule with adjustable numbers in it.

A parameter is one of those adjustable numbers — a dial.

needs model

A prediction is what the rule says before you check the answer.

needs model

Loss is one number saying how wrong the model currently is.

needs prediction

A gradient says which way to turn each dial to make loss smaller.

needs loss

Turn every dial a little downhill, over and over. That is training.

needs gradient

How far you turn the dials each step — too small crawls, too big explodes.

needs gradient-descent

Every setting of the dials has a height; training walks downhill on that terrain.

needs loss

Learning from examples where somebody already wrote the answer down.

needs model

Finding structure in data nobody labelled.

needs supervised

Hiding part of the data and making the model guess it — where modern AI gets its scale.

needs supervised

Using a trained model. No dials move; it just answers.

needs prediction

A list of numbers describing one thing.

A grid of numbers — usually a stack of vectors, or a machine that transforms them.

needs vector

The single operation that eats most of the world's AI electricity.

needs matrix

One number saying how much two vectors point the same way.

needs vector

How fast one number changes when you nudge another.

How to trace a nudge through a long chain of steps. The engine of all training.

needs derivative

A number from 0 to 1 for how much you believe something.

Belief spread across every possible answer, adding to one.

needs probability

How surprised you should expect to be. Low entropy means confident.

needs distribution

The loss used whenever a model picks from a list of options.

needs entropy

Why practitioners take logarithms of everything: multiplication becomes addition, and tiny numbers stop vanishing.

needs probability

One measurable thing you feed the model.

needs model

The right answer, written down by someone, for the model to be scored against.

needs supervised

Hide some data from training, or you will only ever measure memorisation.

needs label

A third slice, used for choosing settings, so the test set stays honest.

needs train-test-split

Learning the noise in your examples instead of the pattern behind them.

needs train-test-split

A model too simple to capture what is actually going on.

needs overfitting

Doing well on things you have never seen. The only success that counts.

needs overfitting

Two ways to be wrong: too rigid, or too jumpy.

needs underfitting

Put features on comparable scales, or training crawls along a canyon floor.

needs feature · learning-rate

When the answer sneaks into the inputs and your results become a lie.

needs train-test-split

When 99% of examples are one class, 99% accuracy means nothing.

needs label

Making more training data by changing what you have in ways that keep the answer true.

needs generalisation

Fit a straight line. The smallest model that genuinely learns.

needs gradient-descent

Bend a line into a probability, and you can classify.

needs linear-regression · probability

Predict whatever the most similar examples did. No training at all.

needs feature

A flowchart of yes/no questions, learned from the data.

needs label

Hundreds of shallow trees voting beat one deep tree.

needs decision-tree

Each new tree fixes the mistakes of the ones before. Still wins on tabular data.

needs decision-tree · gradient

Find the boundary with the widest possible gap around it.

needs logistic-regression

Grouping things nobody labelled, by how close together they sit.

needs unsupervised

Squash many features into a few that keep most of the differences.

needs vector · unsupervised

Penalise complicated answers so the model stops chasing noise.

needs overfitting

Multiply the inputs by weights, add them up, bend the result.

needs linear-regression

The weights say what matters; the bias says where to start.

needs neuron

The bend. Without it, a hundred layers collapse into one straight line.

needs neuron

Keep positives, zero the negatives. Absurdly simple, and it won.

needs activation

Turn any list of scores into probabilities that sum to one.

needs activation · distribution

A row of neurons all looking at the same inputs.

needs neuron

Stack bent layers and you can draw any shape at all.

needs layer · activation

The problem one layer cannot solve — and the reason the field stalled for a decade.

needs mlp

Enough bent pieces can approximate any function you like.

needs mlp

Send the blame backwards through the network with the chain rule.

needs mlp · chain-rule

Turning a word, a user, or an image into a vector of learned numbers.

needs vector · layer

One pass through the data, taken in handfuls.

needs gradient-descent

Use a small random handful each step. Noisier, and far faster.

needs epoch

Let the ball keep some speed so it rolls through flat patches.

needs sgd

Give every dial its own step size, tuned as you go.

needs momentum

Start bold, finish careful.

needs learning-rate

Where the dials start decides whether training ever gets going.

needs mlp

Blame that fades to nothing, or blows up, on its way back through deep networks.

needs backprop

Re-centre the numbers inside the network so training stays stable.

needs vanishing-gradient

Switch off random neurons while training so no single one becomes indispensable.

needs regularisation

Stop when the held-out score turns, not when the training score does.

needs validation

The settings you choose, as opposed to the numbers the model learns.

needs learning-rate

The single most useful diagnostic skill in the whole field.

needs loss · overfitting

Use smaller numbers to train faster, without losing the answer.

needs log-space

Save the dials often; training runs die.

needs epoch

A photograph is a grid of brightness values, nothing more.

needs matrix

Slide a small filter across the image and look for one pattern everywhere.

needs pixel · layer

The small grid of weights that does the looking — learned, not designed.

needs convolution

Shrink the picture, keep the evidence.

needs convolution

Edges become shapes become objects, layer by layer.

needs kernel · pooling

Let the signal skip layers, and suddenly you can train very deep networks.

needs cnn · vanishing-gradient

Learn to remove noise, then start from pure noise and remove it all.

needs cnn · distribution

Cut the image into patches and treat them like words.

needs cnn · transformer

Data where order carries meaning — text, audio, prices, DNA.

needs feature

Chopping text into the pieces a model actually sees. Rarely whole words.

needs sequence

Read one step at a time and carry a memory forward.

needs sequence · mlp

Learned gates that decide what to remember and what to drop.

needs rnn · vanishing-gradient

Read the whole input, then write the whole output.

needs rnn

Squeezing a whole sentence into one vector loses the sentence.

needs seq2seq

Let every position look directly at every other and decide what matters.

needs bottleneck · dot-product

Ask a question, match it against labels, collect the contents.

needs attention

The sentence attending to itself, which is how context gets built.

needs qkv

Several attention patterns at once, each watching for something different.

needs self-attention

Attention has no sense of order, so order has to be added back in.

needs self-attention

Attention, then a small network, twice per layer, with skips. That is the whole thing.

needs multihead · resnet

Hide the future so the model must predict it rather than read it.

needs self-attention

Remember past keys and values so each new token is cheap.

needs causal-mask

The entire training objective: guess the next piece of text.

needs transformer · cross-entropy

Months of next-token prediction over an enormous pile of text.

needs lm-objective · self-supervised

More data, more parameters, more compute — and loss falls predictably.

needs pretraining

Skills nobody trained for, appearing once a model is large enough.

needs scaling-laws

How the model chooses among possible next words, and why it varies.

needs softmax · lm-objective

How much the model can hold in mind at once, and why it costs so much.

needs kv-cache

A model trained to sound right will sound right even when it is wrong.

needs sampling

Only wake the part of the network you need for this token.

needs transformer

Steering a finished model with words alone. No dials move.

needs inference

Show the pattern in the prompt instead of training it in.

needs prompting

Take a trained model and keep training it on your own data.

needs pretraining · loss-curve

Most of what a model learned on other data is still useful on yours.

needs finetuning

Train a tiny patch instead of the whole model. Fits on a free GPU.

needs finetuning · matmul

Store the weights in fewer bits so the model fits on the hardware you have.

needs precision-fp16

Look the facts up and put them in the prompt, instead of training them in.

needs embedding · prompting

Finding the passage that means the same thing, not the one with the same words.

needs embedding · dot-product

Train a small model to copy a big one.

needs transfer

No answer key. Just consequences, and a score you want to raise.

needs gradient-descent

The number the agent is trying to maximise — and the thing it will exploit.

needs rl-basics

Take the known-good option, or gamble on finding better.

needs rl-basics

The rule the agent follows for choosing what to do next.

needs rl-basics

People rank two answers; the model learns which kind to give.

needs reward · finetuning

Making a capable model actually do what was intended.

needs rlhf

Any measure that becomes a target stops being a good measure.

needs reward

Accuracy, precision, recall — and why the wrong one hides your failure.

needs validation

Deciding whether it works, before your users decide for you.

needs metrics

How fast one answer arrives, versus how many you can serve at once.

needs inference

Working out the price of an answer before you offer it to a million people.

needs latency

Serve many requests at once, because the hardware prefers it.

needs latency

The world changes after you ship; the model does not.

needs eval

Seeds, versions, and data snapshots — or your result was luck.

needs checkpoint

Turning raw sensor readings into a usable picture of what is out there.

needs cnn

Finding where the writing is, then working out what it says. The oldest paying job in computer vision.

needs cnn · sequence

Not just what is in the picture, but exactly where, with a box around it.

needs cnn

Labelling every single pixel — road, person, sky — instead of the picture as a whole.

needs detection

Working out how far away things are, from flat images or from time-of-flight sensors.

needs perception

Which way something is facing, and how its parts are arranged. What a drone needs to know about itself.

needs depth

Building a map of somewhere you have never been, while working out where in it you are.

needs pose

Camera, radar, and inertial readings all disagree slightly. Fusion decides who to believe.

needs perception · probability

Going from what the machine sees to what the motors actually do.

needs policy · perception

Train a million times in a simulator, because crashing a real drone costs money.

needs control · rl-basics

Everything that works in the simulator and fails in the rain.

needs sim2real · generalisation

Turning a pressure wave into words, with all the accents and background noise intact.

needs sequence · attention

Going the other way — text into a voice that does not sound like a robot.

needs asr

A machine that learns by acting in the world and living with the consequences.

needs control · self-supervised

A drone deciding in 20 milliseconds is a different problem to a model answering in two seconds.

needs latency · perception

The precise ladder from cruise control to no steering wheel, and where the hard part sits.

needs control · autonomy

A model trained on the past will repeat the past, including its unfairness.

needs generalisation

Working out why the model said that. Genuinely hard, and unsolved.

needs mlp

Models can repeat back training data, including data that was never meant to be public.

needs overfitting

Small, invisible changes to an input can flip a confident answer.

needs generalisation

The honest list, kept up to date, of where these systems fail.

needs hallucination

Where training data came from, and whether anyone agreed to it.

needs pretraining

The difference between a system that suggests and one that acts on its own.

needs inference

Where a human must approve before anything happens, and why that line is drawn.

needs autonomy

Fraud, impersonation, and manufactured evidence — the harms that come from it working, not failing.

needs limits

When seeing is no longer believing, and what that costs a society.

needs diffusion · misuse

Which jobs change, which disappear, and who carries the cost.

needs limits

Training frontier models costs more than most countries will spend, and that concentrates power.

needs scaling-laws

How you supervise something whose reasoning nobody can inspect.

needs interpretability · autonomy