Training

Training starts with a mistake score

A model learns by being wrong in a measurable way. Take the simplest model there is, a straight line with two parameters, a slope w and a shift b. It should predict the dots as well as it can.

The dashed lines are the errors, the distance between each dot and the prediction. Square every error and take the average, and you get one number called the loss. Here it is the mean squared error. Lower is better, and zero would mean every prediction is perfect.

Try it. Move the sliders and bring the loss as low as you can.

The loss landscape

Every pair of values for w and b gives a loss, so the loss forms a landscape. The map shows it from above, dark where the loss is low and bright where it is high. Training means finding the lowest point of this landscape.

With two parameters you can see the whole map at once. A language model has billions of parameters, so its landscape has billions of dimensions and nobody can draw it. Training has to find its way down without a map.

Try it. Click anywhere on the map to put the model there and watch the line on the right.

The gradient points uphill

Without a map you can still feel the slope under your feet. The gradient is the slope of the loss for every parameter, how much the loss grows when that parameter grows a tiny bit. Calculus gives it exactly for any model built from simple operations.

Step against the gradient and you go downhill. Every step changes each parameter by its own slope times a small number called the learning rate. That is gradient descent, the core of almost all training.

Try it. Press the step button a few times and watch the arrow shrink as the ground gets flat.

The learning rate

The learning rate sets the step size. Too small and training crawls. Too big and every step jumps over the valley to the other side, a bit higher each time, until the numbers explode.

Notice how the path first slides down the steep side and then creeps along the flat valley floor. Real landscapes are full of such valleys, which is why practical training uses smarter steps such as momentum and Adam, and lowers the learning rate over time.

Try it. Run all three learning rates and compare the loss curves.

Training a real network

A neural network trains the same way, only with more parameters. Here 2 inputs go through two hidden layers of 8 neurons to one output, 105 parameters in all. The loss now measures how sure the network is about the wrong color.

The gradient of every weight is found by backpropagation. The error at the output is passed backwards through the layers, and the chain rule of calculus gives each weight its share of the blame. The backward pass costs only about twice as much as running the network forward.

Try it. Press play and watch the colors find their shape. The spiral is the hard one.

Learning too well

Low loss on the training data is not the goal. The goal is good answers on new data. A model with too many parameters for its data can bend itself through every training point, noise included, and then fail on anything new. That is called overfitting.

Here a curve learns from 10 noisy points. A second set of points, never shown to it, checks the result. As the curve gets more flexible the training error keeps falling, while the test error first falls and then climbs. Real training watches such a held-out set and stops in time.

Loss

One number that says how wrong the model is

Update

parameter = parameter − learning rate × gradient

Gradients

Backpropagation, the chain rule applied layer by layer

In practice

Small random batches, Adam optimizer, a held-out test set