Rolling Down the Loss

Lesson 5 of 10 · 7 min · Free, no account needed

What you will learn

Feel the slope, step downhill, repeat: that's gradient descent.
The slope of the loss under the ball is the gradient. It points uphill, so the step goes the other way.
The step size is the learning rate. One step: new m = m - learning rate × gradient.
Too small crawls, too big overshoots. A good learning rate is quick and still settles.

A computer can't see the whole error curve.
It's like standing on a hill in fog. Which way would you step to find the bottom? Tap an arrow, then Check.
Same six players, simpler line: only its slope m can change.
Turn the slope. Each slope leaves an error bar on the right. Get the error under 0.3.
Step. Step. Step.
Press Step. The ball feels the slope under it and steps downhill. Keep going to the bottom.
Can it get there in one jump?
Tap a step size to preview the jump. Pick the one that lands closest to the bottom, then Check.
You just reinvented the most important algorithm in AI.
Tap to reveal.
Feel the slope, step downhill, repeat: this is gradient descent.
The slope of the loss under the ball is the gradient. It points uphill, so we step the other way.
The step size is the learning rate. One step: new m = m - learning rate × gradient.
Game: bottom in 6.
The ball starts on the right. Set the learning rate and press Go: it takes 6 steps. First try earns 3 stars.
Be the computer. Build one step.
This is a simpler valley. Put the tiles in order to get the new m, then Check.
Two numbers to tune? Then the hill is a map.
With two knobs the loss is a landscape. Turn the arrow so the ball goes straight downhill, then Check.
m = 2.5, gradient = 3, learning rate = 0.1. Where does the ball land?
Slide the marker on the m axis, then Check.
Race: your learning rate against 0.05.
Both balls start at the same spot. Press Race and see who settles at the bottom first.

Lessons are interactive and need JavaScript. Turn it on to play them.