TwiddleLab / Courses / Artificial Intelligence

Learning by Reward

An agent tries, fails, gets a reward and slowly learns what works.

Advanced · 10 lessons · Free, no account needed

Best after: Search, Games and Plans, Probability by Simulation

Start the course

Lessons

1. Agent, World, Reward (5 min): An agent acts in an environment. Where things stand is the state; what it can do is an action.
2. Trial and Error (7 min): An episode is one try, from the start to an ending.
3. Value: How Good Is This Square? (7 min): The value of a square is the best total the agent can still get from there.
4. Q-Learning (7 min): A Q-value is a note for one move in one square: how good that move turned out, counting what came after.
5. Explore or Exploit? (7 min): Exploit: use the best option you know. Explore: try something else to learn more.
6. Discounting the Future (7 min): The discount factor γ says how much a reward one move later still counts.
7. Reward Hacking (7 min): Reward hacking: an agent finds a way to score high that nobody intended.
8. Learning to Play Games (7 min): Self-play: an agent learns a game by playing against a copy of itself.
9. RL in Chatbots (7 min): RLHF: people rate answers, a reward model learns to score like them, and RL trains the chatbot toward high scores.
10. Project: Train a Critter to Find Food (8 min): An RL project is a recipe: a world, a reward, settings (ε, γ, learning rate) and many episodes.

Lessons are interactive and need JavaScript. Turn it on to play them.