TwiddleLab / Artificial Intelligence / Learning by Reward
Project: Train a Critter to Find Food
What you will learn
- An RL project is a recipe: a world, a reward, settings (ε, γ, learning rate) and many episodes.
- Random moves help it learn but cost reward while it trains.
- Always watch the trained agent: a reward with a loophole gets exploited.
How the lesson goes
Your critter is hungry and the world is full of thorns. Pick a world to start from.
Then design its rewards and let it learn.
Build your world.
Place food, thorns, water and rocks. Follow the three rules on the right.
Design the reward.
Everything starts at 0. The map shows how good each square becomes.
Train your critter with Q-learning.
Your recipe, in the words of this course.
Every RL project chooses these. Next you will tune the last four.
How much should it explore while it learns?
Train three times with different ε and compare the returns.
A friend's critter never eats. Find the loophole.
Watch it, then tap the square where it games its reward.
Each move onto water pays +2. Stepping back and forth between the two water squares earns +2 on every move until time runs out, far more than the food's +10. The fix: water should cost a little, or pay only once.
Classic challenge: train it to average 8 or more.
Tune ε, γ and the learning rate. The last 50 of 500 training episodes must average at least 8.
For which job is a good reward hardest to write down?
A driver must be safe, fast, legal, smooth and polite all at once. Reward speed and it speeds; reward safety only and it never leaves the garage. Chess and mazes have one clear outcome.
What is the return of this 9-move path?
Moves 1 to 8 cost -1 each, and move 9 lands on the food for +20: -8 + 20 = 12.
Your world card.
Replay your trained critter next to one that moves at random.
Previous lesson: RL in Chatbots · Part of the course: Learning by Reward
Lessons are interactive and need JavaScript. Turn it on to play them.
