TwiddleLab / Artificial Intelligence / Learning by Reward

Trial and Error

Lesson 2 of 10 · 7 min · Free, no account needed

What you will learn

• An episode is one try, from the start to an ending.
• The return is the total reward of one episode.
• A policy says which action to take in every state. Keeping what worked makes it better.

Start the lesson

How the lesson goes

1. How do you get good at a game nobody explained to you?

   Start with a critter that knows nothing at all.

2. Let the critter try at random.

   Each try leaves a trail. Green trails found the food.

3. Now let it remember its best try.

   Run 20 tries each way and compare the totals.

4. Three words for what you saw.

   Learning like this is trial and error: try, keep what worked, try again.

5. What is the return of this episode?

   Add up the rewards of all 8 moves.

   Moves 1 to 7 each cost -1 (the bump too: it costs a move even though the critter stays put). Move 8 lands on the food for +10. So -7 + 10 = 3.

6. Write your own policy: an arrow for each square.

   Make the arrows lead from the start to the food with total 4.

7. This policy has three bad arrows. Fix them.

   From every square, the arrows should lead to the food.

   A policy is a rule for every square, so one bad arrow spoils every route that passes through it.

8. In code, a policy is a dictionary.

   Each square is a key and its move is the value. Fill in the blank and run the loop.

9. Follow this policy. Where does the episode end?

   Right, right, up, then right again: straight into the thorns. The critter does not think, it just follows the policy.

10. Speed round: be the policy.

    A square lights up. Tap the best move from it before time runs out.

Previous lesson: Agent, World, Reward · Next lesson: Value: How Good Is This Square? · Part of the course: Learning by Reward

Lessons are interactive and need JavaScript. Turn it on to play them.