TwiddleLab / Artificial Intelligence / How Language Models Work

Guess the Next Word

Lesson 1 of 12 · 5 min · Free, no account needed

What you will learn

• A language model looks at the words so far and gives every possible next word a probability.
• The probabilities for one gap add up to 100%. Some gaps have a clear winner, many don't.
• A chatbot writes one guess at a time: it adds the picked word and guesses again.

Start the lesson

How the lesson goes

1. Your phone keyboard guesses your next word. What would you type?

   You typed "See you". Tap the word you'd type next and see what the keyboard suggests.

2. You vs the model: five phrases.

   Tap the word you'd type next. A point every time you pick the model's tallest bar.

3. Now let the model write by itself.

   Press Step. Each time, it scores every next word and takes the tallest bar.

4. What you just played has names.

   Tap to reveal.

   A language model is a program that takes the words so far and guesses what comes next. Your keyboard has a tiny one. ChatGPT is a giant one.

   It gives every possible next word a probability: how likely it is, from 0% to 100%. The bars show them, and for one gap they add up to 100%.

   Writing is one guess at a time: pick a word, add it, guess again.

5. Be the model: make your own bars.

   Drag the bars to show how likely each word is. They always add up to 100%. Then Check to see the model's bars.

6. The bars are covered. Which word is the model's top pick?

   Tap a word, then Check.

7. Speed round: three phrases, three seconds each.

   Drag the most likely word into the gap before the bar runs out. Can you beat your game score?

Next lesson: Counting Is a Model · Part of the course: How Language Models Work

Lessons are interactive and need JavaScript. Turn it on to play them.