TwiddleLab / Artificial Intelligence / How Language Models Work
From Autocomplete to Assistant
What you will learn
- A base model only continues documents. Asked a quiz question, it may write more quiz questions.
- Fine-tuning on example dialogs teaches it to answer. Human feedback (votes on better answers) makes it more helpful.
- A system prompt is a hidden instruction the app adds to every chat. The model reads it like any other text.
How the lesson goes
A raw model reads "What is the capital of France?" What comes next?
Drag the line you think a raw language model writes next.
Let the raw model write.
Press Write.
You are the rater. Pick the better answer, four times.
Watch the model's bars for the France question after each vote.
How a raw model becomes an assistant.
Tap to reveal.
The model after pretraining is a base model: it continues documents.
Fine-tuning trains it more on example dialogs, so it learns to answer. Human feedback, votes on better answers, makes it more helpful and polite.
On top, the app adds a system prompt: a hidden instruction at the start of every chat.
Put the path in order: raw model to assistant.
Drag the three stages onto the belt, then Check.
This bot said "Ahoy!" Nobody asked for a pirate. Why?
Tap words in the bot's reply to see where they look.
Write the hidden instruction.
Build an instruction from tiles so the bot meets all three goals. Test as often as you like, fewer tries is better.
Your bot against another bot on the same model.
Ask both bots the same questions. Ask at least two.
Which stage gave the assistant each behaviour?
Drag each behaviour onto a stage, then Check.
A mystery: what happens when you hit Send?
Press Send and watch.
Previous lesson: Why It Makes Things Up · Next lesson: What Happens When You Hit Send · Part of the course: How Language Models Work
Lessons are interactive and need JavaScript. Turn it on to play them.
