TwiddleLab / Artificial Intelligence / Learning From Data
Ask Your Neighbors: kNN
What you will learn
- k-nearest neighbours labels a new example by a vote of the k closest examples in the dataset.
- With k = 1, one odd example can decide everything. A middle k gives a calmer vote; a huge k lets far-away examples vote too.
- kNN keeps the whole dataset. Change the examples and you change its answers.
How the lesson goes
The new critter has ears of 4.8 cm and a tail of 7.4 cm. Puff or Zip?
Make a gut call: tap the known critter it looks most like. Then Check.
Ask the 3 closest critters.
Drag the violet critter around the map. Lines go to its 3 nearest neighbours, and they vote.
One neighbour might be a strange one. Ask more.
k is how many of the closest critters get a vote. Slide it.
What you just did is a real machine learning method.
Tap to reveal its name.
It's called k-nearest neighbours, or kNN.
To label a new example, find the k closest examples. Close means the features are similar: the distance between the points is small.
They vote. The kind with the most votes wins. With two kinds, an odd k avoids ties.
There's no rule inside. The dataset itself is the model.
Out-guess the vote.
Six mystery critters, one at a time. Tap what its 3 nearest neighbours will say before the lines appear.
Teach it with your own critters.
Right now k = 3 says Puff for the violet critter. Pick a brush and tap the map to add critters until it says Zip. Then Check.
One odd Zip is fooling kNN.
With k = 1, two of the four test critters get the wrong answer. Try other values of k until all four are right. Then Check.
Code: kNN is a short recipe.
A program does exactly what you did, in this order. Put the stations on the belt. Sorting is the same idea as in Sorting, One Swap at a Time.
Your map, with your critters in it. What does k = 5 say?
Count the colours at the ends of the 5 lines and tap Puff or Zip. Then Check.
Three flicks.
Each critter shows its 3 neighbours for 3 seconds. Drag it to Puffs or Zips, tap a pile, or press Left or Right.
Previous lesson: Data Is a Table · Next lesson: Twenty Questions: Decision Trees · Part of the course: Learning From Data
Lessons are interactive and need JavaScript. Turn it on to play them.
