TwiddleLab / Artificial Intelligence / How a Neural Network Thinks

Networks Solve XOR

Lesson 8 of 10 · 7 min · Free, no account needed

What you will learn

• One neuron draws one straight line, so it can't solve XOR.
• Neurons between the inputs and the output form a hidden layer. Each hidden neuron adds a line.
• Combining lines lets a network draw shapes no single line can: that's nonlinearity.
• Every wire is one weight: 2 inputs, 4 hidden and 1 output make 2 × 4 + 4 × 1 = 12.

Start the lesson

How the lesson goes

1. Light the stairs from both switches.

   The light should be on when exactly one switch is up: that's XOR. Turn the weights and the threshold of one neuron. How close can you get?

2. Add a second neuron: a second line.

   Neuron 1's line is fixed. Drag neuron 2's line so the strip between them holds only the lit corners, then Check.

3. A third neuron listens to the other two.

   Press Play. Each corner goes through the wires, layer by layer.

4. You built a neural network.

   Tap to reveal.

   Neurons between the inputs and the output are a hidden layer: you never see their lamps from outside.

   Each hidden neuron draws one line. The output combines them into a shape.

   A network that can draw more than straight lines has nonlinearity. That's what makes networks powerful.

5. Build your own XOR network.

   The hidden layer has an OR neuron and an AND neuron. Tap the two wires into the output to set their weights, and turn the output threshold.

6. Run the network by hand for A = 1, B = 0.

   Tap every lamp you think lights: first the hidden layer, then the output. Then Check.

7. Game: solve three datasets with as few hidden neurons as you can.

   A real network trains on each one. Pick a dataset, set the neurons, press Train. Match par on all three.

8. Your network is a circuit of logic gates.

   Drag a gate onto each neuron of your XOR network, then Check.

9. A network has 2 inputs, 4 hidden neurons and 1 output. How many weights?

   Every input is wired to every hidden neuron, and every hidden neuron to the output. Don't count thresholds.

10. Race on XOR: one neuron against your network.

    Press Race. Both get the four corners.

Previous lesson: A Neuron Is a Weighted Vote · Next lesson: Why Networks Get Fooled · Part of the course: How a Neural Network Thinks

Lessons are interactive and need JavaScript. Turn it on to play them.