Many Neurons, Many Knobs

Lesson 1 of 10 · 5 min · Free, no account needed

What you will learn

A layer is a group of neurons that all read the same inputs.
Its weights form a grid, the weight matrix W: one row per neuron, one column per input.
Running the layer is W x: each row times the input column, added up. A layer has inputs × neurons weights.

This tiny network has a knob on every wire. A photo app has millions. How does anyone keep track?
Start small: 3 inputs about a critter, 2 neurons.
Connect every input to every neuron.
Tap an input, then a neuron. Each wire gets a cell in the grid.
Run the layer, one cell at a time.
Each weight multiplies its input. A neuron adds up its row.
The grid is the layer's weight matrix.
Writing a layer as a matrix lets a computer run millions of weights the same simple way.
Set neuron 1's weights so it outputs 14.
The inputs stay 2, 1 and 3. Only row 1 changes.
Neuron 1 computes 2 × w1 + 1 × w2 + 3 × w3. Many settings give 14, for example 1, 3, 3 or 2, 1, 3.
A digit picture has 784 pixels. A layer of 10 neurons reads all of them. How many weights?
Weights = inputs × neurons = 784 × 10 = 7840. The matrix W has 10 rows and 784 columns.
Finish the layer: make it output 14 and 7.
Row 1 is yours from before. Tap the row 2 cells. Fewer taps, more stars.

Lessons are interactive and need JavaScript. Turn it on to play them.