TwiddleLab / Artificial Intelligence / How Machines See and Hear

Stacking Filters: Convolution Layers

Lesson 3 of 10 · 7 min · Free, no account needed

What you will learn

• A convolution layer is a set of filters that turns one stack of maps into a new stack of maps.
• Layer 2 reads layer 1's maps, not the pixels: two kinds of edges nearby make a corner.
• Deeper layers combine combos into parts (ears, eyes), then whole things: a feature hierarchy.
• Each layer sees a bigger patch: two 3 × 3 layers see 5 × 5 pixels.

Start the lesson

How the lesson goes

1. An edge isn't an ear. How does a machine get from lines to a critter?

   Start small. A second layer looks at these two edge maps. Tap where you think it lights up, then Check.

2. Send the square through both layers.

   Press Run and watch each layer's maps appear.

3. How much of both edges does layer 2 need? Find the setting where only the corners fire.

   Each edge adds up to 1. The slider is the bar the sum must pass.

4. Stacking filters has names.

   Tap to reveal them.

   A set of filters that turns maps into new maps is a convolution layer.

   Layer 2 is a hidden layer that reads layer 1's maps: edges become corners and curves.

   Stack more layers and they find parts, then whole things: a feature hierarchy.

5. Wire your own layer-2 unit. Make it fire at every corner of both shapes.

   Drag two layer-1 maps into its sockets. Watch the test shapes.

   The unit fires only where both inputs are strong nearby. Two vertical maps are strong together along the sides, blur is strong inside the whole shape, but vertical plus horizontal are strong together only at corners.

6. Game: shape scanner. Predict where your detector fires on 4 shapes.

   Tap every spot you expect, then Scan. Wrong taps cost half a point.

7. Peek inside a vision model. Which layer made each map?

   Drag each map into Layer 1, 2 or 3.

   Each layer can only combine what the layer below found. Edges come from pixels, corners from edges, parts like eyes and ears from many corners and curves.

8. Math: how big a patch does one layer-2 neuron see?

   Size the frame, then Check.

   A layer-2 neuron covers layer-1 cells from one step left to one step right. Each of those reaches one more pixel out. So the width is 3 + 2 = 5. A third layer gives 7, and so on.

9. Build the feature tower, from pixels to a critter.

   Drag each card to a floor, bottom first, then Check.

   Each floor is made by filters looking at the floor below it. Pixels give edges, edges give corners, corners give parts, parts give the whole critter.

Previous lesson: Filters Find Edges · Next lesson: Pooling: Zooming Out · Part of the course: How Machines See and Hear

Lessons are interactive and need JavaScript. Turn it on to play them.