01 · Convolutions 🔲

5 min read

The single idea that makes computer vision work: instead of looking at every pixel independently, slide a small learnable filter across the image to detect local patterns everywhere at once.

1. The 'Why'

Imagine feeding a modest 224×224 color image to a fully-connected nn.Linear layer. You'd first flatten it into a vector of 224 × 224 × 3 ≈ 150,000 numbers, and a single hidden layer of just 1,000 neurons would then need 150 million weights — for one layer. Worse, that dense layer treats every pixel as unrelated to its neighbors, so it has to learn "what an eye looks like in the top-left corner" completely separately from "what an eye looks like in the center." It's both wildly wasteful and blind to the most obvious truth about images: patterns are local, and the same pattern can appear anywhere.

The convolution solves both problems at once. Instead of a giant weight matrix, you learn a tiny filter (or kernel) — say a 3×3 grid of weights — and slide it across the entire image, computing a small weighted sum at each position. That one small filter detects its pattern (an edge, a color blob, a texture) everywhere in the image, so you get translation invariance for free and use a few dozen weights instead of millions. Stack convolutional layers and something remarkable emerges: early layers learn simple features like edges, and deeper layers combine those into eyes, wheels, and eventually whole objects. This hierarchy of learned features, built from the humble sliding filter, is the engine of modern computer vision.

2. Core Concepts

The kernel/filter. A small grid of learnable weights (commonly 3×3). It slides over the input, and at each location computes a dot product with the patch it covers. The kernel's weights are what the layer learns.

Channels. Images have channels (3 for RGB). A conv layer maps in_channels to out_channels; each output channel is a distinct filter detecting a different pattern. Its outputs are called feature maps.

nn.Conv2d(in_channels, out_channels, kernel_size). The 2-D convolution layer. Key extra args: stride (how far the filter jumps each step) and padding (zeros added around the border to control output size).

Output size shrinks (unless you pad). A 3×3 filter with no padding trims 1 pixel off each edge. padding=1 with a 3×3 kernel keeps the spatial size the same — a common "same convolution" trick.

Convolution usually pairs with an activation. As with linear layers, a ReLU follows each conv so the network can model non-linear patterns.

3. Code in Action

A single convolution layer

import torch
import torch.nn as nn

# 3 input channels (RGB) -> 16 output feature maps, using 3x3 filters
conv = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3, padding=1)

# A batch of 8 RGB images, 32x32 pixels: (batch, channels, height, width)
x = torch.randn(8, 3, 32, 32)
out = conv(x)
print(out.shape)     # torch.Size([8, 16, 32, 32]) -> 16 feature maps, size kept by padding=1

How kernel size, stride, and padding change the output

import torch
import torch.nn as nn

x = torch.randn(1, 1, 28, 28)                 # one grayscale 28x28 image

no_pad = nn.Conv2d(1, 4, kernel_size=3)        # padding=0 -> shrinks by 2
print(no_pad(x).shape)                         # torch.Size([1, 4, 26, 26])

same = nn.Conv2d(1, 4, kernel_size=3, padding=1)  # padding=1 -> size preserved
print(same(x).shape)                           # torch.Size([1, 4, 28, 28])

strided = nn.Conv2d(1, 4, kernel_size=3, stride=2, padding=1)  # stride halves size
print(strided(x).shape)                        # torch.Size([1, 4, 14, 14])

Conv + ReLU, the standard pairing

import torch
import torch.nn as nn

block = nn.Sequential(
    nn.Conv2d(3, 16, kernel_size=3, padding=1),   # detect 16 local patterns
    nn.ReLU(),                                    # non-linearity
)
x = torch.randn(4, 3, 64, 64)
print(block(x).shape)     # torch.Size([4, 16, 64, 64])

4. Common Pitfalls

Pitfall 1 — Wrong input layout. Conv2d expects (batch, channels, height, width) — channels-first. If your images are (batch, height, width, channels) (as NumPy/PIL often are), the layer errors or produces nonsense. Use ToTensor() from Module 04, which puts channels first, or permute the axes.

Pitfall 2 — Mismatched in_channels. The in_channels of a conv layer must equal the channel count of its input: 3 for the first layer on RGB, but the out_channels of the previous conv for later layers. Chaining Conv2d(3,16,...) into Conv2d(3,32,...) is a shape error — the second should take 16 in.

Pitfall 3 — Forgetting the batch dimension. A single image is (C, H, W), but the layer needs a batch: (1, C, H, W). Add it with img.unsqueeze(0) (Module 01) before running a lone image through the model.

5. Further Reading & Watch List

➡️ Next: 02 · CNN Architecture