Zoom Out With Every Layer

After stacking several 3x3 convolution layers followed by 2x2 max pooling layers, a single unit deep in the network responds to a much larger region of the original input image than any single 3x3 filter covers directly. What is this effective input region called, and what causes it to grow?

The gradient horizon; it grows because backpropagation accumulates gradients over more time steps

The embedding window; it grows because tokenization merges adjacent pixels into larger patches before the first layer

The receptive field; it grows because stacking convolution and pooling layers lets each successive layer's units aggregate information from a wider area of the original input

The attention span; it grows because self-attention layers directly connect every pixel to every other pixel

The latent bottleneck; it grows because the fully connected output layer compresses all spatial information into one vector