3 / 4 Architectures (CNN / RNN)

Zoom Out With Every Layer

After stacking several 3x3 convolution layers followed by 2x2 max pooling layers, a single unit deep in the network responds to a much larger region of the original input image than any single 3x3 filter covers directly. What is this effective input region called, and what causes it to grow?

A

The gradient horizon; it grows because backpropagation accumulates gradients over more time steps

B

The embedding window; it grows because tokenization merges adjacent pixels into larger patches before the first layer

C

The receptive field; it grows because stacking convolution and pooling layers lets each successive layer's units aggregate information from a wider area of the original input

D

The attention span; it grows because self-attention layers directly connect every pixel to every other pixel

E

The latent bottleneck; it grows because the fully connected output layer compresses all spatial information into one vector