Learning CNNs: architecture, history, and training

Convolutional layers

Convolutional layers are the core building blocks of convolutional neural networks (CNNs). They take an input tensor (channels with spatial height and width) and apply learnable filters (kernels) across local receptive fields to produce an activation/feature map. Each neuron in a convolutional layer processes only a restricted region of the previous layer (its receptive field), and sliding the same kernel across the input generates spatially arranged responses that feed subsequent layers. A key mechanism in convolutional layers is parameter sharing: the same filter weights and bias are reused at different spatial locations (and across the depth slice), which greatly reduces the number of free parameters compared with fully connected layers. This makes CNNs more efficient, helps training stability (e.g., mitigating vanishing/exploding gradients), and supports translation-equivariant behavior (shifted inputs lead to correspondingly shifted feature maps). Convolutional layers also depend on hyperparameters such as kernel size, stride, and padding, which determine the output feature map size. Convolutional layers can be combined with other components such as pooling (for downsampling and local invariance) and nonlinear activations like ReLU. Variants like depthwise separable convolution (depthwise followed by pointwise 1×1 convolution) can speed computation, and dilated (atrous) convolutions can expand the receptive field without increasing parameter count by interleaving “blind” regions.

0/2 modes complete

Pooling layers

0/2 modes complete