Learning CNNs: architecture, history, and training

Convolutional layers

Convolutional layers are the core building blocks of convolutional neural networks (CNNs). They take an input tensor (height, width, and channels) and apply learnable filters (kernels) across local receptive fields, producing an activation/feature map for the next layer. Each neuron in a convolutional layer only “sees” a restricted region of the previous layer (its receptive field), which makes CNNs practical for large inputs like images by greatly reducing the number of parameters compared with fully connected layers. A key idea in convolutional layers is parameter sharing: the same filter weights and bias are reused across different spatial locations. This reduces memory and computation, helps training stability (mitigating vanishing/exploding gradients), and yields translation-equivariant responses (feature maps shift in a predictable way when the input shifts). Convolutional layers also depend on hyperparameters such as kernel size, stride, and padding, which determine the spatial size of the output feature maps. Convolutional layers can be extended or modified for efficiency and larger context. For example, depthwise separable convolutions replace standard convolution with a depthwise spatial convolution per channel followed by a pointwise (1×1) convolution. Dilated (atrous) convolutions expand the receptive field without increasing parameters by inserting “holes” between sampled positions, enabling multi-layer context growth more efficiently.

0/2 modes complete

Pooling layers

0/2 modes complete