Autoencoder definition and two-part structure

Summary

An autoencoder is an unsupervised neural network designed to learn efficient codings of unlabeled data. It consists of two main parts: an encoder that maps the input message to a latent code (a compressed representation), and a decoder that reconstructs the original message from that code. The model is trained to minimize reconstruction error, so the learned code captures the most useful structure in the data. Mathematically, an autoencoder is defined by (1) a latent (encoded) space Z and a decoded (message) space X, and (2) two parametrized function families: an encoder E_φ: X → Z and a decoder D_θ: Z → X. For an input x, the encoder produces a code z = E_φ(x); the decoder then reconstructs x' = D_θ(z). If the latent space has fewer dimensions than the input (an undercomplete autoencoder), the network is forced to compress rather than simply copy, enabling dimensionality reduction and meaningful embeddings for downstream tasks.

Key Points

• 1

  An autoencoder learns an efficient latent representation by training an encoder-decoder pair to reconstruct inputs.

• 2

  The two-part structure is: encoder maps input to a code, and decoder reconstructs the input from the code.

• 3

  Autoencoders are defined using an encoded space Z, a decoded space X, and parametrized functions E_φ and D_θ.

• 4

  Training quality is determined by reconstruction loss, often using a distance function d(x, x').

• 5

  Undercomplete autoencoders (latent dimension smaller than input) act like compression/dimensionality reduction; overcomplete models may learn identity behavior unless constrained.