Learning Representations Across Supervision Types

Representation learning and feature learning overview

Representation learning (also called feature learning) is a set of machine-learning techniques that automatically discovers useful representations from raw data (such as images, text, or sensor signals) for downstream tasks like classification or regression. The goal is to avoid manual feature engineering by learning both the features and how to use them, often improving training speed and task performance compared with feeding raw data directly. Feature learning can be supervised, unsupervised, or self-supervised. In supervised feature learning, labeled data (inputs paired with ground-truth labels) provides an error signal that guides learning; examples include supervised dictionary learning and neural networks (including multilayer architectures that learn hidden-layer representations). In unsupervised feature learning, features are learned from unlabeled data by exploiting structure in the data, often producing lower-dimensional or structured representations; common methods include k-means clustering, PCA, LLE, ICA, and unsupervised dictionary learning (e.g., sparse coding). Self-supervised learning uses unlabeled data but constructs “pseudo-labels” or learning signals from the data itself, enabling supervised-style optimization (e.g., word embeddings and autoencoders, and more recently deep models like CNNs and transformers across many modalities). The overview also highlights deep and multilayer architectures for representation learning, such as restricted Boltzmann machines (RBMs) and autoencoders, where intermediate layers act as learned feature detectors. Self-supervised approaches are typically trained with contrastive and/or generative objectives, and dynamic representation learning extends embeddings to time-evolving systems by accounting for possible misalignment due to arbitrary transformations. Overall, representation learning provides a general framework for extracting task-relevant structure from complex data across many learning paradigms and data types.

0/2 modes complete

Supervised feature learning with labeled data

0/2 modes complete