is a field of AI focused on statistical algorithms that learn from data and generalize to unseen data without explicit programming.

Context: Machine learning definition

Deep learning is a subset of , and ML is a subset of AI.

Context: Relationship between deep learning, ML, and AI

is the ability to perform accurately on new, unseen examples after training on a finite dataset.

Context: Generalization meaning

A measures the discrepancy between model predictions and true outcomes.

Context: Loss function concept

Many ML/deep learning algorithms can be described as minimizing on training data under a theoretical framework.

Context: Empirical risk minimization (ERM)

Minimizing a loss function on training data (empirical risk minimization) causes which leads to improved predictive performance.

Context: Cause→effect: ERM leads to parameter learning and better prediction

Finite training sets and uncertainty about the future cause learning theory to provide rather than absolute guarantees.

Context: Cause→effect: uncertainty leads to probabilistic bounds

provides a theoretical framework for describing when learning algorithms can achieve near-optimal performance with high probability.

Context: PAC learning meaning

Clustering groups similar unlabeled data points into k clusters, which causes data to be represented compactly using .

Context: Cause→effect: clustering leads to centroid-based compact representation

k-means clustering partitions data into k clusters, each represented by a .

Context: k-means clustering term

A system that predicts posterior probabilities of a sequence given its history can be used for optimal data compression because arithmetic coding turns probabilistic predictions into representations.

Context: Cause→effect: posterior prediction enables optimal compression

An optimal compressor can encode symbols effectively given prior history, which causes it to be used for .

Context: Cause→effect: optimal compression enables prediction

Supervised learning trains models using data to predict outputs such as class labels or numeric values.

Context: Supervised learning meaning

Unsupervised learning finds structure in data, with clustering grouping similar points into clusters.

Context: Unsupervised learning meaning

ML overlaps with statistics and data mining, but ML emphasizes generalizable prediction while data mining (KDD) emphasizes discovering previously unknown in data.

Context: Relationship to data mining (KDD) vs ML goals