Loading your learn pack…
Read the summary, then start training!
Diffusion models learn a forward noising process and a reverse denoising process to sample from a target data distribution.
Diffusion models (also called diffusion-based or score-based generative models) are latent variable generative models that learn to generate data by simulating a gradual corruption process and then reversing it. They consist of a forward diffusion process that repeatedly adds Gaussian noise to data until it becomes (approximately) standard Gaussian, and a reverse sampling process that uses a learned neural network (“backbone”) to denoise step-by-step and recover samples from the original data distribution. In the DDPM formulation, the forward process is a Markov chain with a fixed noise schedule, while the reverse process is parameterized by a neural network that predicts the mean (and often variance) of the denoising distribution at each time step. Training is typically done via variational inference, which can be expressed as learning to predict noise (or equivalently the score/gradient of the log-density). In the score-based view, the network is trained to approximate the score function ∇x log ρt of the noisy data distribution at noise level t, enabling generation by integrating a reverse-time stochastic differential equation (SDE) or its discrete-time counterpart. A key result is the equivalence between DDPMs and score-based generative models: with appropriate parameterizations and in the continuous limit, the denoising network’s predicted noise corresponds to the score function, so both formalisms describe the same underlying generative mechanism. This equivalence also explains why diffusion models can be implemented with different but related architectures and sampling strategies (e.g., faster samplers like DDIM), while still relying on the same learned denoising/score estimation principle.
Diffusion models learn a forward noising process and a reverse denoising process to sample from a target data distribution.
DDPMs are trained via variational inference and typically learn to predict noise; the reverse process is modeled as Gaussian denoising steps.
Score-based generative models train a network to approximate the score ∇x log ρt and generate by integrating a reverse-time SDE.
DDPMs and score-based models are equivalent: the predicted noise can be transformed into a score estimate, and the continuous-time limits match.
A generative model that introduces hidden (latent) variables to explain how observed data are produced.
A Markov (or continuous-time) process that gradually adds Gaussian noise to data until it approaches a simple distribution like N(0,I).
A learned process that iteratively removes noise from a noisy sample to generate new data resembling the training distribution.
A diffusion-model framework where the reverse denoising distribution is parameterized by a neural network and trained using variational inference.
A predefined sequence (or function) controlling how much noise is added at each diffusion step.
The neural network (often a U-Net or transformer) that predicts denoising quantities such as noise, mean, or score at each step.
A training approach that optimizes a lower bound on the data likelihood by approximating intractable posteriors.
The gradient of the log probability density, ∇x log ρt, which indicates how to move in input space to increase likelihood under ρt.
A neural network trained to output an estimate of the score function for a distribution at a specific noise level.
A relationship that connects the score ∇x log q(xt) to the conditional expectation E[x0|xt], enabling score estimation from denoising predictions.
A continuous-time model describing dynamics with both drift and random noise, used to formalize diffusion and reverse-time generation.
The SDE whose integration generates samples by reversing the forward diffusion dynamics using the learned score.
The correspondence showing that DDPM noise-prediction training implies a score estimate, and in the continuous limit both approaches yield the same reverse dynamics.
A sampling method that uses a DDPM-trained model to skip steps by making the reverse process partially or fully deterministic, trading speed for controllable quality loss.
“Can you explain what "Diffusion models learn a forward noising process and a reverse denoising process to sample from a target data distribution." means in simple terms?”