Diffusion from Zero: learning generative models from the ground up
Why I started this series as an electrical engineering student with no probability course behind me, how the parts fit together, and the resources that helped most.
THE SERIES
Diffusion from Zero
Building up to diffusion models and Diffusion Policy, one concept at a time.
In this series
Distributions, Bayes' theorem, expectation, and Jensen's inequality: the four ideas behind the ELBO
Why a deep-learning series has to start with probability: what distributions, Bayes' theorem, expectation, and Jensen's inequality each contribute to generative models.
Where MSE and cross-entropy come from: maximum likelihood and KL divergence
Maximum likelihood explains the losses we use every day: a Gaussian assumption gives MSE, a categorical one gives cross-entropy, and both connect to KL divergence.
What each probability actually means: a field guide to VAE and diffusion notation
p(x), p(z), p(x|z), p(z|x), q, and p-theta: what each expression in VAEs and diffusion models describes, explained with concrete examples instead of derivations.
Autoencoders: compression, bottlenecks, and why they aren't quite generative
How an autoencoder compresses and reconstructs its input, why the bottleneck matters, what reconstruction loss means in probability terms, and why a plain autoencoder is hard to sample from.
VAEs and the ELBO: learning a distribution instead of a point
Why an autoencoder isn't enough for generation, how a VAE learns a latent distribution, what each term of the ELBO does, and how the reparameterization trick makes it trainable.
Reading Understanding Diffusion Models: from VAEs to score-based models
Following Calvin Luo's unified perspective in order: the ELBO, hierarchical VAEs, variational diffusion models, the three equivalent prediction targets, and the score-based view.
Reading Diffusion Policy, part 1: from formulation to network design
Sections 1 to 3 of Diffusion Policy: why behavior cloning needs multimodal action distributions, how DDPM becomes a visuomotor policy, and the CNN, transformer, and FiLM design choices.
Reading Diffusion Policy, part 2: why it trains stably, and what the robots show
Sections 4 to 9 of Diffusion Policy: modes and basins, why learning the score removes the normalizing constant, the benchmark and real-robot results, and the limits the authors state.