Normalizing Flow

Build a generator out of invertible steps. Start from a simple distribution, push it through transformations you can run backwards, and you get both samples and their exact probabilities — the one thing GANs cannot give you and diffusion gives only approximately. The price is that every layer must be invertible, which is a severe constraint on what you can build.

The determinant of the Jacobian has to be cheap, so the architecture is designed around that: coupling layers transform half the dimensions conditioned on the other half, making the Jacobian triangular. Exact likelihoods make flows useful for anomaly detection and for hybrid models even where sample quality lags — and it does lag, because invertibility forbids the discarding of information that other architectures rely on.

With x = f(z) invertible, log p(x) = log p(z) − log|det ∂f/∂z|. Coupling layers split x into halves and set y sub ₁  = x sub ₁ , y sub ₂  = x sub ₂ ·exp(s(x sub ₁ )) + t(x sub ₁ ), giving a triangular Jacobian whose log-determinant is Σ s(x sub ₁ ) — linear rather than cubic in dimension, which is what makes the whole family tractable.