RealNVP for physicists

To understand RealNVP, its proper to start from normalizing flow

Understand Bijectors

Suppose we have a Bijector $z=f(x)$

Inference refers to

$x\sim \hat p_X$


Generation referes to

$z\sim p_Z$


Tractable density

The change of probability

$p_X(x) = p_Z(f(x))\left\vert\det \left(\frac{\partial f(x)}{\partial x^T}\right)\right\vert$ $\log p_X(x) = \log\left(p_Z\left(f(x)\right)\right) + \log\left(\left\vert\det \left(\frac{\partial f(x)}{\partial x^T}\right)\right\vert\right)$

The detailed derivation

Let $(a, b) = x$ and $(u, v) = z$

Contributed by Shuo-Hui Li

Here, the matrix inside the determinant is the Jacobian, which can be extended to higher dimensions.

Example: Box-Muller Algorithm

$z_1, z_2\sim\mathcal U(0,1)$ are two uniform random variables Then, $x_1, x_2$ satisfies normal distribution.

Try: get the inverse mapping and prove the result.


A chain of Bijectors, mapping any distribution to normal distribution.