RealNVP for physicists

To understand RealNVP, its proper to start from normalizing flow

https://blog.evjang.com/2018/01/nf1.html

Understand Bijectors

Suppose we have a Bijector $z=f(x)$

Inference refers to

$x\sim \hat p_X$

$z=f(x)$

Generation referes to

$z\sim p_Z$

$x=f^{-1}(z)$

Tractable density

The change of probability

$p_X(x) = p_Z(f(x))\left\vert\det \left(\frac{\partial f(x)}{\partial x^T}\right)\right\vert$ $\log p_X(x) = \log\left(p_Z\left(f(x)\right)\right) + \log\left(\left\vert\det \left(\frac{\partial f(x)}{\partial x^T}\right)\right\vert\right)$

The detailed derivation

Let $(a, b) = x$ and $(u, v) = z$

Contributed by Shuo-Hui Li

Here, the matrix inside the determinant is the Jacobian, which can be extended to higher dimensions.

Example: Box-Muller Algorithm

$z_1, z_2\sim\mathcal U(0,1)$ are two uniform random variables Then, $x_1, x_2$ satisfies normal distribution.

Try: get the inverse mapping and prove the result.

RealNVP

A chain of Bijectors, mapping any distribution to normal distribution.