GiggleLiu - The Numeric Monster

RealNVP for physicists

To understand RealNVP, its proper to start from normalizing flow

https://blog.evjang.com/2018/01/nf1.html

Understand Bijectors

Suppose we have a Bijector z=f(x)z=f(x)

Inference refers to

x∼p^Xx\sim \hat p_X z=f(x)z=f(x)

Generation referes to

z∼pZz\sim p_Z x=f−1(z)x=f^{-1}(z)

Tractable density

The change of probability

pX(x)=pZ(f(x))∣det⁡(∂f(x)∂xT)∣p_X(x) = p_Z(f(x))\left\vert\det \left(\frac{\partial f(x)}{\partial x^T}\right)\right\vert log⁡pX(x)=log⁡(pZ(f(x)))+log⁡(∣det⁡(∂f(x)∂xT)∣)\log p_X(x) = \log\left(p_Z\left(f(x)\right)\right) + \log\left(\left\vert\det \left(\frac{\partial f(x)}{\partial x^T}\right)\right\vert\right)

The detailed derivation

Let (a,b)=x(a, b) = x and (u,v)=z(u, v) = z

Contributed by Shuo-Hui Li

∬u,vpZ(u,v)dAZ=∬a,bpX(a,b)dA \iint\limits_{u,v}{p_Z(u,v)}\mathrm{d}{A_Z} = \iint\limits_{a,b}p_X(a,b)\mathrm{d}{A} dA=dadb=(∂a∂u∂b∂v−∂a∂v∂b∂u)dudv=det⁡∣∂a∂u∂a∂v∂b∂u∂b∂v∣dudv \begin{split} \mathrm{d}{A} &= \mathrm{d}{a}\mathrm{d}{b} \\ &= \left( \frac{\partial{a}}{\partial{u}}\frac{\partial{b}}{\partial{v}}-\frac{\partial{a}}{\partial{v}}\frac{\partial{b}}{\partial{u}}\right) \mathrm{d}{u}\mathrm{d}{v}\\ &=\det{\left\vert\begin{matrix} \frac{\partial{a}}{\partial{u}} & \frac{\partial{a}}{\partial{v}}\\ \frac{\partial{b}}{\partial{u}} & \frac{\partial{b}}{\partial{v}} \end{matrix}\right\vert} \mathrm{d}{u}\mathrm{d}{v} \\ \end{split}

Here, the matrix inside the determinant is the Jacobian, which can be extended to higher dimensions.

∬u,vpZ(u,v)dAZ=∬a,bpX(a,b)det⁡∣Jacobian∣dudv \iint\limits_{u,v}{p_Z(u,v)}\mathrm{d}{A_Z} = \iint\limits_{a,b}p_X(a,b)\det{\left\vert\rm Jacobian \right\vert} \mathrm{d}{u}\mathrm{d}{v} pZ(u,v)=pX(a,b)det⁡∣Jacobian∣ p_Z(u,v) = p_X(a,b)\det{\left\vert\rm Jacobian \right\vert}

Example: Box-Muller Algorithm

z1,z2∼U(0,1)z_1, z_2\sim\mathcal U(0,1) are two uniform random variables

{x1=−2ln⁡z1cos⁡(2πz2),x2=−2ln⁡z1sin⁡(2πz2). \begin{cases} x_1 = \sqrt{-2\ln z_1} \cos(2\pi z_2),\\ x_2 = \sqrt{-2\ln z_1} \sin(2\pi z_2). \end{cases}

Then, x1,x2x_1, x_2 satisfies normal distribution.

Try: get the inverse mapping and prove the result.

RealNVP

A chain of Bijectors, mapping any distribution to normal distribution.

H:(u,v)→(a,b){a=m0(u,v)u+m1(u,v)b=m2(u,v)v+m3(u,v) \begin{equation} H:(u,v)\rightarrow (a,b) \\ \left\{ \begin{array}{rl} a &= m^0(u,v)u+m^1(u,v)\\ b &=m^2(u,v)v+m^3(u,v)\\ \end{array} \right. \end{equation}
CC BY-SA 4.0 GiggleLiu. Last modified: April 04, 2024. Website built with Franklin.jl and the Julia programming language.