This blog covers the Jacobians and Hessians for reversible primitives, they can be used to propagate gradients and Hessians in a reversible programs.

The Definition

For function $\vec{y} = f(\vec{x})$, we define its Jacobian as

Its Hessian is

Jacobian of Reversible Primitives

(1). $a \mathrel+= b$

The inverse is $a \mathrel-= b​$, its Jacobian is the inverse of the matrix above

In the following, we omit the Jacobians and Hessians of inverse functions.

(2). $a\mathrel+=b*c$

(3). $a\mathrel+=b/c​$

(4). $a\mathrel+=b^c​$

(5). $a\mathrel+=e^b$

(6). $a\mathrel+=\log b​$

(7). $a\mathrel+=\sin b​$

(8). $a\mathrel+=\cos b​$

(9). $a \mathrel+= \vert b\vert​$

(10). $a = -a​$

(11). ${\rm SWAP}(a, b) = (b, a)$

(12). ​