Maths for ML / Notation

\mathbf{J} = \frac{\partial \mathbf{f}}{\partial \mathbf{x}}

Jacobian

CalculusAdvanced

Say it “the Jacobian of f”

For a function with vector input and vector output: the matrix of every output's partial derivative with respect to every input.

Example

\mathbf{f}: \mathbb{R}^{n}\to\mathbb{R}^{m} \Rightarrow \mathbf{J}\in\mathbb{R}^{m\times n}

In code

torch.func.jacrev(f)(x)

Where you’ll see it in ML

Backprop multiplies Jacobians layer by layer (the chain rule).

Don’t confuse it with

$\nabla_{\theta} L$ nabla / gradient The vector of all partial derivatives. It points in the direction the function increases fastest.
$\mathbf{H} = \nabla^{2} f$ Hessian The matrix of second derivatives — describes the curvature of a function.

Lessons that use it

Jacobians and Hessians: derivatives as tables

← All 94 symbols