Jacobian Matrix · aiwiki.page

The Jacobian matrix is a matrix containing the first-order partial derivatives of a vector-valued function. For a differentiable function, it represents the derivative as a linear map: multiplying a small input displacement by the Jacobian gives the first-order change in the output. It extends differentiation from single-variable functions to mappings between finite-dimensional spaces and connects multivariable calculus with linear algebra. (math.jhu.edu)

Definition and notation

Let $f:U\subseteq\mathbb{R}^n\to\mathbb{R}^m$ , where $U$ is open, and write

f(x)=\bigl(f_1(x),\ldots,f_m(x)\bigr).

If the relevant partial derivatives exist at $a\in U$ , the Jacobian is

J_f(a)= \begin{pmatrix} \frac{\partial f_1}{\partial x_1}(a)&\cdots& \frac{\partial f_1}{\partial x_n}(a)\\ \vdots&\ddots&\vdots\\ \frac{\partial f_m}{\partial x_1}(a)&\cdots& \frac{\partial f_m}{\partial x_n}(a) \end{pmatrix}.

Thus $J_f(a)$ has $m$ rows and $n$ columns: rows correspond to output components, and columns to input coordinates. Common notations include $Df(a)$ and $f'(a)$ , particularly when differentiability is established. (math.jhu.edu)

Using column vectors, a scalar-valued function has a $1\times n$ Jacobian equal to the transpose of its gradient:

J_f(a)=\nabla f(a)^{\mathsf T}.

For $n=m=1$ , the Jacobian reduces to the ordinary derivative. For an affine mapping $f(x)=Ax+b$ , it is the constant matrix $A$ . These conventions make derivative composition ordinary matrix multiplication. (ocw.mit.edu)

Differentiability and local approximation

The central interpretation is

f(a+h)=f(a)+J_f(a)h+o(\|h\|), \qquad h\to0.

The remainder divided by $\|h\|$ tends to zero. This requirement describes simultaneous changes in all input coordinates, not merely changes along coordinate axes. The Jacobian therefore represents the total derivative when this approximation holds. (math.cmu.edu)

Existence of all partial derivatives at a point does not by itself imply differentiability there. A standard sufficient condition is that the partial derivatives exist in a neighborhood and are continuous at the point. This distinction matters because a matrix of partial derivatives can exist even when it fails to provide a valid first-order approximation. (math.cmu.edu)

For example, consider the polynomial mapping

f(x,y)=(x^2+y,xy).

Direct differentiation gives

J_f(x,y)= \begin{pmatrix} 2x&1\\ y&x \end{pmatrix}.

At $(1,2)$ , an input displacement $(h,k)$ produces

f(1+h,2+k)-f(1,2) =(2h+k,\,2h+k)+(h^2,hk).

The linear terms are exactly $J_f(1,2)(h,k)^{\mathsf T}$ ; the remaining terms are higher-order.

Composition and inverse mappings

For differentiable mappings $f:\mathbb{R}^n\to\mathbb{R}^m$ and $g:\mathbb{R}^m\to\mathbb{R}^p$ , the multivariable chain rule states

J_{g\circ f}(a)=J_g(f(a))J_f(a).

The order matters: the right-hand matrix first converts an input displacement into an intermediate displacement, and the left-hand matrix converts that into an output displacement. Their dimensions are respectively $m\times n$ and $p\times m$ . (live.ocw.mit.edu)

The inverse function theorem links a square Jacobian to local invertibility. If $f$ is continuously differentiable near $a$ and $J_f(a)$ is invertible, then $f$ has a continuously differentiable inverse on suitable neighborhoods, with

J_{f^{-1}}(f(a))=[J_f(a)]^{-1}.

The right-hand side is an inverse matrix. The conclusion is local, not a guarantee of global invertibility. A zero derivative does not automatically exclude an inverse: $x\mapsto x^3$ is invertible, although its inverse is not differentiable at zero. (math.colostate.edu)

Determinants and changes of variables

When $m=n$ , the determinant $\det J_f(a)$ is called the Jacobian determinant. It must be distinguished from the matrix itself; rectangular Jacobians have no ordinary determinant. Its absolute value measures the local volume-scaling factor of a differentiable coordinate transformation, while its sign distinguishes preservation from reversal of orientation. (live.ocw.mit.edu)

For a continuously differentiable bijection $T:U\to V$ between open subsets of $\mathbb{R}^n$ , with continuously differentiable inverse, the change-of-variables formula for a suitable integrand $q$ is

\int_V q(x)\,dx = \int_U q(T(u))\,|\det J_T(u)|\,du.

The absolute value is essential because ordinary volume is unsigned. This generalizes substitution in a one-dimensional integral. (live.ocw.mit.edu)

For polar coordinates,

T(r,\theta)=(r\cos\theta,r\sin\theta), \qquad J_T= \begin{pmatrix} \cos\theta&-r\sin\theta\\ \sin\theta&r\cos\theta \end{pmatrix}.

Its determinant is $r$ , yielding $dx\,dy=r\,dr\,d\theta$ for $r>0$ . The formula is applied on regions where the angular coordinate makes the transformation one-to-one; the origin is a singular point of this coordinate system. (live.ocw.mit.edu)

Numerical computation and machine learning

For a nonlinear system $F(x)=0$ , Newton’s method determines a correction $s_k$ from the linear system

J_F(x_k)s_k=-F(x_k), \qquad x_{k+1}=x_k+s_k.

This solves the local linear approximation rather than the original nonlinear equations directly. Implementations can solve for the correction without explicitly forming the inverse Jacobian. (web.mit.edu)

In automatic differentiation, the full matrix need not be stored. Forward-mode differentiation computes Jacobian–vector products $J_f(x)v$ ; reverse-mode computes transpose-Jacobian products $J_f(x)^{\mathsf T}w$ . These propagate input perturbations and output sensitivities, respectively. Full Jacobians can be assembled column by column or row by row. (docs.jax.dev)

This distinction is important in machine learning. Backpropagation repeatedly applies reverse-mode products to obtain parameter gradients of a scalar loss function, avoiding explicit construction of every intermediate Jacobian. The Hessian matrix, by contrast, contains second derivatives of a scalar function; it can be viewed as the Jacobian of that function’s gradient. (live.ocw.mit.edu)