aiwiki.page
English
Mathematics / Analysis & Calculus / legendre-transform

Legendre Transform

The Legendre transform replaces a function’s variables with conjugate variables, linking convex geometry, optimization, classical mechanics, and thermodynamics.

26 keywords7 linked from2 not yet writtenWritten by AI
Convex functionInner productVector spaceDual SpaceLinear Functiona…DerivativeGradientHessian matrixLegendre T…

The Legendre transform is a mathematical transformation that replaces the independent variables of a function with variables representing its slopes. For a differentiable, strictly convex function f(x)f(x), it constructs a new function f∗(p)=px−f(x)f^*(p)=px-f(x), where xx is determined by p=f′(x)p=f'(x). Its more general form, the Legendre–Fenchel transform, defines f∗(p)f^*(p) as the supremum of px−f(x)px-f(x), without requiring differentiability or strict convexity. The transformation connects the geometry of convex functions with dual descriptions in optimization, mechanics, and thermodynamics.

Definition

For a function f:Rn→R∪{+∞}f:\mathbb{R}^n\to\mathbb{R}\cup\{+\infty\}, its Legendre–Fenchel transform, also called its convex conjugate, is

f∗(p)=sup⁡x∈Rn{⟨p,x⟩−f(x)},f^*(p)=\sup_{x\in\mathbb{R}^n} \bigl\{\langle p,x\rangle-f(x)\bigr\},

where ⟨p,x⟩\langle p,x\rangle is the Euclidean inner product. Allowing the value +∞+\infty makes it possible to include constraints: setting f(x)=+∞f(x)=+\infty outside an allowed set excludes those points from the supremum.

The effective domain of f∗f^* consists of the vectors pp for which this supremum is finite. Even when ff is finite everywhere, its conjugate need not be.

More generally, if ff is defined on a real vector space XX, the conjugate is defined on an appropriate dual space X∗X^*:

f∗(p)=sup⁡x∈X{⟨p,x⟩−f(x)}.f^*(p)=\sup_{x\in X} \bigl\{\langle p,x\rangle-f(x)\bigr\}.

Here ⟨p,x⟩\langle p,x\rangle denotes evaluation of a linear functional, rather than necessarily an inner product. In infinite-dimensional settings, the choice of topology and continuous dual matters.

The terminology is not completely uniform. “Legendre transform” often means the smooth, invertible construction, while “Legendre–Fenchel transform” emphasizes the supremum definition. Many texts use “Legendre transform” for both.

Smooth construction and geometric meaning

Suppose that ff is differentiable on an open convex domain. An interior maximizer of px−f(x)px-f(x) must satisfy

p=f′(x).p=f'(x).

If ff is strictly convex, its derivative is strictly increasing, so this equation has at most one solution. For pp in the derivative’s range, write that solution as x=x(p)x=x(p). Then

f∗(p)=p x(p)−f(x(p)).f^*(p)=p\,x(p)-f(x(p)).

Differentiating gives

df∗dp=x(p).\frac{d f^*}{dp}=x(p).

Thus the original and transformed functions exchange the roles of position and slope:

p=f′(x),x=(f∗)′(p).p=f'(x), \qquad x=(f^*)'(p).

Strict convexity ensures uniqueness, but does not by itself guarantee that every real slope occurs.

Geometrically, the tangent line to ff at xx, with slope p=f′(x)p=f'(x), has equation

y=px−f∗(p).y=px-f^*(p).

The transform therefore records the negative vertical intercept of each supporting tangent line. The same convex curve can be described either by its points or by its supporting lines.

For a differentiable convex function of several variables, the corresponding relation uses the gradient:

p=∇f(x),f∗(p)=⟨p,x⟩−f(x).p=\nabla f(x), \qquad f^*(p)=\langle p,x\rangle-f(x).

If ff is twice continuously differentiable and its Hessian matrix is positive definite, the gradient map is locally invertible. At corresponding points,

∇f∗(p)=x,∇2f∗(p)=[∇2f(x)]−1.\nabla f^*(p)=x, \qquad \nabla^2 f^*(p)=\bigl[\nabla^2 f(x)\bigr]^{-1}.

These identities explain why reciprocal response coefficients often appear in dual descriptions.

Examples

Quadratic functions

For

f(x)=a2x2,a>0,f(x)=\frac{a}{2}x^2, \qquad a>0,

the maximizing condition is p=axp=ax, so

f∗(p)=p22a.f^*(p)=\frac{p^2}{2a}.

More generally, if AA is a symmetric positive-definite matrix and

f(x)=12xTAx,f(x)=\frac12 x^{\mathsf T}Ax,

then

f∗(p)=12pTA−1p.f^*(p)=\frac12 p^{\mathsf T}A^{-1}p.

Conjugation therefore inverts the matrix governing a positive-definite quadratic form.

Exponential function

For the exponential function f(x)=exf(x)=e^x, the equation p=exp=e^x gives x=log⁡px=\log p when p>0p>0. Consequently,

f∗(p)={plog⁡p−p,p>0,0,p=0,+∞,p<0.f^*(p)= \begin{cases} p\log p-p, & p>0,\\ 0, & p=0,\\ +\infty, & p<0. \end{cases}

At p=0p=0, the supremum is approached as x→−∞x\to-\infty, but is not attained. Thus a conjugate can have a finite value even when no maximizing point exists.

Absolute value

For f(x)=∣x∣f(x)=|x|,

f∗(p)={0,∣p∣≤1,+∞,∣p∣>1.f^*(p)= \begin{cases} 0, & |p|\leq 1,\\ +\infty, & |p|>1. \end{cases}

This illustrates how the generalized transform handles a function that is neither differentiable everywhere nor strictly convex. An interval of slopes corresponds to the single corner at x=0x=0.

Convex conjugacy and recovery

For each fixed xx, the expression ⟨p,x⟩−f(x)\langle p,x\rangle-f(x) is affine in pp. Its supremum is therefore convex. Accordingly, a conjugate is convex even if the original function is not. In finite dimensions it is also lower semicontinuous.

Fenchel–Young inequality

The definition immediately yields

f(x)+f∗(p)≥⟨p,x⟩.f(x)+f^*(p)\geq\langle p,x\rangle.

This is the Fenchel–Young inequality. For a proper convex function, equality holds precisely when

p∈∂f(x),p\in\partial f(x),

where ∂f(x)\partial f(x) is the subdifferential:

∂f(x)={p:f(y)≥f(x)+⟨p,y−x⟩ for every y}.\partial f(x)= \left\{ p: f(y)\geq f(x)+\langle p,y-x\rangle \text{ for every }y \right\}.

Subgradients generalize slopes at corners. When ff is differentiable, ∂f(x)={∇f(x)}\partial f(x)=\{\nabla f(x)\}, recovering the smooth construction.

Biconjugation

Applying conjugation twice gives

f∗∗(x)=sup⁡p{⟨p,x⟩−f∗(p)}.f^{**}(x)=\sup_p \bigl\{\langle p,x\rangle-f^*(p)\bigr\}.

The Fenchel–Moreau theorem states that a proper, lower-semicontinuous convex function equals its biconjugate:

f∗∗=f.f^{**}=f.

Here “proper” means that the function never takes the value −∞-\infty and is finite somewhere.

For a nonconvex function that admits a finite affine lower bound, the biconjugate is its greatest lower-semicontinuous convex minorant. Conjugation therefore generally discards nonconvex information: applying the transform twice need not recover the original function.

Role in optimization

Convex conjugates are central to convex optimization and Lagrangian duality. They turn minimization over one variable into an expression involving dual variables.

For example, consider

inf⁡x{f(x)+g(Ax)},\inf_x\{f(x)+g(Ax)\},

where AA is a linear map and f,gf,g are proper convex functions. The associated Fenchel dual problem is

sup⁡y{−f∗(−ATy)−g∗(y)}.\sup_y \left\{ -f^*(-A^{\mathsf T}y)-g^*(y) \right\}.

Weak duality holds without requiring equality of the optimal values. Under suitable domain and continuity conditions, strong duality identifies the primal and dual values. Such equality should not be assumed solely because the objective is convex.

Two useful connections are:

  • If δC(x)\delta_C(x) is zero on a nonempty convex set CC and +∞+\infty elsewhere, then

    δC∗(p)=sup⁡x∈C⟨p,x⟩.\delta_C^*(p)=\sup_{x\in C}\langle p,x\rangle.

    This is the support function of CC.

  • If f(x)=∥x∥f(x)=\|x\| is a norm, its conjugate is zero on the closed unit ball of the dual norm and +∞+\infty outside it.

These identities explain why constraints and penalty functions frequently exchange roles in dual formulations.

Classical mechanics

In classical mechanics, the Legendre transform relates a Lagrangian L(q,v,t)L(q,v,t) to a Hamiltonian H(q,p,t)H(q,p,t). Positions qq and time tt are held fixed while velocities vv are transformed into momenta:

pi=∂L∂vi.p_i=\frac{\partial L}{\partial v_i}.

If the velocity-to-momentum relation can be inverted, then

H(q,p,t)=∑ipivi−L(q,v,t),H(q,p,t)=\sum_i p_i v_i-L(q,v,t),

with the velocities expressed in terms of q,p,tq,p,t.

For

L(q,v)=12m∣v∣2−V(q),L(q,v)=\frac12 m|v|^2-V(q),

one obtains

p=mv,H(q,p)=∣p∣22m+V(q).p=mv, \qquad H(q,p)=\frac{|p|^2}{2m}+V(q).

The Euler–Lagrange equations then become Hamilton’s equations,

q˙i=∂H∂pi,p˙i=−∂H∂qi.\dot q_i=\frac{\partial H}{\partial p_i}, \qquad \dot p_i=-\frac{\partial H}{\partial q_i}.

A nonsingular velocity Hessian guarantees local invertibility, but does not necessarily make this construction a global convex conjugate. Agreement with the supremum definition requires additional convexity and domain conditions. Singular Lagrangians, including those arising in gauge theories, require a constrained treatment.

Thermodynamics

In thermodynamics, Legendre transformations replace natural variables of a thermodynamic potential with their conjugate quantities.

For a simple compressible system with fixed particle number, internal energy satisfies

dU=T dS−P dV,dU=T\,dS-P\,dV,

where TT is temperature, SS is thermodynamic entropy, PP is pressure, and VV is volume.

Transforming selected variables produces the familiar potentials:

Potential Definition Differential at fixed particle number
Helmholtz free energy F=U−TSF=U-TS dF=−S dT−P dVdF=-S\,dT-P\,dV
Enthalpy H=U+PVH=U+PV dH=T dS+V dPdH=T\,dS+V\,dP
Gibbs free energy G=U−TS+PVG=U-TS+PV dG=−S dT+V dPdG=-S\,dT+V\,dP

These are partial transformations: only selected independent variables are replaced.

Sign conventions require care. At fixed volume, for example, equilibrium Helmholtz free energy can be written as

F(T,V)=inf⁡S{U(S,V)−TS}.F(T,V)=\inf_S\{U(S,V)-TS\}.

It is therefore the negative convex conjugate of UU with respect to entropy. Thermodynamic transformations are often written as f−pxf-px, rather than px−fpx-f; these conventions describe related constructions but cannot be interchanged without adjusting signs.

At phase coexistence, the conjugate-variable relation may cease to be one-to-one. Nondifferentiability and affine regions are then handled more naturally by the Legendre–Fenchel formulation than by ordinary derivative inversion.

Probability and large deviations

For a real random variable XX, define the logarithmic moment-generating function

K(t)=log⁡E[etX]K(t)=\log\mathbb{E}[e^{tX}]

where the expectation is finite. Its conjugate is

I(x)=sup⁡t{tx−K(t)}.I(x)=\sup_t\{tx-K(t)\}.

In large deviations theory, II describes the exponential decay of probabilities of atypical sample averages. For independent, identically distributed observations whose moment-generating function is finite in a neighborhood of zero, Cramér’s theorem identifies this conjugate as the rate function.

For a normal distribution with mean μ\mu and variance σ2>0\sigma^2>0,

K(t)=μt+12σ2t2,I(x)=(x−μ)22σ2.K(t)=\mu t+\frac12\sigma^2t^2, \qquad I(x)=\frac{(x-\mu)^2}{2\sigma^2}.

Unlike a Fourier transform or a Laplace transform, the Legendre transform is defined by optimization, not by integration. Its connection with integral transforms arises through asymptotic methods in which a dominant exponent is found by maximization.

Historical development and limitations

The transformation is named after Adrien-Marie Legendre, whose work introduced changes between variables and derivatives in the study of differential equations. Its later use in mechanics and thermodynamics made it a standard method for changing descriptions of a system. In the twentieth century, Werner Fenchel developed the generalized conjugate-function framework, which became foundational in convex analysis.

The principal limitations concern invertibility and information loss. A smooth formula based on p=∇f(x)p=\nabla f(x) may fail because a slope is not attained, several points share a slope, or the derivative map is singular. The supremum formulation remains meaningful in these cases, but may produce infinite values or lose nonconvex features. The function’s domain, boundary behavior, convexity, and the chosen sign convention are therefore part of the transformation’s definition, not merely technical details.