{
  "id": "lagrange-multipliers",
  "title": "Lagrange Multipliers",
  "lang": "en",
  "source_lang": "en",
  "summary": "Lagrange multipliers characterize constrained optima by relating objective gradients to constraint gradients, with applications in geometry, economics, statistics, mechanics, and numerical optimization.",
  "category": "mathematics",
  "subcategory": "applied",
  "type": "concept",
  "aliases": [
    "Method of Lagrange multipliers",
    "Lagrange multiplier method",
    "Undetermined multipliers"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/lagrange-multipliers/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/lagrange-multipliers/",
      "json": "https://aiwiki.page/zh/lagrange-multipliers.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Lagrange multipliers are auxiliary quantities used to study extrema of a function subject to constraints. Instead of eliminating constrained variables, the method incorporates the constraints into a new function, called a Lagrangian, and derives equations involving both the original variables and the multipliers. At a regular constrained optimum, the gradient of the objective is a linear combination of the gradients of the active constraints.\n\nThe method is fundamental to mathematical optimization because many problems involve restrictions that cannot conveniently be removed by substitution: a mechanical system may have fixed distances between its parts, a statistical model may require probabilities to sum to one, or an economic allocation may be limited by a budget. Multipliers identify stationary candidates, encode forces or marginal values associated with restrictions, and connect local differential conditions with global optimization through duality.\n\nNamed after Joseph-Louis Lagrange, the method extends well beyond elementary multivariable calculus. Its modern forms include optimality conditions for inequalities, algorithms for large constrained systems, and variational formulations in which the unknowns are functions rather than finite lists of numbers."
    },
    {
      "heading": "Equality-constrained problems and the multiplier theorem",
      "text": "Consider the problem\n\n\\[\\min_{x\\in\\mathbb R^n} f(x) \\qquad\\text{subject to}\\qquad h_i(x)=0,\\quad i=1,\\ldots,m,\\]\n\nwhere \\(f\\) and the constraint functions \\(h_i\\) are continuously differentiable near a candidate solution. The function \\(f\\) is the objective function, and the points satisfying every constraint form the feasible set. Define the Lagrangian by\n\n\\[L(x,\\lambda)=f(x)+\\sum_{i=1}^{m}\\lambda_i h_i(x),\\]\n\nwhere \\(\\lambda=(\\lambda_1,\\ldots,\\lambda_m)\\in\\mathbb R^m\\). Equality-constraint multipliers have no sign restriction.\n\nThe classical multiplier theorem states that if \\(x^*\\) is a local constrained minimum or maximum and the vectors\n\n\\[\\nabla h_1(x^*),\\ldots,\\nabla h_m(x^*)\\]\n\nare linearly independent, then there is a unique multiplier vector \\(\\lambda^*\\) such that\n\n\\[\\nabla_xL(x^*,\\lambda^*)=0, \\qquad h(x^*)=0.\\]\n\nWriting \\(Dh\\) for the Jacobian matrix whose rows are the transposed constraint gradients, these equations become\n\n\\[\\nabla f(x^*)+Dh(x^*)^{\\mathsf T}\\lambda^*=0, \\qquad h(x^*)=0.\\]\n\nThey comprise \\(n+m\\) scalar equations in \\(n+m\\) unknowns, although this count alone does not imply that solutions exist, are isolated, or can be found explicitly.\n\nThe equations are necessary conditions under the stated regularity assumption, not a complete test for optimality. Their solutions can include constrained minima, constrained maxima, and stationary points that are neither. A local analysis also does not establish the existence of a global optimum. For example, the extreme value theorem guarantees attainment when the feasible set is nonempty and compact and the objective is continuous; the multiplier equations then help identify where attainment can occur.\n\nThe sign convention is arbitrary for equalities. Some presentations define \\(L=f-\\sum_i\\lambda_i h_i\\), which replaces each multiplier by its negative without changing the candidate points. Constraint normalization matters as well: replacing \\(h_i\\) by \\(c_i h_i\\), with \\(c_i\\ne0\\), replaces its multiplier by \\(\\lambda_i/c_i\\). Thus a multiplier is meaningful only together with the precise equation and sign convention to which it belongs."
    },
    {
      "heading": "Geometry, tangent directions, and regularity",
      "text": "The geometric explanation uses the directions in which a feasible point can move. When \\(Dh(x^*)\\) has full row rank \\(m\\), the implicit function theorem implies that the nearby feasible set is a smooth manifold of dimension \\(n-m\\). Its tangent space at \\(x^*\\) is\n\n\\[T_{x^*}=\\{d\\in\\mathbb R^n:Dh(x^*)d=0\\}.\\]\n\nA tangent direction therefore satisfies every linearized constraint. In a problem with one equation in three variables, this space is ordinarily a plane tangent to a constraint surface.\n\nIf \\(x(t)\\) is a differentiable feasible curve through \\(x^*\\), the chain rule gives\n\n\\[\\left.\\frac{d}{dt}f(x(t))\\right|_{t=0} =\\nabla f(x^*)^{\\mathsf T}x'(0).\\]\n\nAt a local constrained extremum, this derivative vanishes for every feasible tangent direction. Consequently, \\(\\nabla f(x^*)\\) belongs to the orthogonal complement of the tangent space. A result from linear algebra identifies that orthogonal complement with the span of the constraint gradients, yielding the multiplier equation.\n\nFor a single regular constraint in the plane, this says that an objective level curve and the constraint curve have parallel normal vectors where they touch at an extremum. A familiar diagram shows nested objective contours becoming tangent to a feasible curve. The general theorem is more precise than the picture: the objective gradient may itself vanish, and several constraints may contribute independent normal directions.\n\nRegularity is essential. Consider\n\n\\[\\min_x x\\qquad\\text{subject to}\\qquad x^2=0.\\]\n\nThe only feasible point is \\(x=0\\), so it is both a constrained minimum and maximum. Nevertheless,\n\n\\[1+\\lambda(2x)=0\\]\n\ncannot hold there. The failure arises because the constraint gradient vanishes, not because the feasible optimum is absent. Rewriting the same feasible set as \\(x=0\\) restores a nonzero gradient and a valid multiplier equation.\n\nRedundant constraints create a different problem. The equations \\(x=0\\) and \\(2x=0\\) describe the same restriction, but their gradients are dependent. Multipliers may then exist without being unique. More general Fritz John conditions introduce a coefficient \\(\\alpha\\) on the objective gradient:\n\n\\[\\alpha\\nabla f(x^*)+\\sum_i\\lambda_i\\nabla h_i(x^*)=0,\\]\n\nwith the coefficients not all zero. Regularity conditions rule out the abnormal case \\(\\alpha=0\\), permitting normalization to the ordinary Lagrange equations."
    },
    {
      "heading": "Representative calculations",
      "text": "A basic example is the optimization of\n\n\\[f(x,y)=xy \\qquad\\text{subject to}\\qquad x^2+y^2=1.\\]\n\nWith\n\n\\[L(x,y,\\lambda)=xy+\\lambda(x^2+y^2-1),\\]\n\nstationarity gives\n\n\\[y+2\\lambda x=0,\\qquad x+2\\lambda y=0,\\qquad x^2+y^2=1.\\]\n\nNeither coordinate can vanish at a stationary solution: if one did, the first two equations would force both to vanish. Eliminating \\(\\lambda\\) therefore gives \\(x^2=y^2\\).\n\nThe four candidates are\n\n\\[(x,y)= \\left(\\frac1{\\sqrt2},\\frac1{\\sqrt2}\\right), \\left(-\\frac1{\\sqrt2},-\\frac1{\\sqrt2}\\right), \\left(\\frac1{\\sqrt2},-\\frac1{\\sqrt2}\\right), \\left(-\\frac1{\\sqrt2},\\frac1{\\sqrt2}\\right).\\]\n\nTheir objective values are \\(1/2,1/2,-1/2,-1/2\\). Because the circle is compact and the constraint gradient is nonzero everywhere on it, every global extremum appears among these candidates. The calculation illustrates why solving the multiplier equations must be followed by classification or comparison.\n\nA higher-dimensional example is the point nearest a given vector \\(a\\) subject to linear equations:\n\n\\[\\min_x\\frac12\\|x-a\\|^2 \\qquad\\text{subject to}\\qquad Ax=b,\\]\n\nwhere the matrix \\(A\\in\\mathbb R^{m\\times n}\\) has full row rank. The equations are\n\n\\[x-a+A^{\\mathsf T}\\lambda=0,\\qquad Ax=b.\\]\n\nSubstitution yields\n\n\\[AA^{\\mathsf T}\\lambda=Aa-b,\\]\n\nand hence\n\n\\[x^*=a-A^{\\mathsf T}(AA^{\\mathsf T})^{-1}(Aa-b).\\]\n\nThe matrix \\(AA^{\\mathsf T}\\) is invertible under the rank assumption. Geometrically, \\(x^*\\) is the orthogonal projection of \\(a\\) onto the affine feasible set; the correction \\(a-x^*\\) lies in its normal space.\n\nMultipliers also recover the variational characterization of eigenvalues and eigenvectors. For a real symmetric matrix \\(Q\\), consider\n\n\\[\\min_x x^{\\mathsf T}Qx \\qquad\\text{subject to}\\qquad x^{\\mathsf T}x=1.\\]\n\nStationarity gives\n\n\\[2Qx+2\\lambda x=0,\\]\n\nso a candidate is a unit eigenvector with eigenvalue \\(-\\lambda\\). The smallest and largest eigenvalues are respectively the minimum and maximum objective values on the unit sphere. Eigenvectors for intermediate eigenvalues can be constrained stationary points without being extrema. This example connects the multiplier method to the spectral theorem and to dimensionality-reduction procedures based on quadratic forms."
    },
    {
      "heading": "Second-order conditions and classification",
      "text": "First derivatives determine whether an objective can change to first order along the feasible set. Classifying a stationary point requires curvature information. Suppose \\(f\\) and the constraints are twice continuously differentiable, the constraint gradients are independent at \\(x^*\\), and \\(\\lambda^*\\) satisfies the stationarity equations. Define the Lagrangian Hessian\n\n\\[H_L=\\nabla^2f(x^*)+ \\sum_{i=1}^{m}\\lambda_i^*\\nabla^2h_i(x^*).\\]\n\nThe constraint terms are indispensable: the curvature of the feasible set contributes to the second derivative of the objective along a feasible curve.\n\nA second-order necessary condition for a local minimum is\n\n\\[d^{\\mathsf T}H_Ld\\ge0 \\qquad\\text{for every }d\\in T_{x^*}.\\]\n\nFor a local maximum, the inequality is reversed. A standard sufficient condition for a strict local minimum is\n\n\\[d^{\\mathsf T}H_Ld>0 \\qquad\\text{for every nonzero }d\\in T_{x^*}.\\]\n\nThus the Lagrangian Hessian need not be a positive-definite matrix on the entire ambient space. Only directions consistent with the linearized constraints enter this test.\n\nTo express the test computationally, let the columns of \\(Z\\) form a basis of the null space of \\(Dh(x^*)\\). Every tangent direction has the form \\(d=Zp\\), and\n\n\\[d^{\\mathsf T}H_Ld=p^{\\mathsf T}(Z^{\\mathsf T}H_LZ)p.\\]\n\nThe matrix \\(Z^{\\mathsf T}H_LZ\\) is called the reduced Hessian. Its positive definiteness establishes strict local minimality under the regularity assumptions. A negative direction rules out a local minimum, while zero eigenvalues can leave the classification unresolved.\n\nFor example, minimize \\(f(x,y)=y\\) subject to\n\n\\[h(x,y)=y-x^2=0.\\]\n\nOn the feasible parabola, the objective equals \\(x^2\\), so the origin is a strict minimum. Yet the objective Hessian is zero everywhere. At the origin, stationarity gives \\(\\lambda=-1\\), and\n\n\\[\\nabla_{xx}^2L= \\begin{pmatrix} 2&0\\\\ 0&0 \\end{pmatrix}.\\]\n\nThe tangent directions are \\((d_x,0)\\), on which the quadratic form is strictly positive whenever \\(d_x\\ne0\\).\n\nIf the reduced Hessian is only semidefinite, higher-order behavior may decide the result. Under the constraint \\(y=0\\), the objectives \\(x^4\\), \\(-x^4\\), and \\(x^3\\) all have zero first and second derivatives at the origin, but give a minimum, a maximum, and neither, respectively. Second-order tests therefore have explicit limits even when all functions are smooth."
    },
    {
      "heading": "Inequalities and the Karush–Kuhn–Tucker conditions",
      "text": "For inequalities, the standard minimization convention is\n\n\\[\\min_x f(x) \\quad\\text{subject to}\\quad h_i(x)=0,\\qquad g_j(x)\\le0.\\]\n\nIts Lagrangian is\n\n\\[L(x,\\lambda,\\mu) =f(x)+\\sum_i\\lambda_i h_i(x)+\\sum_j\\mu_jg_j(x).\\]\n\nThe Karush–Kuhn–Tucker conditions, usually abbreviated KKT conditions, combine stationarity with feasibility and sign restrictions:\n\n\\[\\begin{aligned} \\nabla_xL(x^*,\\lambda^*,\\mu^*)&=0,\\\\ h_i(x^*)&=0,\\\\ g_j(x^*)&\\le0,\\\\ \\mu_j^*&\\ge0,\\\\ \\mu_j^*g_j(x^*)&=0. \\end{aligned}\\]\n\nEquality multipliers remain unrestricted. The final condition is complementary slackness.\n\nAn inequality is active when \\(g_j(x^*)=0\\) and inactive when \\(g_j(x^*)<0\\). Complementary slackness forces an inactive constraint to have zero multiplier. An active inequality can carry a positive multiplier, but it can also have multiplier zero. This distinction matters because the geometrical fact that a boundary is reached does not by itself imply a nonzero first-order value for that restriction.\n\nFor example, minimize \\(f(x)=(x-2)^2\\) subject to \\(x\\le1\\). Here \\(g(x)=x-1\\), and the optimum is \\(x^*=1\\). Stationarity gives\n\n\\[2(x^*-2)+\\mu^*=0,\\]\n\nso \\(\\mu^*=2\\). If the constraint is changed to \\(x\\le3\\), the unconstrained minimizer \\(x^*=2\\) is feasible and the multiplier is zero. If instead the constraint is \\(x\\le2\\), the optimum lies on the boundary but its multiplier is still zero.\n\nFor nonconvex problems, KKT conditions are generally necessary only when a suitable constraint qualification holds. One sufficient qualification is linear independence of the equality gradients together with all active inequality gradients. Weaker conditions cover additional cases. Without any qualification, an optimizer can fail to admit ordinary KKT multipliers, just as it can for singular equality constraints.\n\nIn convex optimization, the role of these equations is stronger. If \\(f\\) and every \\(g_j\\) are differentiable convex functions and the equalities are affine, any feasible point satisfying KKT conditions is globally optimal. Conversely, qualifications such as Slater’s condition—strict feasibility of the inequalities together with feasibility of the affine equalities—ensure multiplier existence under the standard finite-dimensional hypotheses when an optimum is attained. These are global conclusions that do not follow from the elementary stationary-point theorem alone."
    },
    {
      "heading": "Duality, sensitivity, and the value of constraints",
      "text": "Multipliers connect differential optimality conditions with Lagrangian duality. For a constrained minimization problem, define the dual function\n\n\\[q(\\lambda,\\mu)=\\inf_xL(x,\\lambda,\\mu), \\qquad \\mu\\ge0.\\]\n\nThe infimum is taken over the original variable domain, with the displayed constraints incorporated into the Lagrangian rather than imposed separately. At any feasible \\(x\\),\n\n\\[L(x,\\lambda,\\mu)\\le f(x),\\]\n\nbecause the equality terms vanish and each inequality term is nonpositive. Therefore \\(q(\\lambda,\\mu)\\) is a lower bound on the primal optimal value.\n\nThe dual problem maximizes this bound:\n\n\\[\\sup_{\\lambda,\\ \\mu\\ge0}q(\\lambda,\\mu).\\]\n\nIts optimum can never exceed the primal infimum, a fact known as weak duality. Equality of the optimal values is strong duality. Convexity and an appropriate constraint qualification often establish strong duality, whereas nonconvex problems may exhibit a positive duality gap. Solving local stationarity equations in the original variables is consequently not equivalent to solving the dual problem.\n\nMultipliers also describe sensitivity to changes in constraint levels. Consider\n\n\\[v(b)=\\min_x f(x) \\qquad\\text{subject to}\\qquad h(x)=b,\\]\n\nwith\n\n\\[L(x,\\lambda;b)=f(x)+\\lambda^{\\mathsf T}(h(x)-b).\\]\n\nAlong a differentiable local solution branch, stationarity and the differentiated constraints imply\n\n\\[\\nabla_bv(b)=-\\lambda^*(b).\\]\n\nFor the global value function, this formula additionally requires that the branch continues to describe the global optimum and that the relevant differentiability conditions hold. A change of sign convention for \\(L\\) changes the displayed multiplier sign accordingly.\n\nA direct example is\n\n\\[v(b)=\\min_x\\frac12x^2\\qquad\\text{subject to}\\qquad x=b.\\]\n\nThe value is \\(v(b)=b^2/2\\). Stationarity gives \\(x+\\lambda=0\\), so \\(\\lambda^*=-b\\), exactly matching \\(v'(b)=-\\lambda^*=b\\). In economics and operations research, analogous marginal values are often called shadow prices. Their units are objective units divided by the units of the corresponding constraint.\n\nSensitivity can fail to be described by a single derivative when optimizers change abruptly, multipliers are nonunique, or the value function is nonsmooth. In convex problems, optimal multipliers can instead furnish supporting linear bounds on the value function. This retains a precise marginal interpretation without assuming a unique optimizer or a smooth response to perturbation."
    },
    {
      "heading": "Numerical solution and constrained algorithms",
      "text": "Computationally, the multiplier equations form a coupled nonlinear system. Applying Newton’s method to equality-constrained stationarity leads to linear systems of the form\n\n\\[\\begin{pmatrix} H_L & A^{\\mathsf T}\\\\ A & 0 \\end{pmatrix} \\begin{pmatrix} \\Delta x\\\\ \\Delta\\lambda \\end{pmatrix} = - \\begin{pmatrix} \\nabla_xL\\\\ h \\end{pmatrix},\\]\n\nwhere \\(A=Dh(x)\\) and the derivatives are evaluated at the current iterate. The upper equation corrects stationarity, while the lower equation corrects constraint violation. Such matrices are commonly called KKT matrices.\n\nThese matrices are generally indefinite, even near a strict constrained minimum. Solvers therefore use suitable symmetric-indefinite factorizations, null-space methods, or block elimination rather than assuming positive definiteness. If the constraint Jacobian has full row rank and the reduced Hessian is positive definite, the equality-constrained KKT matrix is nonsingular. Nearly dependent constraint gradients, by contrast, can make multiplier estimates highly sensitive to small numerical errors.\n\nSequential quadratic programming constructs a sequence of local quadratic optimization problems. The objective is approximated quadratically and the constraints linearly, with a Lagrangian Hessian or an approximation supplying curvature. Solving each subproblem produces a step in the original variables and estimates of the multipliers. For inequalities, active-set methods identify constraints treated locally as equalities, while interior-point methods maintain slacks and positive dual variables and approach complementary slackness gradually.\n\nPenalty methods provide another connection. A quadratic penalty replaces an equality-constrained problem by\n\n\\[\\min_x f(x)+\\frac{\\rho}{2}\\|h(x)\\|^2.\\]\n\nAt a stationary point,\n\n\\[\\nabla f(x)+Dh(x)^{\\mathsf T}\\bigl(\\rho h(x)\\bigr)=0,\\]\n\nso \\(\\rho h(x)\\) acts as a multiplier estimate. Increasing \\(\\rho\\) can drive constraint violations toward zero, but very large penalties often worsen the conditioning of the numerical problem.\n\nThe augmented Lagrangian method combines an explicit multiplier with a quadratic penalty:\n\n\\[\\mathcal L_\\rho(x,\\lambda) =f(x)+\\lambda^{\\mathsf T}h(x)+\\frac{\\rho}{2}\\|h(x)\\|^2.\\]\n\nA standard iteration approximately minimizes this expression in \\(x\\) and then updates\n\n\\[\\lambda_{k+1}=\\lambda_k+\\rho h(x_{k+1}).\\]\n\nUnder appropriate assumptions and accuracy requirements, multiplier updates allow feasibility to improve without requiring the penalty parameter to grow as rapidly as in a pure penalty scheme. In practice, constraint scaling, derivative accuracy, and numerical stability affect both the computed solution and the reliability of its reported multipliers."
    },
    {
      "heading": "Applications in statistics, information theory, and mechanics",
      "text": "In maximum likelihood estimation, probabilities are often constrained to sum to one. Suppose categorical outcomes have positive counts \\(n_1,\\ldots,n_k\\), with total \\(N\\). Maximizing\n\n\\[\\ell(p)=\\sum_{i=1}^{k}n_i\\log p_i \\qquad\\text{subject to}\\qquad \\sum_i p_i=1\\]\n\ngives, using\n\n\\[L(p,\\lambda)=\\ell(p)+\\lambda\\left(\\sum_i p_i-1\\right),\\]\n\nthe equations\n\n\\[\\frac{n_i}{p_i}+\\lambda=0.\\]\n\nNormalization yields \\(\\lambda=-N\\) and \\(p_i=n_i/N\\). Positive counts ensure an interior solution; zero counts require explicit attention to boundary probabilities and the domain of the logarithm.\n\nIn information theory, maximizing entropy under a prescribed mean produces exponential distributions over a finite state space. For states with values \\(E_i\\), impose\n\n\\[\\sum_i p_i=1,\\qquad \\sum_i p_iE_i=U.\\]\n\nAn interior stationary point of\n\n\\[-\\sum_i p_i\\log p_i -\\alpha\\left(\\sum_i p_i-1\\right) -\\beta\\left(\\sum_i p_iE_i-U\\right)\\]\n\nsatisfies\n\n\\[p_i=\\frac{e^{-\\beta E_i}}{Z(\\beta)}, \\qquad Z(\\beta)=\\sum_j e^{-\\beta E_j}.\\]\n\nThe multiplier \\(\\beta\\) is selected to match the prescribed mean. Entropy’s strict concavity supplies uniqueness when a feasible interior solution exists, so the calculation establishes more than an arbitrary stationary candidate.\n\nIn classical mechanics, multipliers represent generalized constraint forces. For generalized coordinates \\(q\\), mechanical Lagrangian \\(L_{\\mathrm{mech}}(q,\\dot q,t)\\), and ideal holonomic constraints \\(h_i(q,t)=0\\), one convention writes\n\n\\[\\frac{d}{dt}\\frac{\\partial L_{\\mathrm{mech}}}{\\partial\\dot q_a} -\\frac{\\partial L_{\\mathrm{mech}}}{\\partial q_a} = \\sum_i\\lambda_i\\frac{\\partial h_i}{\\partial q_a}.\\]\n\nThe right-hand side is normal to the instantaneous admissible virtual displacements. For a particle restricted to a sphere, it describes a radial reaction force. The multiplier’s numerical relation to a physical force depends on how the constraint equation is normalized.\n\nMultipliers also explain the relationship between constrained fitting and regularization. A problem minimizing a loss function subject to \\(R(x)\\le c\\) has Lagrangian\n\n\\[f(x)+\\mu(R(x)-c),\\qquad\\mu\\ge0.\\]\n\nAt a suitable optimum, \\(\\mu\\) acts like the coefficient of a penalty \\(R(x)\\). This connects constrained and penalized formulations of ridge regression and lasso regression, although nonsmooth penalties require generalized derivatives and the correspondence between bounds and penalty weights need not be one-to-one."
    },
    {
      "heading": "Historical development and variational extensions",
      "text": "The multiplier method belongs to the eighteenth-century development of analytical mechanics and the calculus of variations. Lagrange’s Mécanique analytique, published in 1788, gave systematic analytical treatments of constrained mechanical systems. Undetermined multipliers made it possible to retain convenient coordinates while accounting for restrictions through additional unknown quantities. The historical method was therefore closely associated with eliminating unknown reactions and expressing equilibrium or motion without solving every geometrical constraint explicitly.\n\nThe modern finite-dimensional theorem expresses this idea through differentiable functions, matrix rank, and tangent spaces. Inequality-constrained optimization acquired a broader framework in the twentieth century. William Karush’s 1939 master’s thesis established conditions now recognized as part of the KKT theory; Harold W. Kuhn and Albert W. Tucker presented closely related conditions in 1951. Fritz John’s 1948 work supplied an important formulation accommodating abnormal multipliers when ordinary constraint qualifications fail.\n\nIn variational problems, the unknown is a function and a constraint may itself be an integral. Consider an isoperimetric problem,\n\n\\[\\min_y\\int_a^b F(t,y(t),y'(t))\\,dt\\]\n\nsubject to fixed endpoint values and\n\n\\[\\int_a^b G(t,y(t),y'(t))\\,dt=c.\\]\n\nUnder appropriate differentiability and regularity assumptions, a scalar multiplier \\(\\lambda\\) combines the integrands:\n\n\\[\\widetilde F=F+\\lambda G.\\]\n\nA stationary solution then satisfies the Euler–Lagrange equation\n\n\\[\\frac{\\partial\\widetilde F}{\\partial y} -\\frac{d}{dt}\\frac{\\partial\\widetilde F}{\\partial y'}=0,\\]\n\ntogether with the integral constraint. A single scalar restriction produces a scalar multiplier even though the optimization variable is an entire function.\n\nPointwise constraints on fields generally require multiplier functions rather than constants. For example, the incompressibility condition in fluid mechanics is imposed at every spatial point, and pressure has a multiplier interpretation in variational and weak formulations. Comparable constructions appear in constrained elasticity and mixed formulations of partial differential equations, where the multiplier enforces a physical restriction while becoming an additional unknown field.\n\nIn functional analysis, the general setting involves differentiable maps between Banach spaces. Multipliers belong to an appropriate dual space, and finite-dimensional full-rank assumptions are replaced by conditions involving surjectivity, range closure, or other properties of the constraint derivative. Mixed numerical discretizations must also preserve compatibility between the spaces used for the primary variables and the multipliers. Otherwise, a formally correct continuous multiplier formulation can produce a singular or unstable discrete system."
    }
  ],
  "links": [
    "mathematical-function",
    "gradient",
    "mathematical-optimization",
    "objective-function",
    "feasible-set",
    "linear-independence",
    "jacobian-matrix",
    "extreme-value-theorem",
    "manifold",
    "tangent-space",
    "chain-rule",
    "orthogonal-complement",
    "linear-algebra",
    "matrix",
    "orthogonal-projection",
    "eigenvalues-and-eigenvectors",
    "spectral-theorem",
    "hessian-matrix",
    "positive-definite-matrix",
    "null-space",
    "karush-kuhn-tucker-conditions",
    "convex-optimization",
    "convex-function",
    "lagrangian-duality",
    "sensitivity-analysis",
    "economics",
    "operations-research",
    "newtons-method",
    "sequential-quadratic-programming",
    "condition-number",
    "numerical-stability",
    "maximum-likelihood-estimation",
    "information-theory",
    "information-entropy",
    "classical-mechanics",
    "regularization",
    "loss-function",
    "ridge-regression",
    "lasso-regression",
    "calculus-of-variations",
    "integral",
    "euler-lagrange-equations",
    "partial-differential-equation",
    "functional-analysis",
    "banach-space",
    "dual-space"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}