{
  "id": "quadratic-programming",
  "title": "Quadratic Programming",
  "lang": "en",
  "source_lang": "en",
  "summary": "Quadratic programming optimizes a quadratic objective subject to linear constraints, with applications in statistics, finance, control, and machine learning.",
  "category": "mathematics",
  "subcategory": "applied",
  "type": "concept",
  "aliases": [
    "Quadratic optimization",
    "QP"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/quadratic-programming/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/quadratic-programming/",
      "json": "https://aiwiki.page/zh/quadratic-programming.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Quadratic programming (QP) is a class of mathematical optimization problems in which a quadratic objective function is minimized or maximized subject to linear equality and inequality constraints. It generalizes linear programming by allowing squared variables and products of pairs of variables in the objective. The most important distinction is between convex quadratic programming, for which local optimality implies global optimality, and nonconvex quadratic programming, which can have multiple local minima and is generally much harder to solve globally."
    },
    {
      "heading": "Mathematical formulation",
      "text": "A standard minimization form is\n\n\\[\\begin{aligned} \\operatorname{minimize}_{x\\in\\mathbb{R}^n}\\quad & f(x)=\\frac12 x^\\top Qx+c^\\top x\\\\ \\text{subject to}\\quad & Ax\\leq b,\\\\ & Ex=d. \\end{aligned}\\]\n\nHere:\n\n- \\(x\\) is the vector of decision variables;\n- \\(Q\\in\\mathbb{R}^{n\\times n}\\) describes the quadratic terms;\n- \\(c\\in\\mathbb{R}^n\\) describes the linear terms;\n- \\(A\\) and \\(b\\) specify inequality constraints;\n- \\(E\\) and \\(d\\) specify equality constraints.\n\nThe inequality \\(Ax\\leq b\\) is interpreted componentwise. Variable bounds, such as \\(l\\leq x\\leq u\\), are also linear constraints and can be included explicitly.\n\nThe objective function contains a quadratic form, \\(x^\\top Qx\\). The matrix \\(Q\\) can always be taken as symmetric because\n\n\\[x^\\top Qx=x^\\top\\left(\\frac{Q+Q^\\top}{2}\\right)x.\\]\n\nThe factor \\(1/2\\) is conventional: for symmetric \\(Q\\), the gradient and Hessian matrix are simply\n\n\\[\\nabla f(x)=Qx+c, \\qquad \\nabla^2f(x)=Q.\\]\n\nA constant term may be added to the objective without changing its minimizers. Maximization problems can be converted to minimization problems by negating the objective.\n\nThe feasible set is an intersection of finitely many linear half-spaces and hyperplanes, and therefore is a polyhedron. It may be empty, bounded, or unbounded. An unbounded feasible set does not by itself imply an unbounded objective."
    },
    {
      "heading": "Convexity, existence, and uniqueness",
      "text": "If \\(Q\\) is positive semidefinite, meaning\n\n\\[v^\\top Qv\\geq0 \\quad\\text{for every }v\\in\\mathbb{R}^n,\\]\n\nthen the objective is a convex function. Since linear constraints define a convex set, the resulting problem belongs to convex optimization. Every local minimizer is then a global minimizer, although the minimizer need not be unique.\n\nIf \\(Q\\) is a positive definite matrix, the objective is strictly convex. On any nonempty feasible set defined by linear constraints, it has a unique minimizer. Positive definiteness also makes the objective grow without bound as \\(\\lVert x\\rVert\\) grows, ensuring existence even when the feasible set is unbounded.\n\nPositive semidefiniteness alone does not ensure that the objective is bounded below. For example, \\(f(x)=-x\\), regarded as a quadratic objective with \\(Q=0\\), is unbounded below on \\(x\\geq0\\).\n\nThe curvature of the objective only along feasible directions may matter. In an equality-constrained problem, write\n\n\\[x=x_0+Zz,\\]\n\nwhere \\(Ex_0=d\\) and the columns of \\(Z\\) form a basis for the null space of \\(E\\). The reduced Hessian is\n\n\\[Z^\\top QZ.\\]\n\nIf this matrix is positive semidefinite, the objective is convex on the equality-feasible affine space, even if \\(Q\\) is not positive semidefinite on the whole ambient space. If it is positive definite, the objective is strictly convex on that affine space.\n\nWhen \\(Q\\) has both positive and negative curvature, the problem is generally nonconvex. Linear constraints may nevertheless exclude directions of negative curvature or restrict them sufficiently to produce a finite minimum."
    },
    {
      "heading": "Optimality conditions and duality",
      "text": "For the standard formulation, introduce multipliers \\(\\lambda\\geq0\\) for \\(Ax\\leq b\\) and unrestricted multipliers \\(\\nu\\) for \\(Ex=d\\). The Lagrangian is\n\n\\[L(x,\\lambda,\\nu) = \\frac12x^\\top Qx+c^\\top x +\\lambda^\\top(Ax-b) +\\nu^\\top(Ex-d).\\]\n\nThe Karush–Kuhn–Tucker conditions are\n\n\\[\\begin{aligned} Qx+c+A^\\top\\lambda+E^\\top\\nu&=0, &&\\text{stationarity},\\\\ Ax\\leq b,\\qquad Ex&=d, &&\\text{primal feasibility},\\\\ \\lambda&\\geq0, &&\\text{dual feasibility},\\\\ \\lambda_i(A_ix-b_i)&=0 \\quad\\text{for every }i, &&\\text{complementary slackness}. \\end{aligned}\\]\n\nA constraint is active when it holds with equality. Complementary slackness requires the multiplier of every inactive inequality to be zero.\n\nFor a convex QP, any point satisfying these conditions is globally optimal. Conversely, at an attained optimum of a QP with only linear constraints, suitable multipliers exist, even when strict feasibility fails. Multipliers need not be unique, especially when constraints are redundant.\n\nFor nonconvex QPs, these first-order conditions are necessary at a local optimum but are not sufficient. Second-order conditions examine the curvature along directions compatible with the active constraints.\n\nLagrangian duality provides bounds and optimality certificates. When \\(Q\\) is positive definite, set\n\n\\[r=c+A^\\top\\lambda+E^\\top\\nu.\\]\n\nMinimizing the Lagrangian over \\(x\\) gives the dual problem\n\n\\[\\begin{aligned} \\operatorname{maximize}_{\\lambda,\\nu}\\quad &-\\frac12r^\\top Q^{-1}r-b^\\top\\lambda-d^\\top\\nu\\\\ \\text{subject to}\\quad &\\lambda\\geq0. \\end{aligned}\\]\n\nFor a singular positive semidefinite \\(Q\\), the dual expression additionally requires a range condition and can be written using a generalized inverse. In numerical computation, linear systems are normally solved rather than forming \\(Q^{-1}\\) explicitly."
    },
    {
      "heading": "Illustrative example",
      "text": "Consider\n\n\\[\\begin{aligned} \\operatorname{minimize}\\quad& (x_1-2)^2+(x_2-1)^2\\\\ \\text{subject to}\\quad& x_1+x_2\\leq2,\\\\ &x_1,x_2\\geq0. \\end{aligned}\\]\n\nThe unconstrained minimizer, \\((2,1)\\), is infeasible. The objective measures squared Euclidean distance from that point, so the constrained solution is its orthogonal projection onto the feasible triangle.\n\nThe solution lies on \\(x_1+x_2=2\\). Substituting \\(x_2=2-x_1\\) gives\n\n\\[(x_1-2)^2+(1-x_1)^2.\\]\n\nDifferentiating yields \\(4x_1-6=0\\), hence\n\n\\[x^\\star=\\left(\\frac32,\\frac12\\right), \\qquad f(x^\\star)=\\frac12.\\]\n\nThe constraint \\(x_1+x_2\\leq2\\) is active, while both nonnegativity constraints are inactive. Strict convexity guarantees that this solution is unique."
    },
    {
      "heading": "Solution methods",
      "text": "Equality-constrained methods\n\nWith equality constraints alone, stationarity and feasibility give the block system\n\n\\[\\begin{bmatrix} Q&E^\\top\\\\ E&0 \\end{bmatrix} \\begin{bmatrix} x\\\\ \\nu \\end{bmatrix} = \\begin{bmatrix} -c\\\\ d \\end{bmatrix}.\\]\n\nIf \\(E\\) has full row rank and \\(Q\\) is positive definite on the null space of \\(E\\), this system is nonsingular. Null-space methods eliminate the equalities first; range-space methods instead eliminate some variables or multipliers. Their relative efficiency depends on problem dimensions and matrix structure.\n\nActive-set methods\n\nAn active-set method maintains a working set of inequalities treated as equalities. It solves an equality-constrained subproblem, moves toward its solution, and updates the working set when constraints become binding or when multiplier signs indicate that a constraint should be released.\n\nThese methods are often effective for small or medium-sized problems and for repeated, closely related QPs. A previous solution and working set can provide a useful warm start. Degeneracy and a large number of possible working sets can complicate their behavior.\n\nInterior-point methods\n\nInterior-point methods handle inequalities through barrier terms or perturbed primal–dual optimality equations. For example, a logarithmic barrier for \\(Ax<b\\) gives\n\n\\[\\frac12x^\\top Qx+c^\\top x -\\mu\\sum_i\\log(b_i-A_ix),\\]\n\nwith \\(\\mu>0\\) progressively reduced.\n\nPrimal–dual methods typically solve a structured linear system at each iteration. They are widely used for large convex QPs, particularly when matrix sparsity can be exploited. Appropriate interior-point methods have polynomial-time complexity guarantees for convex QP under standard computational assumptions.\n\nFirst-order and splitting methods\n\nProjected-gradient methods alternate a step in the negative objective gradient with projection onto the feasible set. They are attractive when projection is inexpensive, as with simple box constraints, but projection onto a general polyhedron can itself require solving a QP.\n\nOperator-splitting methods divide the problem into simpler computational steps. They can be useful for large sparse problems and applications requiring moderate accuracy, although performance depends on scaling, parameter choices, and the required solution tolerance.\n\nGlobal methods for nonconvex problems\n\nGeneral nonconvex QP is NP-hard. Global solvers may use branch-and-bound, convex relaxations, cutting planes, and bounds that improve as the feasible region is subdivided.\n\nA local nonlinear optimizer may find a useful feasible point, but local optimality alone does not certify global optimality."
    },
    {
      "heading": "Applications",
      "text": "Least squares and statistical estimation\n\nA constrained least-squares problem,\n\n\\[\\min_x\\frac12\\lVert Bx-y\\rVert_2^2 \\quad\\text{subject to linear constraints},\\]\n\nis a QP because\n\n\\[\\frac12\\lVert Bx-y\\rVert_2^2 = \\frac12x^\\top B^\\top Bx-y^\\top Bx +\\frac12y^\\top y.\\]\n\nHere \\(Q=B^\\top B\\) is positive semidefinite. Nonnegative least squares, constrained regression, and many smoothing problems have this form. Adding an \\(L^2\\) penalty, as in ridge regression, adds a positive semidefinite quadratic term.\n\nPortfolio optimization\n\nIn mean–variance portfolio optimization, portfolio risk is represented by\n\n\\[w^\\top\\Sigma w,\\]\n\nwhere \\(w\\) contains asset weights and \\(\\Sigma\\) is their covariance matrix. Minimizing risk subject to a budget constraint, a target expected return, and linear exposure limits produces a convex QP when \\(\\Sigma\\) is positive semidefinite.\n\nRestrictions involving whole-number holdings or a fixed number of selected assets generally require integer variables and change the problem class.\n\nControl and engineering\n\nModel predictive control repeatedly optimizes predicted behavior over a finite horizon. With linear dynamics, quadratic tracking and effort costs, and linear state and input constraints, each optimization is a QP.\n\nEngineering design, trajectory smoothing, signal reconstruction, and resource allocation also produce QPs when costs are quadratic and restrictions are linear.\n\nMachine learning\n\nThe standard soft-margin support vector machine can be expressed as a convex QP. Its dual uses a positive semidefinite kernel matrix and linear constraints on the dual variables.\n\nQP subproblems also occur within sequential quadratic programming, a method for general nonlinear optimization. Each iteration approximates the objective locally by a quadratic function and the constraints by linear functions."
    },
    {
      "heading": "Related problem classes",
      "text": "A QP has a quadratic objective but linear constraints. A problem with quadratic constraints is instead a quadratically constrained quadratic program; its convexity depends on the constraint functions as well as the objective.\n\nIf some decision variables must be integers, the result is a mixed-integer quadratic program, combining QP with integer programming. Even a convex quadratic objective does not make that discrete problem a convex optimization problem.\n\nLinear programming is the special case \\(Q=0\\). Convex QPs can also be reformulated as conic optimization problems, although a direct QP formulation may preserve useful structure more efficiently."
    },
    {
      "heading": "Historical development",
      "text": "Quadratic programming developed alongside postwar operations research and mathematical programming. Harry Markowitz’s 1952 work on portfolio selection supplied an influential application involving a quadratic measure of risk and linear portfolio constraints.\n\nAlgorithmic work expanded during the 1950s, including Philip Wolfe’s quadratic programming method published in 1959. Later developments in active-set methods, interior-point algorithms, sparse linear algebra, and operator splitting broadened the range of problems that could be solved efficiently."
    },
    {
      "heading": "Numerical limitations",
      "text": "Practical performance depends not only on the number of variables and constraints but also on sparsity, scaling, degeneracy, and the conditioning of the linear systems solved internally.\n\nA large condition number can make computed solutions sensitive to small perturbations. In least-squares formulations, explicitly forming \\(B^\\top B\\) can worsen conditioning; methods that preserve the original factor structure may be more reliable.\n\nNearly dependent constraints can make multipliers unstable even when the primal solution is comparatively stable. Solver tolerances also mean that reported feasibility and optimality are approximate: primal residuals, stationarity residuals, and complementarity residuals must be interpreted at the scale of the problem.\n\nAn indefinite estimated quadratic matrix may indicate a genuinely nonconvex model or numerical error in constructing an intended convex one. Adding a diagonal regularization term can improve conditioning or enforce positive definiteness, but it changes the objective and therefore generally changes the optimization problem."
    }
  ],
  "links": [
    "mathematical-optimization",
    "linear-programming",
    "objective-function",
    "quadratic-form",
    "matrix",
    "gradient",
    "hessian-matrix",
    "feasible-set",
    "polyhedron",
    "convex-function",
    "convex-set",
    "convex-optimization",
    "positive-definite-matrix",
    "null-space",
    "karush-kuhn-tucker-conditions",
    "lagrangian-duality",
    "orthogonal-projection",
    "interior-point-method",
    "branch-and-bound",
    "least-squares",
    "ridge-regression",
    "covariance-matrix",
    "model-predictive-control",
    "support-vector-machine",
    "integer-programming",
    "operations-research",
    "condition-number"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}