{
  "id": "back-substitution",
  "title": "Back Substitution",
  "lang": "en",
  "source_lang": "en",
  "summary": "Back substitution solves an upper triangular system of linear equations by determining unknowns successively from the last equation upward.",
  "category": "mathematics",
  "subcategory": "algebra",
  "type": "concept",
  "aliases": [
    "Backward substitution"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/back-substitution/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/back-substitution/",
      "json": "https://aiwiki.page/zh/back-substitution.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Back substitution is an algorithm for solving a system of linear equations whose coefficient matrix is upper triangular. It determines the last unknown from the last equation, substitutes that value into the preceding equation, and continues upward until all unknowns have been found. In numerical linear algebra, it is a standard final step after a general system has been reduced to triangular form, particularly by Gaussian elimination."
    },
    {
      "heading": "Mathematical formulation",
      "text": "Consider the system\n\n\\[Ux=b,\\]\n\nwhere \\(U=(u_{ij})\\) is an \\(n\\times n\\) upper triangular matrix, so that \\(u_{ij}=0\\) whenever \\(i>j\\). Written explicitly, the system is\n\n\\[\\begin{aligned} u_{11}x_1+u_{12}x_2+\\cdots+u_{1n}x_n&=b_1,\\\\ u_{22}x_2+\\cdots+u_{2n}x_n&=b_2,\\\\ &\\vdots\\\\ u_{nn}x_n&=b_n. \\end{aligned}\\]\n\nIf every diagonal entry \\(u_{ii}\\) is nonzero, the last equation gives\n\n\\[x_n=\\frac{b_n}{u_{nn}}.\\]\n\nThe remaining unknowns are then computed in descending index order:\n\n\\[x_i= \\frac{b_i-\\displaystyle\\sum_{j=i+1}^{n}u_{ij}x_j}{u_{ii}}, \\qquad i=n-1,n-2,\\ldots,1.\\]\n\nAt each stage, all unknowns appearing to the right of \\(x_i\\) have already been determined. This dependency structure is what makes the procedure possible.\n\nThe nonzero-diagonal condition is equivalent to nonsingularity because the determinant of a triangular matrix is\n\n\\[\\det(U)=\\prod_{i=1}^{n}u_{ii}.\\]\n\nThus, under this condition, back substitution produces the unique solution. The same formulas apply over the real numbers, the complex numbers, or any field in which the required divisions are defined."
    },
    {
      "heading": "Example",
      "text": "For the system\n\n\\[\\begin{aligned} 2x_1-x_2+3x_3&=9,\\\\ 4x_2+2x_3&=10,\\\\ 5x_3&=5, \\end{aligned}\\]\n\nthe last equation gives\n\n\\[x_3=1.\\]\n\nSubstituting into the second equation gives\n\n\\[x_2=\\frac{10-2(1)}{4}=2.\\]\n\nFinally,\n\n\\[x_1=\\frac{9+2-3(1)}{2}=4.\\]\n\nThe solution is therefore\n\n\\[x= \\begin{pmatrix} 4\\\\2\\\\1 \\end{pmatrix}.\\]\n\nUnlike elimination, this calculation does not create zeros in the coefficient matrix: it uses zeros that are already present."
    },
    {
      "heading": "Algorithm and computational cost",
      "text": "A direct implementation is:\n\nfor i = n, n-1, ..., 1:\n    s = b[i]\n    for j = i+1, ..., n:\n        s = s - U[i,j] * x[j]\n    x[i] = s / U[i,i]\n\nFor a dense triangular matrix, the algorithm performs\n\n\\[\\frac{n(n-1)}{2}\\]\n\nmultiplications, the same number of subtractions, and \\(n\\) divisions. Counting each of these as one arithmetic operation gives \\(n^2\\) operations in total. Its running time is therefore \\(O(n^2)\\) in big-O notation, compared with \\(O(n^3)\\) for a conventional dense elimination or factorization.\n\nThe solution may overwrite the right-hand-side vector, so only constant auxiliary storage is required beyond the matrix and the storage used for that vector. If \\(U\\) is unit upper triangular, meaning that all diagonal entries equal one, the divisions can be omitted.\n\nFor a sparse matrix, implementations can skip entries known to be zero. The arithmetic cost then depends on the number and arrangement of stored nonzero entries. For an upper triangular matrix with upper bandwidth \\(w\\), the solve requires \\(O(nw)\\) operations."
    },
    {
      "heading": "Role in matrix factorizations",
      "text": "Back substitution is usually one stage of a larger solution method.\n\nGaussian elimination and LU decomposition\n\nGaussian elimination transforms a general system into an equivalent upper triangular system. Back substitution then recovers the unknowns; it is distinct from the elimination stage itself.\n\nIn an LU decomposition with row permutations, a common convention is\n\n\\[PA=LU,\\]\n\nwhere \\(P\\) is a permutation matrix, \\(L\\) is lower triangular, and \\(U\\) is upper triangular. Solving \\(Ax=b\\) then consists of:\n\n- Applying the permutation to obtain \\(Pb\\).\n- Solving \\(Ly=Pb\\) by forward substitution.\n- Solving \\(Ux=y\\) by back substitution.\n\nOnce the factors have been computed, they can be reused for different right-hand sides without repeating the factorization.\n\nQR decomposition and least squares\n\nFor a real matrix \\(A\\) with linearly independent columns, a reduced QR decomposition has the form\n\n\\[A=QR,\\]\n\nwhere \\(Q\\) has orthonormal columns and \\(R\\) is square, upper triangular, and nonsingular. The least-squares problem\n\n\\[\\min_x \\|Ax-b\\|_2\\]\n\nreduces to\n\n\\[Rx=Q^{T}b,\\]\n\nwhich is solved by back substitution. For complex matrices, the conjugate transpose replaces the transpose.\n\nCholesky decomposition\n\nFor a real symmetric positive-definite matrix, the Cholesky decomposition can be written\n\n\\[A=LL^{T}.\\]\n\nThe system is solved by forward substitution with \\(L\\), followed by back substitution with \\(L^{T}\\).\n\nThese methods use triangular solves rather than explicitly forming an inverse matrix, avoiding the additional work of computing an inverse when only solutions are needed."
    },
    {
      "heading": "Numerical accuracy",
      "text": "In floating-point arithmetic, multiplication, subtraction, and division introduce rounding errors. Nevertheless, ordinary back substitution has a standard componentwise backward-error bound. Under the usual arithmetic model, and assuming no overflow or problematic underflow, the computed solution \\(\\widehat{x}\\) satisfies\n\n\\[(U+\\Delta U)\\widehat{x}=b, \\qquad |\\Delta U|\\leq \\gamma_n |U|,\\]\n\nwhere absolute values and inequalities are componentwise, and\n\n\\[\\gamma_n=\\frac{nu}{1-nu}.\\]\n\nHere \\(u\\) is the unit roundoff and \\(nu<1\\) is assumed. This means that the computed result is the exact solution of a nearby triangular system.\n\nThis form of numerical stability does not guarantee a small error in the solution itself. Forward accuracy also depends on the condition number of \\(U\\): an ill-conditioned system can amplify small perturbations substantially.\n\nLarge intermediate values may overflow even when the final mathematical solution is representable. Specialized triangular solvers can introduce scaling to reduce this risk. Moreover, the stability of back substitution does not by itself establish the stability of a preceding factorization."
    },
    {
      "heading": "Singular and rectangular systems",
      "text": "The standard recurrence requires nonzero diagonal entries. If \\(u_{ii}=0\\), division by that entry is impossible. At that stage, the equation instead imposes the compatibility condition\n\n\\[0=b_i-\\sum_{j=i+1}^{n}u_{ij}x_j.\\]\n\nA nonzero right-hand side makes that equation incompatible with the values already assigned; a zero right-hand side leaves \\(x_i\\) undetermined at that step. Earlier equations may impose further constraints.\n\nConsequently, a zero diagonal entry does not by itself distinguish an inconsistent system from one with infinitely many solutions. Handling singular systems requires identifying pivot variables, free variables, and compatibility conditions, using information about matrix rank.\n\nBack substitution also applies to the pivot equations of a rectangular row-echelon system. Free variables are assigned parameters, after which pivot variables are determined in reverse pivot order."
    },
    {
      "heading": "Variants and implementation",
      "text": "With multiple right-hand sides, the problem becomes\n\n\\[UX=B.\\]\n\nEach column can be solved separately, but blocked implementations can process several columns together and exploit efficient matrix multiplication routines.\n\nBlock back substitution partitions the system into triangular blocks. For example,\n\n\\[\\begin{pmatrix} U_{11}&U_{12}\\\\ 0&U_{22} \\end{pmatrix} \\begin{pmatrix} x_1\\\\x_2 \\end{pmatrix} = \\begin{pmatrix} b_1\\\\b_2 \\end{pmatrix}\\]\n\nis solved first for \\(x_2\\), then for \\(x_1\\):\n\n\\[U_{22}x_2=b_2, \\qquad U_{11}x_1=b_1-U_{12}x_2.\\]\n\nThe ordinary scalar algorithm has a sequential dependency between successive unknowns. This limits straightforward parallelization, although block updates, multiple right-hand sides, and independent portions of sparse dependency structures can provide parallel work."
    }
  ],
  "links": [
    "algorithm",
    "system-of-linear-equations",
    "numerical-linear-algebra",
    "gaussian-elimination",
    "triangular-matrix",
    "determinant",
    "real-number",
    "complex-number",
    "field-mathematics",
    "big-o-notation",
    "sparse-matrix",
    "lu-decomposition",
    "qr-decomposition",
    "least-squares",
    "positive-definite-matrix",
    "cholesky-decomposition",
    "inverse-matrix",
    "floating-point-arithmetic",
    "numerical-stability",
    "condition-number",
    "matrix-rank",
    "matrix-multiplication",
    "parallel-computing"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}