{
  "id": "gram-schmidt-process",
  "title": "Gram–Schmidt Process",
  "lang": "en",
  "source_lang": "en",
  "summary": "The Gram–Schmidt process converts an ordered set of linearly independent vectors into an orthonormal set spanning the same subspace.",
  "category": "mathematics",
  "subcategory": "algebra",
  "type": "concept",
  "aliases": [
    "Gram–Schmidt orthogonalization",
    "Gram–Schmidt orthonormalization",
    "Gram–Schmidt algorithm"
  ],
  "generated_at": "2026-10-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/gram-schmidt-process/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/gram-schmidt-process/",
      "json": "https://aiwiki.page/zh/gram-schmidt-process.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "The Gram–Schmidt process is an algorithm in linear algebra that constructs an orthonormal basis from an ordered, linearly independent collection of vectors. It successively subtracts the components of each vector along previously constructed directions, then normalizes the remaining vector. The resulting vectors span the same subspace as the original collection. Applied to matrix columns, the process produces a QR decomposition, connecting its geometric interpretation with computational methods. (github.com)"
    },
    {
      "heading": "Mathematical setting",
      "text": "Let \\(v_1,\\ldots,v_n\\) be vectors in a vector space over the real numbers or complex numbers, equipped with an inner product. Orthogonality means that the inner product of two vectors is zero; orthonormality additionally requires each vector to have unit length. Gram–Schmidt uses the associated norm,\n\n\\[\\|v\\|=\\sqrt{\\langle v,v\\rangle}.\\]\n\nThe construction applies to abstract inner-product spaces, not only coordinate vectors: polynomials and suitable spaces of functions also admit it. (people.tamu.edu)\n\nThe standard assumption is linear independence. If the input is a basis of a finite-dimensional space, the output is an orthonormal basis of that space. More generally, it is a basis of the inputs’ linear span, which may be a proper subspace of the ambient space. (github.com)"
    },
    {
      "heading": "Classical construction",
      "text": "Use the convention that \\(\\langle x,y\\rangle\\) is conjugate-linear in its first argument and linear in its second. Set\n\n\\[u_1=v_1,\\qquad q_1=\\frac{u_1}{\\|u_1\\|}.\\]\n\nFor \\(k=2,\\ldots,n\\), define\n\n\\[u_k=v_k-\\sum_{j=1}^{k-1}q_j\\langle q_j,v_k\\rangle, \\qquad q_k=\\frac{u_k}{\\|u_k\\|}.\\]\n\nEach summand is the orthogonal projection of \\(v_k\\) onto the direction \\(q_j\\). Subtracting their sum leaves the component perpendicular to all previously constructed directions. Omitting normalization gives mutually orthogonal vectors \\(u_k\\), rather than unit vectors \\(q_k\\). (archive.nptel.ac.in)\n\nThe justification follows directly from the inner-product identities. Assuming the earlier \\(q_j\\) are orthonormal, for every \\(i<k\\),\n\n\\[\\langle q_i,u_k\\rangle =\\langle q_i,v_k\\rangle -\\sum_{j<k}\\langle q_i,q_j\\rangle \\langle q_j,v_k\\rangle =0.\\]\n\nMoreover, each new vector is a linear combination of the first \\(k\\) inputs, and the defining equation expresses \\(v_k\\) in terms of the new vectors. Consequently,\n\n\\[\\operatorname{span}(q_1,\\ldots,q_k) =\\operatorname{span}(v_1,\\ldots,v_k).\\]\n\nIndependence ensures \\(u_k\\ne0\\), so normalization is possible. (archive.nptel.ac.in)\n\nFor dependent inputs, a zero residual identifies a vector already in the span of earlier accepted vectors; it can be discarded. The output generally depends on input order, although the final spanned subspace does not. (people.tamu.edu)"
    },
    {
      "heading": "Worked example",
      "text": "Consider, in Euclidean space \\(\\mathbb R^3\\),\n\n\\[v_1=(1,1,0),\\qquad v_2=(1,0,1).\\]\n\nWith the standard dot product, the first normalized vector is\n\n\\[q_1=\\frac{(1,1,0)}{\\sqrt2}.\\]\n\nThe second residual is\n\n\\[u_2=v_2-q_1\\langle q_1,v_2\\rangle =(1,0,1)-\\tfrac12(1,1,0) =(\\tfrac12,-\\tfrac12,1).\\]\n\nThus\n\n\\[q_2=\\frac{(1,-1,2)}{\\sqrt6}.\\]\n\nDirect calculation gives \\(\\langle q_1,q_2\\rangle=0\\) and unit norm for both vectors. This illustrates the general construction: \\(q_1,q_2\\) form an orthonormal basis of the plane spanned by the two inputs. (archive.nptel.ac.in)"
    },
    {
      "heading": "Matrix formulation and least squares",
      "text": "Place \\(n\\) independent vectors in the columns of an \\(m\\times n\\) matrix \\(A\\), with \\(m\\ge n\\). Gram–Schmidt yields\n\n\\[A=QR,\\]\n\nwhere \\(Q=[q_1\\ \\cdots\\ q_n]\\) and \\(R\\) is upper triangular. Its entries are\n\n\\[r_{jk}=\\langle q_j,v_k\\rangle\\quad(j<k), \\qquad r_{kk}=\\|u_k\\|>0.\\]\n\nThe relation \\(Q^*Q=I_n\\) uses the conjugate transpose \\(Q^*\\) and the identity matrix. For real matrices, \\(Q^*=Q^T\\). When \\(m=n\\), \\(Q\\) is an orthogonal matrix in the real case and a unitary matrix in the complex case. (github.com)\n\nFor ordinary least squares, minimizing \\(\\|Ax-b\\|_2\\) reduces, in exact arithmetic, to solving\n\n\\[Rx=Q^*b.\\]\n\nThe fitted vector is \\(QQ^*b\\). This avoids explicitly forming the normal equations \\(A^*Ax=A^*b\\), an important distinction in numerical computation. (ocw.mit.edu)"
    },
    {
      "heading": "Modified Gram–Schmidt and numerical behavior",
      "text": "In floating-point arithmetic, classical Gram–Schmidt can lose orthogonality, particularly when input vectors are nearly dependent. Modified Gram–Schmidt changes the order of operations: initialize \\(w=v_k\\), then successively compute\n\n\\[r_{jk}=\\langle q_j,w\\rangle,\\qquad w\\leftarrow w-q_jr_{jk},\\]\n\nbefore normalizing \\(w\\). Each projection therefore uses the updated residual rather than the original vector. The two versions are equivalent in exact arithmetic but differ in numerical stability. (ocw.mit.edu)\n\nModified Gram–Schmidt generally preserves orthogonality better, but substantial loss can still occur. Reorthogonalization repeats the projection-removal step to reduce remaining components along earlier vectors. Both basic versions require \\(O(mn^2)\\) arithmetic operations. Householder transformations provide another QR construction with stronger orthogonality properties in floating-point computation. (netlib.org)"
    },
    {
      "heading": "Function spaces and iterative methods",
      "text": "Applying Gram–Schmidt to \\(1,x,x^2,\\ldots\\), using\n\n\\[\\langle f,g\\rangle=\\int_{-1}^{1}\\overline{f(x)}g(x)\\,dx,\\]\n\nproduces orthogonal polynomials. With conventional scaling, these are the Legendre polynomials, beginning with \\(1\\), \\(x\\), and \\((3x^2-1)/2\\). Conventional Legendre normalization differs from unit-norm normalization. In a separable Hilbert space, orthogonalizing a countable collection with dense span produces an orthonormal basis in the sense of closed linear span. (people.tamu.edu)\n\nIn numerical linear algebra, Gram–Schmidt also builds bases of Krylov subspaces. Orthogonalizing successive matrix-generated vectors underlies the Arnoldi iteration, which supplies the orthonormal basis used by the generalized minimal residual method for solving linear systems. (netlib.org)"
    }
  ],
  "links": [
    "algorithm",
    "linear-algebra",
    "orthonormal-basis",
    "qr-decomposition",
    "vector-space",
    "real-number",
    "complex-number",
    "inner-product",
    "norm-mathematics",
    "linear-independence",
    "basis-linear-algebra",
    "linear-span",
    "linear-subspace",
    "orthogonal-projection",
    "linear-combination",
    "euclidean-space",
    "matrix",
    "conjugate-transpose",
    "identity-matrix",
    "orthogonal-matrix",
    "unitary-matrix",
    "ordinary-least-squares",
    "floating-point-arithmetic",
    "numerical-stability",
    "householder-transformation",
    "orthogonal-polynomials",
    "legendre-polynomials",
    "hilbert-space"
  ],
  "sources": [
    {
      "title": "github.com/mitmath/1806",
      "url": "https://github.com/mitmath/1806"
    },
    {
      "title": "people.tamu.edu/~f-narcowich/m641/m641_notes/orthonormal_sets2014.html",
      "url": "https://people.tamu.edu/~f-narcowich/m641/m641_notes/orthonormal_sets2014.html"
    },
    {
      "title": "archive.nptel.ac.in/content/storage2/courses/103101009/modules/module2/sec5/1.2…",
      "url": "https://archive.nptel.ac.in/content/storage2/courses/103101009/modules/module2/sec5/1.2.html"
    },
    {
      "title": "github.com/mitmath/numerical_hub",
      "url": "https://github.com/mitmath/numerical_hub"
    },
    {
      "title": "ocw.mit.edu/courses/18-065-matrix-methods-in-data-analysis-signal-processing-an…",
      "url": "https://ocw.mit.edu/courses/18-065-matrix-methods-in-data-analysis-signal-processing-and-machine-learning-spring-2018/resources/lecture-9-four-ways-to-solve-least-squares-problems/"
    },
    {
      "title": "ocw.mit.edu/courses/18-335j-introduction-to-numerical-methods-spring-2019/pages…",
      "url": "https://ocw.mit.edu/courses/18-335j-introduction-to-numerical-methods-spring-2019/pages/week-4/"
    },
    {
      "title": "netlib.org/utk/people/JackDongarra/etemplates/node109.html",
      "url": "https://netlib.org/utk/people/JackDongarra/etemplates/node109.html"
    },
    {
      "title": "netlib.org/linalg/old_html_templates/subsection2.6.3.4.html",
      "url": "https://www.netlib.org/linalg/old_html_templates/subsection2.6.3.4.html"
    }
  ],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}