{
  "id": "singular-value",
  "title": "Singular Value",
  "lang": "en",
  "source_lang": "en",
  "summary": "A singular value is a nonnegative measure of how strongly a matrix or linear operator stretches vectors along an orthogonal principal direction.",
  "category": "mathematics",
  "subcategory": "algebra",
  "type": "concept",
  "aliases": [
    "Matrix singular value"
  ],
  "generated_at": "2026-11-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/singular-value/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/singular-value/",
      "json": "https://aiwiki.page/zh/singular-value.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "A singular value of a real or complex matrix \\(A\\) is a nonnegative square root of an eigenvalue of \\(A^*A\\), where \\(A^*\\) denotes the conjugate transpose. Singular values describe the stretching produced by the associated linear map, independently of the orthonormal coordinates used in its domain and codomain. They are defined for rectangular as well as square matrices and are central to matrix approximation, the solution of linear systems, and the analysis of numerical sensitivity."
    },
    {
      "heading": "Definition and conventions",
      "text": "Let \\(A\\in\\mathbb{C}^{m\\times n}\\), with \\(m,n\\geq 1\\), and put \\(p=\\min(m,n)\\). The singular values of \\(A\\) are conventionally listed as\n\n\\[\\sigma_1(A)\\geq \\sigma_2(A)\\geq\\cdots\\geq\\sigma_p(A)\\geq0,\\]\n\ncounting multiplicities.\n\nThe matrix \\(A^*A\\) is Hermitian and positive semidefinite because\n\n\\[x^*A^*Ax=\\|Ax\\|_2^2\\geq0.\\]\n\nConsequently, all its eigenvalues are real and nonnegative.\n\nMore precisely, if \\(A\\) has rank \\(r\\), then \\(A^*A\\) and \\(AA^*\\) have the same \\(r\\) positive eigenvalues, counting multiplicities. Their positive square roots are the positive singular values of \\(A\\). The conventional list of \\(p\\) singular values is completed with \\(p-r\\) zeros.\n\nThis convention matters for rectangular matrices. If \\(n>m\\), the matrix \\(A^*A\\) has \\(n-m\\) additional zero eigenvalues beyond those represented in the length-\\(p\\) singular-value list. Some treatments instead retain all \\(n\\) square roots of the eigenvalues of \\(A^*A\\), explicitly including those extra zeros.\n\nFor a real matrix, \\(A^*\\) is simply its transpose \\(A^{\\mathsf T}\\)."
    },
    {
      "heading": "Singular value decomposition",
      "text": "The singular value decomposition (SVD) expresses any such matrix as\n\n\\[A=U\\Sigma V^*,\\]\n\nwhere \\(U\\in\\mathbb{C}^{m\\times m}\\) and \\(V\\in\\mathbb{C}^{n\\times n}\\) are unitary matrices, and \\(\\Sigma\\) is an \\(m\\times n\\) rectangular diagonal matrix whose diagonal entries are \\(\\sigma_1,\\ldots,\\sigma_p\\). For real matrices, \\(U\\) and \\(V\\) can be chosen to be orthogonal matrices.\n\nThe columns \\(v_i\\) of \\(V\\) are right singular vectors, and the columns \\(u_i\\) of \\(U\\) are left singular vectors. For \\(1\\leq i\\leq p\\),\n\n\\[Av_i=\\sigma_i u_i, \\qquad A^*u_i=\\sigma_i v_i.\\]\n\nIn particular,\n\n\\[A^*Av_i=\\sigma_i^2v_i, \\qquad AA^*u_i=\\sigma_i^2u_i.\\]\n\nThe singular values are uniquely determined by \\(A\\), once their ordering is fixed. The singular vectors are not generally unique. A repeated singular value permits changes of orthonormal basis within its singular subspaces. Even a simple positive singular value allows a simultaneous sign change of its left and right singular vectors in the real case, or a simultaneous multiplication by a complex number of modulus one in the complex case.\n\nThe SVD establishes that every matrix has singular values, without requiring the matrix to be square, invertible, or diagonalizable."
    },
    {
      "heading": "Geometric interpretation",
      "text": "With the standard inner products, a matrix maps a unit sphere to an ellipsoid, possibly contained in a lower-dimensional subspace. The positive singular values are the lengths of the ellipsoid’s principal semiaxes, and the corresponding left singular vectors give their directions. Zero singular values indicate directions that collapse to zero.\n\nThe factorization \\(A=U\\Sigma V^*\\) separates the transformation into three steps:\n\n- \\(V^*\\) changes orthonormal coordinates in the domain.\n- \\(\\Sigma\\) stretches, compresses, or annihilates coordinate directions.\n- \\(U\\) changes orthonormal coordinates in the codomain.\n\nFor example,\n\n\\[A= \\begin{pmatrix} 3&0\\\\ 0&-2 \\end{pmatrix}\\]\n\nhas singular values \\(3\\) and \\(2\\). It maps the unit circle to an ellipse with semiaxis lengths \\(3\\) and \\(2\\). The negative eigenvalue \\(-2\\) records a reversal of direction, whereas the corresponding singular value records only the stretching magnitude.\n\nFor every nonzero vector \\(x\\), the ratio\n\n\\[\\frac{\\|Ax\\|_2}{\\|x\\|_2}\\]\n\nmeasures the stretching along that vector. Its maximum is\n\n\\[\\sigma_1(A)=\\max_{\\|x\\|_2=1}\\|Ax\\|_2.\\]\n\nIf \\(m\\geq n\\), its minimum is\n\n\\[\\sigma_n(A)=\\min_{\\|x\\|_2=1}\\|Ax\\|_2.\\]\n\nIf \\(m<n\\), this minimum is always zero because the null space is nontrivial, even when all \\(p=m\\) conventionally listed singular values are positive.\n\nSingular values depend on the inner products, or equivalently on how lengths are measured in the domain and codomain. They are invariant under orthonormal coordinate changes, not under arbitrary changes of basis or changes of measurement scale."
    },
    {
      "heading": "Relation to eigenvalues",
      "text": "Eigenvalues describe directions that a square matrix preserves up to scalar multiplication. Singular values describe orthogonal directions of stretching between a domain and a codomain, which may have different dimensions.\n\nFor a normal matrix—one satisfying \\(A^*A=AA^*\\)—the singular values are the absolute values of its eigenvalues, counting multiplicities. This includes real symmetric, complex Hermitian, and unitary matrices.\n\nFor a general matrix, singular values need not equal eigenvalue magnitudes. For example,\n\n\\[A= \\begin{pmatrix} 0&1\\\\ 0&0 \\end{pmatrix}\\]\n\nhas two zero eigenvalues but singular values \\(1\\) and \\(0\\). Thus eigenvalues alone need not describe how much a matrix can stretch a vector.\n\nThe largest eigenvalue magnitude is always at most \\(\\sigma_1(A)\\), but the inequality can be strict."
    },
    {
      "heading": "Fundamental properties",
      "text": "Rank and invertibility\n\nThe number of positive singular values equals the rank:\n\n\\[\\operatorname{rank}(A)=\\#\\{i:\\sigma_i(A)>0\\}.\\]\n\nFor a square \\(n\\times n\\) matrix, invertibility is equivalent to \\(\\sigma_n(A)>0\\). If \\(A\\) is invertible, the singular values of its inverse are the reciprocals of those of \\(A\\), in reverse order:\n\n\\[\\sigma_i(A^{-1})=\\frac{1}{\\sigma_{n-i+1}(A)}.\\]\n\nUnitary invariance and scaling\n\nIf \\(Q\\) and \\(R\\) are unitary matrices of suitable sizes, then\n\n\\[\\sigma_i(QAR)=\\sigma_i(A).\\]\n\nAlso,\n\n\\[\\sigma_i(cA)=|c|\\,\\sigma_i(A).\\]\n\nThe matrices \\(A\\) and \\(A^*\\) have the same conventional singular-value list.\n\nMatrix norms\n\nSeveral important matrix norms can be expressed through singular values.\n\nThe spectral norm, induced by the Euclidean vector norm, is\n\n\\[\\|A\\|_2=\\sigma_1(A).\\]\n\nThe Frobenius norm satisfies\n\n\\[\\|A\\|_F =\\left(\\sum_{i,j}|a_{ij}|^2\\right)^{1/2} =\\left(\\sum_{i=1}^{p}\\sigma_i(A)^2\\right)^{1/2}.\\]\n\nThe nuclear norm, also called the trace norm, is\n\n\\[\\|A\\|_*=\\sum_{i=1}^{p}\\sigma_i(A).\\]\n\nIt is used as a convex surrogate for rank in convex optimization.\n\nMore generally, for \\(1\\leq q<\\infty\\), the Schatten \\(q\\)-norm is\n\n\\[\\|A\\|_{S_q} =\\left(\\sum_{i=1}^{p}\\sigma_i(A)^q\\right)^{1/q}.\\]\n\nDeterminant and volume\n\nFor square \\(A\\),\n\n\\[|\\det A|=\\prod_{i=1}^{n}\\sigma_i(A).\\]\n\nThus the absolute determinant, which gives the factor by which the transformation scales volume, is the product of its principal stretching factors.\n\nFor rectangular \\(A\\) with full column rank,\n\n\\[\\det(A^*A)=\\prod_{i=1}^{n}\\sigma_i(A)^2.\\]\n\nIts square root is the factor by which \\(A\\) scales \\(n\\)-dimensional volume in its image.\n\nVariational characterization\n\nSingular values can be characterized without explicitly constructing an SVD. Let\n\n\\[\\widehat{\\sigma}_1\\geq\\cdots\\geq\\widehat{\\sigma}_n\\]\n\nbe the square roots of all eigenvalues of \\(A^*A\\), including any additional zeros when \\(n>m\\). Then\n\n\\[\\widehat{\\sigma}_k = \\max_{\\substack{S\\subseteq\\mathbb{C}^n\\\\ \\dim S=k}} \\ \\min_{\\substack{x\\in S\\\\ \\|x\\|_2=1}} \\|Ax\\|_2.\\]\n\nHere \\(S\\) ranges over linear subspaces of dimension \\(k\\). This states that the \\(k\\)-th singular value is the largest minimum stretching achievable on a \\(k\\)-dimensional subspace."
    },
    {
      "heading": "Low-rank approximation",
      "text": "Singular values measure how accurately a matrix can be approximated by one of lower rank. Writing\n\n\\[A=\\sum_{i=1}^{r}\\sigma_i u_i v_i^*,\\]\n\ndefine the truncated SVD\n\n\\[A_k=\\sum_{i=1}^{k}\\sigma_i u_i v_i^*, \\qquad 0\\leq k<r.\\]\n\nThe Eckart–Young–Mirsky theorem gives\n\n\\[\\min_{\\operatorname{rank}(B)\\leq k}\\|A-B\\|_2 =\\|A-A_k\\|_2 =\\sigma_{k+1},\\]\n\nand\n\n\\[\\min_{\\operatorname{rank}(B)\\leq k}\\|A-B\\|_F =\\|A-A_k\\|_F =\\left(\\sum_{i=k+1}^{p}\\sigma_i^2\\right)^{1/2}.\\]\n\nRapid decay of singular values therefore indicates that a low-rank approximation can have small error. This underlies applications in image compression, dimensionality reduction, and reduced-order models. If singular values near the truncation point are repeated, the best approximation need not be unique."
    },
    {
      "heading": "Conditioning and perturbations",
      "text": "For an invertible square matrix, the spectral condition number is\n\n\\[\\kappa_2(A) =\\|A\\|_2\\|A^{-1}\\|_2 =\\frac{\\sigma_1(A)}{\\sigma_n(A)}.\\]\n\nA large ratio means that the transformation stretches some directions much more than others, allowing perturbations in data to be strongly amplified when solving the associated linear system.\n\nThe smallest singular value also gives the distance to singularity:\n\n\\[\\min_{A+E\\ \\text{singular}}\\|E\\|_2=\\sigma_n(A).\\]\n\nIndividual singular values obey the perturbation bound\n\n\\[|\\sigma_i(A+E)-\\sigma_i(A)|\\leq\\|E\\|_2.\\]\n\nThey are therefore stable in an absolute-error sense. A very small singular value, however, may have a large relative error even under a small perturbation.\n\nSingular vectors can be much more sensitive when neighboring singular values are close. Stability of singular values should therefore be distinguished from stability of their associated directions.\n\nIn numerical work, numerical rank counts singular values exceeding a tolerance rather than those that are exactly positive. The tolerance depends on data scale, noise, dimensions, and floating-point precision; there is no universal threshold separating signal from error."
    },
    {
      "heading": "Applications",
      "text": "Least squares and inverse problems\n\nFor a least-squares problem\n\n\\[\\min_x\\|Ax-b\\|_2,\\]\n\nthe minimum-norm solution can be written\n\n\\[x=\\sum_{i=1}^{r}\\frac{u_i^*b}{\\sigma_i}v_i.\\]\n\nSmall positive singular values lead to division by small numbers and can amplify errors in \\(b\\). This is a central issue in inverse problems.\n\nRegularization modifies this amplification. Truncated SVD omits terms associated with small singular values. Tikhonov regularization replaces the factor \\(1/\\sigma_i\\) by\n\n\\[\\frac{\\sigma_i}{\\sigma_i^2+\\lambda}, \\qquad \\lambda>0.\\]\n\nThese methods exchange exact fitting in sensitive directions for reduced sensitivity to noise.\n\nPrincipal component analysis\n\nLet \\(X\\) be a real data matrix with centered columns and \\(N>1\\) observations. Its sample covariance matrix is\n\n\\[C=\\frac{1}{N-1}X^{\\mathsf T}X.\\]\n\nThe principal-component variances are therefore\n\n\\[\\lambda_i(C)=\\frac{\\sigma_i(X)^2}{N-1}.\\]\n\nThe right singular vectors give the principal directions, linking singular values directly to principal component analysis. Relative sums of squared singular values describe the proportion of total variance retained by selected components."
    },
    {
      "heading": "Computation",
      "text": "Although singular values are defined through \\(A^*A\\), explicitly forming this product is often undesirable. For a full-column-rank matrix,\n\n\\[\\kappa_2(A^*A)=\\kappa_2(A)^2,\\]\n\nand rounding errors in the product can obscure small singular values.\n\nAlgorithms in numerical linear algebra commonly reduce \\(A\\) to bidiagonal form using orthogonal or unitary transformations, then compute the singular values of the bidiagonal matrix. Such transformations help maintain numerical stability.\n\nFor very large matrices, iterative and randomized methods can estimate selected singular values or construct approximate truncated decompositions without computing a full SVD. Their accuracy and cost depend on the spectrum, matrix structure, and the number of singular values sought."
    },
    {
      "heading": "Extension to operators",
      "text": "In functional analysis, singular values extend naturally to compact operators \\(T\\) between Hilbert spaces. They are the eigenvalues of the positive operator\n\n\\[|T|=(T^*T)^{1/2},\\]\n\nlisted in nonincreasing order with multiplicities, with zeros appended when appropriate.\n\nIf infinitely many singular values are nonzero, they tend to zero. Their decay quantifies approximation by finite-rank operators, extending the role singular values play for matrices.\n\nFor general bounded operators that are not compact, \\(|T|\\) need not have a discrete eigenvalue list. The finite-dimensional description of stretching by a sequence of singular values therefore does not transfer unchanged to every infinite-dimensional operator."
    }
  ],
  "links": [
    "matrix",
    "eigenvalues-and-eigenvectors",
    "conjugate-transpose",
    "linear-map",
    "matrix-rank",
    "matrix-transpose",
    "singular-value-decomposition",
    "unitary-matrix",
    "orthogonal-matrix",
    "inner-product",
    "null-space",
    "inverse-matrix",
    "norm-mathematics",
    "convex-optimization",
    "determinant",
    "linear-subspace",
    "dimensionality-reduction",
    "condition-number",
    "system-of-linear-equations",
    "floating-point-arithmetic",
    "least-squares",
    "inverse-problem",
    "regularization",
    "covariance-matrix",
    "principal-component-analysis",
    "numerical-linear-algebra",
    "numerical-stability",
    "functional-analysis",
    "hilbert-space"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}