A singular value of a real or complex matrix is a nonnegative square root of an eigenvalue of , where denotes the conjugate transpose. Singular values describe the stretching produced by the associated linear map, independently of the orthonormal coordinates used in its domain and codomain. They are defined for rectangular as well as square matrices and are central to matrix approximation, the solution of linear systems, and the analysis of numerical sensitivity.
Definition and conventions
Let , with , and put . The singular values of are conventionally listed as
counting multiplicities.
The matrix is Hermitian and positive semidefinite because
Consequently, all its eigenvalues are real and nonnegative.
More precisely, if has rank , then and have the same positive eigenvalues, counting multiplicities. Their positive square roots are the positive singular values of . The conventional list of singular values is completed with zeros.
This convention matters for rectangular matrices. If , the matrix has additional zero eigenvalues beyond those represented in the length- singular-value list. Some treatments instead retain all square roots of the eigenvalues of , explicitly including those extra zeros.
For a real matrix, is simply its transpose .
Singular value decomposition
The singular value decomposition (SVD) expresses any such matrix as
where and are unitary matrices, and is an rectangular diagonal matrix whose diagonal entries are . For real matrices, and can be chosen to be orthogonal matrices.
The columns of are right singular vectors, and the columns of are left singular vectors. For ,
In particular,
The singular values are uniquely determined by , once their ordering is fixed. The singular vectors are not generally unique. A repeated singular value permits changes of orthonormal basis within its singular subspaces. Even a simple positive singular value allows a simultaneous sign change of its left and right singular vectors in the real case, or a simultaneous multiplication by a complex number of modulus one in the complex case.
The SVD establishes that every matrix has singular values, without requiring the matrix to be square, invertible, or diagonalizable.
Geometric interpretation
With the standard inner products, a matrix maps a unit sphere to an ellipsoid, possibly contained in a lower-dimensional subspace. The positive singular values are the lengths of the ellipsoid’s principal semiaxes, and the corresponding left singular vectors give their directions. Zero singular values indicate directions that collapse to zero.
The factorization separates the transformation into three steps:
- changes orthonormal coordinates in the domain.
- stretches, compresses, or annihilates coordinate directions.
- changes orthonormal coordinates in the codomain.
For example,
has singular values and . It maps the unit circle to an ellipse with semiaxis lengths and . The negative eigenvalue records a reversal of direction, whereas the corresponding singular value records only the stretching magnitude.
For every nonzero vector , the ratio
measures the stretching along that vector. Its maximum is
If , its minimum is
If , this minimum is always zero because the null space is nontrivial, even when all conventionally listed singular values are positive.
Singular values depend on the inner products, or equivalently on how lengths are measured in the domain and codomain. They are invariant under orthonormal coordinate changes, not under arbitrary changes of basis or changes of measurement scale.
Relation to eigenvalues
Eigenvalues describe directions that a square matrix preserves up to scalar multiplication. Singular values describe orthogonal directions of stretching between a domain and a codomain, which may have different dimensions.
For a normal matrix—one satisfying —the singular values are the absolute values of its eigenvalues, counting multiplicities. This includes real symmetric, complex Hermitian, and unitary matrices.
For a general matrix, singular values need not equal eigenvalue magnitudes. For example,
has two zero eigenvalues but singular values and . Thus eigenvalues alone need not describe how much a matrix can stretch a vector.
The largest eigenvalue magnitude is always at most , but the inequality can be strict.
Fundamental properties
Rank and invertibility
The number of positive singular values equals the rank:
For a square matrix, invertibility is equivalent to . If is invertible, the singular values of its inverse are the reciprocals of those of , in reverse order:
Unitary invariance and scaling
If and are unitary matrices of suitable sizes, then
Also,
The matrices and have the same conventional singular-value list.
Matrix norms
Several important matrix norms can be expressed through singular values.
The spectral norm, induced by the Euclidean vector norm, is
The Frobenius norm satisfies
The nuclear norm, also called the trace norm, is
It is used as a convex surrogate for rank in convex optimization.
More generally, for , the Schatten -norm is
Determinant and volume
For square ,
Thus the absolute determinant, which gives the factor by which the transformation scales volume, is the product of its principal stretching factors.
For rectangular with full column rank,
Its square root is the factor by which scales -dimensional volume in its image.
Variational characterization
Singular values can be characterized without explicitly constructing an SVD. Let
be the square roots of all eigenvalues of , including any additional zeros when . Then
Here ranges over linear subspaces of dimension . This states that the -th singular value is the largest minimum stretching achievable on a -dimensional subspace.
Low-rank approximation
Singular values measure how accurately a matrix can be approximated by one of lower rank. Writing
define the truncated SVD
The Eckart–Young–Mirsky theorem gives
and
Rapid decay of singular values therefore indicates that a low-rank approximation can have small error. This underlies applications in image compression, dimensionality reduction, and reduced-order models. If singular values near the truncation point are repeated, the best approximation need not be unique.
Conditioning and perturbations
For an invertible square matrix, the spectral condition number is
A large ratio means that the transformation stretches some directions much more than others, allowing perturbations in data to be strongly amplified when solving the associated linear system.
The smallest singular value also gives the distance to singularity:
Individual singular values obey the perturbation bound
They are therefore stable in an absolute-error sense. A very small singular value, however, may have a large relative error even under a small perturbation.
Singular vectors can be much more sensitive when neighboring singular values are close. Stability of singular values should therefore be distinguished from stability of their associated directions.
In numerical work, numerical rank counts singular values exceeding a tolerance rather than those that are exactly positive. The tolerance depends on data scale, noise, dimensions, and floating-point precision; there is no universal threshold separating signal from error.
Applications
Least squares and inverse problems
For a least-squares problem
the minimum-norm solution can be written
Small positive singular values lead to division by small numbers and can amplify errors in . This is a central issue in inverse problems.
Regularization modifies this amplification. Truncated SVD omits terms associated with small singular values. Tikhonov regularization replaces the factor by
These methods exchange exact fitting in sensitive directions for reduced sensitivity to noise.
Principal component analysis
Let be a real data matrix with centered columns and observations. Its sample covariance matrix is
The principal-component variances are therefore
The right singular vectors give the principal directions, linking singular values directly to principal component analysis. Relative sums of squared singular values describe the proportion of total variance retained by selected components.
Computation
Although singular values are defined through , explicitly forming this product is often undesirable. For a full-column-rank matrix,
and rounding errors in the product can obscure small singular values.
Algorithms in numerical linear algebra commonly reduce to bidiagonal form using orthogonal or unitary transformations, then compute the singular values of the bidiagonal matrix. Such transformations help maintain numerical stability.
For very large matrices, iterative and randomized methods can estimate selected singular values or construct approximate truncated decompositions without computing a full SVD. Their accuracy and cost depend on the spectrum, matrix structure, and the number of singular values sought.
Extension to operators
In functional analysis, singular values extend naturally to compact operators between Hilbert spaces. They are the eigenvalues of the positive operator
listed in nonincreasing order with multiplicities, with zeros appended when appropriate.
If infinitely many singular values are nonzero, they tend to zero. Their decay quantifies approximation by finite-rank operators, extending the role singular values play for matrices.
For general bounded operators that are not compact, need not have a discrete eigenvalue list. The finite-dimensional description of stretching by a sequence of singular values therefore does not transfer unchanged to every infinite-dimensional operator.