A matrix norm is a norm on a vector space of matrices, assigning each matrix a nonnegative number that measures its size. Depending on the norm, this size may describe the magnitude of its entries, its maximum amplification of vectors, or the aggregate size of its singular values. Matrix norms are fundamental in linear algebra and numerical analysis, where they quantify errors, sensitivity, approximation quality, and convergence.
Definition and conventions
Let be either the real or complex numbers. A norm on is a function satisfying:
- Positive definiteness: , with equality if and only if .
- Absolute homogeneity: for every scalar .
- Triangle inequality: .
The third property is the triangle inequality for matrix addition.
Terminology varies. Some authors call any norm on a matrix space a matrix norm. Others additionally require submultiplicativity:
For square matrices, this requirement concerns products within one matrix space. For rectangular matrices, it concerns a compatible family of norms across the dimensions involved.
The distinction is important: a norm on matrix entries need not be submultiplicative. Unless otherwise specified, properties involving products must be checked rather than inferred from the three norm axioms alone.
Induced norms
A matrix represents a linear map from to . Given norms on these two vector spaces, its induced norm, also called its operator norm or subordinate norm, is
The maximum exists because the unit sphere is compact in finite dimensions.
This norm measures the greatest amplification of an input vector:
It is the smallest constant for which this inequality holds for every .
When the domain and codomain use the same type of vector norm, the notation is commonly shortened to . Induced norms obey
In particular, each induced -norm is submultiplicative. The identity matrix satisfies .
Induced -norm
For the vector norm , the induced matrix norm is the largest absolute column sum:
Induced infinity norm
For the vector norm , the induced matrix norm is the largest absolute row sum:
Spectral norm
The induced norm associated with the Euclidean vector norm is the spectral norm:
Here is the conjugate transpose, is the largest singular value, and is the largest eigenvalue of the positive semidefinite matrix .
Geometrically, the spectral norm is the largest factor by which stretches a vector in Euclidean space. It can be obtained from the singular value decomposition. Despite its name, it is not generally the largest absolute eigenvalue of .
Entrywise norms and the Frobenius norm
Matrices can also be treated as vectors of entries. For , the entrywise -norm is
and the entrywise maximum norm is
These are norms, but they generally differ from induced -norms. In particular, “matrix -norm” ordinarily means the maximum column sum, not the sum of all absolute entries. Likewise, “matrix infinity norm” ordinarily means the maximum row sum, not the largest absolute entry.
The maximum-entry norm is not generally submultiplicative with constant . For example, if is the matrix whose entries are all , then
For compatible matrices with inner dimension , the general estimate is
Frobenius norm
The entrywise -norm is the Frobenius norm, also called the Hilbert–Schmidt norm:
It is induced by the matrix inner product
where denotes the trace. Consequently,
The Frobenius norm is submultiplicative:
It is not generally induced by a vector norm on the underlying input and output spaces: for an identity matrix,
whereas every same-space induced norm gives .
Unitarily invariant norms and Schatten norms
A norm is unitarily invariant if
for every pair of appropriately sized unitary matrices . Over the real numbers, the corresponding transformations use orthogonal matrices.
Such norms depend only on the singular values. Important examples belong to the family of Schatten norms:
with
The principal cases are:
: the nuclear norm or trace norm,
: the Frobenius norm;
: the spectral norm.
Thus the notation must distinguish the induced -norm from the Schatten -norm, and the induced -norm from the Schatten -norm.
The nuclear norm is important in convex optimization because it is a convex surrogate for matrix rank. Penalizing it encourages solutions with relatively few significant singular values, although it does not always produce the same solution as direct rank minimization.
Comparison and equivalence
For of rank ,
and
These inequalities follow by comparing the maximum, Euclidean norm, and sum of the singular values.
Other useful bounds are
and
All norms on a fixed finite-dimensional matrix space are equivalent: for any two norms , constants exist such that
for every matrix in that space. They therefore define the same topology and the same convergent sequences.
This does not make norms quantitatively interchangeable. Equivalence constants may depend on dimension and grow without bound when matrix sizes increase.
Eigenvalues, powers, and convergence
The spectral radius of a square matrix is
For every submultiplicative matrix norm,
For a normal matrix, meaning , the spectral norm equals the spectral radius. This equality can fail substantially for nonnormal matrices. For example,
has , but .
Submultiplicativity gives
Thus is a sufficient condition for . The exact condition is ; a particular chosen norm can exceed even when the powers eventually decay.
More generally, Gelfand’s formula states that, for any norm on a fixed finite-dimensional square matrix space,
When in a submultiplicative norm, the matrix geometric series satisfies
For an induced norm, it also yields
Numerical applications
Conditioning and error bounds
For an invertible square matrix, the normwise condition number is
usually defined using an induced norm. In the spectral norm,
Consider a linear system , and an approximate solution with residual . For nonzero , the corresponding induced norm gives
A small residual therefore need not imply a small relative solution error when the matrix is ill-conditioned. Conditioning describes sensitivity of the mathematical problem; it is distinct from the numerical stability of an algorithm.
Low-rank approximation
If
is a singular value decomposition, retaining only the largest singular values produces a matrix of rank at most . The Eckart–Young–Mirsky theorem gives
and
These results underpin matrix compression and methods related to principal component analysis.
Regularization and computational cost
Different norms impose different notions of error or model complexity. A Frobenius penalty distributes a squared penalty across all entries or singular values; a spectral-norm constraint controls worst-case Euclidean amplification; a nuclear-norm penalty encourages low rank. Their roles in regularization are therefore not interchangeable.
For a dense matrix, the induced -norm, infinity norm, maximum-entry norm, and Frobenius norm can be evaluated directly in arithmetic operations. Spectral and nuclear norms generally require singular-value computations or iterative approximations. For large sparse matrices, spectral-norm estimates can often be obtained using products with and , without explicitly forming .