The Moore–Penrose pseudoinverse of a real or complex matrix is the unique matrix satisfying four equations that generalize the defining properties of an ordinary matrix inverse. It exists for every finite-dimensional matrix, including rectangular matrices and singular square matrices. Its central role is to produce the minimum-norm solution of a linear least-squares problem, whether the underlying equations have a unique solution, infinitely many solutions, or no exact solution.
Definition
Let . Its Moore–Penrose pseudoinverse is the matrix satisfying the four Penrose equations:
Here denotes the conjugate transpose of . For real matrices, it is simply the transpose.
The first two equations express generalized inverse relationships. The last two require the products and to be Hermitian. Together, all four conditions determine uniquely.
The notation is also common. The term generalized inverse is broader: a matrix satisfying only , for example, need not be the Moore–Penrose pseudoinverse and need not be unique.
For a square nonsingular matrix,
The pseudoinverse of a zero matrix is the zero matrix of the transposed shape.
Construction by singular value decomposition
The singular value decomposition provides a general construction and establishes existence. Write
where and are unitary matrices, and is rectangular diagonal. If has rank , its positive singular values are
Form by transposing the shape of , replacing each positive singular value by its reciprocal, and leaving zero entries zero. Then
Equivalently, using only the singular vectors associated with positive singular values,
This formula makes the essential operation explicit: the pseudoinverse reverses every nonzero singular direction and assigns zero to directions that cannot be inverted.
Substituting this expression into the four Penrose equations verifies existence. For uniqueness, the Hermitian conditions force and to be specific orthogonal projections; the remaining conditions then fix how acts on their ranges.
Geometric interpretation
Regard as a linear map from to , equipped with their standard inner products. The domain decomposes as
where is the null space and the superscript denotes the orthogonal complement. The codomain decomposes as
The restriction
is bijective. The pseudoinverse inverts this restricted map and maps to zero.
Consequently,
where is the orthogonal projection onto a subspace . The space is the row space, interpreted as a subspace of the domain.
This interpretation explains why the products are not generally identity matrices: directions outside the relevant ranges cannot be recovered. It also shows that the Moore–Penrose construction depends on the inner products. Changing them can change which solution is regarded as minimum-norm.
Linear equations and least squares
Consider a system of linear equations
For every , the vector
minimizes the Euclidean residual:
Among all vectors attaining that minimum residual, it is the unique one with the smallest Euclidean norm.
Consistent systems
An exact solution exists precisely when , equivalently when
In that case, every exact solution has the form
where is the identity matrix. The second term ranges over .
Since is orthogonal to the null space, adding any nonzero null-space component increases the norm. Thus the pseudoinverse selects a single solution even when infinitely many exact solutions exist.
Inconsistent systems
If , no exact solution exists. Instead,
is the orthogonal projection of onto the column space of . The residual
is orthogonal to that space.
All least-squares minimizers again have the form
They satisfy the normal equations
When is rank-deficient, these equations do not uniquely determine ; the minimum-norm requirement selects .
Example
For the single equation
the pseudoinverse is
Hence the minimum-norm solution is
All exact solutions are
whose squared norms are .
Special formulas and algebraic properties
If has full column rank, then is invertible and
If has full row rank, then is invertible and
Neither formula with an ordinary inverse applies without its rank assumption. General identities using pseudoinverses are
Other useful properties include
and
For compatible unitary matrices and ,
Thus the construction behaves naturally under orthonormal changes of coordinates. It does not generally obey the reverse-order law
that identity requires additional conditions. Nor does the pseudoinverse generally distribute over matrix addition.
For a Hermitian matrix, the spectral theorem gives an analogous construction: retain its eigenvectors, replace each nonzero eigenvalue by its reciprocal, and leave zero eigenvalues zero.
Numerical computation and sensitivity
In numerical linear algebra, the SVD is a standard method for computing pseudoinverses and minimum-norm solutions. Rank-revealing forms of QR decomposition and related factorizations are also used.
If the aim is only to obtain , explicitly forming the entire matrix is often unnecessary. A factorization can be applied directly to , reducing storage and sometimes computational cost.
The full-column-rank expression is useful algebraically but can be numerically undesirable. Forming squares the spectral condition number:
This can magnify the effects of rounding errors.
Small singular values and numerical rank
In exact mathematics, every nonzero singular value is inverted. In floating-point arithmetic, extremely small singular values may be indistinguishable from perturbations or rounding error. Software therefore commonly uses a tolerance and treats singular values below it as zero.
The resulting matrix is the exact pseudoinverse of a truncated approximation to , rather than necessarily the exact pseudoinverse of . The tolerance determines the effective numerical rank.
Even when a small singular value is accurately computed, its reciprocal can amplify measurement noise. Numerical stability of the factorization does not eliminate sensitivity inherent in the problem.
Continuity and rank changes
The pseudoinverse varies continuously along perturbations that preserve rank, but continuity can fail when rank changes. For example,
For ,
whereas
The divergence of illustrates why nearly singular problems can be highly sensitive.
Regularization and applications
The pseudoinverse is central to linear regression, constrained linear calculations, and inverse problems. In ordinary least squares, gives a minimum-norm coefficient vector even when the design matrix is rank-deficient. This algebraic choice does not, by itself, make nonidentifiable coefficients uniquely interpretable.
In inverse problems, direct pseudoinversion may amplify noise. Regularization replaces exact inversion with a modified problem. For example, minimizing
gives
This is the basic quadratic penalty used in ridge regression. In singular-vector coordinates, it replaces the reciprocal factor with
For a fixed finite-dimensional matrix,
Truncating small singular values is another regularization strategy. Both approaches modify the treatment of weakly determined directions; neither should be confused with exact pseudoinversion of the original matrix.
History and scope
The construction is named after Eliakim Hastings Moore, who introduced a generalized inverse in 1920, and Roger Penrose, who gave the four-equation characterization in 1955. It became a standard tool connecting linear algebra, least squares, and numerical computation.
The finite-dimensional theory extends to operators between Hilbert spaces. For bounded operators with closed range, the Moore–Penrose inverse is a bounded operator with analogous projection properties. Without closed range, a generalized inverse can be unbounded and need not be defined on the entire codomain, so the finite-dimensional existence statement requires qualification in infinite dimensions.