The Moore–Penrose inverse of a real or complex matrix is the unique matrix satisfying four algebraic conditions that extend the defining properties of an ordinary inverse. It exists for every finite-dimensional matrix, including rectangular matrices and singular square matrices. When is invertible, is exactly its inverse. Otherwise, it reverses the action of on the directions where reversal is possible and assigns zero to the remaining orthogonal directions.
Its central role is in solving linear systems that have either no exact solution or more than one. For every right-hand side , the vector minimizes the residual ; among all vectors attaining that minimum, it has the smallest Euclidean norm. This makes the construction fundamental to least squares, numerical computation, statistical modeling, and inverse problems. Its geometry is described by orthogonal projections, while its most direct computational characterization uses the singular value decomposition.
The name commemorates E. H. Moore, who presented an early form of the generalized inverse in 1920, and Roger Penrose, who characterized it through four matrix equations in 1955. Related work by Arne Bjerhammar appeared in 1951. Although “pseudoinverse” sometimes refers to a broader class of generalized inverses, it commonly means the Moore–Penrose inverse.
Defining equations and geometric meaning
Let , and let denote its conjugate transpose. The Moore–Penrose inverse is the matrix satisfying the four Penrose equations:
For real matrices, the conjugate transpose reduces to the ordinary transpose. The first equation says that reverses sufficiently to reproduce every output of . The second excludes unnecessary action outside the part needed for this reversal. The last two impose orthogonality rather than merely algebraic projection.
To interpret these requirements, regard as a linear map from to , both equipped with their standard inner products. Its domain and codomain have orthogonal decompositions
where denotes the range and the null space. The restriction of to is a bijection onto . The pseudoinverse is its inverse on that range and is zero on . This construction establishes both existence and uniqueness without requiring that be square.
The two products
are orthogonal projections. Specifically, projects onto the column space , while projects onto , the conjugate-transposed row space. They are self-adjoint and idempotent:
Consequently, projects onto , and projects onto . These complementary projections separate attainable data from residual error, and identifiable input directions from directions annihilated by .
The orthogonality requirements distinguish the Moore–Penrose inverse from a general generalized inverse. A matrix satisfying only can reproduce outputs of , but need not produce orthogonal projections or minimum-norm solutions. Many such matrices may exist. The Moore–Penrose inverse is canonical relative to the specified inner products, not independent of geometry: changing the inner products generally changes which solution has minimum norm.
Singular values and explicit constructions
The singular value decomposition gives a direct formula valid for every matrix. Write
where and are unitary matrices, and is rectangular diagonal. If has rank , its positive singular values can be ordered as
Form by replacing each positive singular value with its reciprocal and leaving every zero entry zero. Then
Thus pseudoinversion reverses the input and output singular directions while reciprocating only the nonzero scale factors.
In terms of left and right singular vectors,
The associated projections are
These formulas show that , that the nonzero singular values of are the reciprocals of those of , and that the pseudoinverse of the zero matrix is the zero matrix of transposed dimensions.
Important special cases admit shorter expressions. If has full column rank, then and
If it has full row rank, then and
The matrices and are Gram matrices; the relevant one is invertible precisely under the stated rank condition. These expressions are algebraically useful, although explicitly forming Gram matrices can be numerically undesirable.
A rank factorization provides another exact construction. If
where has full column rank and has full row rank, then
Only matrices of order need to be inverted in this formula. For a nonzero rank-one matrix , it reduces to
For a Hermitian matrix, the spectral theorem supplies an analogous construction: diagonalize the matrix, reciprocate each nonzero eigenvalue, and retain zero eigenvalues as zero. Negative eigenvalues remain negative after reciprocation.
Least-squares solutions and minimum norm
For and , consider
The pseudoinverse selects
Its fitted output is
the orthogonal projection of onto . To see why this minimizes the residual, decompose
For every , the vectors and are orthogonal. The Pythagorean identity therefore gives
The second term is unavoidable; the first vanishes exactly when .
All least-squares minimizers have the form
The second term ranges over , so it changes the input without changing the fitted output. Since , it is orthogonal to every such addition. Hence
This proves that is the unique minimum-norm least-squares solution. The two minimizations are hierarchical: residual norm is minimized first, and solution norm only among residual minimizers.
When , the residual is zero, and the formula describes all exact solutions of . For example,
The equation has infinitely many solutions, but the pseudoinverse chooses . Conversely, for
the pseudoinverse is . Applied to , it returns the arithmetic mean, the scalar minimizing the sum of squared discrepancies from the two observations.
Every least-squares minimizer satisfies the normal equations
If is rank deficient, these equations do not determine a unique vector. The pseudoinverse adds the condition , excluding null-space components. For several right-hand sides collected in a matrix , the same argument applies column by column: minimizes and has minimum Frobenius norm among all minimizers.
Algebraic identities and their limitations
Several familiar inverse identities survive pseudoinversion. For every finite matrix,
For a nonzero scalar ,
If and are unitary matrices of suitable dimensions, then
Thus pseudoinversion behaves naturally under orthonormal changes of coordinates. This property follows either from the Penrose equations or from the singular value decomposition.
These identities do not extend to arbitrary invertible coordinate transformations. Nonunitary transformations distort lengths and angles, changing the minimum-norm criterion. In general,
For example, let and . Then
whereas
Both are right inverses of , but only the first selects solutions of minimum standard Euclidean norm.
The ordinary reverse-order law also fails in general:
A small example is
Here , so , while
Reverse order does hold under additional hypotheses. In particular, it holds when has full column rank and has full row rank, as follows from the rank-factorization formula. It also holds when the relevant factor is unitary.
Useful Gram-matrix identities valid without full-rank assumptions are
and
For an orthogonal projection , one has ; for a unitary matrix, the pseudoinverse is its adjoint. Block-diagonal matrices are pseudoinverted block by block. By contrast, there is no general additive rule: is not usually . These distinctions are important because manipulations valid for invertible matrices can silently fail when intermediate factors lose rank.
Numerical computation and numerical rank
The singular value decomposition is the standard general-purpose basis for computing a pseudoinverse. Its central advantage is that it exposes the directions in which a matrix is nearly singular. In exact arithmetic, every positive singular value is reciprocated. In floating-point arithmetic, however, computed singular values near the scale of rounding error cannot reliably distinguish exact dependence from very weak independence. Software therefore commonly applies a threshold , retaining only when .
A typical scale-dependent threshold has the form
where is machine precision and is an implementation-dependent factor. The resulting matrix is the exact pseudoinverse of a truncated approximation to , not necessarily the exact pseudoinverse of the original mathematical matrix. This distinction becomes significant for poorly conditioned data. A threshold based only on arithmetic precision addresses rounding error; a threshold chosen to suppress measurement noise expresses an additional modeling decision.
When full column rank is known, a thin QR decomposition
gives
For a single right-hand side, the computation can instead form and solve the triangular system , without constructing . Rank-revealing QR and complete orthogonal decompositions can handle rank deficiency, although rank decisions remain necessary. The SVD is generally more informative about singular values, while QR-based approaches can be less expensive.
The algebraic formula is not usually the preferred numerical construction. For a full-column-rank matrix, the spectral condition number satisfies
Forming the normal equations therefore squares the condition number and may lose information about small singular directions before the inverse is computed. Orthogonal factorizations avoid this particular amplification, although no algorithm can eliminate sensitivity inherent in the underlying problem.
For a dense matrix with , standard SVD computation has arithmetic cost on the order of . Explicitly storing the pseudoinverse requires entries and may destroy sparsity. Large-scale methods therefore often compute indirectly. Krylov-subspace least-squares methods such as LSQR, in exact arithmetic and with a zero initial vector, select the minimum-norm least-squares solution upon convergence. Early stopping, preconditioning, and finite precision can alter that behavior and must be distinguished from exact pseudoinversion.
Perturbation, discontinuity, and regularization
Pseudoinversion is sensitive to small nonzero singular values because
for a nonzero rank- matrix. If the matrix is fixed but the right-hand side changes by , then
More specifically, noise along produces an input error along , amplified by . The difficulty is therefore directional, not simply a matter of the overall magnitude of the data.
Unlike ordinary inversion on the set of nonsingular matrices, pseudoinversion is not continuous across rank changes. Consider
For ,
but
Although , the pseudoinverses diverge. Restricted to matrices of a fixed rank, the pseudoinverse varies continuously and smoothly. A deficient-rank matrix nevertheless has higher-rank matrices arbitrarily close to it, so unrestricted perturbations can cause singular behavior.
In an inverse problem, a small singular value may encode a genuine but weakly observed direction. Exact pseudoinversion can amplify noise in that direction to an unacceptable degree. Regularization modifies the inverse action rather than attempting to recover every nonzero singular component. Truncated SVD uses
This eliminates poorly resolved directions entirely. The threshold controls a trade-off between suppressing noise and discarding signal; it is not determined by the Penrose equations.
A smooth alternative is quadratic, or Tikhonov, regularization:
Its singular-direction factors are
rather than . For fixed finite-dimensional ,
and, at the matrix level,
The regularized solution at positive is generally not the Moore–Penrose solution of the original problem. Its purpose is precisely to replace unstable exact inversion with a controlled approximation.
Statistical and scientific applications
In linear regression, a real design matrix relates observations to coefficients through
The ordinary least-squares coefficient vector can be chosen as
If the columns of are linearly independent, this equals . If they are dependent, coefficients are not uniquely determined by least squares, but the fitted vector remains unique:
The hat matrix is the orthogonal projection onto the model’s column space, and
The minimum-norm convention does not resolve statistical nonidentifiability. Under ,
which equals only when lies in the row space of . A linear contrast is estimable from the mean response when . If
then
Zero variance in a null-space direction reflects the convention that the estimator sets that component to zero, not information supplied by observations.
Weighted problems demonstrate the importance of specifying the geometry. For a positive-definite matrix , minimizing
can be reduced to ordinary least squares by applying to and . If solution size is measured by , with positive definite, the minimum--norm weighted least-squares solution is
Such formulations arise when observation errors have unequal variances or when input coordinates have different physical scales.
In differential kinematics, a robot’s joint velocities and task-space velocity are related by a Jacobian matrix:
The command has minimum Euclidean joint-velocity norm among exact solutions, or among least-squares solutions when the requested velocity is unattainable. The term describes motions that do not change task-space velocity to first order. Near kinematic singularities, small singular values can produce large commands, motivating damped least-squares variants.
In graph theory, a connected undirected graph has a singular Laplacian matrix , whose null space is spanned by the constant vector. Its pseudoinverse selects the mean-zero solution of whenever has zero sum. For a resistor network with positive edge conductances, the effective resistance between vertices and is
The missing inverse direction corresponds to the arbitrary additive constant in electric potential.
Hilbert-space operators and infinite-dimensional limits
The geometric construction extends from matrices to bounded linear operators between Hilbert spaces, but finite dimensionality conceals an important condition: every finite-dimensional range is closed. Let be bounded. If is closed, then
The restriction of to is a bijection onto . Its inverse, extended by zero on , is a bounded operator . It has the same minimum-norm and projection interpretations as a matrix pseudoinverse.
If the range is not closed, a Moore–Penrose inverse can still be defined as a densely defined operator with domain
but it is unbounded. This changes the existence of least-squares solutions. An element may lie in the closure of without lying in the range itself. Then the infimum of is zero, yet no vector attains it. Finite-dimensional least squares always has a minimizer because a matrix range is closed.
A concrete example acts on the space of square-summable sequences:
This operator is bounded and injective, and its range is dense but not closed. Its generalized inverse acts as
on the domain for which the resulting sequence is square summable. For the unit coordinate vector , , proving that is unbounded. The sequence belongs to but not to , since its formal preimage is the nonsummable constant sequence.
For an operator admitting a singular system,
the finite-dimensional formula becomes
where the series converges only for appropriate . In particular, its range component must satisfy
Compact operators of infinite rank have singular values tending to zero, so their inverses on the range are unbounded. This phenomenon appears in integral equations and other problems in functional analysis. Finite discretizations replace unbounded inversion with matrices having increasingly small singular values, linking the operator-theoretic closed-range condition directly to numerical ill-conditioning and the need for regularization.