A singular or rectangular matrix does not have an ordinary two-sided inverse, yet it can still have a well-defined pseudoinverse. The singular value decomposition constructs one by inverting the nonzero singular values and setting the remaining entries of the transposed rectangular diagonal to zero; zero singular directions are not inverted. Four equations characterize the resulting matrix without referring to the particular SVD used to construct it.
The first two equations say that the original map and the proposed inverse reproduce each other in a specific sense. They are not enough to select the pseudoinverse: the other two require the associated projections to be orthogonal. We will show existence from an SVD and prove uniqueness by examining the image, kernel and orthogonal complements of those projections.
The equations and their sizes
Definition 4.3.1
(The Moore–Penrose equations)
Let , with positive finite . A matrix satisfies the Moore–Penrose equations for if
The products and are square but act in different spaces: acts on and on . The transpose conditions are therefore not interchangeable. For complex matrices the corresponding characterization uses conjugate transpose instead; this chapter proves the real case.
If is square and invertible, the first equation implies by multiplying by on both sides. If , the second equation forces . Thus the equations include both ordinary inversion and the rank-zero extreme. For a general rectangular matrix, however, neither nor must be an identity on its entire space.
Existence from an SVD
Theorem 4.3.2
(Existence of the pseudoinverse)
Every has a matrix , built from a full SVD by inverting only the positive singular values, that satisfies all four Moore–Penrose equations.
Proof
(Checking the equations on the rectangular diagonals)
Take a full singular value decomposition , where and are orthogonal. If , the rectangular diagonal has positive entries and all remaining entries zero. Define to be the rectangular diagonal matrix whose first diagonal entries are , with all other entries zero. Set
No zero singular value is inverted. When , both and are zero. Directly on the rectangular diagonals,
Moreover is the diagonal matrix with ones followed by zeros, while is the analogous matrix. Both are symmetric. Inserting and into the products proves all four equations for . This establishes existence, including rank zero and rectangular shapes.
A fact about symmetric idempotents
Lemma 4.3.3
(Symmetric idempotents are orthogonal projections)
If a real square matrix satisfies and , then with , and is the orthogonal projection onto .
Proof
(Splitting as )
Suppose a real square matrix satisfies and . Every vector decomposes as . The first term lies in , while the second lies in because . The two subspaces are perpendicular: if and , then . Their intersection is zero, so
Consequently is the unique orthogonal projection onto its image. The decomposition proves uniqueness: a vector has only one component in the image whose difference from it is perpendicular to that image. Symmetry is essential for the word “orthogonal”; idempotence alone only gives a direct decomposition.
The product is the orthogonal projection onto the column space of . Similarly projects onto the span of the right singular vectors for positive singular values, which is . These two projections usually have different sizes and different domains.
The four equations force two projections
Proposition 4.3.4
(The four equations force two projections)
If satisfies all four equations for , then is the orthogonal projector onto , is the orthogonal projector onto , and .
Proof
(Idempotent, symmetric, with the right image and kernel)
Now let be any matrix satisfying all four equations. From ,
Together with the transpose equations, and are symmetric idempotents. The image of is exactly : one inclusion is immediate, and the other follows from . Hence is the orthogonal projector onto .
The kernel of is exactly . If then ; conversely, implies . Since is a symmetric idempotent, it is the orthogonal projector onto . In particular, these two projectors are determined by ; they do not depend on which candidate we started with.
We also need to know where sends vectors. The equation gives for every , so . This inclusion is the step that makes the uniqueness argument work; it must not be assumed merely from the dimensions.
Uniqueness by the inverse restriction
Theorem 4.3.5
(Uniqueness of the pseudoinverse)
For each there is exactly one satisfying the four Moore–Penrose equations. It is denoted , and it does not depend on the SVD used to construct it.
Proof
(Two matrices that agree on an image and on its complement)
The restriction of to maps bijectively onto . It is injective because a vector in both and is zero. It is onto because every decomposes as with , and .
If , then , since is the projector onto that image. Also . Therefore is the unique vector in mapped to by . If , then , so ; the previous inclusion also puts it in , hence .
Every has a unique decomposition into an image component and a perpendicular component. Linearity therefore determines uniquely on the whole space. Any two matrices satisfying the four equations agree on both components and are equal. Since the SVD construction gives one such matrix, it is the unique one, denoted ; in particular, different valid SVD choices give the same pseudoinverse.
Proof X-ray
Where each equation does its work
The first equation makes and idempotent, and the two transpose equations make them symmetric; together they turn oblique splittings into orthogonal ones. The key move is the inclusion , which comes from and is the step that cannot be read off from the dimensions. Once and are known to be the projectors determined by , the vector is pinned down: it is the unique preimage in of the image component of , and zero on the perpendicular component. The argument closes because those two components span and is linear. Drop either transpose equation and the projection need not be orthogonal, so the argument breaks.
This also explains the earlier minimum-norm result. For any , is the unique projection of onto the column space, and lies in . All least-squares coefficients differ from it by an element of , so Pythagoras makes the unique one with smallest norm. The four equations characterize the operator; the minimum-norm statement describes what it does on a right-hand side.
Why two equations are insufficient
Counterexample mode
Two equations do not determine the pseudoinverse
Claim: any with and equals . This is false. Let and . Direct multiplication gives and , so is a reflexive generalized inverse under these two equations. But is not symmetric, and . Its image/kernel splitting is oblique rather than the required orthogonal one. The two reconstruction equations alone therefore do not select the Moore–Penrose inverse.
One should also distinguish verifying a proposed pseudoinverse from merely solving one system. A matrix might give the minimum-norm answer for a particular by coincidence and still fail one of the four equations. Conversely, once the four identities have been checked for the correct sizes, uniqueness proves that the matrix works for every right-hand side.
How to check the equations in practice
Begin by writing the dimensions of and the proposed . The reconstruction products and have the same shapes as and , respectively; the symmetry checks concern two square matrices of potentially different sizes. A dimension mismatch is already enough to reject a proposed formula. Then check the four identities separately. Establishing that is symmetric does not by itself establish that is symmetric, nor does either symmetry condition automatically give the two reconstruction identities.
A convenient verification can use previously established identities. For instance, if , then and follow immediately, and the first symmetry condition is automatic. One still needs to check that is symmetric. This shortcut applies only when the displayed product is actually an identity; it cannot be assumed for a rank-deficient matrix. The examples below illustrate both a rank-one rectangular case and a full-row-rank case.
Concept lensStructural
The pseudoinverse as an inverse on a restricted pair of subspaces
The uniqueness proof gives a description without any equations. The restriction of to is a bijection onto , and is the inverse of that bijection on , extended linearly by on . So undoes wherever can be undone and discards what cannot reach. The four equations are the matrix form of this sentence.
Worked examples
Worked example 4.3.6
(A rank-one rectangular matrix)
Let and . Find , its two projections, and the minimum-norm least-squares coefficient for .
Solution. Write with and . Their squared norms are 6 and 5, so the single positive singular value is and
The products are
They are symmetric projections onto and respectively. Since and by the displayed rank-one products, all four equations hold. For , and the fitted vector is . Its residual is , perpendicular to , with squared norm . The coefficient is perpendicular to , confirming minimum norm.
Worked example 4.3.7
(A full-row-rank system)
Let and . Find a pseudoinverse and the minimum-norm exact solution.
Solution. Here is invertible. Set
Multiplication gives and
The second matrix is symmetric, so , and both symmetry conditions hold. By uniqueness . The vector solves . The kernel is spanned by , perpendicular to that solution, so it is the minimum-norm exact solution. The formula involving applies here because full row rank was verified; it is not a formula for an arbitrary matrix.
Worked example 4.3.8
(The zero matrix: a rank-zero edge case)
Let be the zero matrix. No singular value is positive, so is the zero matrix and is the zero matrix. All four equations hold: , , and and are zero matrices of sizes and , hence symmetric. The shape of is the transposed shape, and the second equation is what forces here.
Common mistakes and subtle points
Common mistake
(Using an inverse formula without checking the rank)
The formulas and are valid only when has full column rank, respectively full row rank. For the matrix is singular, but still exists.
Common mistake
(Checking one symmetry and calling it done)
and are square matrices of different sizes when . Symmetry of says nothing about ; for one has for , which is automatically symmetric, yet must still be checked and has size .
Exercises
Quick check
(Find for .)
Invert only the nonzero diagonal entries.
Solution · Answer
. Then is symmetric, and .
Quick check
(If has orthonormal columns, show that .)
Check the four equations for .
Solution · Answer
Orthonormal columns mean . Then , , is symmetric, and is symmetric. By uniqueness .
Quick check
(Find for the row matrix .)
Write with and , or use .
Solution · Answer
is invertible, so . Indeed and is symmetric.