A matrix is both a collection of rows and columns and a map from an input space to an output space. Four subspaces describe what it loses and what it can produce. Two live in the input space, and two live in the output space. Their orthogonality explains not only rank and nullity, but also why a vector can certify that a system has no solution.
We work over the real numbers with the usual dot product. Orthogonal-complement statements use this inner product; the analogous complex results require conjugate transpose. Here projection is used to interpret the four subspaces.
Four spaces, two ambient spaces
Let and let . We regard each row of as a column vector in by transposing it.
Definition 9.5.1
(The four fundamental subspaces)
The four fundamental subspaces of are
The last is called the left null space: belongs to it exactly when .
A row of has entries, whereas a column has entries; writing the ambient spaces first prevents an attempt to take a dot product between vectors of incompatible lengths. Even if , row and column space need not be the same subspace.
The column space consists of all outputs . The kernel consists of all inputs sent to zero. The row space records the linear combinations of the row measurements; the left null space records linear dependencies among those measurements, because says the weighted sum of the rows with coefficients is zero.
Row space and the kernel
Theorem 9.5.2
(The row space is the orthogonal complement of the kernel)
For every ,
with respect to the usual dot product on . Consequently every has a unique decomposition with and .
Proof
(One inclusion by dot products, the other by counting dimensions)
If , each row of has dot product zero with , since the entries of are precisely those dot products. Every linear combination of the rows is therefore perpendicular to . This proves .
For the reverse inclusion, use dimensions. Rank–nullity gives . An orthonormal basis of the kernel can be extended to one of ; the added vectors form a basis of the orthogonal complement. Hence . The row space also has dimension by the row-rank/column-rank theorem. A subspace contained in another subspace of the same finite dimension is equal to it, proving the claim. The equality of the two ranks is a separate theorem, proved by rank factorization; it is not being inferred merely because rows and columns look similar.
For the decomposition, for , and .
Proof X-ray
Which step uses which hypothesis
The proof has two halves of quite different character. The inclusion is a one-line computation: the th entry of is the dot product of row with . It uses no property of beyond arithmetic.
The reverse inclusion cannot be done element by element, since nothing yet produces an element of from a vector perpendicular to the kernel. The proof instead squeezes: it shows that the two subspaces have the same finite dimension , and then a subspace of a space of equal finite dimension is the whole space. The two numbers come from different sources. The value for comes from rank–nullity, and turning it into uses an orthonormal basis, hence a positive definite inner product. The value for the row space needs row rank equal to column rank. If either dimension count were unavailable, the argument would stop at the inclusion.
The proof closes because both spaces are finite dimensional. The positive definiteness is not decoration: a bilinear form that lets a nonzero vector be perpendicular to itself gives , and then the decomposition into breaks down, as the counterexample under Common mistakes shows.
There is a useful consequence. Any input has a unique orthogonal decomposition
Applying removes the second component: . Thus the row-space component contains all information about the input that the map can recover. A nonzero null component is a genuine change to the input that produces no change in the output.
Concept lensGeometric
The kernel as an intersection of hyperplanes
Write for the rows of . Each equation describes a hyperplane through the origin with normal vector (or all of if ). The kernel is the intersection of these hyperplanes, and a vector perpendicular to the kernel is exactly a combination of the normals . So says: the span of the normal vectors of a family of hyperplanes through the origin is the orthogonal complement of their common intersection. For the kernel is the line in , and the row space is the perpendicular line spanned by .
Column space and the left kernel
Theorem 9.5.3
(Column space, left null space and the four dimensions)
For of rank ,
and
Every has a unique decomposition with and .
Proof
(Apply the row-space identity to the transpose)
The row space of is , and its kernel is . The previous theorem applied to therefore gives , equivalently , and the unique decomposition of . The matrix also has rank , since row rank equals column rank. Its domain is , so rank–nullity gives . The other three dimensions were found above.
The first component is the orthogonal projection of onto the attainable outputs. The second is perpendicular to every attainable output. The formula for is rank–nullity for , and the equality of the two ranks is essential there. These formulas include the zero matrix: its row and column spaces are zero, and both kernels are their full ambient spaces.
The two decompositions concern different spaces. A diagram that pairs row space with null space and column space with left null space is useful only if the domains of the arrows are kept visible.
Concept lensStructural
The invertible part of a matrix
Restrict to . The restriction maps onto , because for every . It is injective, because : a vector perpendicular to itself is zero. Both spaces have dimension . So is an invertible map between two -dimensional spaces, plus a kernel discarded in the input and a left kernel never reached in the output.
Consistency and a certificate of failure
Theorem 9.5.4
(Certificate of inconsistency)
The system has a solution if and only if
Equivalently, exactly one of the following holds: has a solution, or there is with and .
Proof
(Left-null vectors cannot see the column space)
The system is solvable exactly when , and by the previous theorem, which is the displayed condition.
If some satisfies but , then is impossible: indeed, if , then , a contradiction. Conversely, if is not in the column space, its nonzero left-null component is such a certificate: .
The strict positivity here uses the real Euclidean inner product. It is not a purely formal transpose statement over every field. The finite-dimensional projection argument also shows why checking only a few arbitrarily selected left-null vectors is not enough: they must span the whole left null space. A basis gives finitely many scalar conditions; if all hold, every linear combination also annihilates .
For a compatible , choose one solution . Every solution is with . Among them, the unique solution in is obtained by removing the null component of . Orthogonality makes it the minimum-norm solution: adding any nonzero kernel vector increases the squared norm by its squared length. For an incompatible , the same construction solves and yields the minimum-norm least-squares coefficient. This is an interpretation of the projection theorem, not a claim that has become consistent.
How to find bases without mixing them up
Reduce to RREF. Its nonzero rows, transposed, form a basis of the row space because row operations preserve row space. Solve the homogeneous reduced system to obtain a basis of . The pivot positions identify independent columns, but a basis of the original column space must use the corresponding columns of , not the columns of its RREF. Row operations can change the column space as a set.
To find the left null space, solve ; equivalently, find all linear relations among the original rows. Check dimensions against , , and . These counts are checks, not substitutes for showing that the listed vectors actually belong to the requested spaces and are independent. At rank zero, an empty basis is appropriate for each zero space; at full row or column rank, the relevant kernel basis may be empty.
Worked examples
Worked example 9.5.5
(A consistency certificate)
Let . Find bases for the four subspaces and decide whether and are attainable.
The second row is twice the first, so and a row-space basis is . The equation gives kernel basis . The first column gives a column-space basis . Solving gives left-null basis . Their dimensions are , consistent with , .
For , the left-null dot product is ; indeed and all solutions are
For , . The same left-null vector certifies inconsistency; no row reduction of the augmented system is needed once this certificate is known.
Worked example 9.5.6
(Decomposing an input and an output)
Let . Find bases for the four subspaces, decompose into row and null components, and find the projection of onto the column space.
The third row is the sum of the first two, and those rows are independent. Thus , with row-space basis and . Solving gives kernel basis and . The first two columns , form a column-space basis; the remaining columns are and . The left null space is spanned by .
Write the row component as . Requiring to be perpendicular to both basis vectors gives
Hence and . Their sum is ; direct multiplication gives .
For the output, and , so . Therefore . The nonzero left component certifies that is inconsistent; it is also the least-squares residual for the fitted output. The input maps to , and all least-squares coefficients are obtained by adding the displayed kernel vectors.
Worked example 9.5.7
(Full column rank: a trivial kernel)
Let , so , . The two columns are independent, so : the row space is all of , and has the empty set as a basis. The column space is the plane spanned by and , and has dimension . Solving means and , giving the basis vector .
Test : , so is attainable, and indeed , so . For , , so has no solution. Since the kernel is trivial, the solution for is unique: there is no null component to add. For , the least-squares problem has , as , and .
Common mistakes
Common mistake
(Taking the column space from the reduced matrix)
Let and let be with its rows swapped. The row spaces are equal, since they are spanned by the same vector , but the column spaces are different coordinate axes: , . Row operations preserve the row space and the rank, but not the column space as a set. A basis of must therefore use the pivot columns of the original matrix.
Common mistake
(Pairing the kernel with the column space)
For the kernel lies in and the column space in , so the question whether they are perpendicular does not even make sense. The orthogonal pairs are in the input space and in the output space.
Common mistake
(Reading dimensions off the wrong space)
The left null space has dimension , not . For a matrix of rank , the kernel has dimension but the left null space has dimension . The sum is , and the sum is .
The real inner product is also needed in the first theorem, and the following example shows it cannot be dropped.
Counterexample mode
Over the complex numbers with the plain dot product the splitting fails
Claim: for every matrix , every input splits as , whatever the scalars. Take the complex matrix and the bilinear dot product with no conjugation. Then satisfies , so , while is the row of , so . Both spaces are the line , because each has dimension . The sum is that same line, which does not contain , since is impossible. So has no decomposition . The repair is to use the conjugate transpose, so that and a nonzero vector is never perpendicular to itself, or to stay with real vectors as in the theorem.
Quick checks
Quick check
(A matrix is with rank . State the dimension and the ambient space of each of the four fundamental subspaces.)
Use , , .
Solution · Answer
has dimension ; has dimension ; has dimension ; has dimension .
Quick check
(For , find a vector that certifies that is not in the column space.)
Solve and test .
Solution · Answer
, and gives . Then . If had a solution, then ; so is not attainable. Indeed, every output of has equal entries.
Quick check
(For , decompose into a row-space component and a kernel component.)
The row space is spanned by , and the kernel by .
Solution · Answer
Solve : and , so . Then , . The two components are perpendicular, since , and .