An equation may fail to have a solution even when there are vectors with . In finite dimensions the range of a linear map is always closed, so this distinction is easy to overlook. In a Hilbert space it governs whether an exact solution exists, whether a least-squares problem attains its minimum, and whether approximate data can require increasingly large solutions.
Throughout, are real or complex Hilbert spaces and is bounded and linear. Inner products are linear in the first variable. Its adjoint satisfies . The kernel is closed by continuity; the range need not be. We use the Hilbert projection theorem and the bounded inverse theorem, with their usual completeness hypotheses.
Orthogonality identifies the closure
Proposition 2.2.1
(Range of and kernel of )
Similarly .
Proof
(Take orthogonal complements twice)
A vector is perpendicular to every exactly when for every , which is equivalent to (take ). This proves the first equality. For any subspace of a Hilbert space, : projection onto shows that a vector perpendicular to has zero component in . Taking the orthogonal complement of the first equality gives the second. Apply it to and use for the last formula.
The closure cannot be omitted in general. Orthogonality to says that can be approximated by outputs; it does not alone say that some maps exactly to .
A quantitative criterion for a closed range
Let . Every has a unique decomposition with and . Moreover and . Thus removes the nonuniqueness caused by the kernel.
Theorem 2.2.2
(Closed-range criterion)
is closed if and only if there exists such that
Proof
(Bounded inverse on the orthogonal complement of the kernel)
If the range is closed, it is a Hilbert space. Restrict to . This is a bounded bijection between Banach spaces: injectivity follows from , and the decomposition above gives surjectivity. The bounded inverse theorem gives for some , which is the estimate with because and .
Conversely, suppose the estimate holds and converges in . Replace each by its component ; the images are unchanged. Since , its distance to is its norm, so for all
Thus is Cauchy, converges to some , and continuity gives . Every limit of range elements belongs to the range, which is therefore closed.
Proof X-ray
The kernel is the missing qualification
A lower bound on cannot control a component that annihilates. Projecting onto first makes the restricted map injective. Closedness then makes its range complete, allowing the bounded inverse theorem; conversely, the lower bound turns a Cauchy sequence of outputs into a Cauchy sequence of those canonical preimages.
The estimate controls distance to the kernel, and this qualification is not cosmetic.
Common mistake
(Bounding the norm instead of the distance to the kernel)
The criterion is not . If has a nonzero kernel, adding arbitrarily large kernel vectors leaves unchanged, so no such bound can hold, even though the range may well be closed. Only when is injective is the distance to the kernel simply .
Worked example 2.2.3
(A functional on with a kernel)
Let , . Its kernel is the line spanned by , and is spanned by . The distance from to the kernel is the length of the component along :
The criterion holds with , consistent with being closed. No bound with holds, since gives .
The adjoint has closed range at the same time
Theorem 2.2.4
(Closed range theorem)
is closed if and only if is closed.
Proof
(Transfer the lower bound to the adjoint)
Suppose is closed. By the preceding criterion, for each there is a unique with and . Therefore
so , including the trivial case . Since when the range is closed, decompose any as with and . Then and . The bound above therefore gives for arbitrary , and the criterion for proves closed. The reverse implication follows by the same argument applied to , because .
This result gives a qualitative equivalence.
Common mistake
(Closed range does not mean onto or invertible)
The closed range theorem does not assert that is onto, invertible, or compact. A closed range may be a proper subspace, and an operator with closed range may have a nonzero kernel; the next example has both features.
Worked example 2.2.5
(The two shifts on )
The right shift satisfies , so the criterion holds with and , the sequences with first coordinate zero, is closed but proper. From
its adjoint is the left shift . Here and , so also satisfies the criterion with . Indeed is onto but not injective, while is injective but not onto.
Best approximation and the normal equation
Definition 2.2.6
(Least-squares minimizer and normal equation)
For , a vector is a least-squares minimizer for if . The equation , that is , is the normal equation.
Theorem 2.2.7
(Least squares and closed range)
A vector is a least-squares minimizer for if and only if it solves the normal equation. If is closed, minimizers exist for every ; they form a translate of , and exactly one of them, the one of smallest norm, lies in . Conversely, if every has a minimizer, then is closed.
Proof
(Projection onto the range and Pythagoras)
If solves the normal equation, then for every the residual is perpendicular to , because . Pythagoras gives
so is a minimizer, and is another minimizer exactly when . Conversely, if is a minimizer, then has a minimum at for each and each real ; its derivative there is , and replacing by in the complex case also kills the imaginary part. Hence , which is the normal equation.
Suppose is closed. The projection theorem gives a unique minimizing ; any with is a minimizer, so the set of minimizers is . It meets in exactly one point , and for we have , so has the smallest norm. Finally, suppose every target has a minimizer, and let . Then , so a minimizer satisfies . Hence , and the range is closed.
Worked example 2.2.8
(A least-squares problem with a kernel)
Let be and . Here , so the normal equation reads , and every minimizer has with arbitrary. The projected output is , the least residual is , of norm , and the minimum-norm minimizer is , the one lying in . The normal equation identifies the family but does not determine the kernel component.
Concept lensStructural
The minimum-norm minimizer as one bounded operator
When is closed, the rule sending to its minimum-norm minimizer is itself a bounded linear operator, the pseudoinverse of . With the orthogonal projection onto and the restricted bijection,
for any constant of the closed-range criterion. If is also injective, makes the self-adjoint operator bounded below, hence invertible, and the normal equation gives .
A dense range can still miss its target
Without closed range the last theorem breaks down in the most concrete way.
Counterexample mode
An infimum of zero that is never attained
The tempting claim is that a least-squares problem in a Hilbert space always has a minimizer, because the projection theorem supplies a closest point. On define . It is bounded and injective. Every finite-support sequence is in its range, so the range is dense. But is not in the range: a preimage would have every coordinate equal to one, which is not square-summable. The truncations lie in the range and satisfy , so , yet attaining zero would solve the impossible equation . The projection theorem projects onto a closed subspace; the repair is the hypothesis that is closed, and here it is not.
The canonical preimages of the truncations are , whose norms grow like . There is no contradiction with continuity of ; its inverse on the range is unbounded. Indeed has norm one but its preimage has norm , so that inverse also admits no bounded extension.
The normal equation also has no solution for this particular target. Indeed, is self-adjoint and injective, so would imply . Thus an infimum can exist without either an exact solution or a least-squares minimizer.
Practice with solutions
Quick check
(Let be the right shift on . Find and verify .)
Use the formula for the left shift.
Solution · Answer
, so its kernel is the span of . The range of consists of sequences with first coordinate zero, whose orthogonal complement is the same span.
Quick check
(If is injective and for some , prove its range is closed. Must it be all of ?)
Follow a Cauchy sequence of outputs back to its preimages.
Solution · Answer
If , then , so the preimages are Cauchy and converge to some ; continuity gives , so the limit is in the range. It need not be onto: the right shift is an isometry with proper range.
Quick check
(For , , and , solve the normal equation and identify the smallest-norm minimizer.)
Compute first; it maps a scalar to .
Solution · Answer
, so the solutions are the line ; every one is an exact solution, because . The smallest-norm one lies in , so it is , of norm .
Quick check
(For the diagonal operator , calculate and explain why no positive lower bound can hold.)
Compare with .
Solution · Answer
but . Since is injective, its kernel is zero and the distance to the kernel is , so a bound would give for every , which is impossible.