Why completeness matters
In finite-dimensional linear algebra a sequence of increasingly accurate approximations often has a limit in the space under study. In infinite dimensions this must be proved. Functional analysis separates the algebraic operation of adding functions or sequences from the metric question of whether a limiting object still belongs to the same space. The distinction underlies both approximation and the solution of operator equations.
Definition 1.1.1
(Norm, Cauchy sequence and Banach space)
Let the scalar field be or . A norm on a vector space is a map such that exactly when , , and . It induces the metric .
- A sequence is Cauchy if for every there is such that whenever .
- A Banach space is a normed space in which every Cauchy sequence converges in that norm to an element of the space.
The norm is part of the structure: completeness can change when the norm changes, even when the underlying vector space stays the same.
Example: a normed space that is not complete
Worked example 1.1.2
(Continuous functions under the integral norm)
Let have the integral norm . For define to be zero on , one on , and linear between. The integral distance from the step function is the area of a triangle of base and height one, namely . Thus , so is Cauchy, although itself is not continuous.
If converged in this norm to a continuous , then , so . Continuity would force on and on , which is impossible at . A Cauchy sequence need not converge merely because its terms are continuous.
By contrast, with the supremum norm is Banach: a Cauchy sequence in that norm converges uniformly, because it converges pointwise in the complete scalar field and the Cauchy bound persists in the limit. Its uniform limit is continuous. The proof uses both completeness of the scalars and the uniform-limit theorem.
Why finite-dimensional spaces behave differently
Definition 1.1.3
(Equivalent norms)
Two norms and on the same vector space are equivalent if there are positive constants such that for every .
Either norm then bounds the other, so they have the same convergent and Cauchy sequences. The constants must work for every vector: a bound depending on does not establish equivalence.
Theorem 1.1.4
(Finite-dimensional norm theorem)
Every two norms on a finite-dimensional real or complex vector space are equivalent. Consequently every finite-dimensional normed space is complete, every finite-dimensional subspace of a normed space is closed, and every linear map from a finite-dimensional normed space into any normed space is bounded.
Proof
(Comparison with the coordinate Euclidean norm)
The zero-dimensional case is immediate. Otherwise choose a basis and write . Define . For any norm , the triangle and Cauchy–Schwarz inequalities give
In particular, , so is continuous in the coordinate Euclidean norm. The coordinate unit sphere is compact, including in the complex case (identify with ). The continuous function therefore attains a minimum there. Since no vector on the sphere is zero, . Scaling gives for all . Applying this to two norms proves their equivalence; there is no circular appeal to compactness in the unknown norm.
A Cauchy sequence in is Cauchy in coordinates, hence converges coordinatewise in the complete scalar field; the upper bound shows convergence in . Thus the finite-dimensional space is Banach. If it is a subspace of a normed space , a sequence in it that converges in is Cauchy in the inherited norm. It therefore converges to a point in the subspace, which must equal the ambient limit by uniqueness. Finally, for a linear , set . Then , proving boundedness.
Proof X-ray
Where finite dimension enters
The upper bound uses only the triangle inequality, but it needs the expansion to be a finite sum, so that is a finite number. That bound is the key move: it makes the unknown norm continuous for the Euclidean topology, where compact sets are known by the Heine–Borel theorem. Finite dimension is used a second time when the Euclidean unit sphere is declared compact, and the norm axiom " only for " is used to make the minimum strictly positive. The three corollaries then need no further compactness: completeness and closedness are transferred from coordinates by the two-sided bound, and boundedness of is the same upper-bound estimate applied to the vectors . In infinite dimensions both uses fail: the vectors of satisfy , so the Euclidean sphere contains a sequence with no convergent subsequence.
The theorem genuinely needs its dimension hypothesis, as the following example shows.
Counterexample mode
Two inequivalent norms on one space
Claim: any two norms on a vector space are equivalent. On , the vector space of finite-support scalar sequences, both and are norms. Let have its first entries equal to and the rest zero. Then , while . Thus no constant can satisfy for all , and in one norm but not in the other. The correct statement requires finite dimension; in infinite dimensions even the notion of convergence can change with the norm.
The closedness corollary is often the one used in later arguments, because it needs no completeness of the ambient space.
Quick check
(Why is the range of a linear map from a finite-dimensional space closed, even when its codomain is not complete?)
Look at the range itself as a normed space.
Solution · Answer
The range is itself finite-dimensional, hence complete in its inherited norm. A sequence in it converging in the codomain is Cauchy in the inherited norm, hence also converges to a point of the range; uniqueness of limits identifies the two limits, so the limit lies in the range.
Linear operators and their norms
Definition 1.1.5
(Bounded operator and operator norm)
A linear map between normed spaces is bounded if there is such that for every . Its operator norm is
Proposition 1.1.6
(Bounded linear maps are the continuous ones)
For a linear map between normed spaces, the following are equivalent: is bounded; is continuous at zero; is continuous everywhere.
Proof
(Scaling into a small ball)
A bound gives , so is (Lipschitz) continuous everywhere, in particular at zero. Conversely, continuity at zero gives a radius such that implies . Scaling any nonzero to , which has norm , yields .
Worked example 1.1.7
(Integration is bounded, differentiation is not)
On with the supremum norm, satisfies , so . Equality is attained at , where has supremum one, so .
The derivative map on with the supremum norm is linear but unbounded: has norm at most one while gives . No constant satisfies for all . Algebraic linearity alone does not imply continuity in infinite dimensions.
Operator norms turn , the space of bounded linear maps, into a normed space, and its completeness depends only on the target.
Theorem 1.1.8
(Completeness of the operator space)
If is a Banach space, then is a Banach space in the operator norm, for every normed space .
Proof
(Pointwise limit with a uniform bound)
Let be operator-norm Cauchy. For each , , so the sequence is Cauchy in ; define . Passing to limits in proves linearity. Given , choose so that for . Letting in gives for every . Thus is bounded and has norm at most ; in particular is bounded and in operator norm.
Common mistake
(Pointwise convergence of operators is not norm convergence)
The proof obtains from the operator-norm Cauchy hypothesis, not from for each . Pointwise convergence alone is weaker. On , let keep the first coordinates and set the rest to zero. For every , , yet , so for every .
Inner products and projection
Definition 1.1.9
(Inner product and Hilbert space)
An inner product space has an inner product , linear in the first argument and conjugate-linear in the second, positive definite and conjugate symmetric. It induces . A Hilbert space is an inner product space that is complete in this norm.
The Cauchy–Schwarz inequality makes the inner product jointly continuous. The parallelogram identity
is a necessary condition for a norm to come from an inner product. For example, the norm on fails it at and : the left side is and the right side is , so no inner product induces that norm.
Why the inner-product estimate works
The inequality is more than a convenient bound: it helps establish convergence of the inner-product series and that small changes of vectors produce small changes of their inner product. It follows directly from positivity, with no completeness assumption.
Theorem 1.1.10
(Cauchy–Schwarz inequality)
In a real or complex inner product space, . When , equality holds precisely when for some scalar (including ); if , equality holds for every .
Proof
(Expanding a nonnegative square)
If both sides are zero. Otherwise set . Using linearity in the first argument and conjugate-linearity in the second,
Rearrangement proves the estimate, and equality holds exactly when , that is, . The triangle inequality follows by expanding and bounding the real part of the cross term. Thus the inner product really does induce a norm.
For joint continuity, if and , then is bounded and
The bounded factor matters: convergence of each factor separately does not justify replacing a varying vector by a fixed one without an estimate.
A Hilbert space built from sequences
Define as the real or complex sequences for which . For , finite-dimensional Cauchy–Schwarz bounds each finite sum of by . These nonnegative partial sums are increasing and bounded, so converges absolutely. It therefore defines an inner product, linear in its first argument, with induced norm . Finite-dimensional Cauchy–Schwarz also gives the triangle inequality for finite truncations; taking limits shows .
Theorem 1.1.11
(The sequence space is complete)
With the inner product , the space is a Hilbert space.
Proof
(Coordinate limits and truncated sums)
Take a Cauchy sequence in this norm. Each coordinate is Cauchy since ; let its scalar limit be . Given , choose so that whenever . For every finite , first let in the finite sum:
Letting shows and its norm is at most . Since , it follows that , and the same bound proves in norm.
Common mistake
(Coordinatewise convergence is not convergence in )
Finding the coordinate limits is only the first step of the proof. The standard vectors satisfy in every coordinate, but for every , so does not converge to in . What closes the proof is the uniform Cauchy bound, which controls all coordinates together; it is applied to a finite sum before , so no interchange of two limits is needed.
Quick check
(Let . Does converge in , and to what? What happens in the finite-support subspace ?)
Compare with the full sequence .
Solution · Answer
Yes. Since converges, lies in , and . The same sequence is Cauchy in but has no limit there, because its only possible limit has infinitely many nonzero entries. This illustrates why is not complete.
Completeness is exactly what makes nearest points exist in closed subspaces.
Theorem 1.1.12
(Projection theorem)
Let be a nonempty closed linear subspace of a Hilbert space . For each there is a unique minimizing , and it is characterized by .
Proof
(Minimizing sequence, then first-order condition)
Put and choose with . The parallelogram identity applied to and gives
Since , the last norm is at least , so the right side is at most and is Cauchy. Completeness and closedness give a limit , and continuity gives . For any , the real function has its minimum at , so ; in the complex case also use to obtain , hence orthogonality. Conversely, if , then for the Pythagorean identity gives , so is the only minimizer. This also proves uniqueness.
Proof X-ray
What the parallelogram identity buys
A minimizing sequence exists in any metric setting; the difficulty is that it need not converge. The key move is the parallelogram identity, which converts "both and are nearly optimal" into "they are close to each other". It uses that the midpoint is again in , so only convexity of is needed at this step. Completeness of then supplies a limit, and closedness of keeps that limit in ; each hypothesis is used exactly once. The characterization is a separate first-order argument along each direction ; real only controls the real part of , which is why the complex case tests as well. Uniqueness comes for free from the Pythagorean identity. Without an inner product the existence part may survive but uniqueness can fail: in with , every point with is at distance one from , the least possible distance to the horizontal axis.
The closedness hypothesis cannot be removed, and a concrete point shows what goes wrong.
Counterexample mode
A dense subspace has no nearest points
Claim: every linear subspace of a Hilbert space contains a nearest point to each . Take , and . The truncations satisfy , so . Every has finitely many nonzero entries while has infinitely many, so and . The infimum is not attained. The projection theorem requires to be closed; here is a dense proper subspace.
For a closed subspace, writing gives with the second term in . This decomposition proves is linear and ; its norm is one when .
Theorem 1.1.13
(Riesz representation theorem)
Every bounded linear functional on a Hilbert space has a unique such that for all , and .
Proof
(A normal vector to the kernel)
If , take . Otherwise is closed, because is continuous. Choose with : start with a vector on which is nonzero, note that and , and scale. For any , , hence , that is, . Taking represents . Cauchy–Schwarz gives , and equality follows by evaluating at when . If two vectors represent , their difference has zero inner product with every vector, in particular with itself, and hence is zero.
Concept lensGeometric
A functional as a family of parallel hyperplanes
For the representer turns into geometry. The level sets are parallel closed hyperplanes with common normal , and . If , then , so Pythagoras gives , with equality at . Hence
a functional of large norm is one whose level sets are packed closely together.
Worked example 1.1.14
(A functional on and its representer)
On , let . With , Cauchy–Schwarz gives , so the series converges absolutely and . Since is real, , so is the representer. The theorem gives , and the value is attained at , where .
Common mistake
(Forgetting the conjugate in the representer)
With the convention used here, the representer sits in the second slot and enters conjugated. For on with , the representer is , since ; the guess gives . Likewise, on complex the functional with is represented by , not by . The norm is unaffected.
Practice with solutions
Exercise 1.1.15
(Let be a closed subspace of a Banach space . Prove that is a Banach space with the inherited norm.)
FoundationProof
Solution(4)
- 1
Start with a Cauchy sequence in
Let be Cauchy in . The norm of is the norm of , so is also Cauchy in .
- 2
Use completeness of
There is with .
Because: is a Banach space.
- 3
Use closedness of
Key step
Since every lies in , the limit lies in .
Because: A closed set contains the limits of its convergent sequences.
- 4
Conclude
∎The sequence converges to in the inherited norm. Every Cauchy sequence in converges in , so is Banach.
Exercise 1.1.16
(On with the supremum norm let . Find .)
Core
Solution(3)
- 1
Bound by the triangle inequality
, so .
- 2
Find a function attaining the bound
Key step
For , and .
Because: The bound is sharp when and with .
- 3
Conclude
.
Exercise 1.1.17
(In , find the nearest point to in the subspace spanned by , and the distance from to that subspace.)
Core
Solution(3)
- 1
Project onto the line
Put . Then and , so .
Because: For a one-dimensional subspace spanned by , the projection is .
- 2
Verify the orthogonality condition
Key step
, and .
Because: The projection theorem characterizes the nearest point by .
- 3
Compute the distance
By Pythagoras, . The nearest point is and the distance is .