Fourier projection works because the error is perpendicular to every retained frequency. That geometry is not specific to oscillating functions. It belongs to every complete inner-product space. Once we identify the geometric ingredient, we can solve best-approximation problems on closed subspaces, represent continuous linear measurements by vectors, and distinguish an orthonormal family from a complete system of coordinates.
There are two hypotheses to watch throughout. A set must be closed if a limiting minimizer is to remain admissible, and the ambient inner-product space must be complete if a minimizing Cauchy sequence is to have a limit at all. Neither condition can be replaced by a formal calculation of perpendicular directions.
Inner-product geometry and conventions
Let be a real or complex Hilbert space: a complete inner-product space with . We take the inner product to be linear in its first variable and conjugate-linear in its second. In particular,
The second identity is the parallelogram law, obtained by expanding both squares. We use Cauchy–Schwarz and the triangle inequality. They follow from minimizing and expanding a square, respectively; no projection theorem is needed to establish them.
Definition 2.4.1
(Distance, convexity and orthogonal complement)
For a nonempty set , write . A set is convex when it contains every segment between two of its points. If is a linear subspace, define
This is a closed linear subspace, because each condition is the zero set of a continuous linear functional of . A subspace need not be closed. We always say explicitly when closedness is required.
Best approximation to a closed convex set
Theorem 2.4.2
(Projection onto a closed convex set)
If is nonempty, closed and convex, every has a unique nearest point . It is characterized by
For and ,
In a real space the real-part notation can be omitted.
Proof
(Minimizing sequences are Cauchy)
Let and choose with . The midpoint belongs to , so the parallelogram law gives
The right side tends to zero. Completeness gives a limit , closedness puts it in , and continuity of the norm proves . Applying the same inequality to two minimizers proves uniqueness.
For the characterization, the point belongs to when . Minimality implies
Divide by and let to obtain the inequality. Conversely, if that inequality holds for all , expansion gives , so is the unique minimizer.
For stability, use in the inequality for , and in the one for . Adding the two inequalities and rearranging yields . Cauchy–Schwarz gives the norm bound, including the case without division. A projection onto a convex set is therefore stable, but it need not be linear.
Proof X-ray
The midpoint turns minimizing into Cauchy
A bounded minimizing sequence need not have a convergent subsequence in an infinite-dimensional space. The proof does not assume such compactness. Instead, the midpoint constraint and the parallelogram identity force any two sufficiently good candidates to be close. Completeness supplies their common limit. This is why the Hilbert geometry can prove existence where a finite-dimensional compactness argument would fail.
Common mistake
(Projection onto a convex set is not linear)
Only projection onto a closed subspace is linear. Let in . Then and , so . The variational inequality characterizes the nearest point; linearity appears only when is a subspace, where stays in for every .
Closed subspaces and perpendicular decomposition
Theorem 2.4.3
(The orthogonal projection theorem)
For a closed linear subspace ,
Thus every has a unique decomposition with and . The map is linear, , and . Its operator norm is one if and zero otherwise. For every linear subspace , closed or not,
Proof
(Perpendicular residual and its consequences)
Apply the convex projection theorem to . Since for every real and , the variational inequality implies . In a complex space apply it also to ; this makes the imaginary part zero. Thus . Conversely, any such decomposition is minimizing by Pythagoras. The intersection is zero, proving uniqueness.
Adding or scaling two decompositions gives another decomposition of the required kind, so uniqueness proves linearity. A vector already in projects to itself, proving idempotence. Pythagoras gives
This gives the operator-norm upper bound; a nonzero vector of gives equality after normalization.
Finally by continuity of the inner product. Certainly . If belongs to the latter space, decompose with and . Then , so . This proves the reverse inclusion. In particular, density of a linear span is equivalent to the absence of a nonzero vector perpendicular to it.
Counterexample mode
A dense admissible subspace can have no nearest point
Let and let be the subspace of finitely supported sequences. For , truncation shows . No vector of attains that distance, because distance zero would force equality with . The subspace is dense and linear, but not closed. Projecting onto its closure gives , which is not an admissible point of the original problem. Replacing a set by its closure can change whether the infimum is attained.
Continuous linear measurements are inner products
Let denote the bounded linear functionals on , with . This is the continuous dual, not the full algebraic dual.
Theorem 2.4.4
(Riesz representation in a Hilbert space)
For every there is a unique such that
With our convention, the correspondence from to is conjugate-linear in a complex space and linear in a real space.
Proof
(Normalize a vector perpendicular to the kernel)
The zero functional is represented by zero. If , its kernel is a closed subspace. Choose with and set . Then and . Put , so . For every , the vector belongs to the kernel. Consequently
Thus gives the representation. Uniqueness follows by testing the difference of two representing vectors against itself. Cauchy–Schwarz gives , and if , taking gives equality. Finally, is represented by , since the second argument of the inner product is conjugate-linear. This explains the scalar convention rather than concealing it in notation.
Proof X-ray
One perpendicular direction carries the whole functional
Continuity is used once, to make closed; that is what lets the projection theorem, which uses completeness, produce a nonzero perpendicular to the kernel. The decisive observation is that the kernel has codimension one: every splits as plus a kernel vector, so the single direction records all of . Testing that splitting against turns the value into an inner product.
Orthonormal families and genuine bases
Definition 2.4.5
(Orthonormal family and orthonormal basis)
A family is orthonormal if is one for and zero otherwise. We call it an orthonormal basis when its closed linear span is all of . This is not a Hamel basis: infinite norm-convergent expansions are allowed, rather than only finite algebraic combinations.
For nonnegative numbers indexed by an arbitrary set, define . The space consists of scalar families with finite sum of squares in this sense.
Theorem 2.4.6
(Orthogonal expansion and complete coordinates)
For an orthonormal family, put . Then
The nonzero coefficients of each fixed vector form an at most countable set, and the series converges in norm independently of their order. The following conditions are equivalent: the family is a basis; its perpendicular space is zero; every vector equals the displayed series; Parseval equality holds for every vector. For a basis, the coefficient map is a linear isometric bijection from onto .
Proof
(Finite subsums, then a limit over finite sets)
For a finite , let . Direct orthogonality gives
Hence the coefficient square sum is bounded by . For each positive integer , only finitely many coefficients can have magnitude at least : otherwise finite subsums would be arbitrarily large. Their countable union contains every nonzero coefficient. Enumerate that support. Orthogonality shows that differences of partial sums have squared norm equal to the corresponding coefficient tail. They are Cauchy, so completeness gives a limit .
The construction does not depend on the enumeration. Given , choose finitely many coefficients outside which the total square sum is less than . Every finite subsum containing those coefficients is within of . This also defines the meaning of the indexed series when is uncountable. Each coefficient of equals that of , so is perpendicular to all , and therefore to by continuity. The projection theorem identifies . Along the finite subsets ordered by inclusion, tends in norm to and the coefficient square sums increase to their defined supremum. Passing to this limit in the finite energy identity proves the displayed residual formula. This is a limit over finite subsets, not an enumeration of an uncountable index set.
If , the residual vanishes for every vector, giving both expansion and Parseval, and the perpendicular space is zero. Conversely, Parseval for every makes the residual formula force , hence ; expansion for every also puts every vector in . Finally the family and its closed span have the same perpendicular space. If that space is zero, the orthogonal decomposition leaves no residual for any , again giving . This proves all the equivalences. To prove surjectivity, start with . The same finite-threshold argument makes its support countable. Its orthogonal partial sums are Cauchy and converge to a vector with coefficients . If the family is a basis, uniqueness follows because the only vector perpendicular to all basis vectors is zero. Norm preservation follows from Parseval.
Common mistake
(Bessel is not Parseval for an incomplete family)
An orthonormal family always gives Bessel's inequality, but Parseval equality needs the family to be a basis. In the family is orthonormal, yet has every coefficient zero, so . The missing quantity is exactly the residual in the theorem.
For completeness, an orthonormal basis exists in every Hilbert space if we use Zorn's lemma as the standard set-theoretic input. Order orthonormal subsets by inclusion; this collection is nonempty because it contains the empty family, also covering . The union of a chain remains orthonormal, so a maximal family exists. If its closed span were proper, projection would give a nonzero perpendicular residual, whose normalization could be adjoined, a contradiction. Conversely, a complete family cannot be enlarged by another orthogonal unit vector.
In a separable space there is a direct construction. Take a countable dense sequence and process it in order: subtract its projection onto the finite span of the vectors already retained, and retain the normalized residual whenever it is nonzero. Every processed vector lies in the span obtained by that stage, so the resulting finite or countable orthonormal family has dense span. Conversely, a countable orthonormal basis gives a countable dense set by taking finite combinations with rational coefficients, or rational real and imaginary parts. Any orthonormal family in a separable space is countable: distinct members have distance , and disjoint balls of radius around them must contain distinct points of a countable dense set.
Concept lensStructural
What the Fourier theorem still had to prove
The abstract theorem cannot establish that the linear span of a particular orthonormal family is dense. For the Fourier application, take , where is normalized Lebesgue measure, and . Here we use two standard measure-theoretic inputs: is complete, and continuous functions on the circle are dense in . They require proof separately and do not follow from the abstract theorem above.
To see why finite linear combinations of the exponentials are then dense, let be continuous and use the finite Fejér kernel
It is nonnegative and has integral one. Its convolution with is a trigonometric polynomial. If , then whenever the circular distance from to zero is at least . Splitting the convolution integral into the near and far parts shows uniform convergence to : uniform continuity controls the near part, and the displayed bound makes the far mass tend to zero. Thus continuous functions, and hence all of , are approximated in norm by trigonometric polynomials. The abstract theorem now gives Parseval and norm reconstruction for the full exponential family. This argument does not assert pointwise convergence of ordinary Fourier partial sums.
Worked examples
Worked example 2.4.7
(Projection onto a continuous linear constraint)
Let in , let be a scalar, and impose . This affine hyperplane is nonempty because belongs to it, closed by continuity of the functional, and convex by linearity. Its nearest point to is
Indeed satisfies the constraint, and is a multiple of , perpendicular to the direction space . Pythagoras then proves minimality. The formula works over complex scalars because the inner product is linear in the first variable. It exhibits how a continuous measurement determines both a correction direction and its size.
Worked example 2.4.8
(Best linear approximation to a quadratic)
In , the vectors and are orthogonal, with squared norms one and . For , its projection onto their span is the constant : the residual has integral zero and is orthogonal to the odd function . Hence
The minimizing coefficients are uniquely , . The vector has not been normalized; a general coefficient formula must divide by its squared norm, or first normalize the vector. Orthogonal and orthonormal are different conditions.
Worked example 2.4.9
(An uncountable basis, but countable support per vector)
For an uncountable set , the space is Hilbert with its coordinate inner product. To check completeness, let be Cauchy and bounded in norm by . Each coordinate has a limit . For finite , passing to the limit gives , so . If for all sufficiently large , take the coordinate limit in each finite subsum and then its supremum to get .
The coordinate unit vectors form an orthonormal basis: each vector has countable support, and its finite tails tend to zero in norm. The space is not separable, because its uncountably many unit vectors have pairwise distance . There is no contradiction. Each vector uses only countably many coordinates, but no one countable set of coordinates suffices for every vector.
Practice
Quick check
(For the closed ball with , prove if , and otherwise. If , show this projection is not linear.)
Solution · Answer
Inside the ball the distance is zero. Outside, the reverse triangle inequality gives for every admissible , and attains this bound. Uniqueness follows from the convex projection theorem, or its proved equality characterization. If is a unit vector, while . Thus the map is not linear. Stability and uniqueness do not imply linearity for a general convex constraint.
Quick check
(In , project onto the span of and . Explain why simply summing gives the wrong answer.)
Solution · Answer
Write . Orthogonality of to and gives and . Thus , so and the squared error is . The proposed direct sum is ; it is not the projection. The usual coefficient formula applies to an orthonormal family, whereas these vectors have nonzero inner product and squared norms two.
Quick check
(Can a bounded linear functional on agree with on every finitely supported vector? Prove your answer both by a norm estimate and by Riesz representation.)
Solution · Answer
Let have its first entries equal to and all others zero. Its norm is one, but its coordinate sum is , contradicting any fixed bounded-functional norm. Alternatively, a representing vector would have to satisfy for every , so , which is not square-summable. Finite formulas do not automatically extend continuously to the completion.
Quick check
(On complex , let . Find its representing vector and norm. What vector represents under the convention used here?)
Solution · Answer
Sequence Cauchy–Schwarz proves the series defines a bounded functional. Its representing vector is , because . Thus , using . The functional is represented by , not by . The sign is forced by conjugate-linearity in the second argument.
Quick check
(Let with the norm, let in its completion, and let . Prove is closed in , but has no nearest point in . Compute the distance and explain which hypothesis of the projection theorem fails.)
Solution
The functional is bounded on by Cauchy–Schwarz, so its kernel is relatively closed there. In the complete space , we have and . Projection onto the closed hyperplane is therefore
This is not finitely supported: for every , its th coordinate is . It is nevertheless approximable by points of : truncate it to and set . Then , and , imply . Thus the infimum over is . Equality for a vector of would make it the unique minimizer in the complete hyperplane, hence equal to , which is impossible. The set is closed in its ambient space, but that ambient space is not complete.
Exact scope
The note proves Hilbert best approximation, bounded-functional representation and orthonormal-coordinate theory, including arbitrary index sets. It does not construct Hilbert completions from arbitrary pre-inner-product spaces, prove the spectral theorem, develop weak compactness, treat unbounded operators or replace the separate density argument needed for any particular proposed basis. The general existence proof uses Zorn's lemma explicitly; the separable construction does not. These are the boundaries of the proved results, not gaps concealed by the word basis.