Advanced Analysis, Notes 6: Banach spaces (basics, the Hahn-Banach Theorems)
by Orr Shalit
Recall that a norm on a (real or complex) vector space is a function that satisfies for all and all scalars the following:
A vector space with a norm on it is said to be a normed space. Inner product spaces are normed spaces. However, many norms of interest are not induced by an inner product. In fact:
Exercise A: A norm is induced by an inner product if and only if it satisfies the parallelogram law:
Instead of solving this exercise, you might prefer to read this old paper where Jordan and von Neumann prove this.
Using Exercise A, it is not hard to show that some very frequently occurring norms, such as the sup norm on or the operator norm on , are not induced by inner products. The latter example shows that even if one is working in the setting of Hilbert spaces one is led to study other normed spaces. We now begin our study of normed spaces and, particular, Banach spaces.
Our main objects of interest will now be normed vector spaces, over either the real or the complex numbers. Every norm naturally induces a metric. A Banach space is a normed vector space which is complete with respect to the norm.
Example 1: Let be a topological space. The space of bounded continuous functions equipped with the sup norm is a Banach space. This is usually proved in a course in topology. At least when is a metric space, the proof is similar to the proof from a course in analysis which shows that the uniform limit of continuous functions is continuous.
Example 2: Let , and let , equipped with the norm . Then is a Banach space. At the outset it may not be clear that satisfies the triangle inequality, or that is closed under addition (unless , in which case it is obvious). These two facts follow from Minkowski’s inequality. Once these facts are established, the proof that is complete is obtained from the proof that is complete (see Notes 1) by replacing by .
Example 3: Let be the space of bounded sequences with the norm . Then is a Banach space.
Proposition 4: Every normed space can be embedded isometrically in a unique Banach space which contains as a dense subset.
The proof of Proposition 4 is just like Theorem 5 in Notes 1, only simpler. The unique Banach space containing is called the completion of .
Exercise B: Let denote the vector space of all finitely supported sequences. For , equip with the norm. Compute the completion of .
Exercise C: A function is said to have compact support if is compact. A function is said to vanish at infinity if for all there is a compact such that for all . Let denote the space of all compactly supported continuous functions, and let denote the space of all continuous functions vanishing at infinity. Prove that the completion of is .
Example 5: If is a measure space and , then one may define the Lebesgue spaces as the space of all measurable functions on for which . If and is the counting measure then we get Example 2 (the counting measure is the measure that assigns to each set the value the number of elements in ).
Example 6: For those who are not yet very comfortable with measure spaces, another way to obtain (instances of) the previous example is as follows. Let be an open subset in . Denote by the continuous functions of compact support on . Define a norm on by . Then can be identified with (and can also be defined as) the completion of with respect to this norm.
Exercise D: Prove that is separable. Prove that is separable for and non-separable for .
2. Bounded operators and dual spaces
Example 7: Let and be normed spaces. We define the space of bounded linear operators from to , , just as we have in the Hilbert space case (see these notes). We equip with the operator norm . It is really simple to show that the operator norm is truly a norm: . It is also easy to show that Proposition 1 from Notes 4 holds: a linear map between to normed spaces is bounded if and only if it is continuous, if and only if it is continuous at some point. A slightly deeper fact is the following.
Proposition 8: If is a Banach space, then is a also a Banach space.
Proof: Suppose that is a Cauchy sequence in . For all , is a Cauchy sequence in the complete space . Let . It is easy to see that the map is linear and bounded. Even though we wrote down , we still did not show that in , because this involves showing that . If is chosen large so that is smaller that for , then for all ,
Thus for , thus .
One of the most important instances of the above proposition is when is equal to or .
Definition 9: Let be a normed space. We denote the space of all bounded linear functionals (i.e., bounded linear maps into the scalar field) by . is called the dual space or the conjugate space of .
Remark: In some parts of the literature, the dual space is denoted by .
- Every Hilbert space can be identified (anti-linearly) with its dual.
- : every gives rise to an by way of .
- If , and satisfies , then , as above.
- We will later see that .
We will discuss some more examples in the next set of notes, where we will also prove .
Having now several examples of Banach spaces, it is of interest to know what is the relationship between these spaces. Recall that Hilbert spaces which had orthonormal bases of the same cardinality turned out to be isomorphic, so they all enjoy the same structure and the same geometry. When it comes to Banach spaces the landscape is much richer. Although we hardly have any tools to begin mapping the landscape at this point, we raise the issue now, so we can return to it from time to time as we proceed.
A bounded operator is said to be an isomorphism if it is bijective and if it has a bounded inverse. It is said to be an isometry if for all . and are said to be isomorphic if there is an isomorphism between them, and isometrically isomorphic (or simply isometric) if there is an isometric isomorphism between them.
Note that the notion of isomorphism that we defined here is much weaker than the one we defined for Hilbert spaces (although within the class of Hilbert spaces they are equivalent). Indeed, a Hilbert space cannot be isometrically isomorphic to a normed space that is not Hilbert, but it certainly can be isomorphic. In fact, all -dimensional normed spaces are isomorphic to with the standard inner product.
At this point we do not have much more to say about which of our examples of Banach spaces is isomorphic to which. Here is something simple that we can say.
Exercise E: Prove that is not isomorphic to for all .
4. Topological vector spaces
Before moving on, I have to say that Banach spaces, though by far more general than Hilbert spaces, do not cover all the kinds of topological vector spaces that arise in analysis or in applications.
Consider for example the space , where . When is not compact there exist unbounded continuous functions on it. Thus one cannot define the sup norm for all elements in . However, for every compact subset of and every , we can define . If is a sequence of compact sets such that , then we may define a measure of distance between elements in by
One shows that is a complete metric on , and that a sequence converges to in if and only if uniformly on every compact subset of .
It can be shown that there is no norm one can define on that induces this metric. Other examples of interesting function spaces which are not linearly homeomorphic to Banach spaces are or (the space of holomorphic functions on ) when .
A vector space which is also a complete metric space with a translation invariant metric which has a convex local base at is called a Frechet space. There are even more general topological vector spaces of interest in analysis, especially locally convex topological vector spaces. Some of the theory we shall develop can also be developed for these spaces. I chose to stick to the more concrete Banach spaces for didactic purposes. We will discuss some specific locally convex topologies when they arise in the study of Banach spaces and their operators.
5. The Hahn-Banach extension theorems
Definition 10: Let be a real vector space. A sub-linear functional is a function such that
- for all .
- for all and .
Theorem 11 (HB extension theorem, sub-linear functional version): Let be a real vector space, and let be a sub-linear functional on . Suppose that is a subspace, and that is a linear functional on such that for all . Then there exists a linear functional on such that and for all .
Proof: We may suppose that is not trivial, otherwise the result is trivial. The first part of the proof involves extending only to a slightly larger subspace. The second part of the proof then involves using the first part to show that can be extended all the way up to .
Let be nonzero, and define . We will extend to a functional on such that for all . Since is independent of , we are free to define to be any real number, and that determines uniquely an extension of . The issue here is to choose so that is smaller than on . What we need is
for all . We may divide (*) by , and we may replace by any multiple of , to convert condition (*) to
for all . Working backwards, we see that if there exists any such that
then we can define , and that defines the extension on that is dominated by . But for all ,
so , so such a exists, and the first part of the proof is complete.
And now, the second part of the proof. If is finite dimensional, then we proceed by induction. If is not finite dimensional, we use Zorn’s lemma as follows. Let be the collection of all pairs such that
- is a linear subspace of containing .
- is a linear functional on that extends .
- for all .
The pair is in , so is not empty. Let be partially ordered by the rule if and only if and . It is easy to see that every chain in has an upper bound, thus by Zorn’s lemma has a maximal element . Now, must be , otherwise we would be able to extend it further by the first part of the theorem. Thus, is the required extension of , and the proof is complete.
Theorem 12 (HB extension theorem, bounded functional version): Let be a subspace of a normed space , and let . Then there exists such that and .
Proof: We prove the theorem first for a real space. Define for all . This is easily seen to be a sub-linear functional on , which dominates on . By the HB extension theorem, there exists extending such that for all , thus . Since extends , this is clearly an equality.
Suppose now that is a normed space over the complex numbers. Then it is also a normed space over the reals. Define . Then is a bounded real functional on , and . By the previous paragraph, extends to a bounded real functional on such that . The natural hope is that there is a complex linear functional such that which does what we want. But a short computation shows that the real part of a complex functional defines it completely, in the sense that . Thus we have no other option but to define . It is now a matter of some computations to show that such a defined is linear and extends . To see that , it suffices to show that (why?).
Let , and write . Then . But this is a real number, so its imaginary part vanishes and we get , which proves .
Corollary 13: Let be a normed space and . Then there exists such that and .
Proof: Let , define on by , and extend using Hahn-Banach.
Corollary 14: Let be an element in a normed space for which for all . Then .
6. The Hahn-Banach separation theorems
Theorem 15 (HB sepratation theorem, subspace-point): Let be a linear subspace of a normed space , and let such that . Then there exists , such that , and .
Proof: On we define a linear functional by . By Hahn-Banach, we will be done once we show that . But (when ), thus
Corollary 16: Let be a subspace in a normed space. A point is in if and only for all that vanishes on .
Corollary 16 is very useful in approximation theory. There is a proof of the Stone-Weierstrass Theorem that is based on this corollary (and also the Krein-Milman theorem, which we shall learn in a future lecture). Just to give you another taste, Corollary 16 (together with some complex analysis) can also be used to prove the following theorem of Muntz.
Theorem 17 (Muntz approximation theorem): Let be an increasing sequence of positive reals. Let . Then the linear space spanned by the monomials is dense in if and only if .
In particular, polynomials in prime powers of are dense in , isn’t that neat? See Theorem 3 on this post for the part of the proof of Muntz’ theorem (the sufficiency part – which is the more interesting part) that uses Hahn-Banach.
We return to our discussion of separation theorems. Let us fix for the rest of the post a normed space over the reals. Since every complex space is also a real space, the following separation theorems can be applied to complex spaces (with the modification that every appearing in the statements and definitions be replaced by its real part ).
The conclusion of Corollary 16 can be restated by saying that there is a functional that “separates” a subspace from a point. To make the notion of separation more geometrically intuitive, we introduce the following terminology.
Definition 18: A hyperplane in is a subset of the form , where is a linear functional on and . If , we say that the hyperplane separates from if
We say that this hyperplane strictly separates from if there is some such that
It is instructive to draw a picture that goes with this definition. Though a statement about a hyperplane separating two sets is just a statement about functionals, it is convenient to use the geometric terminology. To make our geometric vocabulary complete, we need the following result.
Exercise F: Prove that a hyperplane is closed if and only if it is not dense, and this happens if and only if is continuous.
To obtain the separation theorems, we will make use of the following device. Recall the definition of convex set from Notes 2.
Definition 19 (the Minkowski functional): Let be a convex and open set. Define by
is called the Minkowski functional of .
Lemma 20: Let be a convex and open set containing . Then is a sub-linear functional, , and there exists some such that for all .
Proof: There is some such that the closed ball , thus for every , , hence . That takes care of the last assertion.
Assume that . The set is open, so for sufficiently small , whence .
Assume that . Thus there is some for which . But since is on the ray connecting and and is convex, . Thus .
Clearly, . For , can be written as .
We proceed to prove that is sub-additive. Let and . From the definition and are in . Then every convex combination
( ) will also be in . We now choose the value of cleverly, so that the coefficients of and will be the same, and in that way we will get something times . The solution of the equation
is , and plugging that value of in (*) gives . Thus, . Letting tend to we obtained the result.
Theorem 21 (HB separation theorem, convex-open): Let be two nonempty disjoint convex sets, such that is open. Then there exists a closed hyperplane which separates and .
Proof: Let us first prove the case where consists of one point. By translation invariance of all the notions involved, we may assume that and . We want to use the HB extension theorem (Theorem 11). Let be the Minkowski functional of . Let be the linear functional sending every to . We will show that for all . Now, , so if , then . If , then . By the HB extension theorem can be extended to a functional on such that for all . This satisfies and for , so the hyperplane separates and (Question: why is this hyperplane closed?).
Now let us treat the general case. The set is convex and open, and does not contain . By the previous paragraph, there is a closed hyperplane that separates and as follows
so for all . If then separates and .
Theorem 22 (HB separation theorem, compact-closed): Let be two nonempty disjoint convex sets, such that is closed and is compact. Then there exists a closed hyperplane which strictly separates and .
Proof: As above, we consider , call this set . As above, is convex and does not contain . Since is compact, it follows that is closed. Let be a small ball disjoint from . By Theorem 21, there is a functional and such that
for all . It follows that and can be strictly separated by some hyperplane defined by .