Advanced Analysis, Notes 1: Hilbert spaces (basics)
by Orr Shalit
In this lecture and the next few lectures we will study the basic theory of Hilbert spaces. Hilbert spaces are usually studied over or over . In this course, whenever we consider Hilbert spaces, we shall consider only complex Hilbert spaces, that is, spaces over . The are two reasons for this. First, the results in this post hold equally well for real Hilbert spaces with similar proofs. Second, in some topics that we will discuss later the nice results only hold for complex spaces. So we will ignore real Hilbert spaces because they are essentially the same and also because they are fundamentally different!
Remark: The only situation I know where it is really important to concentrate on real Hilbert spaces when doing convex analysis (there must be others that I don’t know of). On the other hand, it is often convenient – indeed, we already did so in this course – to study real Banach spaces.
1. Basics of the basics
Definition 1: Let be a complex vector space. is said to be an inner product space if there exists a function that satisfies the following conditions:
From properties 3 and 4 above we have the following property too: .
Henceforth will denote an inner product space.
- Let with the standard inner product: .
- Let (the continuous functions on the interval) with the inner product: .
Remark: The reader might not be familiar with Riemann integration of complex valued functions. There is really nothing to it: if is a continuous, complex valued functions where and are its real and imaginary parts, then
If , one defines the norm of to be the nonnegative real number given by
Theorem 2 (Cauchy-Schwarz inequality): For all ,
Equality holds if and only if and are linearly dependent.
Proof: Exercise: prove the Cauchy-Schwarz inequality. Try also to come up with a proof for the inequality (and not he “equality holds if and only if” part) that only uses properties 2,3 and 4 of inner product (from Definition 1). We will have occasion to use this.
Theorem 3 (triangle inequality): For all ,
Equality holds if and only if one of the vectors is a positive multiple of the other.
Proof: One expands and then uses Cauchy-Schwarz :
Wee see that satisfies all the familiar properties of a norm:
- and .
It follows that induces a metric on by
Definition 4: is said to be a Hilbert space if it is complete with respect to the (metric induced by the) norm.
Recall that (in this setting) a sequence in is said to be a Cauchy sequence if as , and that is said to be complete if every Cauchy sequence actually converges to a point in .
Examples: Let us review the examples we saw above.
- . This space is complete w.r.t. the norm (as one should know from a basic course in topology), so it is a Hilbert space.
- . This space is not complete w.r.t. the norm, so this inner product space is not a Hilbert space. Indeed, let be the continuous function that is equal to zero between and , equal to between and , and linear on . Then if , then , thus this series is Cauchy. However, one can show that there is no continuous function on the interval for which .
Example: This is the fundamental example of a Hilbert space. Let (or, for short, ) denote the set of all sequences of all square summable complex numbers. That is
We endow with the natural vector space operations, and with the inner product . The norm then has to be .
There are several things to prove here. First, we have to show that is a vector space; second, we to show that our definition of the inner product makes sense; third, we have to show that is complete.
The only non-trivial thing to check with regard to being a vector space is that if , then . Using the triangle inequality in we find
for all . Letting we see that . In a similar manner, one uses the Cauchy-Schwarz inequality in to verify that converges absolutely for all , and thus the inner product is well defined.
Suppose that is a Cauchy sequence in . That means that
as . In particular, for all , we have that , thus is a Cauchy sequence of complex numbers. By completeness of , for each there is some such that converges to . Define . We need to show that , and that converges to in the norm of .
Now every Cauchy sequence in any metric space is bounded, so there is an such that for all . Fix some , and choose some such that for all . Then for every integer ,
for all . Letting , we find that
for all , and therefore . Letting be arbitrarily large, we conclude that and that for all . Since was arbitrary, we also get that . That completes the proof that is a Hilbert space.
Remark: It is convenient at times to consider the close relatives of defined as follows. Let be a set. Then we define to be
with the inner product . (In case is uncountable then one has to be careful how this sum is defined. This issue will be discussed later on).
Some of you may have forgotten what an important role the completeness property of played in the first real analysis courses (hint: it was probably used nine out of every ten times in which some series or sequence was proved to be convergent, but you had no clue as to what it was converging to). In infinite dimensional analysis it is very hard to get far without completeness. Most of the great theorems (but not all!) depend crucially on completeness of the spaces involved. That, together with the example of above, is the reason for the importance of the following theorem.
Theorem 5: Let be an inner product space. There exists a Hilbert space , and a linear map such that
- For all , .
- is dense in .
If is another Hilbert space and is a linear map satisfying the above two conditions, then there is a bijective linear map such that for all and for all .
- The Hilbert space is said to be the completion of .
- The last assertion of the theorem says that is essentially unique, up to the obvious isomorphism appropriate for Hilbert spaces.
- Since preserves all of the structure that enjoys, one usually identifies with and then the theorem is stated as follows: There exists a unique Hilbert space which contains as a dense subset.
Proof: Let be the vector space consisting of all Cauchy sequences in . To be precise, let be the vector space of all sequences in
with the usual addition and scalar multiplication: if and , then
Now let be the linear subspace of consisting of all Cauchy sequences (one checks that it is indeed a linear subspace). Our goal now is to play around with until it becomes the sought after Hilbert space . First thing let us try to define an inner product on . If and , then we define
The limit really does exist: every Cauchy sequence (in any metric space) is bounded, so the identity
together with Cauchy-Schwarz implies that is a Cauchy sequence of complex numbers. It is easy to check that the form satisfies conditions 2,3 and 4 of Definition 1. However, it does not satisfy condition 1, so it does not make into an inner product space.
We define an equivalence relation (check that it is) on by if and only if (in other words if and only if ), and we denote by the set of all equivalence classes. can also be thought of as the quotient of by the linear subspace of sequences that converge to . It is well known (and easy to show) that is a vector space, and now we will make it into a Hilbert space. Let us denote by the equivalence class of . By an identity similar to (*) above, the form is well defined (i.e., does not depend on the representative), and it follows that inherits from properties 2,3 and 4 of the definition of inner product. The form also satisfies condition 1, since if then is equivalent to the zero element of , hence (an alternative way to see that is well defined is to use the fact that (like any form satisfying 2,3 and 4 of Definition 1) satisfies the Cauchy-Schwarz inequality – which you were asked to do right after Theorem 2 above). To show that is a Hilbert space it remains to show that it is complete. I’ll put in some details in the next few paragraphs, but it is a good idea for students to try to skip the next paragraphs (up to the next section) and fill in the details themselves.
Suppose that is a Cauchy sequence in . As always with Cauchy sequences, to show that it converges to a limit, it suffices to show that a subsequence converges to a limit. Thus we may assume without loss of generality that for all .
To show that converges in , we will construct a for which . Choose a representative for for all . We are free to choose any representative we wish, so let us use this freedom. Note that if is any integer, and , then . Therefore, we may always modify a representative by discarding as many initial terms in the sequence as we need without changing the equivalence class. So we choose in such a way that for all .
Now define by . This sequence is actually Cauchy, that is, . Indeed, Since there is some such that . Thus, if , then
Now and because of the way in which we chose the representatives. The middle term it smaller than by choice of , thus
which shows that is a Cauchy sequence. To see that , we check, similarly to the above, that
Now we define by letting be , where is the sequence with for all . Then it is clear that is well defined and linear and that for all . Moreover, if and for all , then for we have . So is dense in .
We now come to the final assertion. Let and be as described in the theorem. We define a map by . From the properties of , it follows that is a linear map and that
It follows that is an isometry (i.e., a metric preserving map) between dense subsets of two complete metric spaces, and thus it extends uniquely to a map (which we also denote by ) between and . That completes the proof.
3. The space
Let denote the space of piece-wise continuous functions on the interval , and define the following form:
(The integral we require is the standard (complex valued) Riemann integral or equivalently the Darboux integral). Note that this isn’t exactly an inner product on because it may happen that for a function which is not identically zero; for example if at finitely many points and zero elsewhere. However, this problem is easily fixed, because
Exercise A: if and only if the set where is nonzero is finite.
So to fix the problem, say that if and are equal at all points of except finitely many. The set of equivalence classes may then be given the structure of an inner product space, with
It is customary to denote the set of equivalence classes of piece-wise continuous functions also as , and to consider elements of this space as functions, with the provision of declaring two functions to be “equal” if they are equivalent.
Exercise B: Show that for every , there is at most one in its equivalence class. Thus, our new equality in is consistent with the equality of functions in .
Exercise C: Prove that is dense in .
4. The space
By Theorem 5, every inner product space can be completed to a Hilbert space, and when working with infinite dimensional inner product spaces it is often advantageous to pass to the completion. Let us look at the most important example of this.
In many problems in classical analysis or mathematical physics, such as Fourier series, differential equations, or integral equation, it seems that the natural space of interest is or . Experience has led mathematicians to feel that it is helpful to introduce on these spaces the inner product
and to use the induced norm as a measure of size in this space. However, neither of these spaces is complete (we have seen that is not).
Exercise D: Show that with respect to the above inner product is not complete.
Consider the inner product space with the above inner product. By Theorem 5, has a unique completion; denote the completion by .
Exercise E: Prove that is dense in , and hence deduce that is also “equal” to the abstract completion of (as a first step, clarify to yourself in what sense is “in” ).
Now, was defined in Theorem 5 rather abstractly, as the quotient space of a space of sequences of elements in . The space can be defined in a completely different manner, in rather concrete function-theoretic terms, as the space of square integrable Lebesgue measurable functions on the interval. This is the way this space is defined in a course in measure theory (in fact this is how it is usually defined). After one defines the space , one can prove that it is complete, and that the continuous functions (or the piece-wise continuous functions) are dense in this space. However, the uniqueness part of Theorem 5 promises us that we get the same Hilbert space as we do from the construction we carried out above. Note that we did not get for free any measure theoretic result, it is the theorems in measure theory (completeness of and density of the piece-wise continuous functions in it) that allow us to conclude that the measure theoretic construction and the abstract construction give rise to the “same” Hilbert space.
At this point we do not require any knowledge from measure theory, rather, we will stick with our definition of as an abstract completion, and we will derive some of its function theoretic nature from this. This approach is good enough for many applications of the theory.
Let us decide that we will call every element a function. Since , it is clear that we will be identifying some functions – we were already identifying functions at the level of , and since we added new functions we are sure to have at least as many identifications (in fact, we will have much more). Now, if we cannot really say what is the value of at a point (since already in we cannot do this), but has some other function-like aspects.
First of all, is “square integrable” on . To be precise, we define the integral of on to be
We see that is and are functions in , then we consider and as being equal if .
Second, is actually “integrable” on , that is, we can define
where is the constant function on , which is in , and therefore in .
Lastly, we can define the integral of on every sub-interval :
where is the piece-wise continuous function that is equal to on and zero elsewhere.
All these definitions are consistent with the definitions of integral of piece-wise continuous functions. Moreover, for a continuous function , if we know for every interval then we can completely recover ; for a piece-wise continuous function , if we know for every interval then we can recover the equivalence class of in . It follows (with some work) that if then the collection of quantities for all determines , and consequently we can say that a function in is uniquely determined by its integrals over all intervals.
Exercise F: Show that a function in is completely determined by the values of its integrals over sub-intervals of .
You are meant to prove the above exercise using only our definition of as the abstract completion of and the subsequent definition of integral over a sub-interval. It is also a fact in measure theory, that any “function” is nothing more and nothing less then the totality of values .
5. The spaces
In a way similar to the above, if is a subset in of the form , then we define to be the completion of the space with respect to the inner product
The details are similar to the one dimensional case and we do not dwell on them.
spaces can be defined on spaces of a more general nature, but that is best done with some measure theory. The reason that we are taking our rather unusual approach is that students are allowed to take the course “Measure Theory” concurrently with “Advanced Analysis”, so we are trying in the beginning to get along with nothing from measure theory. But it is also interesting for me to see how far we get with only basic analysis and “pure” Hilbert space methods.