Advanced Analysis, Notes 16: Hilbert function spaces (basics)

by Orr Shalit

In the final week of the semester we will study Hilbert function spaces (also known as reproducing kernel Hilbert spaces) with the goal of presenting an operator theoretic proof of the classical Pick interpolation theorem. Since time is limited I will present a somewhat unorthodox route, and ignore much of the beautiful function theory involved. BGU students who wish to learn more about this should consider taking Daniel Alpay’s course next semester. Let me also note the helpful lecture notes available from Vern Paulsen’s webpage and also this monograph by Jim Agler and John McCarthy (in this post and the next one I will refer to these as [P] and [AM] below).

(Not directly related to this post, but might be of some interest to students: there is an amusing discussion connected to earlier material in the course (convergence of Fourier series) here).

1. Hilbert function spaces

A Hilbert function space on a set X is a Hilbert space H which is a subspace of the vector space of all functions X \rightarrow \mathbb{C}, with one additional and crucial property:

For every x \in X the functional of point evalution at x, given by f \mapsto f(x), is a bounded functional on H.

This crucial property connects between the function theoretic properties of H as a space of functions, and the Hilbert space structure of H and therefore also the operator theory on H. Hilbert function spaces are also known as reproducing kernel Hilbert spaces (RKHS), for reasons that will become clear very soon.

When we say that H is a subspace of \mathbb{C}^X, we mean two things: 1) H is a subset of \mathbb{C}^X; and 2) H inherits its linear space structure from \mathbb{C}^X. In particular, the zero function is the zero element of H, and two elements of H which are equal as functions are equal as elements of H.

Example: Perhaps the most interesting examples of Hilbert function spaces are those that occur when X is an open subset of \mathbb{C}^N, and now we’ll meet one of the most beloved examples. Let D denote the open unit disc in the complex plane \mathbb{C}. Recall that every function f analytic in D has a power series representation f(z) = \sum_{n=0}^\infty a_n z^n. We define the Hardy space H^2 to be the space of analytic functions on the disc with square summable Taylor coefficients, that is

H^2 = \{f(z) = \sum_{n=0}^\infty a_n z^n : \sum|a_n|^2 < \infty \}.

This space is endowed with the inner product

(*)  \langle \sum a_n z^n , \sum b_n z^n \rangle = \sum a_n \overline{b_n}.

Before worrying about function theoretic issues, note that the space of formal power series with square summable coefficients is obviously isomorphic (as an inner product space) to \ell^2 when endowed with the inner product (*). But if a formal power series has square summable coefficients then its coefficients are bounded, so it is convergent in the open unit disc. Thus H^2 is a space of analytic functions which as an inner product space is isomorphic to \ell^2, and is therefore complete.

To show that H^2 is a Hilbert function space on D, we need to show that for every w \in D, the functional f \mapsto f(w) is bounded. Well, if this functional is bounded, then by the Riesz representation theorem there exists some k_w \in H^2 such that f(w) = \langle f, k_w \rangle. Take another look at (*), recall that f(w) = \sum a_n w^n, and putting these two things together it is not hard to come up with the guess that k_w(z) = \sum_{n=0}^\infty \overline{w}^n z^n. Since |w|<1, we have that \sum |\overline{w}^n|^2 < \infty, so k_w \in H^2. Plugging k_w in (*) we see that \langle f, k_w \rangle = f(w), so that this functional is indeed bounded.

There are a great many other Hilbert function spaces of rather different function theoretic nature. Some of them are mentioned and studied in [P] and in [AM].

Example: The Hilbert space \ell^2 = \ell^2(\mathbb{N}) is a space of functions from \mathbb{N} to \mathbb{C}, and point evaluation is clearly bounded. Thus \ell^2 is a Hilbert function space on set \mathbb{N}. I have never heard of anything useful that came out of this point of view.

Exercise A.1: The Bergman space L^2_a(D) is the space of all analytic functions in the disc which are square integrable on the disc:

L^2_a(D) = L^2(D) \cap O(D)

(where O(D) denotes the space of functions analytic in D). Show directly (that is, not by guessing the kernel functions) that point evaluation is bounded.

Exercise A.2: The Segal-Bargmann space is the space of all entire functions f for which \int_\mathbb{C} |f(x+iy)|^2 e^{-x^2-y^2}dxdy < \infty. Show that this space is a Hilbert function space.

2. Reproducing kernels

Let H be a Hilbert function space on a set X. As point evaluation is a bounded functional, there is, for every x \in X, an element k_x \in H such that f(x) = \langle f, k_x \rangle for all f \in H. The function k_x is called the kernel function at x. We define a function on X \times X by

k(x,y) = k_y(x) .

The function k is called the reproducing kernel H or simply the kernel of H. The terminology comes from the fact that the family of functions k_y = k(\cdot, y) enable one to reproduce the values of any f \in H via the relationship f(y) = \langle f, k_y \rangle. Note that k(x,y) = \langle k_y, k_x \rangle.

Example: The kernel function for H^2 is

k(z,w) = k_w(z) = \sum \overline{w}^n z^n = \frac{1}{1-z\overline{w}} .

Exercise B: What is the kernel function of \ell^2(\mathbb{N})? What about the Bergman space L^2_a(D)? What about the Segal-Bargmann space?

The kernel k has the following important property. If a_1, \ldots, a_n \in \mathbb{C} and x_1, \ldots, x_n \in X, then

\sum_{i,j=1}^n a_i \overline{a_j} k(x_j,x_i) = \sum_{i,j=1}^n a_i \overline{a_j} \langle k_{x_i}, k_{x_j} \rangle = \|\sum_{i=1}^n a_i k_{x_i}\|^2 \geq 0.

It follows that for every choice of points x_1, \ldots, x_n \in X, the matrix \big(k(x_j, x_i) \big)_{i,j=1}^n is positive semi-definite. A function with this property is called a positive definite kernel (sometimes it is called a positive semi-definite kernel, but let us lighten terminology). Every Hilbert function space gives rise to a positive definite kernel — its reproducing kernel. In fact, there is a bijective correspondence between positive definite kernels and Hilbert function spaces.

Theorem 1: Let k be a positive definite kernel on X. Then there exists a unique Hilbert function space H on X such that k is the reproducing kernel of H.

Remark: Let me be a little bit more precise regarding the uniqueness statement, because in the context of Hilbert space one might mistakingly understand that this uniqueness is up to some kind of Hilbert space isomorphism. The existence statement is that there exists a subset H of the space \mathbb{C}^X of all functions on X, which is a Hilbert function space, and has k as a reproducing kernel. The uniqueness assertion is that this set H \subseteq \mathbb{C}^X is uniquely determined.

Proof: Denote k_x = k(\cdot, x). Let G_0 be the linear subspace of \mathbb{C}^X spanned by the functions \{k_x : x \in X\}. Equip G_0 with the sesquilinear form

\langle \sum a_i k_{x_i}, \sum b_j k_{y_j} \rangle_0 = \sum a_i \overline{b_j} k(y_j, x_i) .

From the definition and the fact that k is a positive definite kernel, the form \langle \cdot, \cdot \rangle_0 satisfies properties 2–4 from the definition of inner product (see Definition 1 in this post). That is, it is almost an inner product, the only property missing is \langle g, g\rangle = 0 \Rightarrow g = 0. As mentioned before (see Theorem 2 in that same old post) this means that the Cauchy-Schwartz inequality holds for this form on G_0. From this it follows that the space N = \{g \in G_0 : \langle g, g\rangle_0 = 0\} is a subspace, and also that \langle f,g \rangle_0 = 0 for all f \in G_0 and g \in N. This implies that the quotient space G = G_0 / N can be equipped with the inner product

\langle \dot{f}, \dot{g} \rangle = \langle f, g \rangle_0 .

As usual, we are using the notation \dot{f} to denote the equivalence class of f in G_0 / N. That this is well defined and an inner product follows from what we said about N. Now complete G to obtain a Hilbert space H.

We started with a space of functions on X, but by taking the quotient and completing we are now no longer dealing with functions on X. To fix this, we define a map F: H \rightarrow \mathbb{C}^X by

(**) F(h)(x) = \langle h, \dot{k}_x \rangle .

This is a linear map and it is injective because the set \{\dot{k}_x : x \in X\} spans G. Note that

(***) F(\dot{k_x}) (y) = \langle \dot{k}_x, \dot{k}_y \rangle = k(y,x).

We may push the inner product from H to F(H), and we obtain a Hilbert space of functions in \mathbb{C}^X. Point evaluation is bounded by definition (see (**)), and the kernel function at x is F(\dot{k}_x).  By (***), this F(\dot{k}_x) = k_x. Let us now identify between H and F(H). It follows that H is spanned by the kernel functions \{k_x : x \in X\}. The kernel function of this Hilbert function space H is given by

\langle \dot{k}_y, \dot{k}_x \rangle = \langle k_y, k_x \rangle = k(x,y) ,

as required.

Now suppose that L is another Hilbert function space on X that has k as a reproducing kernel. By definition, the kernel functions are contained in L, and since H is spanned by the kernel functions, H \subseteq L. However, if f is in the orthogonal complement of H in L then f(x) = \langle f, k_x \rangle = 0 for all x \in X, thus f \equiv 0. This shows that L = H, completing the proof.

Remark: Sometimes Hilbert function spaces are referred to as reproducing kernel Hilbert spaces (RKHS for short), this terminology being justified by the preceding discussion.

Exercise C: Let H be a RKHS with kernel k. Let x_1, \ldots, x_n \in X. Show that the matrix \big( k(x_i, x_j) \big)_{i,j=1}^n is strictly positive definite (meaning that it is positive semi-definite and invertible) iff the functions k_{x_1}, \ldots, k_{x_n} are linearly indendent, iff the evaluation functionals at the points x_1, \ldots, x_n are linearly independent as functionals on H.

Exercise D: Show how the proof of the above theorem is simplified when k is assumed strictly positive definite (meaning that the matrix \big( k(x_i, x_j) \big)_{i,j=1}^n is strictly positive definite for any choice of points). Show how at every stage of the process one has a space of functions on X.

Exercise E: Rewrite the above proof of the theorem (for k not necessarily strictly positive) so that it involves no quotient-taking and no abstract completions.

3. H^2 as a “subspace” of L^2

We now continue to study the space H^2, and give a useful formula for the norm of elements in this space.

Let f(z) = \sum_{n=0}^\infty a_n z^n \in H^2. For any r\in (0,1) denote f_r(z) = f(rz). The power series of f_r is given by

f_r(z) = \sum_{n=0}^\infty a_n r^n z^n .

This series converges to f_r uniformly on the closed unit disc. In fact, f_r extends to an analytic function on a neighborhood of the closed unit disc, and the series converges uniformly and absolutely on a closed disc that is slightly larger than the unit disc. It follows that the following computations are valid:

\frac{1}{2\pi}\int_0^{2 \pi} |f_r(e^{it})|^2 dt = \sum_{m,n=0}^\infty \frac{1}{2 \pi} \int_0^{2 \pi} a_m r^m e^{imt} \overline{a_n} r^n e^{-int} dt =

= \sum_{n=0}^\infty r^{2n}|a_n|^2 = \|f_r\|^2_{H^2} .

It would be nice to say that for every f \in H^2

\|f\|_{H^2} = \big( \frac{1}{2 \pi} \int_0^{2\pi} |f(e^{it})|^2 dt \big)^{1/2} ,

and this is almost true, but we have to be careful because elements in H^2 are not defined on \mathbb{T} = \{z \in \mathbb{C} : |z|=1\}. However,

\|f-f_r\|^2 = \sum (1-r^n)^2|a_n|^2

and this tends to 0 as r \rightarrow 1 by the dominated convergence theorem. In particular

\|f\|^2 = \lim_{r\rightarrow 1} \|f_r\|^2 = \lim_{r\rightarrow 1}\frac{1}{2\pi}\int_0^{2 \pi} |f(re^{it})|^2 dt .

We will use this formula below. Much more is true in fact: the functions f_r converge almost everywhere to a function f^* \in L^2(\mathbb{T}), and \|f\|_{H^2} = \|f^*\|_{L^2}. Moreover, H^2 can be identified via the map f \mapsto f^* with the subspace of L^2(\mathbb{T}) consisting of all functions with Fourier series that are supported on the non-negative integers: if f(z) = \sum_{n=0}^\infty a_n z^n \in H^2, then f^*(e^{it}) = \sum_{n=0}^\infty a_n e^{int}. For details, see [AM]. What we will need is a little weaker fact:

Fact: A function f analytic on D is in H^2 if and only if

\lim_{r\rightarrow 1} \int_0^{2\pi}|f(re^{it})|^2 dt < \infty .

Proof: We already showed that H^2 functions have this property. Now let f be analytic in the disc, and suppose it has power series representation f(z) = \sum_{n=0}^\infty a_n z^n. As before

\frac{1}{2\pi}\int_0^{2\pi}|f(re^{it})|^2 = \sum_{n=0}^\infty r^{2n}|a_n|^2 .

By the monotone convergence theorem the right hand side converges to \sum_{n=0}^\infty |a_n|^2, so the limit of integrals is finite if and only if f \in H^2.

4. Multiplier algebras

Let H be a Hilbert function space on X, to be fixed in the following discussion. Now we define a class of natural operators on H.

The multiplier algebra of H, denoted Mult(H), is defined as

Mult(H) = \{f : X \rightarrow \mathbb{C} : \forall h \in H . fh \in H\} .

In this definition fh simply denotes the usual pointwise product of two functions: fh(x) = f(x) h(x). For every f \in Mult(H) we define an operator M_f : H \rightarrow H by

M_f h = fh .

It is not unusual to identify M_f and f, and we will do so too, but sometimes it is helpful to distinguish between the function f and the linear operator M_f.

Proposition 2: For all f \in Mult(H), we have M_f \in B(H).

Proof: Obviously M_f is a well defined linear operator. To show that it is bounded we use the closed graph theorem. Let h_n \rightarrow h \in H, and suppose that M_f h_n = f h_n\rightarrow g \in H. Since point evaluation is continuous we have for all x \in X

h_n(x) \rightarrow h(x)

and

f(x) h_n(x) \rightarrow g(x)

thus f h(x) = g(x) for all x, meaning that M_f h = g, so the graph of M_f is closed.

Thus, \{M_f : f \in Mult(H)\} is a subalgebra of B(H). We norm Mult(H) by pulling back the norm from B(H) (this means \|f\|_{Mult(H)}:= \|M_f\|), and this turns Mult(H) into a normed algebra. We will see below that it is a Banach algebra.

Important assumption: In fact, there is a situation when \|f\|_{Mult(H)} is not a norm, but only a semi-norm. This doesn’t arise when for every x \in X there exists a function h \in H such that h(x) \neq 0. Thus we will assume this henceforth. This assumption holds in most (but not all) cases of interest, and in particular in the case of H^2(D). It is equivalent to the assumption that k_x(x) = k(x,x) = \|k_x\|^2 \neq 0 for all x \in X.

One of the instances when it is helpful to distinguish between a multiplier f and the multiplication operator M_f that it induces is when discussing adjoints.

Proposition 3: Let f\in Mult(H). For all x \in X, the kernel function k_x is an eigenvector for M_f^* with eigenvalue \overline{f(x)}. Conversely, if T \in B(H) is an operator such that T k_x = \lambda(x) k_x for all x \in X, then there is an f \in Mult(H) such that \lambda(x) = \overline{f(x)} for all x and T = M_f^*.

Proof: We let x \in X and compute for all h \in H

f(x)h(x) = \langle M_f h, k_x \rangle = \langle h, M_f^* k_x \rangle

but f(x) h(x) = \langle h, \overline{f(x)} k_x \rangle. Thus M_f^* k_x = \overline{f(x)} k_x.

For the converse, suppose T k_x = \lambda(x) k_x for all x \in X. Define f(x) = \overline{\lambda(x)}. For all h \in H, we have T^*h \in H too. We now compute

\langle T^* h , k_x \rangle = \langle h, T k_x \rangle = \overline{\lambda(x)} \langle h, k_x \rangle = f(x) h(x).

It follows that T^* h = fh \in H, so f \in Mult(H). It also follows that T^* = M_f.

Corollary 4: Mult(H) contains only bounded functions, and \sup_{X}|f| \leq \|M_f\|.

Proof: This follows from the Propositions 2 and 3, since the eigenvalues of M_f^* are contained in \sigma(M_f^*), which is contained in the disc of radius \|M_f^*\| = \|M_f\|.

Corollary 5: \{M_f : f \in Mult(H)\} is closed in the weak operator topology of B(H), and, in particular, Mult(H) is a Banach algebra.

Remark: The weak operator topology is the topology on B(H) in which a net T_n \rightarrow T if and only for all f,g \in H, \langle T_n f, g \rangle \rightarrow \langle Tf, g \rangle. The reader may safely replace this with the norm topology of B(H) for the purposes of these lectures.

Proof: If T_n is a net in \{M_f : f \in Mult(H)\} converging to T in the weak operator topology, then T_n^* \rightarrow T^* WOT. Then for all x \in X, T_n^* k_x = \lambda_n(x) k_x converges weakly to T^* k_x, so k_x is an eigenvalue for T^*. By Proposition 3, T \in \{M_f : f \in Mult(H)\}.

Example: Let’s find out who is the multiplier algebra of H^2. Since 1 \in H^2, we have that Mult(H) \subseteq H^2, and in particular every multiplier in Mult(H) is an analytic function on D. By Corollary 4 we have that every multiplier is bounded on the disc. Thus

Mult(H^2) \subseteq H^\infty \equiv H^\infty(D) .

The right hand side denotes the algebra of bounded analytic functions in D. On the other hand, by the formula developed in Section 3, we have for f \in H^\infty and h \in H^2,

\lim_{r\rightarrow 1}\frac{1}{2 \pi}\int_0^{2\pi} |f(re^{it})h(re^{it})|^2 dt \leq \|f\|^2_\infty \lim_{r\rightarrow 1} \frac{1}{2\pi} \int_0^{2\pi} |h(re^{it})|^2 dt = \|f\|^2_\infty \|h\|^2_{H^2} .

By the fact at the end of Section 3, we obtain fh \in H^2 and \|fh\|_{H^2} \leq \|f\|_\infty \|h\|_{H^2}, so f \in Mult(H^2) and \|M_f\| \leq \|f\|_\infty.

Exercise F: What is the the multiplier algebra of L^2_a(D)? Of \ell^2(\mathbb{N})? Can you cook up a non-trivial RKHS that has a trivial (scalars only) multiplier algebra?

5. First magic trick

Note that as a consequence of the previous example and Corollary 5 we have that H^\infty(D) is a Banach algebra. It follows readily that the limit of any locally uniformly convergent sequence of analytic functions (on any open set) is also analytic. I leave it to the reader to check just how much complex analysis we used to obtain this result, and to compare with the other proof they know of this fact.