elementary properties of hilbert...

FUNCTIONAL ANALYSIS LECTURE NOTES

CHAPTER 1. HILBERT SPACES

CHRISTOPHER HEIL

1. Elementary Properties of Hilbert Spaces

Notation 1.1. Throughout, F will denote either the real line R or the complex plane C.All vector spaces are assumed to be over the field F.

Definition 1.2 (Semi-Inner Product, Inner Product). If X is a vector space over the field F,then a semi-inner product on X is a function 〈·, ·〉 : X × X → F such that

(a) 〈x, x〉 ≥ 0 for all x ∈ X,

(b) 〈x, y〉 = 〈y, x〉 for all x, y ∈ X, and

(b) Linearity in the first variable: 〈αx+βy, z〉 = α〈x, z〉+β〈y, z〉 for all x, y, z ∈ X andα, β ∈ F.

Exercise 1.3. Immediate consequences are:

(a) Anti-linearity in the second variable: 〈x, αy + βz〉 = α〈x, y〉 + β〈x, z〉.

(b) 〈x, 0〉 = 0 = 〈0, y〉.

(c) 〈0, 0〉 = 0.

Definition 1.4. If a semi-inner product 〈·, ·〉 satisfies:

〈x, x〉 = 0 =⇒ x = 0,

then it is called an inner product, and X is called an inner product space or a pre-Hilbertspace.

Notation 1.5. There are many different standard notations for a semi-inner product, e.g.,

〈x, y〉 = [x, y] = (x, y) = 〈x|y〉,

to name a few. We shall prefer the notation 〈x, y〉.

Date: February 20, 2006.

These notes closely follow and expand on the text by John B. Conway, “A Course in Functional Analysis,”

Second Edition, Springer, 1990.

1

2 CHRISTOPHER HEIL

Exercise 1.6. Prove the following.

(a) The dot product x · y = x1y1 + · · · + xnyn is an inner product on Cn.

(b) If w1, . . . , wn ≥ 0 are fixed scalars, then the weighted dot product 〈x, y〉 = x1y1w1 +· · ·+ xnynwn is a semi-inner product on Cn. It is an inner product if wi > 0 for each i.

(c) If A is an n × n positive semi-definite matrix (Ax · x ≥ 0 for all x ∈ Cn), then〈x, y〉 = Ax · y is a semi-inner product on Cn, and it is an inner product if A is positivedefinite (meaning Ax ·x > 0 for all x 6= 0). The weighted dot product is just the special casewhere A is diagonal.

(d) Show that if 〈·, ·〉 is an arbitrary inner product on Cn, then there exists a positivedefinite matrix A such that 〈x, y〉 = Ax · y. Hint: Let e1, . . . , en be the standard basis. Then〈x, y〉 = y1〈x, e1〉 + · · · + yn〈x, en〉. For each k, the mapping x 7→ 〈x, ek〉 is linear.

Exercise 1.7. Let I be a countable index set (e.g., I = N or Z). Let w : I → [0,∞). Let`2w = `2

w(I) be the weighted `2 space defined by

`2w = `2

w(I) =

x = (xi)i∈I :∑

i∈I

|xi|2 w(i) < ∞

.

Show that

〈x, y〉 =∑

i∈I

xi yi w(i)

defines a semi-inner product on `2w. If w(i) > 0 for all i then it is an inner product.

If w(i) = 1 for all i, then we simply call this space `2.The series defining 〈x, y〉 converges because of the Cauchy–Schwarz inequality.

Example 1.8. Let (X, Ω, µ) be a measure space (X is a set, Ω is a σ-algebra of subsets ofof X, and µ : Ω → [0,∞] is a countably additive measure). Define

L2(X) =

f : X → F :

∫

X

|f(x)|2 dµ(x) < ∞

,

where we identify functions that are equal almost everywhere, i.e.,

f = g a.e. ⇐⇒ µx ∈ X : f(x) 6= g(x) = 0.

Then

〈f, g〉 =

∫

X

f(x) g(x)dµ(x)

defines an inner product on L2(X).Other notations for L2(X) are L2(µ), L2(X, µ), L2(dµ), L2(X, dµ), etc.The space `2

w(I) is a special case of L2(X), where X = I and µ is a weighted countingmeasure on I.

CHAPTER 1. HILBERT SPACES 3

Exercise 1.9. Every subspace of an inner product space is itself an inner product space(using the same inner product).

Hence every subspace of `2w or L2(X) is also an inner product space. For example,

V = L2(Rn) ∩ C(Rn) = f ∈ L2(Rn) : f is continuous

is a subspace of L2(Rn) and hence is also an inner product space. List some other subspacesof some particular `2

w or L2(X) spaces.

Notation 1.10 (Associated Semi-Norm or Norm). If 〈·, ·〉 is a semi-inner product on X,then we will write

‖x‖ = 〈x, x〉1/2.

We will see later that ‖ · ‖ defines a semi-norm on X, and that it is a norm on X if 〈·, ·〉 isan inner product on X.

Lemma 1.11 (Polar Identity). If 〈·, ·〉 is a semi-inner product on X, then for all x, y ∈ Xwe have

‖x + y‖2 = ‖x‖2 + 2 Re 〈x, y〉 + ‖y‖2.

Proof. Using the fact that z + z = 2Re z, we have

‖x + y‖2 = 〈x + y, x + y〉 = ‖x‖2 + 〈x, y〉 + 〈y, x〉 + ‖y‖2

= ‖x‖2 + 2 Re 〈x, y〉 + ‖y‖2.

Theorem 1.12 (Cauchy-Bunyakowski-Schwarz Inequality). If 〈·, ·〉 is a semi-inner producton X, then

∀ x, y ∈ X, |〈x, y〉| ≤ ‖x‖ ‖y‖.

Moreover, equality holds if and only if there exist scalars α, β ∈ F, not both zero, such that‖αx + βy‖ = 0.

Proof. If x, y ∈ X and α ∈ F, then

0 ≤ ‖x − αy‖2 = 〈x − αy, x − αy〉

= 〈x, x〉 − α〈y, x〉 − α〈x, y〉 + αα〈y, y〉

= ‖x‖2 − α〈y, x〉 − α〈x, y〉 + |α|2 ‖y‖2.

Write 〈y, x〉 = beiθ where b ≥ 0 and θ ∈ R (θ = 0 if F = R). Set α = e−iθt where t ∈ R.Then for this α we have

0 ≤ ‖x‖2 − e−iθtbeiθ − eiθtbe−iθ + t2 ‖y‖2 = ‖x‖2 − 2bt + ‖y‖2 t2.

This is a quadratic polynomial in t. In order for this polynomial to be everywhere nonneg-ative, it can have at most one real root, which means that the discriminant must be ≤ 0,i.e.,

(−2b)2 − 4 ‖y‖2 ‖x‖2 ≤ 0.

4 CHRISTOPHER HEIL

Hence

|〈x, y〉|2 = |〈y, x〉|2 = b2 ≤ ‖x‖2 ‖y‖2.

Exercise: Supply the proof for the case of equality.

Corollary 1.13. If 〈·, ·〉 is a semi-inner product on X, then ‖x‖ = 〈x, x〉1/2 defines a semi-norm on X, which means that:

(a) ‖x‖ ≥ 0 for all x ∈ X,

(b) ‖αx‖ = |α| ‖x‖ for all x ∈ X and α ∈ F,

(c) Triangle Inequality: ‖x + y‖ ≤ ‖x‖ + ‖y‖ for all x, y ∈ X.

If 〈·, ·〉 is an inner product on X then ‖ · ‖ is a norm on X, which means that in additionto (a)–(c) above, we also have:

(d) ‖x‖ = 0 =⇒ x = 0.

Proof. (a), (b), (d) Exercises.

(c) Using the Polar Identity, we have

‖x + y‖2 = 〈x + y, x + y〉 = ‖x‖2 + 2 Re 〈x, y〉 + ‖y‖2

≤ ‖x‖2 + 2 |〈x, y〉|+ ‖y‖2

≤ ‖x‖2 + 2 ‖x‖ ‖y‖+ ‖y‖2

=(

‖x‖ + ‖y‖)2

.

Definition 1.14 (Distance). Let 〈·, ·〉 be an inner product on X. Then the distance from xto y in X is

d(x, y) = ‖x − y‖.

Exercise 1.15. d(·, ·) defines a metric on X.

Definition 1.16. Many of the results in this chapter are valid not only for inner productspaces, but for any space which possesses a norm. Assume that X is a vector space. Asemi-norm on X is a function ‖ · ‖ : X → [0,∞) such that statements (a)-(c) above hold.If, in addition, statement (d) holds, then ‖ · ‖ is a norm, and X is called a normed space,normed linear space, or normed vector space.

Definition 1.17 (Convergence). Let X be a normed linear space (such as an inner productspace), and let fn

∞n=1 be a sequence of elements of X.


(a) We say that fn converges to f ∈ X, and write fn → f , if

limn→∞

‖f − fn‖ = 0,

i.e.,

∀ ε > 0, ∃N > 0 such that n > N =⇒ ‖f − fn‖ < ε.

(b) We say that fn is Cauchy if

∀ ε > 0, ∃N > 0 such that m, n > N =⇒ ‖fm − fn‖ < ε.

Exercise 1.18. Let X be a normed linear space. Prove the following.

(a) Reverse Triangle Inequality:∣

∣‖x‖ − ‖y‖∣

∣ ≤ ‖x − y‖.

(b) Continuity of the norm: xn → x =⇒ ‖xn‖ → ‖x‖.

(c) Continuity of the inner product: If X is an inner product space then

xn → x, yn → y =⇒ 〈xn, yn〉 → 〈x, y〉.

(d) All convergent sequences are bounded, and the limit of a convergent sequence is unique.

(e) Cauchy sequences are bounded.

(f) Every convergent sequence is Cauchy.

(g) There exist inner product spaces for which not every Cauchy sequence is convergent.

Definition 1.19 (Hilbert Space). An inner product space H is called a Hilbert space if it iscomplete, i.e., if every Cauchy sequence is convergent. That is,

fn∞n=1 is Cauchy in H =⇒ ∃ f ∈ H such that fn → f.

The letter H will always denote a Hilbert space.

Definition 1.20 (Banach Space). A normed linear space X is called a Banach space if it iscomplete, i.e., if every Cauchy sequence is convergent. We make no assumptions about themeaning of the symbol X, i.e., it need not denote a Banach space. A Hilbert space is thusa Banach space whose norm is associated with an inner product.

Example 1.21. Cn, `2w (w strictly positive), and L2(X) are all Hilbert spaces (using the

default inner products).

Exercise 1.22. Prove that Cn and `2w are Hilbert spaces.

6 CHRISTOPHER HEIL

Definition 1.23 (Convergent Series). Let fn∞n=1 be a sequence of elements of a normed

linear space X. Then the series∑∞

n=1 fn converges and equals f ∈ X if the partial sums

sN =∑N

n=1 fn converge to f . That is,

‖f − sN‖ =∥

∥

∥f −

N∑

n=1

fn

∥

∥

∥→ 0 as N → ∞.

Exercise 1.24. Let X be an inner product space, and suppose that the series∑∞

n=1 fn

converges in X. Show that if g ∈ X, then

⟨

∞∑

n=1

fn, g⟩

=∞

∑

n=1

〈fn, g〉.

Note that this is NOT an immediate consequence of the definition of inner product. You mustuse both the fact that the inner product is linear in the first variable AND the continuity ofthe inner product (Exercise 1.18) to prove this result.

Exercise 1.25 (Absolutely Convergent Series). Let X be a Banach space and let fn∞n=1

be a sequence of elements of X. Prove that if∑∞

n=1 ‖fn‖ < ∞ then the series∑∞

n=1 fn

converges in X. We say that such a series is absolutely convergent.Hint: You must show that the sequence of partial sums sN converges. Since X is a

Banach space, you just have to show that this sequence is Cauchy.

Exercise 1.26 (Unconditionally Convergent Series). Let X be a Banach space and letfn

∞n=1 be a sequence of elements of X. The series f =

∑∞n=1 fn is said to converge uncon-

ditionally if every rearrangement of the series converges. That is, f =∑∞

n=1 fn convergesunconditionally if for each bijection σ : N → N the series

∞∑

n=1

fσ(n)

converges. It can be shown that in this case, the series will converge to f , i.e., f =∑∞

n=1 fσ(n)

for every permutation σ.Prove that if a series f =

∑∞n=1 fn converges absolutely, then it converges unconditionally.

In finite dimensions the converse is true, but we will see later that the converse fails in infinitedimensions (see Exercise 4.16).

Proposition 1.27. Let X be a Banach space and let fn∞n=1 be a sequence of elements of

X. Then the series f =∑∞

n=1 fn converges unconditionally if and only if it converges withrespect to the net of finite subsets of N, i.e., if

∀ ε > 0, ∃ finite F0 ⊆ N such that ∀ finite F ⊇ F0,∥

∥

∥f −

∑

n∈F

fn

∥

∥

∥< ε.


Definition 1.28 (Topology). Let X be a normed linear space.

(a) The open ball in X centered at x ∈ X with radius r > 0 is

B(x, r) = Br(x) = y ∈ X : ‖x − y‖ < r.

(b) A subset U ⊆ X is open if

∀ x ∈ U, ∃ r > 0 such that B(x, r) ⊆ U.

(c) A subset F ⊆ X is closed if X \ F is open.

Definition 1.29 (Limit Points, Closure, Density). Let X be a normed linear space and letA ⊆ X.

(a) A point f ∈ A is called a limit point of A if there exist fn ∈ A with fn 6= f such thatfn → f .

(b) The closure of A is the smallest closed set A such that A ⊆ A. Specifically,

A =⋂

F ⊆ X : F is closed and F ⊇ A.

(c) We say that A is dense in X if A = X.

Exercise 1.30. (a) The closure of A equals the union of A and all limit points of A:

A = A ∪ x ∈ X : x is a limit point of A = z ∈ X : ∃ yn ∈ A such that yn → z.

(b) If X is a normed linear space, then the closure of an open ball B(x, r) is the closed

ball B(x, r) = y ∈ X : ‖x − y‖ ≤ r.

(c) Prove that A is dense if and only if

∀ x ∈ X, ∀ ε > 0, ∃ y ∈ A such that ‖x − y‖ < ε.

Example 1.31. The set of rationals Q is dense in the real line R.

Proposition 1.32. Let X be a normed linear space and let F ⊆ X. Then

F is closed ⇐⇒ F contains all its limit points.

Proof. ⇒. Suppose that F is closed but that there exists a limit point f that does not belongto F . By definition, there must exist fn ∈ F such that fn → f . However, f ∈ X \ F , whichis open, so there exists some r > 0 such that B(f, r) ⊆ X \F . Yet there must exist some fn

with ‖f − fn‖ < r, so this fn will belong to X \ F , which is a contradiction.

⇐. Exercise.

8 CHRISTOPHER HEIL

Exercise 1.33. In finite dimensions, all subspaces are closed sets. This is not true in infinitedimensions.

(a) Prove that

c00 = x = (x1, . . . , xN , 0, 0, . . . ) : N > 0, x1, . . . , xN ∈ F

is a subspace of `2(N) that is not closed. Prove that c00 is dense in `2(N).

(b) Prove thatc0 = x = (xk)

∞k=1 : lim

k→∞xk = 0

is a dense subspace of `2(N).

(c) Prove that Cc(R), the space of continuous, compactly supported functions on R, is asubspace of L2(R) that is dense and not closed. The support of a function f : R → C isthe closure in R of the set x ∈ R : f(x) 6= 0. Thus a function is compactly supported ifnonzero only within a bounded subset of R.

(d) Let E be a (Lebesgue) measurable subset of Rn. Let M = f ∈ L2(Rn) : supp(f) ⊆E. Prove that M is a closed subspace of L2(Rn).

Proposition 1.34. Let H be a Hilbert space and let M be a subspace of H. Then M isitself a Hilbert space (using the inner product from H) if and only if M is closed.

Let X be a Banach space and let M be a subspace of X. Then M is itself a Banach space(using the norm from X) if and only if M is closed.

Proof. ⇒. Suppose that M is a Hilbert space. Let f be any limit point of M , i.e., supposefn ∈ M and fn → f . Then fn is a convergent sequence in H, and hence is Cauchy inH. Since each fn belongs to M , it is also Cauchy in M . Since M is a Hilbert space, fnmust therefore converge in M , i.e., fn → g for some g ∈ M . However, limits are unique, sof = g ∈ M . Therefore M contains all its limit points and hence is closed.

⇐. Exercise.

Exercise 1.35. Find examples of inner product spaces that are not Hilbert spaces.

Remark 1.36. In constructing the examples for the preceding exercise, you will probablylook for examples of inner product spaces X which are subspaces of a larger Hilbert spaceH. Is every inner product space a subspace of a larger Hilbert space? The answer is yes, itis also possible to construct a Hilbert space H ⊇ X, called the completion of X.

Theorem 1.37 (Parallelogram Law). If X is an inner product space, then

∀ f, g ∈ X, ‖f + g‖2 + ‖f − g‖2 = 2(

‖f‖2 + ‖g‖2)

.

Proof. Exercise.


Exercise 1.38. Show that `p for p 6= 2 and Lp(R) for p 6= 2 are not inner product spacesunder the default norms (show that the Parallelogram Law fails).

Exercise 1.39. Suppose that X is a Banach space over the complex field C, and the norm‖ · ‖ of X satisfies the Parallelogram Law. Prove that

〈f, g〉 =1

4

(

‖f + g‖2 − ‖f − g‖2 + i‖f + ig‖2 − i‖f − ig‖2)

is an inner product on X, and that ‖f‖2 = 〈f, f〉.

2. Orthogonality

Definition 2.1. Let H be a Hilbert space.

(a) Two vectors f , g ∈ H are orthogonal (written f ⊥ g) if 〈f, g〉 = 0.

(b) A sequence of vectors fii∈I is an orthogonal sequence if 〈fi, fj〉 = 0 whenever i 6= j.

(c) A sequence of vectors fii∈I is an orthonormal sequence if it is orthogonal and eachvector is a unit vector, i.e.,

〈fi, fj〉 = δij =

1, i = j,

0, i 6= j.

Exercise 2.2. (a) If fii∈I is an orthogonal sequence of nonzero vectors, then it is finitelylinearly independent, i.e., every finite subset is linearly independent.

(b) Let I = N. Define en = (δmn)∞m=1, i.e., en is the sequence having a 1 in the nthcomponent and zeros elsewhere. Show that en

∞n=1 is an orthonormal sequence in `2 (called

the standard basis of `2).

(c) Let X = [0, 1] and let µ be Lebesgue measure. Define en(x) = e2πinx. Show thatenn∈Z is an orthonormal sequence in L2[0, 1].

Theorem 2.3 (Pythagorean Theorem). If f1, . . . , fn ∈ H are orthogonal, then

∥

∥

∥

n∑

k=1

fk

∥

∥

∥

2

=

n∑

k=1

‖fk‖2.

Proof. Exercise.

Definition 2.4 (Convex Set). Let X is a vector space and K ⊆ X, then K is convex if

x, y ∈ K, 0 ≤ t ≤ 1 =⇒ tx + (1 − t)y ∈ K.

Thus, the entire line segment between x and y is contained in K (including the midpoint12x + 1

2y in particular).

10 CHRISTOPHER HEIL

Exercise 2.5. (a) Every subspace of a vector space X is convex.

(b) If X is a normed linear space, then open and closed balls in X are convex.

Definition 2.6. Let X be a normed linear space and let A ⊆ H. The distance from a pointx ∈ H to the set A is

dist(x, A) = inf‖x − y‖ : y ∈ A.

Theorem 2.7 (Closest Point Property). If H is a Hilbert space and K is a nonempty closed,convex subset of H, then given any h ∈ H there exists a unique point k0 ∈ K that is closestto h. That is, there is a unique point k0 ∈ K such that

‖h − k0‖ = dist(h, K) = inf‖h − k‖ : k ∈ K.

Proof. Letd = dist(h, K) = inf‖h − k‖ : k ∈ K.

By definition, there exist kn ∈ K such that ‖h − kn‖ → d, and furthermore we have d ≤‖h − kn‖ for each n. Therefore, if we fix any ε > 0 then we can find an N such that

n > N =⇒ d2 ≤ ‖h − kn‖2 ≤ d2 + ε2.

By the Parallelogram Law,

‖(h − kn) − (h − km)‖2 + ‖(h − kn) + (h − km)‖2 = 2(

‖h − kn‖2 + ‖h − km‖

2)

.

Hence,∥

∥

∥

km − kn

2

∥

∥

∥

2

=1

4‖(h − kn) − (h − km)‖2 =

‖h − kn‖2

2+

‖h − km‖2

2−

∥

∥

∥h −

km + kn

2

∥

∥

∥

2

.

However, km+kn

2∈ K since K is convex, so ‖h − km+kn

2‖ ≥ d and therefore

−∥

∥

∥h −

km + kn

2

∥

∥

∥

2

≤ −d2.

Also, if m, n > N then‖h − kn‖

2, ‖h − km‖2 ≤ d2 + ε2.

Therefore,∥

∥

∥

km − kn

2

∥

∥

∥

2

≤d2 + ε2

2+

d2 + ε2

2− d2 = ε2.

So, ‖km − kn‖ ≤ 2ε for all m, n > N , which says that the sequence kn is Cauchy. SinceH is complete, this sequence must converge, i.e., kn → k0 for some k0 ∈ H. But kn ∈ K forall n and K is closed, so we must have k0 ∈ K.

Since h − kn → h − k0, we have

‖h − k0‖ = limn→∞

‖h − kn‖ = d,

and thus ‖h − k0‖ ≤ ‖h − k‖ for every k ∈ K.Exercise: Prove that k0 is the unique point closest to h by assuming that there exists

another point k′0 that satisfies ‖h−k′

0‖ ≤ ‖h−k‖ for every k ∈ K, and derive a contradiction.

Draw a picture and think about the midpointk0+k′

0

2.


Notation 2.8 (Notation for Closed Subspaces). Since we will often deal with closed sub-spaces of a Hilbert space, we declare that the notation

M ≤ H

means that M is a closed subspace of the Hilbert space H. The letter H will always denotea Hilbert space.

Theorem 2.9. Let M ≤ H, and fix h ∈ H. Then the following statements are equivalent.

(a) h = p + e where p is the (unique) point in M closest to h.

(b) h = p + e where p ∈ M and e ⊥ M (i.e., e ⊥ f for every f ∈ M).

Proof. (a) ⇒ (b). Let p be the (unique) point in M closest to h, and let e = p − h. Chooseany f ∈ M . We must show that 〈f, e〉 = 0.

Since M is a subspace, p + λf ∈ M for any scalar λ ∈ F. Hence,

‖h − p‖2 ≤ ‖h − (p + λf)‖2 = ‖(h − p) − λf‖2

= ‖h − p‖2 − 2 Re 〈λf, h − p〉 + |λ|2 ‖f‖2

= ‖h − p‖2 − 2 Reλ〈f, e〉 + |λ|2 ‖f‖2.

Therefore,

∀λ ∈ F, 2 Reλ〈f, e〉 ≤ |λ|2 ‖f‖2.

If we consider λ = t > 0, then we can divide through by t to get

∀ t > 0, 2 Re 〈f, e〉 ≤ t ‖f‖2.

Letting t → 0+, we conclude that Re 〈f, e〉 ≤ 0. Similarly, taking λ = t < 0 and lettingt → 0−, we obtain Re 〈f, e〉 ≥ 0.

If F = R then we are done. If F = C, then we take λ = it with t > 0 and λ = it witht < 0 to show that Im 〈f, e〉 = 0 as well.

(b) ⇒ (a). Suppose that h = p + e where p ∈ M and e ⊥ M . Choose any f ∈ M . Thenp − f ∈ M , so h − p = e ⊥ p − f . Therefore, by the Pythagorean Theorem,

‖h − f‖2 = ‖(h − p) + (p − f)‖2 = ‖h − p‖2 + ‖p − f‖2 ≥ ‖h − p‖2.

Hence p is the point in M that is closest to h.

Definition 2.10 (Orthogonal Projection). Let M ≤ H.

(a) For each h ∈ H, the point p ∈ M closest to H is called the orthogonal projection ofh onto M .

(b) Define P : H → H by Ph = p, the orthogonal projection of h onto M . Then theoperator P is called the orthogonal projection of H onto M . Note that by definitionand Theorem 2.9, h = Ph + e where e = h − Ph ⊥ M .

12 CHRISTOPHER HEIL

Definition 2.11 (Orthogonal Complement). Let A be a subset (not necessarily a subspace)of a Hilbert space H. The orthogonal complement of A is

A⊥ = x ∈ H : x ⊥ A = x ∈ H : 〈x, y〉 = 0 ∀ y ∈ A.

Exercise 2.12. (a) If A ⊆ H, then A⊥ is a closed subspace of H (even if A itself is not).

(b) 0⊥ = H and H⊥ = 0.

Definition 2.13 (Notation for Operators). Let X, Y be normed linear spaces. Let T : X →Y be a function (= operator = transformation). We write either T (x) or Tx to denote theimage of an element x ∈ X.

(a) T is linear if T (αx + βy) = αT (x) + βT (y) for every x, y ∈ X and α, β ∈ F.

(b) T is injective if T (x) = T (y) implies x = y.

(c) The kernel or nullspace of T is ker(T ) = x ∈ X : T (x) = 0.

(c) The range of T is range(T ) = T (x) : x ∈ X.

(d) The rank of T is the dimension of its range, i.e., rank(T ) = dim(range(T )). Inparticular, T is finite-rank if range(T ) is finite-dimensional.

(d) T is surjective if range(T ) = Y .

(e) T is a bijection if it is both injective and surjective.

(f) We use the notation I or IH to denote the identity map of a Hilbert space H ontoitself.

Exercise 2.14. Show that if T : X → Y is linear and continuous, then ker(T ) is a closedsubspace of X and that range(T ) is a subspace of Y . Must range(T ) be a closed subspace?

Theorem 2.15 (Properties of Orthogonal Projections). Let M ≤ H, and let P be theorthogonal projection of H onto M . Then the following statements hold.

(a) h − Ph ⊥ M for every h ∈ H.

(b) h − Ph ∈ M⊥ for every h ∈ H.

(c) ‖h − Ph‖ = dist(h, M) for every h ∈ H.

(d) P is a linear transformation of H into itself.

(e) ‖Ph‖ ≤ ‖h‖ for every h ∈ H.

(f) P is idempotent, i.e., P 2 = P .

(g) ker(P ) = M⊥.

(h) range(P ) = M .

(i) I − P is the orthogonal projection of H onto M⊥.


Proof. (a) Follows from the definition of orthogonal projection and Theorem 2.9.

(b) Follows from (a) and the definition of M⊥.

(e) Choose any h ∈ H. Then h = Ph + e with Ph ∈ M and e ∈ M⊥. Hence Ph ⊥ e, soby the Pythagorean Theorem, ‖h‖2 = ‖Ph‖2 + ‖e‖2 ≥ ‖Ph‖2.

(g) Suppose that h ∈ ker(P ), i.e., Ph = 0. Then h = h − Ph ⊥ M , so h ∈ M⊥.Conversely, suppose that h ∈ M⊥. Then h = 0 + h with 0 ∈ M and h ∈ M⊥, so we must

have Ph = 0 and hence h ∈ ker(P ).

The remaining parts are left as exercises.

Corollary 2.16 (Double Complements). If M ≤ H, then (M⊥)⊥ = M .

Proof. Choose any x ∈ M . Then 〈x, y〉 = 0 for every y ∈ M⊥. Thus x ⊥ M⊥, so x ∈ (M⊥)⊥.Conversely, suppose that x ∈ (M⊥)⊥. Since M is closed, we can write x = p + e with

p ∈ M and e ∈ M⊥. Since x ∈ (M⊥)⊥ we have 〈x, e〉 = 0. But we also have p ∈ M ⊆ (M⊥)⊥,so 〈e, p〉 = 0. Therefore ‖e‖2 = 〈e, e〉 = 〈x − p, e〉 = 〈x, e〉 − 〈p, e〉 = 0. Hence e = 0, sox = p ∈ M .

Definition 2.17 (Span, Closed Span). Let A be a subset of a normed linear space X.

(a) The finite span (or linear span or just the span) of A, denoted span(A), is the set ofall finite linear combinations of elements of A:

span(A) =

n∑

k=1

αkxk : n > 0, xk ∈ A, αk ∈ F

.

In particular, if A is a countable sequence, say A = xk∞k=1, then

span(xk∞k=1) =

n∑

k=1

αkxk : n > 0, αk ∈ F

.

(b) The closed finite span (or closed linear span or closed span) of A, denoted span(A)or ∨A, is the closure of the set of all finite linear combinations of elements of A:

∨A = span(A) = span(A) = z ∈ X : ∃ yn ∈ span(A) such that yn → z.

Beware: This does NOT say that

span(A) =

∞∑

k=1

αkxk : xk ∈ A, αk ∈ F

.

It does not even imply that every element of span(A) can be written x =∑∞

k=1 αkxk

for some xk ∈ A, αk ∈ F. What does it say about span(A)? If we consider the caseof a countable sequence A = xk

∞k=1, then we have

span(xk∞k=1) =

z ∈ X : ∃αk,n ∈ F such thatn

∑

k=1

αk,nxk → z as n → ∞

.

14 CHRISTOPHER HEIL

That is, an element z lies in the closed span if there exist αk,n ∈ F such thatn

∑

k=1

αk,nxk → z as n → ∞.

In contrast, to say that x =∑∞

k=1 αkxk means thatn

∑

k=1

αkxk → x as n → ∞,

i.e., the scalars αk must be independent of n.

(c) If X is a Banach space then we say that a subset A ⊆ X is complete if span(A) isdense in X, or equivalently, if span(A) = X.

Beware of the two different meanings that we have assigned to the word complete!

Exercise 2.18. Let en∞n=1 be the standard basis for `2(N). Prove that c00 = span(en

∞n=1)

and conclude that en∞n=1 is complete in `2(N).

Exercise 2.19. Let A be any subset of a Hilbert space H.

(a) (A⊥)⊥ = span(A).

(b) A is complete if and only if A⊥ = 0.

Note that, by definition, A⊥ = 0 is equivalent to the statement that x ⊥ A =⇒ x = 0.Thus, this exercise says that A is complete if and only if the only element orthogonal to everyelement of A is 0.

3. The Riesz Representation Theorem

Definition 3.1 (Continuous and Bounded Operators). Let X, Y be normed linear spaces,and let L : X → Y be a linear operator.

(a) L is continuous at a point x ∈ X if xn → x in X implies Lxn → Lx in Y .

(b) L is continuous if it is continuous at every point, i.e., if xn → x in X impliesLxn → Lx in Y for every x.

(c) L is bounded if there exists a finite K ≥ 0 such that

∀ x ∈ X, ‖Lx‖ ≤ K ‖x‖.

Note that ‖Lx‖ is the norm of Lx in Y , while ‖x‖ is the norm of x in X.

(d) The operator norm of L is

‖L‖ = sup‖x‖=1

‖Lx‖.


(e) We let B(X, Y ) denote the set of all bounded linear operators mapping X into Y ,i.e.,

B(X, Y ) = L : X → Y : L is bounded and linear.

If X = Y = X then we write B(X) = B(X, X).

(f) If Y = F then we say that L is a functional. The set of all bounded linear functionalson X is the dual space of X, and is denoted

X ′ = B(X, F) = L : X → F : L is bounded and linear.

Exercise 3.2. Let X, X, Y be normed linear spaces. Let L : X → Y be a linear operator.

(a) L is injective if and only if ker L = 0.

(b) L is bounded if and only if ‖L‖ < ∞.

(c) If L is bounded then ‖Lx‖ ≤ ‖L‖ ‖x‖ for every x ∈ X (note that three differentmeanings of the symbol ‖ · ‖ appear in this statement!).

(d) If L is bounded then ‖L‖ is the smallest value of K such that ‖Lx‖ ≤ K‖x‖ holdsfor all x ∈ X.

(e) ‖L‖ = sup‖x‖≤1

‖Lx‖ = supx6=0

‖Lx‖

‖x‖.

(f) B(X, Y ) is a subspace of the vector space V containing ALL functions A : X → Y .Moreover, the operator norm is a norm on the space B(X, Y ), i.e.,

i. ‖L‖ ≥ 0 for all L ∈ B(X, Y ).

ii. ‖L‖ = 0 if and only if L = 0 (the zero operator that sends every element of Xto the zero vector in Y ).

iii. ‖αL‖ = |α| ‖L‖ for every L ∈ B(X, Y ) and every α ∈ F.

iv. ‖L + K‖ ≤ ‖L‖ + ‖K‖ for every L, K ∈ B(X, Y ).

(g) Part (f) shows that B(X, Y ) is itself a normed linear space, and we will see later thatit is a Banach space if Y is a Banach space. However, in addition to vector additionand scalar multiplication operations, there is a third operation that we can performwith functions: composition. Prove that the operator norm is submultiplicative, i.e.,if A ∈ B(X, Y ) and B ∈ B(Y, Z), then BA ∈ B(X, Z), and furthermore,

‖BA‖ ≤ ‖B‖ ‖A‖. (3.1)

In particular, when X = Y = Z, we see that B(X) is closed under composition. Thespace B(X) is an example of an algebra.

(h) If L ∈ X ′ = B(X, F) then ‖L‖ = sup‖x‖=1

|Lx|.

Exercise 3.3. Let M ≤ H, and let P be the operator of orthogonal projection onto M .Find ‖P‖.

16 CHRISTOPHER HEIL

Exercise 3.4. Let X, Y be normed linear spaces. Prove that if L : X → Y is linear andX is finite-dimensional (take Y = Cn or Rn if you like), then L is bounded. Hint: Ifx = (x1, . . . , xn) ∈ Cn then x = x1e1 + · · · + xnen where e1, . . . , en is the standard basisfor Cn. Use the Triangle Inequality. If X is any finite-dimensional vector space, do the samebut with an arbitrary basis for X, and use the fact (that we will prove later) that all normson a finite-dimensional space are equivalent.

The next lemma is a standard fact about continuous functions. L−1(U) denotes the inverseimage of U ⊆ Y , i.e., L−1(U) = x ∈ X : Lx ∈ U.

Lemma 3.5. Let X, Y be normed linear spaces. Let L : X → Y be linear. Then

L is continuous ⇐⇒(

U open in Y =⇒ L−1(U) open in X)

.

Proof. ⇒. Suppose that L is continuous and that U is an open subset of Y . We will showthat X \ L−1(U) is closed by showing that it contains all its limit points.

Suppose that x is a limit point of X \L−1(U). Then there exist xn ∈ X \L−1(U) such thatxn → x. Since L is continuous, this implies Lxn → Lx. However, xn /∈ L−1(U), so Lxn /∈ U ,i.e., Lxn ∈ Y \ U , which is a closed set. Therefore Lx ∈ Y \ U , and hence x ∈ X \ L−1(U).Thus X \ L−1(U) is closed, so L−1(U) is open.

⇐. Exercise.

Theorem 3.6 (Equivalence of Bounded and Continuous Linear Operators). Let X, Y benormed linear spaces, and let L : X → Y be a linear mapping. Then the following statementsare equivalent.

(a) L is continuous at some x ∈ X.

(b) L is continuous at x = 0.

(c) L is continuous.

(d) L is bounded.

Proof. (c) ⇒ (d). Suppose that L is continuous but unbounded. Then ‖L‖ = ∞, sothere must exist xn ∈ X with ‖xn‖ = 1 such that ‖Lxn‖ ≥ n. Set yn = xn/n. Then‖yn − 0‖ = ‖yn‖ = ‖xn‖/n → 0, so yn → 0. Since L is continuous and linear, this impliesLyn → L0 = 0, and therefore ‖Lyn‖ → ‖0‖ = 0. But

‖Lyn‖ =1

n‖Lxn‖ ≥

1

n· n = 1

for all n, which is a contradiction. Hence L must be bounded.

(d) ⇒ (c). Suppose that L is bounded, so ‖L‖ < ∞. Suppose that x ∈ X and thatxn → x. Then ‖xn − x‖ → 0, so

‖Lxn − Lx‖ = ‖L(xn − x)‖ ≤ ‖L‖ ‖xn − x‖ → 0,

i.e., Lxn → Lx. Thus L is continuous.

The remaining implications are left as exercises.


Exercise 3.7. Let X, Y be normed linear spaces. Suppose that L : X → Y is linear andsatisfies ‖Lx‖ = ‖x‖ for all x ∈ X. Such an operator is said to be an isometry or norm-preserving. Prove that L is injective and find ‖L‖. Find an example of an isometry that isnot surjective. Contrast this with the fact that if A : Cn → Cn is linear, then A is injectiveif and only if it is surjective.

Exercise 3.8. (a) Define L : `2(N) → `2(N) by L(x) = (x2, x3, . . . ) for x = (x1, x2, . . . ) ∈`2(N). Prove that this left-shift operator is bounded, linear, surjective, not injective, and isnot an isometry. Find ‖L‖.

(b) Define R : `2(N) → `2(N) by R(x) = (0, x1, x2, x3, . . . ) for x = (x1, x2, . . . ) ∈ `2(N).Prove that this right-shift operator is bounded, linear, injective, not surjective, and is anisometry. Find ‖R‖.

(c) Compute LR and RL. Contrast this computation with the fact that in finite dimen-sions, if A, B : Cn → Cn are linear maps (hence correspond to multiplication by n × nmatrices), then AB = I implies BA = I and conversely.

Exercise 3.9. Let X be a Banach space and Y a normed linear space. Suppose thatL : X → Y is bounded and linear. Prove that if there exists c > 0 such that ‖Lx‖ ≥ c‖x‖for all x ∈ X, then L is injective and range(L) is closed.

Exercise 3.10. For each h ∈ H, define a linear functional Lh : H → F by

Lh(x) = 〈x, h〉, x ∈ H.

Prove the following.

(a) Lh is linear.

(b) ker(Lh) = h⊥.

(c) ‖Lh‖ = ‖h‖, so Lh is a bounded linear functional on H. Therefore Lh ∈ H ′ =B(H, F).

Thus, each element h of H determines an element Lh of H ′. In the Riesz RepresentationTheorem we will prove that these are all the elements of H ′.

Example 3.11 (Dual Space of Cn). Consider H = Cn (with F = C). Let h = (h1, . . . , hn)T ∈H = Cn (think of h as a column vector). Then Lh(x) = x · h = x1h1 + · · · + xnhn defines abounded linear functional on Cn.

Suppose that L : Cn → C is any linear functional on Cn (it will necessarily be boundedsince Cn is finite-dimensional, see Exercise 3.4). Any linear mapping from Cn to Cm is givenby multiplication by an m × n matrix. So in this case there must be a 1 × m matrix (i.e., arow vector) A =

[

a1 · · · an

]

such that

Lx = Ax =[

a1 · · · an

]

x1...

xn

= x1a1 + · · ·+ xnan = x · h = Lh(x)

18 CHRISTOPHER HEIL

where h = (a1, . . . , an)T.

Thus we see that every element of (Cn)′ is an Lh for some h ∈ Cn.

Exercise 3.12. Define T : Cn → (Cn)′ by

T (h) = Lh, h ∈ Cn.

Prove directly that T is anti-linear, i.e.,

T (αh + βk) = αT (h) + βT (k),

and that T is an isometry, i.e., ‖T (h)‖ = ‖h‖ for each h. All isometries are injective, andin the preceding example we showed that T is surjective, so T is an anti-linear isometricbijection of Cn onto (Cn)′.

If we define S : Cn → (Cn)′ by S(h) = Lh, then S a linear isometric bijection of Cn onto(Cn)′. Such an S is called an isomorphism, and thus Cn and (Cn)′ are isomorphic.

Theorem 3.13 (Riesz Representation Theorem). For each h ∈ H, define Lh : H → F by

Lh(x) = 〈x, h〉, x ∈ H.

(a) For each h ∈ H, we have Lh ∈ H ′ and ‖Lh‖ = ‖h‖.

(b) If L ∈ H ′, then there exists a unique h ∈ H such that L = Lh.

(c) The mapping T : H → H ′ defined by T (h) = Lh is an anti-linear isometric bijectionof H onto H ′.

Proof. (a) See Exercise 3.10.

(b) Choose any L ∈ H ′. If L = 0 then h = 0 is the required vector, so assume that L 6= 0(i.e., L is not the zero operator). Then L does not map every vector to zero, so the kernelof L is a closed subspace of H that is not all of H.

Choose any z /∈ ker(L), and write

z = p + e, p ∈ ker(L), e ∈ ker(L)⊥.

Note that L(p) = 0 by definition, and therefore we must have L(e) 6= 0 (since L(z) 6= 0).Set u = e/L(e), and note that u ∈ ker(L)⊥ and L(u) = 1.

Given x ∈ H, since L is linear and L(u) = 1 we have

L(

x − L(x) u)

= L(x) − L(x) L(u) = 0.

Therefore x − L(x) u ∈ ker(L). However, u ⊥ ker(L), so

0 = 〈x − L(x) u, u〉 = 〈x, u〉 − 〈L(x) u, u〉 = 〈x, u〉 − L(x) ‖u‖2.

Hence,

L(x) =1

‖u‖2〈x, u〉 =

⟨

x,u

‖u‖2

⟩

= 〈x, h〉 = Lh(x)

whereh =

u

‖u‖2.


Thus L = Lh, and from part (a), we have that ‖L‖ = ‖Lh‖ = ‖h‖.It remains only to show that h is unique. Suppose that we also had L = Lh′ . Then for

every x ∈ H we have

〈x, h − h′〉 = 〈x, h〉 − 〈x, h′〉 = Lh(x) − Lh′(x) = L(x) − L(x) = 0.

Consequently, h − h′ = 0.

(c) Parts (a) and (b) show that T is surjective and that T is norm-preserving. Therefore,we just have to show that T is anti-linear.

Let h ∈ H and let c ∈ F. We must show that T (ch) = cT (h), i.e., that Lch = cLh. Thisfollows immediately from the fact that for each x ∈ H, we have

Lch(x) = 〈x, ch〉 = c 〈x, h〉 = c Lh(x).

The proof that Lh+k = T (h + k) = T (h) + T (k) = Lh + Lk is left as an exercise.

Corollary 3.14. If L : `2(I) → F is a bounded linear functional, then there exists a uniqueh = (hi)i∈I ∈ `2(I) such that

L(x) =∑

i∈I

xi hi, x = (xk)i∈I ∈ `2(I).

Corollary 3.15. If (X, Ω, µ) is a measure space and L : L2(X) → F is a bounded linearfunctional, then there exists a unique h ∈ L2(X) such that

L(f) =

∫

X

f(x) h(x) dµ(x), f ∈ L2(X).

4. Orthonormal Sets of Vectors and Bases

Definition 4.1 (Hamel Basis). Let V be a vector space. We say that fii∈I is a basis forV if it is both finitely linearly independent and its finite span is all of V . That is, everyvector in V equals a unique finite linear combination of the fi (aside from trivial zero terms),

i.e., every f 6= 0 can be written f =∑N

k=1 ckfik for a unique choice of indices i1, . . . , iN andnonzero scalars c1, . . . , cN . Because the word “basis” is heavily overused, we shall refer tosuch a basis as a Hamel basis or a vector space basis. For most vector spaces, Hamel basesare only known to exist because of the Axiom of Choice; in fact, the statement “Every vectorspace has a Hamel basis” is equivalent to the Axiom of Choice.

As in finite dimensions, it can be shown that any two Hamel bases for a vector space Vmust have the same cardinality. This cardinality is called the (vector space) dimension of V .

Example 4.2 (Existence of Unbounded Linear Functionals). We can use the Axiom ofChoice to show that there exist unbounded linear functionals L : X → F whenever X is aninfinite-dimensional normed linear space.

Let fii∈I be a Hamel basis for an infinite-dimensional normed linear space X, normalizedso that ‖fi‖ = 1 for every i ∈ I. Let J0 = j1, j2, . . . be any countable subsequence of I.

20 CHRISTOPHER HEIL

Define L : X → C by setting L(fjn) = n for n ∈ N and L(fi) = 0 for i ∈ I\J0. Then extend

L linearly to all of X: if f =∑N

k=1 ckfik is the unique representation of f using nonzero

scalars c1, . . . , cN , then define L(f) =∑N

k=1 ckL(fik). This L is a linear functional on X, butsince ‖fjn

‖ = 1 yet |L(fjn)| = n, we have ‖L‖ = sup‖f‖=1 |L(f)| = ∞.

Thus, if L : X → Y and Y is finite-dimensional, we cannot conclude that L must necessar-ily be bounded. Contrast this with the fact that if L : X → Y and X is finite-dimensional,then L must be bounded (see Exercise 3.4).

Remark 4.3. Aside from the preceding result, Hamel bases are not going to be much useto us. For example, consider the space `2 = `2(N), and let enn∈N be the standard basisfor `2. Note that enn∈N is NOT a Hamel basis for `2! Its finite linear span is the propersubspace c00. Thus enn∈N is a Hamel basis for c00, but not for `2. In fact, a Hamel basisfor an infinite-dimensional Hilbert or Banach space must be uncountable.

The main point is that since we have a norm, there is no reason to restrict to only finitelinear combinations. Given a sequence of vectors fii∈N and scalars (ci)i∈N, we can consider“infinite linear combinations”

∞∑

i=1

ci fi.

HOWEVER, we must be very careful about convergence issues: the series above will notconverge for EVERY choice of scalars ci. On the other hand, if our sequence fi is orthog-onal, or even better yet, orthonormal, then we will see that the convergence issues becomeeasy to deal with, and we can make sense of what it means to have a “basis” which allows“infinite linear combinations.”

HOWEVER, there still remains an issue of how many vectors we need in this “basis”—forsome Hilbert spaces (the separable Hilbert spaces) we will be able to use a basis that requiresonly countably many vectors, as was implicitly assumed in the discussion in the precedingparagraph. But for others, a “basis” may need uncountably many vectors, even allowing“infinite (but still countable) linear combinations.” In some fields (e.g., the geometry ofBanach spaces literature), it is customary to only use the word basis for the case of countablebases. Conway does not follow this custom, which is fine as long as you realize that otherauthors use the term “basis” differently.

Conway’s definition of a basis for a Hilbert space follows. The definition does not seem tosay anything about linear combinations, infinite or otherwise—we shall see the connectionlater. To emphasize that Conway’s definition is not universally accepted, I will refer to hisdefinition of basis as a Conway basis.

Definition 4.4 (Conway Basis). Let H be a Hilbert space. Then a Conway basis for H isa maximal orthonormal set. That is, a subset E ⊆ H is a Conway basis if

(a) E is orthonormal, and

(b) there does not exist an orthonormal set E ′ ⊆ H such that E ⊆ E ′.


Exercise 4.5. Zorn’s Lemma is an equivalent formulation of the Axiom of Choice. Learnwhat Zorn’s Lemma says, and use it to prove that every Hilbert space has a Conway basis.Then ask all your friends the classic riddle: What is yellow and equivalent to the Axiom ofChoice?1 That one is nearly as good as the classic: What is purple and commutes?2

Exercise 4.6. Show that enn∈N is a Conway basis for `2(N).

Exercise 4.7. Let I be any set. Define `2(I) to be the set of all functions x : I → F suchthat x(i) 6= 0 for at most a countable number of i and such that

‖x‖2 =∑

i∈I

|x(i)|2 < ∞

(there are at most countably many nonzero terms in the above sum, so this series with apossibly uncountable index set just means to sum the countably many nonzero terms). Then`2(I) is a Hilbert space with respect to the inner product

〈x, y〉 =∑

i∈I

x(i) y(i).

For each i ∈ I, define ei : I → F by

ei(j) = δij =

1, i = j,

0, i 6= j.

Then eii∈I is called the standard basis for `2(I). Prove that it is a Conway basis for `2(I).

Instead of “Conway basis,” the following gives us a terminology that does not require usto use the word basis.

Lemma 4.8. Let fii∈I be a sequence of elements of H. Then:

fii∈I is orthonormal and complete ⇐⇒ fii∈I is a Conway basis.

Proof. ⇒. Suppose that fii∈I is orthonormal and complete. Then by Exercise 2.19, thereis no nonzero g ∈ H that is orthogonal to every fi. Therefore there can be no orthonormalset that properly contains fii∈I .

⇐. Exercise.

Definition 4.9 (Schauder basis). Now we give the “correct” definition of a basis.Let X be a Banach space. Then a (countable!) sequence fii∈N is a Schauder basis, or

just a basis, for X if for each f ∈ X there exist unique scalars cii∈N such that

f =

∞∑

i=1

cifi.

1Zorn’s Lemon!2An abelian grape.

22 CHRISTOPHER HEIL

Note in particular that this means that∑N

i=1 cifi → f as N → ∞.To be a little more precise, the definition of Schauder basis really includes the requirement

that if we let ci(f) denote the unique scalars such that f =∑∞

i=1 ci(f)fi, then the mappingf 7→ ci(f) must be a continuous linear functional on X for each i. However, the StrongBasis Theorem proves that this is automatically true when X is a Banach space, so we haveomitted that requirement in our definition.

Exercise 4.10. The standard basis enn∈N is an orthonormal Schauder basis for `2(N).

Exercise 4.11. (See also Exercise 10 on p. 98 of Conway.) Let fii∈N be a Schauder basisfor a Banach space X.

(a) Prove that fii∈N is complete in X, i.e., that the set of finite linear combinations isdense in X. Note: The converse is false, i.e., a complete sequence need not be a Schauderbasis (see Example 4.18). On the other hand, we will see that in a separable Hilbert space,a complete orthonormal sequence IS a Schauder basis.

(b) Let

S =

N∑

i=1

rifi : N > 0, Re (ri), Im (ri) ∈ Q

.

Prove that S is a countable, dense subset of X. A Banach space which has a countable,dense subset is said to be separable.

Remark 4.12. Thus, if a Banach space has a Schauder basis, then it must be separable.Does every separable Banach space have a Schauder basis? This was a longstanding openproblem in Banach space theory, called the Basis Problem. It was settled by Enflo in 1973:there exist separable, reflexive Banach spaces which do not possess any Schauder bases.

Exercise 4.13. Let X be a Banach space. Suppose there exists an uncountable sequencexii∈I of elements of X such that ‖xi − xj‖ ≥ ε > 0 for all i 6= j ∈ I. Prove that X is notseparable.

Exercise 4.14. Use the preceding exercise to prove the following.

(a) Prove that `∞(N) is not separable.

(b) Prove that L∞(R) is not separable.

(c) Prove that if I is uncountable then `2(I) is not separable. More generally, any Hilbertspace which contains an uncountable orthonormal sequence is not separable.

We will show that, in a separable Hilbert space, any complete orthonormal sequence is aSchauder basis for H. If H is nonseparable, something very similar happens; we just have tobe careful with what we mean by infinite series with uncountably many terms (only countablymany terms can be nonzero). First, however, we will present the results for separable Hilbertspaces.


Theorem 4.15. Let enn∈N be any orthonormal sequence in a Hilbert space H. Then thefollowing statements hold.

(a) Bessel’s Inequality:

∞∑

n=1

|〈f, en〉|2 ≤ ‖f‖2.

(b) If f =

∞∑

n=1

cnen converges, then cn = 〈f, en〉.

(c)

∞∑

n=1

cnen converges ⇐⇒

∞∑

n=1

|cn|2 < ∞.

(d) If∞

∑

n=1

cnen converges then it converges unconditionally, i.e., it converges regardless of

the ordering of the indices.

(e) f ∈ span(enn∈N) ⇐⇒ f =∞

∑

n=1

〈f, en〉 en.

(f) If f ∈ H, then

Pf =∞

∑

n=1

〈f, en〉 en

is the orthogonal projection of f onto span(enn∈N).

Proof. (a) Choose f ∈ H. For each N define

fN = f −N

∑

n=1

〈f, en〉 en.

Show that fN ⊥ e1, . . . , eN (exercise). Therefore, by the Pythagorean Theorem, we have

‖f‖2 =∥

∥

∥fN +

N∑

n=1

〈f, en〉 en

∥

∥

∥

2

= ‖fN‖2 +

N∑

n=1

‖〈f, en〉 en‖2 ≥

N∑

n=1

|〈f, en〉|2.

Since this is true for each finite n, Bessel’s inequality follows.

(b) Exercise.

(c) ⇐. Suppose that∑∞

n=1 |cn|2 < ∞. Set

sN =

N∑

n=1

cnen, tN =

N∑

n=1

|cn|2.

We know that tN is a convergent (hence Cauchy) sequence of scalars, and we must showthat sN is a convergent sequence of vectors. We have for N > M that

‖sN − sM‖2 =∥

∥

∥

N∑

n=M+1

cnen

∥

∥

∥

2

=N

∑

n=M+1

‖cnen‖2 =

N∑

n=M+1

|cn|2 = |tN − tM |.

24 CHRISTOPHER HEIL

Since tN is Cauchy, we conclude that sN is Cauchy and hence converges.

⇒. Exercise.

(d), (e) Exercise.

(f) By Bessel’s inequality and part (c), we know that the series defining Pf converges, sowe just have to show that it is the orthogonal projection of f onto span(en). Check that〈f − Pf, en〉 = 0 for every n (exercise). By linearity, conclude that f − Pf ⊥ span(en).By continuity of the inner product, conclude that f − Pf ⊥ span(en) (exercise).

Exercise 4.16. Use Theorem 4.15 to construct an example of a series∑

cnen in a Hilbertspace that converges unconditionally but not absolutely (compare Exercise 1.26).

The next theorem shows that any countable orthonormal sequence that is complete mustbe a Schauder basis, and vice versa. Such an orthonormal Schauder basis is usually justcalled an orthonormal basis (or just ONB) for H.

Theorem 4.17. Let enn∈N be any orthonormal sequence in a Hilbert space H. Then thefollowing statements are equivalent.

(a) enn∈N is a Schauder basis for H, i.e., for each f ∈ H there exist unique scalars(cn)n∈N such that f =

∑∞n=1 cnen.

(b) For each f ∈ H, f =

∞∑

n=1

〈f, en〉 en.

(c) enn∈N is complete (i.e, it is a Conway basis for H).

(d) Plancherel/Parseval Equality: For each f ∈ H, ‖f‖2 =

∞∑

n=1

|〈f, en〉|2.

(e) Parseval/Plancherel Equality: For each f , g ∈ H, 〈f, g〉 =∞

∑

n=1

〈f, en〉〈en, g〉.

Proof. (b) ⇒ (e). Suppose that (b) holds, and choose any f , g ∈ H. Then

〈f, g〉 =⟨

∞∑

n=1

〈f, en〉 en, g⟩

=∞

∑

n=1

⟨

〈f, en〉 en, g⟩

=∞

∑

n=1

〈f, en〉〈en, g〉,

where we have used Exercise 1.24 to move the infinite series outside of the inner product.

(c) ⇒ (b). If en is complete, then its closed span is all of H, so by Theorem 4.15(e) wehave f =

∑∞n=1〈f, en〉 en for every f ∈ H.


(d) ⇒ (b). Suppose that ‖f‖2 =∑∞

n=1 |〈f, en〉|2 for every f ∈ H. Fix f , and define

sN =∑N

n=1〈f, en〉 en. Then, by direct calculation and the by the Pythagorean Theorem,

‖f − sN‖2 = ‖f‖2 − 〈f, sN〉 − 〈sN , f〉 + ‖sN‖

2

= ‖f‖2 −N

∑

n=1

|〈f, en〉|2 −

N∑

n=1

|〈f, en〉|2 +

N∑

n=1

|〈f, en〉|2

= ‖f‖2 −

N∑

n=1

|〈f, en〉|2 → 0 as N → ∞.

Hence f =∑∞

n=1〈f, en〉 en.

The remaining implications are exercises.

Example 4.18. Let enn∈N be an orthonormal basis for a separable Hilbert space H. Definefn = e1 + en/n. Then fnn∈N is complete (exercise) but it is not a Schauder basis for H(harder).

Exercise 4.19. Let H be a Hilbert space. Show that H is separable if and only if there existsa countable sequence xnn∈N that is an orthonormal basis for H. Hint: By Exercise 4.5 weknow that there exists a complete orthonormal sequence for H.

We state without proof the facts about complete orthonormal systems in nonseparableHilbert spaces.

Theorem 4.20. Let eii∈I be an orthonormal sequence in a Hilbert space H (note that Imight be uncountable). Then the following statements hold.

(a) If f ∈ H then 〈f, ei〉 6= 0 for at most countably many i.

(b) For each f ∈ H,∑

i∈I

|〈f, ei〉|2 ≤ ‖f‖2.

(c) For each f ∈ H,∑

i∈I

〈f, ei〉 ei converges with respect to the net of finite subsets of I

(see Proposition 1.27 for the meaning of net of finite subsets).

Theorem 4.21. Let eii∈I be an orthonormal sequence in a Hilbert space H. Then thefollowing statements are equivalent.

(a) eii∈I is complete (i.e., is a Conway basis for H).

(b) For each f ∈ H, f =∑

i∈I

〈f, ei〉 ei with respect to the net of finite subsets of I.

(c) For each f ∈ H, ‖f‖2 =∑

i∈I

|〈f, ei〉|2 (only countably many terms are nonzero).

26 CHRISTOPHER HEIL

The (Hilbert space) dimension of H is the cardinality of a complete orthonormal sequence.It can be shown that any two complete orthonormal sequences must have the same cardinal-ity. Further, H is separable if and only if it has a countable complete orthonormal sequence(which by Theorem 4.17 will be an orthonormal Schauder basis for H).

5. Isomorphic Hilbert Spaces and the Fourier Transform for the Circle

Definition 5.1. If H1, H2 are Hilbert spaces, then L : H1 → H2 is an isomorphism if L isa linear bijection that satisfies

∀ f, g ∈ H1, 〈Lf, Lg〉 = 〈f, g〉.

In this case we say that H1 and H2 are isomorphic, and write H1∼= H2.

If H1 = H2 = H and L : H → H is an isomorphism, then we say that L is unitary.

Proposition 5.2. Let L : H1 → H2 be a linear mapping. Then the following statements areequivalent.

(a) L is inner-product-preserving, i.e., 〈Lf, Lg〉 = 〈f, g〉 for all f , g ∈ H1.

(b) L is norm-preserving (an isometry), i.e., ‖Lf‖ = ‖f‖ for all f ∈ H1.

Proof. We need only show that (b) implies (a). Assume that L is an isometry, and fix f ,g ∈ H. Then for any scalar λ ∈ F we have by the Polar Identity and the fact that L isisometric that

‖f‖2 + 2 Re λ 〈f, g〉 + |λ|2 ‖g‖2 = ‖f + λg‖2

= ‖Lf + λLg‖2

= ‖Lf‖2 + 2 Re λ 〈Lf, Lg〉 + |λ|2 ‖Lg‖2

= ‖f‖2 + 2 Re λ 〈Lf, Lg〉 + |λ|2 ‖g‖2.

Thus Re λ 〈Lf, Lg〉 = Re λ 〈f, g〉 for every λ ∈ F. This implies (take λ = 1 and λ = i) that〈Lf, Lg〉 = 〈f, g〉.

In particular, every isomorphism is an isometry and hence is automatically injective.

Corollary 5.3. Let L : H1 → H2 be given. Then L is an isomorphism if and only if it is alinear surjective isometry.

Theorem 5.4. All separable infinite-dimensional Hilbert spaces are isomorphic.

Proof. Let H1 and H2 be separable infinite-dimensional Hilbert spaces. Then H1 has acountable orthonormal basis enn∈N, and H2 has a countable orthonormal basis hnn∈N.If f ∈ H1 then by Theorem 4.17 we have f =

∑∞n=1〈f, en〉 en and ‖f‖2 =

∑∞n=1 |〈f, en〉|

2.Since this is finite and since hn is an ONB, the series Lf =

∑∞n=1〈f, en〉 hn converges, and

‖Lf‖2 =∑∞

n=1 |〈f, en〉|2 = ‖f‖2.

Exercise: Finish the proof by showing that L is a linear surjective isometry.


In particular, every separable Hilbert space is isomorphic to `2(N).The proof can be extended to show that any two Hilbert spaces which have the same

dimension are isomorphic.

Example 5.5 (Fourier Series). Now we give one of the most important examples of anorthonormal basis.

We saw in Exercise 2.2 that if we define en(x) = e2πinx, then enn∈Z is an orthonormalsequence in L2[0, 1] (the space of square-integrable complex-valued functions on the domain[0, 1], or we can consider these functions to be 1-periodic functions on the domain R).

It is a fact that enn∈Z is actually complete in L2[0, 1] and hence is an orthonormal basisfor L2[0, 1]. The Fourier coefficients of f ∈ L2[0, 1] are

f(n) = 〈f, en〉 =

∫ 1

0

f(x) e−2πinx dx, n ∈ Z.

By Theorem 4.17, we have

f =∑

n∈Z

〈f, en〉 en =∑

n∈Z

f(n) en (5.1)

(the series converges unconditionally, so it does not matter what ordering of the index set Z

that we use to sum with). Equation 5.1 is called the Fourier series of f .However, note that the series in 5.1 converges in the norm of the Hilbert space, i.e., in

L2-norm. That is, the partial sums converge in L2-norm, i.e.,

limN→∞

∥

∥

∥f −

N∑

n=−N

f(n) en

∥

∥

∥

2= 0,

or,

limN→∞

∫ 1

0

∣

∣

∣

∣

f(x) −N

∑

n=−N

f(n) e2πinx

∣

∣

∣

∣

2

dx = 0.

We cannot conclude from this that the equality

f(x) =∑

n∈Z

f(n) e2πinx

holds pointwise. In fact, one of the deepest results of Fourier series is the Carleson–HuntTheorem, which proves the conjecture of Lusin that if f ∈ Lp[0, 1] where 1 < p ≤ ∞, thenthe Fourier series of f converges to f a.e.

Exercise 5.6. Show that the mapping F : L2[0, 1] → `2(Z) given by F(f) = f = f(n)n∈Z

is exactly the isomorphism constructed by Theorem 5.4 for the case H1 = L2[0, 1] andH2 = `2(Z). The operator F is the Fourier transform for the circle (thinking of functions inL2[0, 1] as being 1-periodic, the domain [0, 1] is topologically a circle).

28 CHRISTOPHER HEIL

Exercise 5.7. Prove the following (easy) special case of the Riemann–Lebesgue Lemma: If

f ∈ L2[0, 1] then f(n) → 0 as |n| → ∞. The full Riemann–Lebesgue Lemma for the circlestates the same conclusion holds if we only assume that f ∈ L1[0, 1].

Definition 5.8. In honor of Fourier series, if enn∈I is any orthonormal basis of a separableHilbert space H, then 〈f, en〉n∈I is the sequence of (generalized) Fourier coefficients of fand f =

∑

n∈I〈f, en〉 en is the (generalized) Fourier series of f .

Exercise 5.9. The Plancherel formula with respect to the orthonormal basis enn∈Z forL2[0, 1] is

‖f‖22 =

∑

n∈Z

|f(n)|2.

Use the Plancherel formula to derive a formula for π by applying it to the function f =χ[0,1/2) − χ[1/2,1).

elementary properties of hilbert...

Documents