A voyage through space(s) (Part IV)

We have alas reached the final leg of our journey, we will continue with the idea of a normed vector space and then we will describe Banach spaces and Hilbert spaces.

Associated with the norm on our vector space $V$ , is the metric

$d(x,y)=\norm{x-y}.$

So $V$ is endowed with a metric topology in which a sequence $\{x_n\}$ converges to $x$ in $V$ iff

$\norm{x_n-x}\rightarrow0.$

For example, consider the sequence of sequences (take a moment to ponder what we are talking about!) given by:

$\vb{x}_1=(1,0,0\dots),$

$\vb{x}_2=(1/2,1/2,0\dots),$

$\vb{x}_3=(1/3,1/3,1/3,0,0\dots),$

and so on. Now we will consider whether or not this sequence converges. The first question you have to ask yourself is what could this converge to? Hopefully it is obvious that it converges to $\vb{x}=(0,0,0,0...)$ . Let’s say we live in $\ell_1$ , and so for our sequence $\vb{x}_n$ to converge to $\vb{x}$ we need that $\norm{x_n-x}_1\rightarrow 0$ . But does this happen? Let’s find out. Recall that the 1-norm is $\sum_{i=1}^{N}\abs{a_i}$ and applying this we get

(1) $\begin{align*} \norm{x_n-x}_1&=\sum_{i=1}^{n}\abs{x_n-x}\\ &=\sum_{i=1}^{n}\abs{\frac{1}{n}-0}\\ &=n\times\frac{1}{n}\\ &=1 \end{align*}$

And so, as $n\rightarrow\infty$ , $\norm{x_n-x}_1\rightarrow 1$ and hence the sequence does not converge in $\ell_1$ . Doing the same thing, it can be shown that it does converge in $\ell_2$ and $\ell_{\infty}$ .

Now that we have a notion of convergence for these normed vector spaces, we have all those properties that we had previously for metric spaces, in particular, those of Cauchy sequences and completeness. Recall that every complete space, is one in which every Cauchy sequence converges. Now to expand upon normed linear spaces, every complete normed linear space is known as a Banach space. From those examples we mentioned in the previous post, since we proved in part 2 of this series that the reals are complete, and we saw that the reals is an example of a normed linear space, it turns out the reals, are a Banach space. Other examples of Banach spaces are $\ell_2$ and $\ell_{\infty}$ .

Now a Hilbert space is a Banach space in which the norm is derived from an inner product space. What is an inner product? Well, the simplest inner product you have probably been dealing with is the dot product. But more generally, an inner product is a function $<,>$ that satisfies:

$<x,x> \geq 0$ and $<x,x>=0 \iff x=0$
$<x,y+z>=<x,y>+<y,z>$
$<x,\alpha y>=\alpha<x,y>$
$<x,y>=\overline{<y,x>}$

Note that $<x,x>=\norm{x}^2$ and so the norm on any inner product space is

$\norm{x}=\sqrt{<x,x>},$

and it is easily verified that this is indeed a norm. (Refer back to the three norm axioms in the previous post!)

As a final example, we will prove that the space $\ell_2$ , with the inner product $<x,y>=\sum_{i=1}^{\infty}x_iy_i$ is indeed a Hilbert space. Now since we are already given that $<x,y>$ is an inner product, I leave it to you to verify that it actually is by making sure those 4 axioms are satisfied. What we need to do now is prove that the space is complete with respect to this norm. So the way we will do this is outlined as follows:

Pick any Cauchy sequence in $\ell_2$
Find a possible candidate to which it can converge to
Show that that this candidate is indeed in $\ell_2$
Show that the Cauchy sequence actually converges to our candidate with respect to the given norm

This last step is important, although it seems that it is obvious that our sequence must converge to it from step 2, we have not actually shown that it does with respect to the given norm, until we do step 4. I’ll leave steps 3 and 4 for you to figure out! Before we can go on, we need to talk about notation, because when we begin discussing sequences of sequences, things get very complicated.

Given a sequence of sequences $(x)_n$ , it follows that each $x_1,~x_2,~x_3 etc$ is itself a sequence, and so the elements of each of these sequences need to be indexed in some way. Let’s use superscripts to indicate the position in each individual sequence. So for example, $x_3^2$ would mean the second element of the third sequence, $x_i^n$ would mean the n’th element of the i’th sequence. Doing so, we can see that the i’th sequence is indexed as

$(x_i^1,x_i^2,x_i^3,x_i^4...)$

Now that we have this bit of notation trickery up our sleeves, let’s begin this proof.

Proof:

Let $\epsilon>0$ .

1. Pick any Cauchy sequence in $\ell_2$ .

Let $x_i$ be a Cauchy sequence such that $x_i \in \ell_2 \forall I$ and for $i,j>N$ ,

$\norm{x_i-x_j}<\epsilon$

2. Find a possible candidate to which it can converge to.

Since $x_n$ is Cauchy, and remembering that our norm is given by our inner product defined above, we get that

$\sum_n^{\infty}((x_i-x_j)^n)^2<\epsilon^2<\epsilon$

given $\epsilon<1$ . Here I have dealt with the square root that comes with the norm. From this it follows that for all $n$ ,

$((x_i-x_j)^n)^2<\epsilon$

and so

$|(x_i-x_j)^n|<\epsilon$

and so, each $(x)^n$ is a Cauchy sequence in $\mathrm{R}$ . Now you might be confused what this all means, so here is a little picture to help you, with the sequence $(x)^n$ highlighted.