Homework 3

Let $\{e_n, n \in \mathbb{N}\}$ be a complete orthonormal sequence in a real Hilbert space $H$. Let $\mathcal{M}$ be a subspace of $H$, spanned by $\{e_k, k \in S\}$, for some finite set $S \subset \mathbb{N}$. That is,

$\mathcal{M} = \{v \in H : \exists \alpha_k \in \mathbb{R}, k \in S, \quad v = \sum_{k \in S} \alpha_k e_k\}$

Let $x \in H$ be given. Find $x^* \in \mathcal{M}$ which is the solution to the following:

$\min_{x_0 \in \mathcal{M}} \|x - x_0\|,$

in terms of $x$, and $\{e_n, n \in \mathbb{N}\}$.

Hint: By our arguments in class, any vector in $H$ can be written as $x = \sum_{n \in \mathbb{N}} \langle x, e_n \rangle e_n$.

One practically important basis is the class of Haar functions (wavelets). We defined in class Haar functions as follows.

$\Psi_{0,0}(x) = \begin{cases} 1, & \text{if } 0 \le x \le 1 \\ 0 & \text{else} \end{cases}$

and for $n \in \mathbb{Z}_+, k \in \{0, 1, 2, \ldots, 2^n - 1\}$,

$\Phi_{n,k}(x) = \begin{cases} 2^{n/2}, & \text{if } k2^{-n} \le x < (k + 1/2)2^{-n} \\ -2^{n/2}, & \text{if } (k+1/2)2^{-n} \le x \le (k+1)2^{-n} \\ 0 & \text{else} \end{cases}$

Show that the family

$\{\Psi_{0,0}, \Phi_{n,k}, n \in \mathbb{Z}_+, k \in \{0, 1, 2, \ldots, 2^n - 1\}\}$

is complete and orthonormal in $L_2([0,1]; \mathbb{R})$ and is a sequence (that is, countable).

Hint: You may use/assume the fact that the space of $\mathbb{R}$-valued continuous functions on $[0,1]$ are dense in $L_2([0,1]; \mathbb{R})$.

Let $C([0,1]; \mathbb{R})$ denote the normed linear space of continuous functions from $[-1,1]$ to $\mathbb{R}$ under the supremum norm. We observed earlier that polynomials can be used to approximate any function in this space with arbitrary precision, under the supremum norm (Weierstrass Theorem).

Given this, repeating the arguments we made in class, argue that the family of polynomials $\{1, t, t^2, \cdots\}$ can be used to form a complete orthonormal sequence in $L_2([0,1]; \mathbb{R})$. This also establishes that $L_2([0,1]; \mathbb{R})$ is separable.

These polynomials are not orthonormal, but we could orthonormalize them via the Gram-Schmidt procedure. This leads to what is known as the Legendre Polynomials.

Let $f$ be a linear functional on a normed linear space $X$ (mapping $X$ to $\mathbb{R}$). Suppose that we define $f$ to be bounded if there is a constant $M$ such that $|f(x)| \le M\|x\|$ for all $x \in X$.

The smallest such $M$ is called the norm of $f$ and is denoted by $\|f\|$. The space of all bounded, linear functionals on $X$ is called the dual space of $X$.

Show that a linear functional on a normed linear space is bounded if and only if it is continuous.

Let $f$ be a linear and continuous functional on $l_2(\mathbb{N}; \mathbb{R})$; thus, mapping $l_2(\mathbb{N}; \mathbb{R})$ to $\mathbb{R}$.

Show that for every such $f$, there exists a vector $\kappa \in l_2(\mathbb{N}; \mathbb{R})$ such that

$f(x) = \langle x, \kappa \rangle, \quad \forall x \in l_2(\mathbb{N}; \mathbb{R}),$

where the inner-product is the one giving rise to the $l_2$-norm in $l_2(\mathbb{N}; \mathbb{R})$.

In this problem we will show that the usual notion of convergence (that is strong convergence) implies weak convergence for a Hilbert space. Let $x_n \in L_2(\mathbb{R}_+; \mathbb{R})$.

a) Show that if $x_n \to x$, then $\langle x_n, f \rangle \to \langle x, f \rangle \quad \forall f \in L_2(\mathbb{R}_+; \mathbb{R})$, that is convergence in strong sense implies convergence in weak sense.

b) Possibly via a counterexample, show that $x_n \to x$ weakly does not imply convergence in the strong sense.

In class we observed that the space of polynomials is a dense subset of the space of continuous functions $C([0,1])$ under the supremum norm. One class of polynomials which can be used to constructively provide an approximation is the family of Bernstein polynomials, defined as follows: Let for some $f \in C([0,1])$:

$B_{n,f}(t) = \sum_{k=0}^{n} f\left(\frac{k}{n}\right) \binom{n}{k} t^k (1-t)^{n-k}$

In this exercise, you are asked to write a Matlab (or an alternative program) function which admits a trigonometric function $f \in C([0,1])$, and $n$ as its inputs and generates the Bernstein polynomial approximation of the signal.

You may take $f$ to be for example $f(t) = 2\sin(\frac{1}{4}\pi t) + 3\cos(\frac{1}{2}\pi t)$.

Given a trigonometric function of your choice, compute the Bernstein approximations of order $n$, where $n$ can take three values and verify that the supremum difference

$\sup_{t \in [0,1]} |f(t) - B_{n,f}(t)|,$

decreases as $n$ increases.