%% chapter-07.tex %% Approximation Theory: Chapter 7 %% Neal L. Carothers %% Bowling Green State University %% Bowling Green, Ohio 43403 %% carother@math.bgsu.edu %% http://www.bgsu.edu/~carother/ \input 680-setup.tex \chaptertitle = {Fourier Series} \centerline{\hfil\tf A Brief Introduction to Fourier Series\hfil} \vskip-\baselineskip \line{\sc Math 680 \hfil 7/18/94} \noindent The {\sl Fourier series\/} of a $2\pi$-periodic, integrable function $f$ is $${{a_0}\over2}+\sum_{k=1}^\infty\bigl(a_k\cos kx+b_k\sin kx\bigr),$$ where the coefficients are defined by $$a_k={1\over\pi}\int_{-\pi}^\pi f(t)\,\cos kt\,dt \qquad\hbox{and}\qquad b_k={1\over\pi}\int_{-\pi}^\pi f(t)\,\sin kt\,dt.$$ Please note that each of these integrals is well-defined and finite; indeed, $$|a_k|\le{1\over\pi}\int_{-\pi}^\pi |f(t)|\,dt$$ and so, for example, we would have $|a_k|\le2\Vert f\Vert$ for $f\in C^{2\pi}$. We write the partial sums of the series as $$s_n(f)(x)={{a_0}\over2}+\sum_{k=1}^n\bigl(a_k\cos kx+b_k\sin kx\bigr).$$ Now while $s_n(f)$ need not converge pointwise to $f$ (in fact, it may even diverge at a given point), and while $s_n(f)$ is not typically a good uniform approximation to $f$, it is still a very natural choice for an approximation to $f$ in the ``least-squares'' sense (which we'll make precise shortly). Said in other words, the Fourier series for $f$ provides a useful {\sl representation\/} for $f$ even if it fails to converge pointwise to $f$. \noindent{\bf Observations} \item{\bf1.} The functions $1,\cos x,\cos 2x,\ldots$, $\sin x,\sin 2x,\ldots$, are {\sl orthogonal\/} on $[-\pi,\pi\,]$. That is, $$\int_{-\pi}^\pi \cos mx\,\cos nx\,dx= \int_{-\pi}^\pi \sin mx\,\sin nx\,dx= \int_{-\pi}^\pi \cos mx\,\sin nx\,dx=0$$ for any $m\ne n$ (and the last equation even holds for $m=n$), $$\int_{-\pi}^\pi \cos^2 mx\,dx=\int_{-\pi}^\pi \sin^2 mx\,dx=\pi$$ for any $m\ne0$, and, of course, $\int_{-\pi}^\pi 1\,dx=2\pi$. \item{\bf2.} What this means is that if $T(x)={{a_0}\over2}+\sum_{k=1}^n\bigl(a_k\cos kx+b_k\sin kx\bigr)$, then, $$\int_{-\pi}^\pi T(x)\,\cos mx\,dx=a_m\int_{-\pi}^\pi \cos^2 mx\,dx =\cases{\pi a_m &$(m\ne0)$\cr 2\pi a_0 &$(m=0)$.}$$ That is, if $T\in{\cal T}_n$, then $T$ is actually equal to its own Fourier series. \item{\bf3.} The partial sum operator $s_n(f)$ is a {\sl linear projection\/} from $C^{2\pi}$ onto ${\cal T}_n$. \item{\bf4.} If some $f\in C^{2\pi}$ has $a_k=b_k=0$ for all $k$, then $f\equiv0$. \item{} Indeed, by linearity, this means that $$\int_{-\pi}^\pi f(x)\,T(x)\,dx=0$$ for any trig polynomial $T$. But from Weierstrass's second theorem we know that $f$ is the uniform limit of some sequence of trig polynomials $(T_n)$. Thus, $$\int_{-\pi}^\pi f(x)^2\,dx=\Limonn\int_{-\pi}^\pi f(x)\,T_n(x)\,dx=0.$$ Since $f$ is continuous, this easily implies that $f\equiv0$. \item{\bf5.} If $f$, $g\in C^{2\pi}$ have the same Fourier series, then $f\equiv g$. \item{\bf6.} The coefficients $a_k$ and $b_k$ minimize the expression $$\varphi(a_0,a_1,\ldots,b_n) =\int_{-\pi}^\pi \bigl[f(x)-s_n(f)(x)\bigr]^2\,dx.$$ \item{} It's not hard to see, for example, that $${{\partial\,\varphi}\over{\partial\,a_k}} =\int_{-\pi}^\pi 2\bigl[f(x)-s_n(f)(x)\bigr]\cos kx\,dx=0$$ precisely when $a_k$ satisfies $$\int_{-\pi}^\pi f(x)\,\cos kx\,dx=a_k\int_{-\pi}^\pi \cos^2 kx\,dx.$$ \item{\bf7.} The partial sum $s_n(f)$ is the best approximation to $f$ out of ${\cal T}_n$ {\sl relative to the\/} $L_2$ {\sl norm} $$\Vert f\Vert_2=\left({1\over\pi}\int_{-\pi}^\pi f(x)^2\,dx\right)^{1/2}.$$ (Some authors prefer $1/2\pi$ in place of $1/\pi$.) Moreover, $$\Vert f-s_n(f)\Vert_2^2={1\over\pi}\int_{-\pi}^\pi f(x)^2\,dx -{\textstyle{1\over4}}a_0^2-\sum_{k=1}^n\bigl(a_k^2+b_k^2\bigr) =\Vert f\Vert_2^2-\Vert s_n(f)\Vert_2^2.$$ \item{\bf8.} It follows from {\bf7} that $${1\over\pi}\int_{-\pi}^\pi s_n(f)(x)^2\,dx ={\textstyle{1\over4}}a_0^2+\sum_{k=1}^n\bigl(a_k^2+b_k^2\bigr) \le{1\over\pi}\int_{-\pi}^\pi f(x)^2\,dx.$$ In other words, $\Vert s_n(f)\Vert_2\le\Vert f\Vert_2$. In particular, the Fourier coefficients of any $f\in C^{2\pi}$ are square summable. (Why?) \item{\bf9.} It follows from {\bf8} and Weierstrass's second theorem that $s_n(f)\to f$ {\sl in the\/} $L_2$ {\sl norm\/} whenever $f\in C^{2\pi}$. \item{} Indeed, given $\eps>0$, choose a trig polynomial $T$ such that $\Vert f-T\Vert<\eps$. Then, since $s_n(T)=T$ for large enough $n$, we have $$\eqalign{ \Vert f-s_n(f)\Vert_2&\le \Vert f-T\Vert_2+\Vert s_n(T-f)\Vert_2\cr &\le2\Vert f-T\Vert_2\le2\sqrt{2}\,\Vert f-T\Vert<2\sqrt{2}\,\eps. }$$ By way of comparison, let's give a simple class of functions whose Fourier partial sums provide good {\sl uniform\/} approximations. \proclaim Theorem. If\/ $f\,''\in C^{2\pi}$, then the Fourier series for $f$ converges uniformly to $f$. \proof First notice that integration by-parts gives us an estimate on the order of growth of the Fourier coefficients: $$\pi a_k = \int_{-\pi}^\pi f(x)\,\cos kx\,dx =\int_{-\pi}^\pi f(x)\,d\left({{\sin kx}\over k}\right) =-{1\over k}\int_{-\pi}^\pi f\,'(x)\,\sin kx\,dx$$ (because $f$ is $2\pi$-periodic). Thus, $|a_k|\le2\Vert f\,'\,\Vert/k\to0$ as $k\to\infty$. Now we integrate by-parts again: $$-\pi ka_k=\int_{-\pi}^\pi f\,'(x)\,\sin kx\,dx =\int_{-\pi}^\pi f\,'(x)\,d\left({{\cos kx}\over k}\right) ={1\over k}\int_{-\pi}^\pi f\,''(x)\,\cos kx\,dx$$ (because $f\,'$ is $2\pi$-periodic). Thus, $|a_k|\le2\Vert f\,''\,\Vert/k^2\to0$ as $k\to\infty$. More importantly, this inequality (along with the Weierstrass $M$-test) implies that the Fourier series for $f$ is both uniformly and absolutely convergent: $$\left|{{a_0}\over2}+\sum_{k=1}^\infty\bigl(a_k\cos kx+b_k\sin kx\bigr)\right| \le \left|{{a_0}\over2}\right|+\sum_{k=1}^\infty\bigl(|a_k|+|b_k|\bigr) \le C\sum_{k=1}^\infty{1\over{k^2}}.$$ But why should the series actually converge to $f\,$? Well, if we call the sum $$g(x)={{a_0}\over2}+\sum_{k=1}^\infty\bigl(a_k\cos kx+b_k\sin kx\bigr),$$ then $g\in C^{2\pi}$ (why?)\ and $g$ has the same Fourier coefficients as $f\,$! Hence, $g=f$.~\qed Our next chore is to find a closed expression for $s_n(f)$. For this we'll need a couple of trig identities; the first two need no explanation. $$\displaylines{ \cos kt\cos kx + \sin kt\sin kx = \cos k(t-x)\cr 2\cos\alpha\sin\beta = \sin(\alpha+\beta)-\sin(\alpha-\beta)\cr {\textstyle{1\over2}}+\cos\theta+\cos2\theta+\cdots+\cos n\theta= {{\sin\,(n+{1\over2})\,\theta}\over{2\sin{1\over2}\theta}}\cr }$$ Here's a short proof for the third: $$\sin{\textstyle{1\over2}}\theta + \sum_{k=1}^n2\cos k\theta\,\sin{\textstyle{1\over2}}\theta =\sin{\textstyle{1\over2}}\theta+\sum_{k=1}^n\bigl[\, \sin\,(k+{\textstyle{1\over2}})\,\theta -\sin\,(k-{\textstyle{1\over2}})\,\theta\,\bigr] =\sin\,(n+{\textstyle{1\over2}})\,\theta.$$ The function $$D_n(t)={{\sin\,(n+{1\over2})\,t}\over{2\sin{1\over2}\,t}}$$ is called {\sl Dirichlet's kernel}. It plays an important role in our next calculation. Now we're ready to re-write our formula for $s_n(f)$. $$\openup2\jot\eqalign{ s_n(f)(x)&={\textstyle{1\over2}}a_0+ \sum_{k=1}^n\bigl(a_k\cos kx+b_k\sin kx\bigr)\cr &={1\over\pi}\int_{-\pi}^\pi f(t)\left[{\textstyle{1\over2}}+ \sum_{k=1}^n\cos kt\cos kx + \sin kt\sin kx\right]dt\cr &={1\over\pi}\int_{-\pi}^\pi f(t)\left[{\textstyle{1\over2}}+ \sum_{k=1}^n\cos k(t-x)\right]dt\cr &={1\over\pi}\int_{-\pi}^\pi f(t)\cdot {{\sin\,(n+{1\over2})\,(t-x)}\over{2\sin{1\over2}\,(t-x)}}\,dt\cr &={1\over\pi}\int_{-\pi}^\pi f(t)\,D_n(t-x)\,dt ={1\over\pi}\int_{-\pi}^\pi f(x+t)\,D_n(t)\,dt.\cr }$$ It now follows easily that $s_n(f)$ is linear in $f$ (because integration against $D_n$ is linear), that $s_n(f)\in{\cal T}_n$ (because $D_n\in{\cal T}_n$), and, in fact, that $s_n({\cal T}_m)={\cal T}_{\min(m,n)}$. In other words, $s_n$ is indeed a linear projection onto ${\cal T}_n$. While we know that $s_n(f)$ is a good approximation to $f$ in the $L_2$ norm, a better understanding of its effectiveness as a uniform approximation will require a better understanding of the Dirichlet kernel $D_n$. Here are a few pertinent facts: {\baselineskip=24 true pt \proclaim Lemma. \ {\rm(a)} \ $D_n$ is even, \item{\rm(b)} \ $\displaystyle{1\over\pi}\int_{-\pi}^\pi D_n(t)\,dt ={2\over\pi}\int_0^\pi D_n(t)\,dt=1$, \item{\rm(c)} \ $|D_n(t)|\le n+{1\over2}$ \ and \ $D_n(0)=n+{1\over2}$, \item{\rm(d)} \ $\displaystyle{{{|\sin\,(n+{1\over2})\,t\,|}\over t}} \ \le \ |D_n(t)|\le {\pi\over{2t}}$ \ for \ $0n} a_kT_k\right\Vert \ \le \ \sum_{k>n}|a_k| \ \le \ 2\,\Vert \varphi\,''\,\Vert\,\sum_{k>n}{1\over{k^2}}.$$ Lebesgue's theorem should remind you of our ``fancy'' version of Bernstein's theorem; if we knew that $E_n^T(f)\log n\to0$ as $n\to\infty$, then we'd know that $s_n(f)$ converged uniformly to $f$. Our goal, then, is to improve our estimates on $E_n^T(f)$, and the idea behind these improvements is to {\sl replace $D_n$ by a better kernel\/} (with regard to uniform approximation). Before we pursue anything quite so delicate as an estimate on $E_n^T(f)$, though, let's investigate a simple (and useful) replacement for $D_n$. Since the sequence of partial sums $(s_n)$ need not converge to $f$, we might try looking at their arithmetic means (or Ces\`aro sums): $$\sigma_n={{s_0+s_1+\cdots+s_{n-1}}\over n}.$$ (These averages typically have better convergence properties than the partial sums themselves. Just consider $\sigma_n$ in the (scalar) case $s_n=(-1)^n$, for example.) Specifically, we set $$\eqalign{ \sigma_n(f)(x) &={1\over n}(s_0(f)+\cdots+s_{n-1}(f))\cr &={1\over\pi}\int_{-\pi}^\pi f(x+t)\, \left[{1\over n}\sum_{k=0}^{n-1}D_k(t)\right]\,dt ={1\over\pi}\int_{-\pi}^\pi f(x+t)\,K_n(t)\,dt, }$$ where $K_n=(D_0+D_1+\cdots+D_{n-1})/n$ is called {\sl Fej\'er's kernel}. The same techniques we used earlier can be applied to find a closed form for $\sigma_n(f)$ which, of course, reduces to simplifying $(D_0+D_1+\cdots+D_{n-1})/n$. As before, we begin with a trig identity: $$\eqalign{ 2\sin\theta\,\sum_{k=0}^{n-1}\sin\,(2k+1)\theta &=\sum_{k=0}^{n-1}\bigl[\,\cos2k\theta-\cos\,(2k+2)\theta\,\bigr]\cr &=1-\cos2n\theta=2\sin^2n\theta.\cr }$$ Thus, $$K_n(t) \ = \ {1\over n}\sum_{k=0}^{n-1} {{\sin\,(2k+1)\,t/2}\over{2\sin\,(t/2)}} \ = \ {{\sin^2(nt/2)}\over{2n\sin^2(t/2)}}.$$ Please note that $K_n$ is even, {\sl nonnegative}, and ${1\over\pi}\int_{-\pi}^\pi K_n(t)\,dt=1$. Thus, $\sigma_n(f)$ is a positive, linear map from $C^{2\pi}$ onto ${\cal T}_n$ (but it's not a projection---why?), satisfying $\Vert\sigma_n(f)\Vert_2\le\Vert f\Vert_2$ (why?). Now the arithmetic mean operator $\sigma_n(f)$ is still a good approximation $f$ in $L_2$ norm. Indeed, $$\Vert f-\sigma_n(f)\Vert_2 ={1\over n}\left\Vert\,\sum_{k=0}^{n-1}(f-s_k(f))\,\right\Vert_2 \le{1\over n}\sum_{k=0}^{n-1}\Vert f-s_k(f)\Vert_2 \to0$$ as $n\to\infty$ (since $\Vert f-s_k(f)\Vert_2\to0$\/). But, more to the point, $\sigma_n(f)$ is actually a good {\sl uniform\/} approximation to $f$, a fact that we'll call {\sl Fej\'er's theorem\/}: \proclaim Theorem. If\/ $f\in C^{2\pi}$, then\/ $\sigma_n(f)$ converges uniformly to $f$ as $n\to\infty$. Note that since $\sigma_n(f)\in{\cal T}_n$, Fej\'er's theorem implies Weierstrass's (second) theorem. Curiously, Fej\'er was only 19 years old when he proved this result (about 1900) while Weierstrass was 75 at the time he proved his approximation theorems. We'll give two proofs of Fej\'er's theorem; one with details, one without. But both follow from quite general considerations. First: {\baselineskip=24 true pt \proclaim Theorem. Suppose that $k_n\in C^{2\pi}$ satisfies \item{\rm(a)} \ $k_n\ge0$, \item{\rm(b)} \ $\displaystyle{{1\over\pi}\int_{-\pi}^\pi k_n(t)\,dt=1}$, and \item{\rm(c)} \ $\displaystyle\int_{\delta\le|t|\le\pi}k_n(t)\,dt\to0$ for every $\delta>0$. \vskip0pt\noindent Then, \ $\displaystyle{1\over\pi}\int_{-\pi}^\pi f(x+t)\,k_n(t)\,dt\uniformto f(x)$ \ for each $f\in C^{2\pi}$. \par} \proof Let $\eps>0$. Since $f$ is uniformly continuous, we may choose $\delta>0$ so that $|f(x)-f(x+t)|<\eps$, for any $x$, whenever $|t|<\delta$. Next, we use the fact that $k_n$ is nonnegative and integrates to $1$ to write $$\openup2\jot\eqalign{ \left|f(x)-{1\over\pi}\int_{-\pi}^\pi f(x+t)\,k_n(t)\,dt\,\right| \ &= \ {1\over\pi}\left|\int_{-\pi}^\pi \bigl[f(x)-f(x+t)\bigr]\,k_n(t)\,dt\,\right|\cr &\le \ {1\over\pi}\int_{-\pi}^\pi \bigl|f(x)-f(x+t)\bigr|\,k_n(t)\,dt\cr &\le \ {\eps\over\pi}\int_{|t|<\delta}k_n(t)\,dt \ + \ {{2\Vert f\Vert}\over\pi}\int_{\delta\le|t|\le\pi}k_n(t)\,dt\cr &<\eps+\eps=2\eps, }$$ for $n$ sufficiently large.~\qed To see that Fej\'er's kernel satisfies the conditions of the Theorem is easy: In particular, (c) follows from the fact that $K_n(t)\uniformto0$ on the set $\delta\le|t|\le\pi$. Indeed, since $\sin(t/2)$ increases on $\delta\le t\le\pi$ we have $$K_n(t) \ = \ {{\sin^2(nt/2)}\over{2n\sin^2(t/2)}} \ \le \ {1\over{2n\sin^2(\delta/2)}} \ \to0.$$ Our second proof, or sketch, really, is based on a variant of the Bohman-Korovkin theorem for $C^{2\pi}$, due to Korovkin. In this setting, the three ``test cases'' are $$f_0(x)=1,\qquad f_1(x)=\cos x,\qquad\hbox{and}\qquad f_2(x)=\sin x.$$ \proclaim Theorem. Let $(L_n)$ be a sequence of positive, linear maps on $C^{2\pi}$. If\/ $L_n(f)\uniformto f$ for each of the three functions $f_0(x)=1$, $f_1(x)=\cos x$, and $f_2(x)=\sin x$, then $L_n(f)\uniformto f$ for every $f\in C^{2\pi}$. We won't prove this theorem; rather, we'll check that $\sigma_n(f)\uniformto f$ in each of the three test cases. Since $s_n$ is a projection, this is painfully simple! $$\eqalign{ \sigma_n(f_0)&=\textstyle{1\over n}\,(f_0+f_0+\cdots+f_0)=f_0,\cr \sigma_n(f_1)&=\textstyle{1\over n}\,(0+f_1+\cdots+f_1) ={{n-1}\over n}\cdot f_1\uniformto f_1,\cr \sigma_n(f_2)&=\textstyle{1\over n}\,(0+f_2+\cdots+f_2) ={{n-1}\over n}\cdot f_2\uniformto f_2.\cr }$$ Kernel operators abound in analysis; for example, Landau's proof of the Weierstrass theorem uses the kernel $L_n(x)=c_n(1-x^2)^n$. And, in the next section, we'll encounter {\sl Jackson's kernel\/} $J_n(t)=\sin^4nt/n^3\sin^2t$, which is essentially the square of Fej\'er's kernel. While we will have no need for a general theory of such operators, please note that the key to their utility is the fact that they're nonnegative! Lastly, a word or two about Fourier series involving complex coefficients. Most modern textbooks consider the case of a $2\pi$-periodic, integrable function $f:\R\to\C$ and define the Fourier series of $f$ by $$\sum_{k=-\infty}^\infty c_ke^{ikt},$$ where now we have only one formula for the $c_k$'s: $$c_k={1\over{2\pi}}\int_{-\pi}^\pi f(t)\,e^{-ikt}\,dt,$$ but, of course, the $c_k$'s may well be complex. This somewhat simpler approach has other advantages; for one, the exponentials $e^{ikt}$ are now an orthonormal set (relative to the normalizing constant $1/2\pi$). And, if we remain consistent with this choice and define the $L_2$ norm by $$\Vert f\Vert_2 =\left({1\over{2\pi}}\int_{-\pi}^\pi |f(t)|^2\,dt\right)^{1/2},$$ then we have the simpler estimate $\Vert f\Vert_2\le\Vert f\Vert$ for $f\in C^{2\pi}$. The Dirichlet and Fejer kernels are essentially the same in this case, too, except that we would now write $s_n(f)(x)=\sum_{k=-n}^nc_ke^{ikx}$. Given this, the Dirichlet and Fej\'er kernels can be written $$\openup2\jot\eqalign{ D_n(x)=\sum_{k=-n}^ne^{ikx}&=1+\sum_{k=1}^n(e^{ikx}+e^{-ikx})\cr &=1+2\sum_{k=1}^n\cos kx\cr &={{\sin\,(n+{1\over2})\,x}\over{\sin{1\over2}\,x}}\cr }$$ and $$\openup2\jot\eqalign{ K_n(x)&={1\over n}\sum_{m=0}^{n-1}\sum_{k=-m}^me^{ikx} =\sum_{k=-n}^n\left(1-{|k|\over n}\right)\,e^{ikx}\cr &={1\over n}\sum_{m=0}^{n-1}{{\sin\,(m+{1\over2})\,x}\over{\sin{1\over2}\,x}}\cr &={{\sin^2(nt/2)}\over{n\sin^2(t/2)}}.\cr }$$ In other words, each is twice its real coefficient counterpart. Since the choice of normalizing constant ($1/\pi$ versus $1/2\pi$, and sometimes even $1/\sqrt{\pi}$ or $1/\sqrt{2\pi}\,$) has a (small) effect on these formulas, you may find some variation in other textbooks. \bye %% end of chapter-07.tex