MATH 522 (Analysis II) Definitions and Theorems

Ruixuan Tu
ruixuan.tu@wisc.edu

Joris Ross & Andreas Seeger, 25 January 2023

Walter Rudin: Principles of Mathematical Analysis

MATH 521 Notes

More in this series

Prerequisites

• $$\log$$ function (from SDL)
• $$b^y=x \iff y=\log_b(x)$$
• $$a^{\log_a(N)}=N$$
• $$\log(1)=0$$
• $$\log(s)+\log(t)=\log(st)$$
• $$\log(n)-\log(d)=\log\left( \frac{n}{d} \right)$$
• $$p\log(m)=\log(m^p)$$
• $$\log_a(b)=\frac{\log_c(b)}{\log_c(a)}$$
• $$\log_a(b)=\frac{1}{\log_b(a)}$$
• Taylor series (from MIT and U South Carolina)
• (geometric series) $$\forall x\in (-1, 1)$$: $$\frac{1}{1-x}=1+x+x^2+x^3+x^4+\cdots = \sum_{n=0}^{\infty} x^n$$
• (geometric series, finite sum) $$\sum_{k=0}^{n-1}ar^k=\begin{cases} a\left( \frac{1-r^n}{1-r} \right) & \text{if } r\not= 1 \\ an & \text{if } r=1 \end{cases}$$
• $$\forall x\in \mathbb{R}$$: $$e^x = 1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+\frac{x^4}{4!}+\cdots =\sum_{n=0}^{\infty} \frac{x^n}{n!}$$
• $$\forall x\in \mathbb{R}$$: $$\cos(x)=1-\frac{x^2}{2!}+\frac{x^4}{4!}-\frac{x^6}{6!}+\frac{x^8}{8!}-\cdots=\sum_{n=0}^{\infty} (-1)^n \frac{x^{2n}}{(2n)!}$$
• $$\forall x\in \mathbb{R}$$: $$\sin(x)=x-\frac{x^3}{3!}+\frac{x^5}{5!}-\frac{x^7}{7!}+\frac{x^9}{9!}-\cdots=\sum_{n=0}^{\infty}(-1)^n \frac{x^{2n+1}}{(2n+1)!}$$
• $$\forall x\in (-1, 1]$$: $$\ln(1+x)=x-\frac{x^2}{2}+\frac{x^3}{3}-\frac{x^4}{4}+\frac{x^5}{5}-\cdots=\lim_{n=1}^{\infty}(-1)^{n+1} \frac{x^n}{n}$$
• $$\forall x\in [-1, 1]$$: $$\tan^{-1}(x)=x-\frac{x^3}{3}+\frac{x^5}{5}-\frac{x^7}{7}+\frac{x^9}{9}-\cdots =\sum_{n=0}^{\infty} \frac{(-1)^n}{2n+1} x^{2n+1}$$
• $$\forall x\in (-1, 1)$$: $$(1+x)^p=\sum_{n=0}^{\infty} {p\choose n} x^n$$

Lecture 1 (0125)

1. [Def] uniformly convergence of function: We say that $$f_n: X\to \mathbb{R} \text{ or } \mathbb{C}$$ is uniformly convergent with limit $$f: X\to \mathbb{R} \text{ or } \mathbb{C}$$ if $$\forall \varepsilon>0, \exists N\in \mathbb{N}$$ such that $$\forall n\geq N, \forall x\in X, \left| f_n(x)-f(x) \right| < \varepsilon$$. Note: Def 1.23
2. [Def] uniformly convergence of function: Equivalently, $$f_n\to f$$ uniformly if $$n\to \sup_{x\in X} \left| f_n(x)-f(x) \right|$$ is convergent to $$0$$ or $$\left\{ \sup_{x\in X} \left| f_n(x)-f(x) \right| \right\}_{n\in \mathbb{N}}$$. Note: Lemma 1.24
3. [Def] uniformly convergence of function: $$\forall \varepsilon>0, \exists N_{\varepsilon}\in \mathbb{N}, \forall x\in X, \forall n\geq N_{\varepsilon}: \left| f_n(x)-f(x) \right| < \varepsilon$$. Note: Def 1.23
4. [Def] pointwise convergence of function differs in $$\forall \varepsilon>0, \forall x\in X, \exists N_{\varepsilon, x}\in \mathbb{N}, \forall n\geq N_{\varepsilon, x}: \left| f_n(x)-f(x) \right| < \varepsilon$$. Note: Def 1.23
5. [Def] uniform convergence of series of function: $$\sum_{k=1}^{\infty} f_k(x)$$ converges uniformly if $$S_n(x)=\sum_{k=1}^{n} f_k(x)$$ converges uniformly
6. [Def] uniformly Cauchy of series of function: A sequence $$(f_n)_{n\in \mathbb{N}}$$ of functions on a set $$X$$ is uniformly Cauchy iff $$\forall \varepsilon>0, \exists N\in \mathbb{N}$$ such that $$\forall n, m\geq N, \forall x\in X, \left| f_n(x)-f_m(x) \right| < \varepsilon$$, i.e., $$\sup_{x\in X} \left| f_n(x)-f_m(x) \right| < \varepsilon$$. Note: Lemma 1.25
7. [Thm] A sequence is uniformly Cauchy iff it is uniformly convergent
8. [Observation] If $$f_n$$ is uniformly Cauchy, then $$\forall x\in X$$, the sequence of numbers $$f_n(x)$$ is Cauchy

Lecture 2 (0127)

1. [Thm] to estimate integrals
• With $$a<b$$, $$\left| \int_{a}^{b} f(x) ~dx \right| \leq \int_{a}^{b} \left| f(x) \right| ~dx$$
• With $$f_1, f_2$$ integrable, if $$f_1(x) \leq f_2(x)$$, then $$\forall x\in [a, b], \int_{a}^{b} f_1(x) ~dx \leq \int_{a}^{b} f_2(x) ~dx$$
2. [Thm] Weierstrass $$M$$-test: Given a numerical sequence $$a_k \geq 0$$ such that $$\sum_{k=1}^{\infty} a_k$$ converges; given a sequence of function $$f_k: X\to \mathbb{R} \text{ or } \mathbb{C}$$ such that $$\left| f_k(x) \right| \leq a_k$$; then $$\sum_{k=1}^{\infty} f_k(x)$$ converges uniformly on $$X$$

Lecture 3 (0130)

1. [Thm] Taylor’s theorem on polynomial approximation: $$f(a+h)=f(a)+f'(a)h+f''(a)\frac{h^2}{2}+\cdots$$ (an error bound better than $$h^2$$)
2. [Thm] Given $$f_n: X\to \mathbb{R} \text{ or } \mathbb{C}$$, where $$X$$ is a metric space. Assume (i) $$f_n$$ converges uniformly on $$X$$ to $$f$$ and (ii) $$f_n$$ is continuous at a point $$a\in X$$. Then $$f$$ is continuous at $$a$$
3. [Thm] fundamental theorem of calculus: $$f(x)=f(x_0)+\int_{x_0}^{x} f'(t) ~dt$$

Lecture 4 (0201)

1. [Thm] Taylor’s theorem (or Maclaurin’s theorem): Assume $$f\in C^{k+1}$$ () on interval $$I$$, $$a\in \operatorname{Int}(I)$$. Then $$\forall a+h\in I$$ we have $$f(a+h)=\sum_{j=0}^{k}\frac{h^j}{j!}f^{(j)}(a)+E_kf(a, h)$$ where $$E_kf(a, h)=\frac{a^{k+1}}{(k+1)!}\int_{0}^{1} (k+1)(1-s)^k f^{(k+1)}(a+sh) ~ds$$. One can write $$E_kf(a, h)=\frac{h^{k+1}}{(k+1)!}f^{(k+1)}(\xi)$$, $$\xi$$ between $$a$$ and $$a+h$$. Note: Thm A.17

Lecture 5 (0203)

1. [Thm] Taylor’s theorem for polynomial of degree $$\leq n$$: $$P(x)=\sum_{k=0}^{n} \frac{P^{(k)}(a)}{k!} (x-a)^k + E_n(x)$$, $$E_n(x)=\frac{x^{n+1}}{(n+1)!} P^{(n+1)}(\xi)$$
2. [Def] exponential: $$e=\lim_{n\to \infty} \left( 1+\frac{1}{n} \right)^n$$; $$e=\sum_{k=0}^{\infty} \frac{1}{k!}$$; $$e$$ is the unique number $$a$$ which means the integral $$\int_{0}^{a} \frac{dt}{t} = 1$$; $$\log(x)=\int_{1}^{x} \frac{dt}{t} ~dx$$
3. [Thm] Taylor’s polynomial for $$e^x$$: $$e^x=\sum_{k=0}^{n} \frac{x^k}{k!} + \frac{x^{n+1}}{(n+1)!}e^{\xi}$$ for some $$\xi$$ between 0 and $$x$$; the remainder converges to $$0$$, uniformly on $$[-b, b]$$, but does not converge uniformly to $$0$$ on $$(-\infty, \infty)$$
4. [Thm] $$e=\sum_{k=0}^{n} \frac{1}{k!} + \frac{e^{\xi}}{(n+1)!}$$ where $$\xi$$ is between $$0$$ and $$1$$
5. [Thm] $$e$$ is not a rational number
6. [Thm] Taylor series for $$x\in \mathbb{R} \text{ or }\mathbb{C}$$: $$e^x=\sum_{k=0}^{\infty}\frac{x^k}{k!}$$; $$\cos(x)=\sum_{k=0}^{\infty} \frac{(-1)^k x^{2k}}{(2k)!}$$; $$\sin(z)=\sum_{k=0}^{\infty} \frac{(-1)^k x^{2k+1}}{(2k+1)!}$$
7. [Thm] $$e^{ib}=\cos b + i \sin b$$ for $$z=ib$$ without real part

Lecture 6 (0206)

1. [Def] Cauchy product of power series: $$\left( \sum_{i=0}^{\infty} a_i x^i \right)\left( \sum_{j=0}^{\infty} b_j x^j \right)=\sum_{k=0}^{\infty} c_k x^k$$ where $$c_k=\sum_{l=0}^{k} a_l b_{k-l}$$
2. [Def] power series: $$\sum_{n=0}^{\infty} a_n z^n$$ where $$z$$ is a complex (or just real) number
3. [Thm] complex multiplication:
• $$(a+ib)(c+id)=(a+c)+(b+d)i=ac-bd+(ad-bc)i$$
• with $$a+ib=r(\cos \alpha+i\sin \alpha)$$, $$c+id=R(\cos \beta+i\sin \beta)$$, $$(a+ib)(c+id)=Rr\left( \cos(\alpha+\beta) + i \sin(\alpha+\beta) \right)$$
• $$z^n=r^n(\cos \alpha+i\sin \alpha)^n=r^n(\cos n\alpha+i\sin n\alpha)$$ by $$\cos\alpha + i\sin \alpha=e^{i\alpha}$$
4. [Thm] about convergence: $$a_n\in \mathbb{C}$$. There is a unique $$R\in [0, \infty)$$ such that
1. $$\sum_{n=0}^{\infty} a_n z^n$$ is convergent for all $$z$$ with $$|z|<R$$
2. $$\sum_{n=0}^{\infty} a_n z^n$$ is divergent for all $$z$$ with $$|z|>R$$
radius of convergence $$R$$ can be computed as $$R=\frac{1}{\operatorname{\lim\sup}_{n\to\infty} |a_n|^{\frac{1}{n}}}$$

Lecture 7 (0208)

1. [Lemma] $$\sum_{k=0}^{\infty} (-1)^k \frac{x^{2k+1}}{2k+1} = \sum_{k=0}^{\infty} (-1)^k \int_{0}^{x} t^{2k} ~dt = \int_{0}^{x} \sum_{k=0}^{\infty} (-t^2)^k ~dt = \int_{0}^{x} \frac{1}{1+t^2} ~dt = \arctan x$$, and the sum is uniformly convergent on $$[-1, 1]$$
2. [Note] even if we have uniform convergence on $$[-b, b]$$ for all $$b<1$$, we may not have uniform convergence on $$(-1, 1)=\cup_{b<1} [-b, b]$$
3. [Thm] If the series converges uniformly on $$[-1, 1]$$ then the limit sum is a continuous function
4. [Thm] If we can differentiate the power series $$f(x)=\sum_{n=0}^{\infty} a_n x^n$$, then $$f$$ is differentiable and $$f'(x)=\sum_{n=1}^{\infty} a_n n x^{n-1}=\sum_{n=0}^{\infty} a_{n+1} (n+1) x^n$$
5. [Thm] $$\sum_{n=0}^{\infty} a_{n+1} (n+1) x^n$$ has the same radius of convergence and therefore converges uniformly in $$\left[ -R(1-\varepsilon), R(1-\varepsilon) \right]$$
6. [Thm] Power series is its own Taylor series

Lecture 8 (0210)

1. [Thm] Abel’s theorem: If $$\sum_{k=0}^{\infty} a_k$$ converges, then $$\sum_{k=0}^{\infty} a_k = \lim_{r\to 1^-} \sum_{k=0}^{\infty} a_k r^k$$
2. [Thm] Abel’s summation formula (summation by parts): $$\sum_{n=p}^{q} a_n b_n=A_q b_q - A_{p-1} b_p+\sum_{n=p}^{q-1} A_n (b_n - b_{n+1})$$ where $$A_n = \sum_{k=0}^{n} a_k$$
3. [Thm] Another Abel’s summation formula: $$\sum_{k=0}^{n}f_k g_k=f_0 \sum_{k=0}^{n} g_k - \sum_{j=0}^{n-1} (f_{j+1}-f_j) \sum_{k=0}^{j} g_k$$ (from Wikipedia)

Lecture 9 (0213)

1. [Thm] $$\sum_{k=0}^{n} \cos kx+i \sum_{k=0}^{n} \sin kx=\sum_{k=0}^{n} e^{ikx}=\frac{1-(e^{ix})^{n+1}}{1-(e^{ix})}\leq \frac{\left\vert 1-(e^{ix})^{n+1} \right\vert}{\left\vert 1-(e^{ix}) \right\vert}\leq \frac{2}{\left\vert 1-(e^{ix}) \right\vert}$$
2. [Def] norm: On a vector space $$\mathbb{V}$$ a norm $$x\mapsto \left\| x \right\|$$ is a function $$\mathbb{V}\to [0, \infty)$$ s.t.
1. $$\left\| x \right\|=0$$ iff $$x=0$$
2. $$\left\| cx \right\|=\left\vert c \right\vert \left\| x \right\|$$ for all scalars in $$\mathbb{R}$$ or $$\mathbb{C}$$
3. $$\left\| x+y \right\|\leq \left\| x \right\| + \left\| y \right\|$$
3. [Rmk] If we have a norm, $$d(x, y)=\left\| x-y \right\|$$
4. [Thm] $$\max \left\vert x_i \right\vert \leq \left( \sum_{i=1}^{n} \left\vert x_i \right\vert ^2 \right) ^\frac{1}{2} \leq \sum_{i=1}^{n} \left\vert x_i \right\vert \leq n \max \left\vert x_i \right\vert$$
5. [Def] $$\ell^p$$ spaces:
1. $$\ell^1(\mathbb{N})$$: the function on $$\mathbb{N}$$ $$n\mapsto f(n)$$ (or we could write $$a_n$$) with the property $$\sum_{n=1}^{\infty} \left\vert a_n \right\vert$$ is finite
2. $$\ell^1(\mathbb{N})$$ is the space of (absolutely) summable sequences $$\sum_{n=1}^{\infty} \left\vert a_n \right\vert$$
3. $$\ell^{\infty}(\mathbb{N})$$: the space of bounded sequences $$\sup_{n\in \mathbb{N}} \left\vert a_n \right\vert$$

Lecture 10 (0215)

1. [Def] Euler’s constant or Euler-Mascheroni constant: $$\lim_{n\to \infty} \left( \sum_{k=1}^{n} \frac{1}{k} - \ln(x) \right) = \gamma$$, also as $$0.5\dots$$
2. [Thm] Subsets of a metric space $$(X, d)$$, $$Y\subset X$$, $$Y$$ is a metric space, with the metric $$d|_{Y\times Y}$$
3. [Def] open ball: open ball on metric space $$X$$, centered at a point $$y\in Y$$ with radius $$r$$: $$B_X(y, r)=\left\{ x\in X: d(x, y)<r \right\}$$
4. [Col] $$B_Y(y, r)=\left\{ w\in Y, d(w, y)<r \right\}=B_X(y, r)\cap Y$$
5. [Thm] Let $$Y$$ be a subset of $$X$$ with the metric inherited from $$X$$. Then a set $$\mathcal{U}\subset Y$$ is open in $$Y$$ if and only if there is an open set $$O$$ with respect to the metric $$X$$ in $$X$$ such that $$O\cap Y=\mathcal{U}$$
6. [Def] compact metric space: A metric space $$(X, d)$$ is compact if the following holds: Whenever $$X=\cup_{\alpha\in A} O_\alpha$$ with $$O_\alpha$$ open ($$X$$ as a union of open sets), then there are finitely many $$\alpha_1, \dots, \alpha_n\in A$$, $$X=\cup_{i=1}^{n} O_{\alpha_i}$$
7. [Def] compact subset $$K$$ of metric space $$X$$: $$K\subset X$$ is compact if every cover of $$K$$ with open sets has a finite subcover. Meaning: Whenever $$K\subset \cup_{\alpha \in A} O_\alpha$$ with $$O_\alpha$$ open in $$X$$, then there are indices $$\alpha_1, \dots, \alpha_n\in A$$ such that $$K\subset \cup_{i=1}^{n} O_{\alpha_i}$$. Note: consider $$K$$ with the metric inherited from $$X$$, so $$K$$ is a metric space
8. [Thm] Heine-Borel: A subset $$K$$ of $$\mathbb{R}^n$$ is compact if and only if $$K$$ is bounded and closed.
9. [Def] bounded set: $$K\subset X$$ is bounded if $$\exists C$$ such that $$d(x, w)\leq C$$ for all $$x, w\in K$$. In a metric space a set $$Y$$ is bounded if $$\left\| y \right\|<C$$ for all $$y\in Y$$
10. [Note] consider $$X=\ell^{\infty}(\mathbb{N})$$ all bounded sequence, $$n\to f(n)$$, $$f\in \ell^{\infty}(\mathbb{N})$$ if $$\sup_{n\in N} \left\vert f(n) \right\vert <\infty$$

Lecture 11 (0217)

1. [Def] sequentially compact: A metric space $$X$$ is sequentially compact if every sequence in $$X$$ has a convergent subsequence (a.k.a. Bolzano-Weierstrass property)
2. [Def] subsequence: $$b_n$$ is a subsequence of $$a_n$$ if there is a strictly increasing function $$K: \mathbb{N}\to \mathbb{N}$$ such $$b_n=a_{K(n)}$$ ($$K(n)$$ is index-sequence of the subsequence)
3. [Def] Cantor diagonal subsequence: $$d_n=a_n^{(n)}=a_{k(n)}^{(0)}$$, $$k(n)=K_1\circ K_2\circ \cdots \circ K_n(n)$$, with definition $$a_n^{(l)}=a_{K_l(n)}^{(l-1)}=\cdots=a_{K_1\circ K_2\circ \cdots \circ K_l(n)}^{(0)}$$, and $$(d_n)$$ is a subsequence of $$\left( a_n^{(1)} \right), \left( a_n^{(2)} \right), \left( a_n^{(3)} \right), \dots$$
4. [Def] totally bounded: A metric space $$(X, d)$$ is totally bounded if for every $$\varepsilon>0$$, $$X$$ is a union of finitely many balls of radius $$\varepsilon$$
5. [Def] complete metric space: a metric space $$M=(X, d)$$ is called complete (or a Cauchy space) if every Cauchy sequence of points in $$M$$ has a limit that is also in $$M$$ (Wikipedia)
6. [Thm] The following are equivalent for a given metric space $$X$$
1. $$X$$ is compact (every cover with open sets has a finite subcover)
2. $$X$$ is sequentially compact
3. $$X$$ is totally bounded and complete (every Cauchy sequence converges)

Lecture 12 (0220)

1. [Thm] The following are equivalent, $$X$$ is a metric space
1. $$X$$ is compact
2. $$X$$ is sequentially compact
3. $$X$$ is totally bounded and complete
Proof: 1 $$\implies$$ 2 $$\implies$$ 3 $$\implies$$ 2 $$\overset{3}{\implies}$$ 1
2. [Lemma] Consider a fixed cover $$(O_\alpha)$$ with open sets. There exists an $$\varepsilon>0$$ so that all balls of radius $$\varepsilon$$ are contained in at least one of the $$O_\alpha$$

Lecture 13 (0222)

1. [Thm] Let $$X$$ be a compact metric space and $$f: X\to \mathbb{R}$$ be continuous. Then
1. $$f$$ is bounded, i.e., $$\sup_{x\in X} \left\vert f(x) \right\vert <\infty$$
2. $$f$$ has a maximum and a minimum on $$X$$ (there is an $$x_0\in X$$, $$f(x_0)=\sup_{x\in X} f(x)$$ and $$\tilde{x_0}\in X$$, $$f(\tilde{x_0})=\inf_{x\in X} f(x)$$)
3. $$f$$ is uniformly continuous, i.e., for every $$\varepsilon>0$$ there is $$\delta$$ such that $$\left\vert f(x_1)-f(x_2) \right\vert <\varepsilon$$ if $$d(x_1, x_2)<\delta=\sigma(\varepsilon)$$ [Note: $$\sigma(\varepsilon)$$ does not depend on $$x\in K$$]
2. [Example] None of these may hold if $$X$$ is not compact: $$X=(0, 1)$$, (i) $$f(x)=\frac{1}{x}$$, (ii) $$f(x)=\sin\left( \frac{1}{x} \right)$$
3. [Thm] Heine-Borel theorem: Every bounded and closed set $$E$$ in $$\mathbb{R}^n$$ is compact
4. [Def] pointwise continuous: A map $$f:X\to Y$$ is called continuous at $$x\in X$$ if for every $$\varepsilon > 0$$ there exists $$\delta > 0$$ such that if $$d_X(x, y)<\delta$$, then $$d_Y(f(x), f(y))<\varepsilon$$ (from Note 1.19)

Lecture 14 (0224)

1. [Lemma] If $$X$$ is compact, if $$K$$ is closed in $$X$$ then $$K$$ is compact
2. [Thm] Consider $$\mathscr{c}_0=$$ vector space of sequences $$a$$ which converge to 0, i.e., $$\lim_{n\to\infty} a_n=0$$ if and only if $$\lim_{j\to\infty} \sup_{a\in K} \left\vert a_j \right\vert =0$$

Lecture 15 (0227)

1. [Thm] A subset $$A$$ of $$\mathscr{c}_0$$ is totally bounded if the sequence $$n\to \sup_{a\in A} a_n$$ converges to zero

2. [Lemma] Suppose $$A\subset X$$ and assume that for every $$\varepsilon > 0$$, there is a finite number of balls $$B(z, \varepsilon)$$, $$z\in Z$$ ($$Z$$: finite set in $$\mathscr{c}_0$$), $$\# z<\infty$$. $$A\subset \cup_{z\in Z} B(z, \varepsilon)$$, then $$A$$ is totally bounded

3. [Prop] Given $$\varepsilon>0$$, find a finite collection of balls $$B(z, \frac{\varepsilon}{2})$$, $$z\in Z$$ (finite). Let $$Z'\subset Z$$ be the collection of those $$z\in Z$$ for which $$B(z, \frac{\varepsilon}{2})\cap A$$

For $$z\in Z'$$ there is an $$a_z\in A\cap B(z, \frac{\varepsilon}{2})$$, $$B(a_z, \varepsilon)\supset B(a_z, \frac{\varepsilon}{2})$$

If $$y\in B(z, \frac{\varepsilon}{2}), d(y, z)<\frac{\varepsilon}{2}, d(a_z, z)<\frac{\varepsilon}{2}$$, then by triangle inequality $$d(y, a_z)<\varepsilon$$

4. [Def] pointwise bounded functions: A class $$A$$ of functions is pointwise bounded if for every $$x$$ there is $$M_x<\infty$$ such that $$\sup_{f\in A} \left\vert f(x) \right\vert \leq M_x$$

5. [Def] uniformly bounded functions: $$A$$ is uniformly bounded if $$\sup_{x\in X} \sup_{f\in A} \left\vert f(x) \right\vert \leq M_C$$

6. [Rmk] If $$A$$ is uniformly bounded, $$A$$ may not be totally bounded

7. [Fact] totally bounded $$\implies$$ uniformly bounded (Linrong)

8. [Def] equicontinuous: $$A\subset C(K)$$ is equicontinuous if for every $$\varepsilon$$ there exists a $$\delta$$ such that $$\left\vert f(x)-f(y) \right\vert < \varepsilon$$ for all $$f\in A$$ and for all $$x, y$$ such $$d(x, y)<\delta$$

9. [Thm] Arzelà–Ascoli: $$A\subset C(K)$$ is totally bounded if and only if $$A$$ is pointwise bounded and equicontinuous

Proof: $$w_{\delta}(f)=\sup_{d(x, y)< \delta} d(f(x), f(y))$$, $$\sup_{f\in A} w_{\delta}(f)\to 0$$ as $$\delta \to 0$$

10. [Rmk] Arzelà–Ascoli theorem characterizes the totally bounded sets $$f: K\to \mathbb{R} \text{ or } \mathbb{C}$$ in the sense $$C(K)$$ (continuous functions on a compact set $$K$$), $$\left\| f \right\|=\sup_{x\in K} \left\vert f(x) \right\vert$$

11. [Rmk] If $$A\subset C(K)$$ is equicontinuous, then $$A$$ is pointwise bounded if and only if it is uniformly bounded

Lecture 18 (0306)

1. [Def] contraction: If $$(X, d)$$ is a metric space then $$T: X\to X$$ is called a contraction if there exists $$\alpha<1$$ such that $$d(T(x), T(y))\leq \alpha d(x, y)$$

2. [Thm] Banach fixed point theorem (aka contraction principle): If $$(X, d)$$ is complete, then any contraction have a unique fixed point, i.e., a unique $$x\in X$$ such that $$T(x)=x$$

[Proof] method of successive approximation (construct a sequence, pick $$x_0$$, then $$x_1=T(x_0), x_2=T(x_1), \dots, x_{n+1}=T(x_n)$$); $$d(x_{n+1}, x_n)=d(T(x_n), T(x_{n-1}))\leq \alpha d(x_n, x_{n-1})\leq \alpha \cdot \alpha^{n-1} d(x_1, x_0)$$ $$\implies$$ $$d(x_n, x_{n-1})\leq \alpha^{n-1} d(x_0, x_1)$$

Lecture 19 (0308)

1. [Def] initial value problem for differential equation

Pick a point $$(x_0, y_0)$$ to be initial point,

• $$y'(x)=F(x, y(x))$$
• $$y(x_0)=y_0$$ (initial condition)

For a function $$(x, y)\mapsto F(x, y)$$ continuous near $$x_0, y_0$$, we define the initial value problem

2. [Thm] Peano existence theorem: If $$F$$ is continuous near $$(x_0, y_0)$$, then the initial value problem has a solution (based on compactness)

3. [Thm] Picard–Lindelöf theorem: If in addition there is a condition $$\left\vert F(x, y)-F(x, \tilde{y}) \right\vert \leq C\left\vert y-\tilde{y} \right\vert$$ for $$(x, y)$$ and $$(x, \tilde{y})$$ near $$(x_0, y_0)$$; then the solution is unique (based on the contraction principle)

4. [Thm] Picard-Lindelöf theorem: Let $$D\subseteq \mathbb{R}\times \mathbb{R}^n$$ be a closed rectangle with $$(t_0, y_0)\in D$$. Let $$f: D\to \mathbb{R}^n$$ be a function that is continuous in $$t$$ and Lipschitz continuous in $$y$$. Then there exists some $$c>0$$ such that the initial value problem has a unique solution $$y(t)$$ on the interval $$\left[ x_0-\delta, x_0+\delta \right]$$ (from Wikipedia)

5. [Def] Lipschitz continuity: Given two metric spaces $$(X, d_X)$$ and $$(Y, d_Y)$$, a function $$f: X\to Y$$ is called Lipschitz continuous if there exists a real constant $$K\geq 0$$ such that, for all $$x_1$$ and $$x_2$$ in $$X$$, $$d_Y(f(x_1), f(x_2))\leq Kd_X(x_1, x_2)$$. Any such $$K$$ is a Lipschitz constant. If $$0\leq K<1$$ and $$f$$ maps a metric space to itself, the function is called a contraction (from Wikipedia)

6. [Def] Lipschitz condition: Function $$f(t, y)$$ satisfies a Lipschitz condition in the variable $$y$$ on a set $$D\subseteq \mathbb{R}^2$$ if a constant $$L>0$$ exists with $$\left\vert f(t, y_1)-f(t, y_2) \leq L \left\vert y_1-y_2 \right\vert \right\vert$$ whenever $$(t, y_1)$$, $$(t, y_2)$$ are in $$D$$. $$L$$ is Lipschitz constant (from UCB Math)

7. [Lemma] For a continuous function $$x\to y(x)$$ defined on $$\left[ x_0-\delta, x_0+\delta \right]$$ with values in $$\left[ y_0-b, y_0+b \right]$$. The following two conditions are equivalent:

1. $$y$$ is of class $$C^1$$ ($$y'$$ is continuous) on $$\left[ x_0-\delta, x_0+\delta \right]$$ and satisfies $$y'(x)=F(x, y(x))$$, $$y(x_0)=y_0$$
2. $$y$$ satisfies the integral equation $$y(x)=y_0+\int_{x_0}^{x} F(t, y(t)) ~dt$$

Lecture 20 (0310)

1. [Cor] Let $$R$$ be the rectangle $$\left\{ (x, y): \left\vert x-x_0 \right\vert \leq a, \left\vert y-y_0 \right\vert \leq b \right\}$$ and assume $$F$$ is a continuous function on $$R$$, $$M=\max{(x, y)\in R} \left\vert F(x, y) \right\vert$$. For $$\delta$$ in Picard-Lindelöf, $$\delta< \min\left\{ a, \frac{b}{M} \right\}$$; weakly (without contraction), additionally $$\delta < \frac{1}{C}$$

Lecture 21 (0320)

1. [Def] methods of successive approximation: including fixed-point iteration, Picard-Lindelöf theorem, Euler polynomial method
2. [Def] Euler polynomial method:
1. $$y'(y)=F(t, y(t))$$, $$y'(t_0)=F(t, y_0)$$
2. Given a partition of $$[t_0, t_0+a_\star]$$ s.t. $$t_0<t_1<\cdots<t_N=t_0+a_\star$$
3. Define $$y(t)=\begin{cases} y_0 + F(t_0, y_0)(t-t_0) & t_0 \leq t \leq t_1 \\ y(t_1) + F(t_1, y(t_1))(t-t_1) & t_1 \leq t \leq t_2 \\ y(t_2) + F(t_2, y(t_2))(t-t_2) & t_2 \leq t \leq t_3 \\ \vdots \end{cases}$$

Lecture 23 (0324)

1. [Def] $$\ell^p$$ norms on $$\mathbb{R}^n$$ or $$\mathbb{C}^n$$: $$\left\| x \right\|_{p}=\left( \sum_{i=1}^{n} \left\vert x_i \right\vert ^p \right) ^{1/p}$$ ($$p=2$$: standard Euclidean norm), $$\left\| x \right\|_{\infty}=\max_{i=1, \dots, n} \left\vert x_i \right\vert$$
2. [Thm] The function $$x\to \left\| x \right\|_{p}$$ is a norm on $$\mathbb{R}^n$$ if and only if $$p\geq 1$$
3. [Thm] Minkowski’s inequality: if $$p\geq 1$$ and $$f, g\in \ell^p$$, $$\left\| f+g \right\|_{p}\leq \left\| f \right\|_{p}+\left\| g \right\|_{p}$$ with equality for $$p>1$$ iff $$f$$ and $$g$$ are positively linearly dependent (i.e., proportional), in particular: (Wikipedia, Wolfram MathWorld)
1. $$\left[ \int_{a}^{b} \left\vert f(x)+g(x) \right\vert ^p dx \right] ^ {1/p}\leq \left[ \int_{a}^{b} \left\vert f(x) \right\vert ^p dx \right] ^ {1/p} + \left[ \int_{a}^{b} \left\vert g(x) \right\vert ^p dx \right] ^ {1/p}$$
2. with $$a_k, b_k>0$$, $$\left[ \sum_{k=1}^{n} \left\vert a_k+b_k \right\vert ^p \right] ^{1/p}\leq \left( \sum_{k=1}^{n} \left\vert a_k \right\vert ^p \right) ^{1/p} + \left( \sum_{k=1}^{n} \left\vert b_k \right\vert ^p \right) ^{1/p}$$
4. [Thm] Cauchy-Schwarz inequality: $$\left\vert \left< u, v \right> \right\vert \leq \left< u, u \right> \cdot \left< v, v \right>$$
1. in $$\mathbb{R}^n$$: $$\left\vert \sum_i x_i y_i \right\vert \leq \left( \sum_i \left\vert x_i \right\vert ^2 \right) ^{1/2} \left( \sum_i \left\vert y_i \right\vert ^2 \right) ^{1/2}$$
5. [Thm] Hölder’s inequality: for $$p, q\in [1, \infty]$$ with $$\frac{1}{p}+\frac{1}{q}=1$$, then $$\left\| fg \right\|_{1}\leq \left\| f \right\|_{p} \left\| g \right\|_{q}$$, in particular, $$\left\vert \sum_{i=1}^{n} x_i y_i \right\vert \leq \left( \sum_{i=1}^{n} \left\vert x_i \right\vert ^p \right) ^{1/p} \left( \sum_{i=1}^{n} \left\vert y_i \right\vert ^q \right) ^{1/q}$$; proved by generalized AM-GM inequality
6. [Thm] original AM-GM inequality: $$\forall x, y>0$$: $$\frac{x+y}{2}\geq \sqrt{xy}$$ (arithmetic-geometric)
7. [Thm] generalized AM-GM inequality: if $$a, b\geq 0$$ and $$\theta\in (0, 1)$$, then $$a^{1-\theta}b^{\theta}\leq (1-\theta) a + \theta b$$ with $$a, b>0$$, s.t., $$\left( \frac{b}{a} \right) ^{\theta} \leq (1-\theta)+\theta \cdot \frac{b}{a}$$

Lecture 24 (0327)

1. [Def] equivalence of norms: Given a vector space $$\mathbb{V}$$ and two norms on $$\mathbb{V}$$ $$\left\| \cdot \right\|_{a}$$, $$\left\| \cdot \right\|_{b}$$. The two norms are equivalent if there exist constant $$c, C >0$$ such that $$c\left\| v \right\|_{a}\leq \left\| v \right\|_{b}\leq C\left\| v \right\|_{a}$$ for all $$v\in \mathbb{V}$$
2. [Cor] equivalence of norms defines an equivalence relation $$\left\| v \right\|_{a}\sim \left\| v \right\|_{b}$$, as $$\frac{1}{C}\left\| v \right\|_{b}\leq \left\| v \right\|_{a} \leq \frac{1}{c}\left\| v \right\|_{b}$$
3. [Thm] on a finite dimensional vector space over $$\mathbb{R}$$ or $$\mathbb{C}$$, all norms are equivalent

Lecture 25 (0329)

1. [Thm] if $$p_1\leq p_2$$, $$\left\| x \right\|_{\ell^{p_2}} \leq \left\| x \right\|_{\ell^{p_1}}$$
2. [Note] $$\ell^p(\mathbb{N})$$ are complete normed space, i.e., Banach space
3. [Def] linear operator/function: given $$V$$, $$W$$ vector spaces, $$T: V\to W$$ is linear if $$T(\alpha v+\beta w)=\alpha T(v)+\beta T(w)$$
4. [Note] sometimes we write $$Tv$$ for $$T(v)$$
5. [Thm] $$V$$, $$W$$ vector spaces both over $$\mathbb{R}$$ or $$\mathbb{C}$$, $$T: V\to W$$ linear, then the following are equivalent
1. $$T$$ is continuous everywhere
2. $$T$$ is continuous at $$0$$
3. there is a constant $$C$$ such that $$\left\| Tv \right\|_{W}\leq C\left\| v \right\|_{V}$$
Proof: 1 $$\implies$$ 2 $$\implies$$ 3 $$\implies$$ 1

Lecture 26 (0331)

1. [Thm] linear operator $$T: V\to W$$, $$T$$ is continuous $$\iff$$ $$\left\| Tv \right\|_{W}=C\left\| v \right\|_{V}$$, $$C$$ is independent of $$v$$
2. [Note] jargon: we refer to a continuous linear operator as a “bounded operator” because $$T$$ maps bounded sets to bounded sets
3. [Thm] if $$V$$ is finite dimensional, then $$T: V\to W$$ is always continuous
4. [Def] operator norm $$\left\| \cdot \right\|_{\operatorname{op}}$$: for a continuous linear operator $$T: V\to W$$, the smallest possible constant in $$\left\| Tv \right\|_{W}\leq C\left\| v \right\|_{V}$$ for all $$v$$ is called the operator norm
5. [Def] $$\mathcal{L}(V, W)$$: set of all bounded linear operators $$T: V\to W$$ s.t. $$S+T$$ is the operator $$v\mapsto Sv+Tv$$, $$v\mapsto cTv$$ linear, $$c\in \mathbb{R}$$ or $$\mathbb{C}$$
1. [Prop] $$\left\| Sv \right\|_{W}=C_S \left\| v \right\|_{V}$$, $$\left\| Tv \right\|_{W}=C_T \left\| v \right\|_{V}$$ $$\implies$$ $$\left\| Sv+Tv \right\|_{W}=(C_S+C_T) \left\| v \right\|_{V}$$
2. [Prop] operator norm: $$\left\| T \right\|_{op}=\sup_{v\not ={0}}\frac{\left\| Tv \right\|_{W}}{\left\| v \right\|_{V}}=\sup_{v\not ={0}}\left\| Tv \right\|_{W}=$$ maximum norm on the image of $$T$$, when resticted to the unit sphere

Lecture 27 (0403)

1. TODO: two-two norm
2. [Def] Riemann derivative: $$f: (c, d)\to \mathbb{R}$$, $$a\in (c, d)$$, $$f$$ is differentiable at $$a$$ if the derivative $$\lim_{h\to 0}\frac{f(a+h)-f(a)}{h}=\lim_{x\to a}\frac{f(x)-f(a)}{x-a}=f'(a)$$ exists

Lecture 28 (0405)

1. [Def] Fréchet derivative: given normed spaces $$V$$, $$W$$, $$\Omega\subset V$$ open set, $$a\in \Omega$$; define $$f:\Omega\to W$$ is Fréchet differentiable at $$a$$, if there exists a bounded linear operator $$T: V\to W$$ s.t. $$\lim_{h\to 0} \frac{\left\| f(a+h)-f(a)-T[h] \right\|_{W}}{\left\| h \right\|_{V}}=0$$; in other words, with $$a+h\in \Omega$$, $$f(a+h)=f(a)+T[h]+o(\left\| h \right\|_{V})$$ s.t. $$\lim_{h\to 0}\frac{o(\left\| h \right\|_{V})}{\left\| h \right\|_{V}}=0$$ in $$W$$. This $$T$$ is unique and is called the Fréchet derivative of $$f$$ at $$a$$, denoted $$Df_a$$ or $$f'(a)$$
2. [Example] $$V=\mathbb{R}^n$$, $$W=\mathbb{R}^m$$, $$T:V\to W$$, if $$f:\Omega\to W$$ is differentiable there is an $$m\times n$$-matrix $$A$$ so $$Df_a[h]=Ah$$, and the matrix is the derivative