MATH 522 (Analysis II)
Definitions and Theorems

Ruixuan Tu
University of Wisconsin-Madison

Joris Ross & Andreas Seeger, 25 January 2023

Walter Rudin: Principles of Mathematical Analysis

MATH 521 Notes

More in this series


  • \(\log\) function (from SDL)
    • \(b^y=x \iff y=\log_b(x)\)
    • \(a^{\log_a(N)}=N\)
    • \(\log(1)=0\)
    • \(\log(s)+\log(t)=\log(st)\)
    • \(\log(n)-\log(d)=\log\left( \frac{n}{d} \right)\)
    • \(p\log(m)=\log(m^p)\)
    • \(\log_a(b)=\frac{\log_c(b)}{\log_c(a)}\)
    • \(\log_a(b)=\frac{1}{\log_b(a)}\)
  • Taylor series (from MIT and U South Carolina)
    • (geometric series) \(\forall x\in (-1, 1)\): \(\frac{1}{1-x}=1+x+x^2+x^3+x^4+\cdots = \sum_{n=0}^{\infty} x^n\)
    • (geometric series, finite sum) \(\sum_{k=0}^{n-1}ar^k=\begin{cases} a\left( \frac{1-r^n}{1-r} \right) & \text{if } r\not= 1 \\ an & \text{if } r=1 \end{cases}\)
    • \(\forall x\in \mathbb{R}\): \(e^x = 1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+\frac{x^4}{4!}+\cdots =\sum_{n=0}^{\infty} \frac{x^n}{n!}\)
    • \(\forall x\in \mathbb{R}\): \(\cos(x)=1-\frac{x^2}{2!}+\frac{x^4}{4!}-\frac{x^6}{6!}+\frac{x^8}{8!}-\cdots=\sum_{n=0}^{\infty} (-1)^n \frac{x^{2n}}{(2n)!}\)
    • \(\forall x\in \mathbb{R}\): \(\sin(x)=x-\frac{x^3}{3!}+\frac{x^5}{5!}-\frac{x^7}{7!}+\frac{x^9}{9!}-\cdots=\sum_{n=0}^{\infty}(-1)^n \frac{x^{2n+1}}{(2n+1)!}\)
    • \(\forall x\in (-1, 1]\): \(\ln(1+x)=x-\frac{x^2}{2}+\frac{x^3}{3}-\frac{x^4}{4}+\frac{x^5}{5}-\cdots=\lim_{n=1}^{\infty}(-1)^{n+1} \frac{x^n}{n}\)
    • \(\forall x\in [-1, 1]\): \(\tan^{-1}(x)=x-\frac{x^3}{3}+\frac{x^5}{5}-\frac{x^7}{7}+\frac{x^9}{9}-\cdots =\sum_{n=0}^{\infty} \frac{(-1)^n}{2n+1} x^{2n+1}\)
    • \(\forall x\in (-1, 1)\): \((1+x)^p=\sum_{n=0}^{\infty} {p\choose n} x^n\)

Lecture 1 (0125)

  1. [Def] uniformly convergence of function: We say that \(f_n: X\to \mathbb{R} \text{ or } \mathbb{C}\) is uniformly convergent with limit \(f: X\to \mathbb{R} \text{ or } \mathbb{C}\) if \(\forall \varepsilon>0, \exists N\in \mathbb{N}\) such that \(\forall n\geq N, \forall x\in X, \left| f_n(x)-f(x) \right| < \varepsilon\). Note: Def 1.23
  2. [Def] uniformly convergence of function: Equivalently, \(f_n\to f\) uniformly if \(n\to \sup_{x\in X} \left| f_n(x)-f(x) \right|\) is convergent to \(0\) or \(\left\{ \sup_{x\in X} \left| f_n(x)-f(x) \right| \right\}_{n\in \mathbb{N}}\). Note: Lemma 1.24
  3. [Def] uniformly convergence of function: \(\forall \varepsilon>0, \exists N_{\varepsilon}\in \mathbb{N}, \forall x\in X, \forall n\geq N_{\varepsilon}: \left| f_n(x)-f(x) \right| < \varepsilon\). Note: Def 1.23
  4. [Def] pointwise convergence of function differs in \(\forall \varepsilon>0, \forall x\in X, \exists N_{\varepsilon, x}\in \mathbb{N}, \forall n\geq N_{\varepsilon, x}: \left| f_n(x)-f(x) \right| < \varepsilon\). Note: Def 1.23
  5. [Def] uniform convergence of series of function: \(\sum_{k=1}^{\infty} f_k(x)\) converges uniformly if \(S_n(x)=\sum_{k=1}^{n} f_k(x)\) converges uniformly
  6. [Def] uniformly Cauchy of series of function: A sequence \((f_n)_{n\in \mathbb{N}}\) of functions on a set \(X\) is uniformly Cauchy iff \(\forall \varepsilon>0, \exists N\in \mathbb{N}\) such that \(\forall n, m\geq N, \forall x\in X, \left| f_n(x)-f_m(x) \right| < \varepsilon\), i.e., \(\sup_{x\in X} \left| f_n(x)-f_m(x) \right| < \varepsilon\). Note: Lemma 1.25
  7. [Thm] A sequence is uniformly Cauchy iff it is uniformly convergent
  8. [Observation] If \(f_n\) is uniformly Cauchy, then \(\forall x\in X\), the sequence of numbers \(f_n(x)\) is Cauchy

Lecture 2 (0127)

  1. [Thm] to estimate integrals
    • With \(a<b\), \(\left| \int_{a}^{b} f(x) ~dx \right| \leq \int_{a}^{b} \left| f(x) \right| ~dx\)
    • With \(f_1, f_2\) integrable, if \(f_1(x) \leq f_2(x)\), then \(\forall x\in [a, b], \int_{a}^{b} f_1(x) ~dx \leq \int_{a}^{b} f_2(x) ~dx\)
  2. [Thm] Weierstrass \(M\)-test: Given a numerical sequence \(a_k \geq 0\) such that \(\sum_{k=1}^{\infty} a_k\) converges; given a sequence of function \(f_k: X\to \mathbb{R} \text{ or } \mathbb{C}\) such that \(\left| f_k(x) \right| \leq a_k\); then \(\sum_{k=1}^{\infty} f_k(x)\) converges uniformly on \(X\)

Lecture 3 (0130)

  1. [Thm] Taylor’s theorem on polynomial approximation: \(f(a+h)=f(a)+f'(a)h+f''(a)\frac{h^2}{2}+\cdots\) (an error bound better than \(h^2\))
  2. [Thm] Given \(f_n: X\to \mathbb{R} \text{ or } \mathbb{C}\), where \(X\) is a metric space. Assume (i) \(f_n\) converges uniformly on \(X\) to \(f\) and (ii) \(f_n\) is continuous at a point \(a\in X\). Then \(f\) is continuous at \(a\)
  3. [Thm] fundamental theorem of calculus: \(f(x)=f(x_0)+\int_{x_0}^{x} f'(t) ~dt\)

Lecture 4 (0201)

  1. [Thm] Taylor’s theorem (or Maclaurin’s theorem): Assume \(f\in C^{k+1}\) () on interval \(I\), \(a\in \operatorname{Int}(I)\). Then \(\forall a+h\in I\) we have \(f(a+h)=\sum_{j=0}^{k}\frac{h^j}{j!}f^{(j)}(a)+E_kf(a, h)\) where \(E_kf(a, h)=\frac{a^{k+1}}{(k+1)!}\int_{0}^{1} (k+1)(1-s)^k f^{(k+1)}(a+sh) ~ds\). One can write \(E_kf(a, h)=\frac{h^{k+1}}{(k+1)!}f^{(k+1)}(\xi)\), \(\xi\) between \(a\) and \(a+h\). Note: Thm A.17

Lecture 5 (0203)

  1. [Thm] Taylor’s theorem for polynomial of degree \(\leq n\): \(P(x)=\sum_{k=0}^{n} \frac{P^{(k)}(a)}{k!} (x-a)^k + E_n(x)\), \(E_n(x)=\frac{x^{n+1}}{(n+1)!} P^{(n+1)}(\xi)\)
  2. [Def] exponential: \(e=\lim_{n\to \infty} \left( 1+\frac{1}{n} \right)^n\); \(e=\sum_{k=0}^{\infty} \frac{1}{k!}\); \(e\) is the unique number \(a\) which means the integral \(\int_{0}^{a} \frac{dt}{t} = 1\); \(\log(x)=\int_{1}^{x} \frac{dt}{t} ~dx\)
  3. [Thm] Taylor’s polynomial for \(e^x\): \(e^x=\sum_{k=0}^{n} \frac{x^k}{k!} + \frac{x^{n+1}}{(n+1)!}e^{\xi}\) for some \(\xi\) between 0 and \(x\); the remainder converges to \(0\), uniformly on \([-b, b]\), but does not converge uniformly to \(0\) on \((-\infty, \infty)\)
  4. [Thm] \(e=\sum_{k=0}^{n} \frac{1}{k!} + \frac{e^{\xi}}{(n+1)!}\) where \(\xi\) is between \(0\) and \(1\)
  5. [Thm] \(e\) is not a rational number
  6. [Thm] Taylor series for \(x\in \mathbb{R} \text{ or }\mathbb{C}\): \(e^x=\sum_{k=0}^{\infty}\frac{x^k}{k!}\); \(\cos(x)=\sum_{k=0}^{\infty} \frac{(-1)^k x^{2k}}{(2k)!}\); \(\sin(z)=\sum_{k=0}^{\infty} \frac{(-1)^k x^{2k+1}}{(2k+1)!}\)
  7. [Thm] \(e^{ib}=\cos b + i \sin b\) for \(z=ib\) without real part

Lecture 6 (0206)

  1. [Def] Cauchy product of power series: \(\left( \sum_{i=0}^{\infty} a_i x^i \right)\left( \sum_{j=0}^{\infty} b_j x^j \right)=\sum_{k=0}^{\infty} c_k x^k\) where \(c_k=\sum_{l=0}^{k} a_l b_{k-l}\)
  2. [Def] power series: \(\sum_{n=0}^{\infty} a_n z^n\) where \(z\) is a complex (or just real) number
  3. [Thm] complex multiplication:
    • \((a+ib)(c+id)=(a+c)+(b+d)i=ac-bd+(ad-bc)i\)
    • with \(a+ib=r(\cos \alpha+i\sin \alpha)\), \(c+id=R(\cos \beta+i\sin \beta)\), \((a+ib)(c+id)=Rr\left( \cos(\alpha+\beta) + i \sin(\alpha+\beta) \right)\)
    • \(z^n=r^n(\cos \alpha+i\sin \alpha)^n=r^n(\cos n\alpha+i\sin n\alpha)\) by \(\cos\alpha + i\sin \alpha=e^{i\alpha}\)
  4. [Thm] about convergence: \(a_n\in \mathbb{C}\). There is a unique \(R\in [0, \infty)\) such that
    1. \(\sum_{n=0}^{\infty} a_n z^n\) is convergent for all \(z\) with \(|z|<R\)
    2. \(\sum_{n=0}^{\infty} a_n z^n\) is divergent for all \(z\) with \(|z|>R\)
    radius of convergence \(R\) can be computed as \(R=\frac{1}{\operatorname{\lim\sup}_{n\to\infty} |a_n|^{\frac{1}{n}}}\)

Lecture 7 (0208)

  1. [Lemma] \(\sum_{k=0}^{\infty} (-1)^k \frac{x^{2k+1}}{2k+1} = \sum_{k=0}^{\infty} (-1)^k \int_{0}^{x} t^{2k} ~dt = \int_{0}^{x} \sum_{k=0}^{\infty} (-t^2)^k ~dt = \int_{0}^{x} \frac{1}{1+t^2} ~dt = \arctan x\), and the sum is uniformly convergent on \([-1, 1]\)
  2. [Note] even if we have uniform convergence on \([-b, b]\) for all \(b<1\), we may not have uniform convergence on \((-1, 1)=\cup_{b<1} [-b, b]\)
  3. [Thm] If the series converges uniformly on \([-1, 1]\) then the limit sum is a continuous function
  4. [Thm] If we can differentiate the power series \(f(x)=\sum_{n=0}^{\infty} a_n x^n\), then \(f\) is differentiable and \(f'(x)=\sum_{n=1}^{\infty} a_n n x^{n-1}=\sum_{n=0}^{\infty} a_{n+1} (n+1) x^n\)
  5. [Thm] \(\sum_{n=0}^{\infty} a_{n+1} (n+1) x^n\) has the same radius of convergence and therefore converges uniformly in \(\left[ -R(1-\varepsilon), R(1-\varepsilon) \right]\)
  6. [Thm] Power series is its own Taylor series

Lecture 8 (0210)

  1. [Thm] Abel’s theorem: If \(\sum_{k=0}^{\infty} a_k\) converges, then \(\sum_{k=0}^{\infty} a_k = \lim_{r\to 1^-} \sum_{k=0}^{\infty} a_k r^k\)
  2. [Thm] Abel’s summation formula (summation by parts): \(\sum_{n=p}^{q} a_n b_n=A_q b_q - A_{p-1} b_p+\sum_{n=p}^{q-1} A_n (b_n - b_{n+1})\) where \(A_n = \sum_{k=0}^{n} a_k\)
  3. [Thm] Another Abel’s summation formula: \(\sum_{k=0}^{n}f_k g_k=f_0 \sum_{k=0}^{n} g_k - \sum_{j=0}^{n-1} (f_{j+1}-f_j) \sum_{k=0}^{j} g_k\) (from Wikipedia)

Lecture 9 (0213)

  1. [Thm] \(\sum_{k=0}^{n} \cos kx+i \sum_{k=0}^{n} \sin kx=\sum_{k=0}^{n} e^{ikx}=\frac{1-(e^{ix})^{n+1}}{1-(e^{ix})}\leq \frac{\left\vert 1-(e^{ix})^{n+1} \right\vert}{\left\vert 1-(e^{ix}) \right\vert}\leq \frac{2}{\left\vert 1-(e^{ix}) \right\vert}\)
  2. [Def] norm: On a vector space \(\mathbb{V}\) a norm \(x\mapsto \left\| x \right\|\) is a function \(\mathbb{V}\to [0, \infty)\) s.t.
    1. \(\left\| x \right\|=0\) iff \(x=0\)
    2. \(\left\| cx \right\|=\left\vert c \right\vert \left\| x \right\|\) for all scalars in \(\mathbb{R}\) or \(\mathbb{C}\)
    3. \(\left\| x+y \right\|\leq \left\| x \right\| + \left\| y \right\|\)
  3. [Rmk] If we have a norm, \(d(x, y)=\left\| x-y \right\|\)
  4. [Thm] \(\max \left\vert x_i \right\vert \leq \left( \sum_{i=1}^{n} \left\vert x_i \right\vert ^2 \right) ^\frac{1}{2} \leq \sum_{i=1}^{n} \left\vert x_i \right\vert \leq n \max \left\vert x_i \right\vert\)
  5. [Def] \(\ell^p\) spaces:
    1. \(\ell^1(\mathbb{N})\): the function on \(\mathbb{N}\) \(n\mapsto f(n)\) (or we could write \(a_n\)) with the property \(\sum_{n=1}^{\infty} \left\vert a_n \right\vert\) is finite
    2. \(\ell^1(\mathbb{N})\) is the space of (absolutely) summable sequences \(\sum_{n=1}^{\infty} \left\vert a_n \right\vert\)
    3. \(\ell^{\infty}(\mathbb{N})\): the space of bounded sequences \(\sup_{n\in \mathbb{N}} \left\vert a_n \right\vert\)

Lecture 10 (0215)

  1. [Def] Euler’s constant or Euler-Mascheroni constant: \(\lim_{n\to \infty} \left( \sum_{k=1}^{n} \frac{1}{k} - \ln(x) \right) = \gamma\), also as \(0.5\dots\)
  2. [Thm] Subsets of a metric space \((X, d)\), \(Y\subset X\), \(Y\) is a metric space, with the metric \(d|_{Y\times Y}\)
  3. [Def] open ball: open ball on metric space \(X\), centered at a point \(y\in Y\) with radius \(r\): \(B_X(y, r)=\left\{ x\in X: d(x, y)<r \right\}\)
  4. [Col] \(B_Y(y, r)=\left\{ w\in Y, d(w, y)<r \right\}=B_X(y, r)\cap Y\)
  5. [Thm] Let \(Y\) be a subset of \(X\) with the metric inherited from \(X\). Then a set \(\mathcal{U}\subset Y\) is open in \(Y\) if and only if there is an open set \(O\) with respect to the metric \(X\) in \(X\) such that \(O\cap Y=\mathcal{U}\)
  6. [Def] compact metric space: A metric space \((X, d)\) is compact if the following holds: Whenever \(X=\cup_{\alpha\in A} O_\alpha\) with \(O_\alpha\) open (\(X\) as a union of open sets), then there are finitely many \(\alpha_1, \dots, \alpha_n\in A\), \(X=\cup_{i=1}^{n} O_{\alpha_i}\)
  7. [Def] compact subset \(K\) of metric space \(X\): \(K\subset X\) is compact if every cover of \(K\) with open sets has a finite subcover. Meaning: Whenever \(K\subset \cup_{\alpha \in A} O_\alpha\) with \(O_\alpha\) open in \(X\), then there are indices \(\alpha_1, \dots, \alpha_n\in A\) such that \(K\subset \cup_{i=1}^{n} O_{\alpha_i}\). Note: consider \(K\) with the metric inherited from \(X\), so \(K\) is a metric space
  8. [Thm] Heine-Borel: A subset \(K\) of \(\mathbb{R}^n\) is compact if and only if \(K\) is bounded and closed.
  9. [Def] bounded set: \(K\subset X\) is bounded if \(\exists C\) such that \(d(x, w)\leq C\) for all \(x, w\in K\). In a metric space a set \(Y\) is bounded if \(\left\| y \right\|<C\) for all \(y\in Y\)
  10. [Note] consider \(X=\ell^{\infty}(\mathbb{N})\) all bounded sequence, \(n\to f(n)\), \(f\in \ell^{\infty}(\mathbb{N})\) if \(\sup_{n\in N} \left\vert f(n) \right\vert <\infty\)

Lecture 11 (0217)

  1. [Def] sequentially compact: A metric space \(X\) is sequentially compact if every sequence in \(X\) has a convergent subsequence (a.k.a. Bolzano-Weierstrass property)
  2. [Def] subsequence: \(b_n\) is a subsequence of \(a_n\) if there is a strictly increasing function \(K: \mathbb{N}\to \mathbb{N}\) such \(b_n=a_{K(n)}\) (\(K(n)\) is index-sequence of the subsequence)
  3. [Def] Cantor diagonal subsequence: \(d_n=a_n^{(n)}=a_{k(n)}^{(0)}\), \(k(n)=K_1\circ K_2\circ \cdots \circ K_n(n)\), with definition \(a_n^{(l)}=a_{K_l(n)}^{(l-1)}=\cdots=a_{K_1\circ K_2\circ \cdots \circ K_l(n)}^{(0)}\), and \((d_n)\) is a subsequence of \(\left( a_n^{(1)} \right), \left( a_n^{(2)} \right), \left( a_n^{(3)} \right), \dots\)
  4. [Def] totally bounded: A metric space \((X, d)\) is totally bounded if for every \(\varepsilon>0\), \(X\) is a union of finitely many balls of radius \(\varepsilon\)
  5. [Def] complete metric space: a metric space \(M=(X, d)\) is called complete (or a Cauchy space) if every Cauchy sequence of points in \(M\) has a limit that is also in \(M\) (Wikipedia)
  6. [Thm] The following are equivalent for a given metric space \(X\)
    1. \(X\) is compact (every cover with open sets has a finite subcover)
    2. \(X\) is sequentially compact
    3. \(X\) is totally bounded and complete (every Cauchy sequence converges)

Lecture 12 (0220)

  1. [Thm] The following are equivalent, \(X\) is a metric space
    1. \(X\) is compact
    2. \(X\) is sequentially compact
    3. \(X\) is totally bounded and complete
    Proof: 1 \(\implies\) 2 \(\implies\) 3 \(\implies\) 2 \(\overset{3}{\implies}\) 1
  2. [Lemma] Consider a fixed cover \((O_\alpha)\) with open sets. There exists an \(\varepsilon>0\) so that all balls of radius \(\varepsilon\) are contained in at least one of the \(O_\alpha\)

Lecture 13 (0222)

  1. [Thm] Let \(X\) be a compact metric space and \(f: X\to \mathbb{R}\) be continuous. Then
    1. \(f\) is bounded, i.e., \(\sup_{x\in X} \left\vert f(x) \right\vert <\infty\)
    2. \(f\) has a maximum and a minimum on \(X\) (there is an \(x_0\in X\), \(f(x_0)=\sup_{x\in X} f(x)\) and \(\tilde{x_0}\in X\), \(f(\tilde{x_0})=\inf_{x\in X} f(x)\))
    3. \(f\) is uniformly continuous, i.e., for every \(\varepsilon>0\) there is \(\delta\) such that \(\left\vert f(x_1)-f(x_2) \right\vert <\varepsilon\) if \(d(x_1, x_2)<\delta=\sigma(\varepsilon)\) [Note: \(\sigma(\varepsilon)\) does not depend on \(x\in K\)]
  2. [Example] None of these may hold if \(X\) is not compact: \(X=(0, 1)\), (i) \(f(x)=\frac{1}{x}\), (ii) \(f(x)=\sin\left( \frac{1}{x} \right)\)
  3. [Thm] Heine-Borel theorem: Every bounded and closed set \(E\) in \(\mathbb{R}^n\) is compact
  4. [Def] pointwise continuous: A map \(f:X\to Y\) is called continuous at \(x\in X\) if for every \(\varepsilon > 0\) there exists \(\delta > 0\) such that if \(d_X(x, y)<\delta\), then \(d_Y(f(x), f(y))<\varepsilon\) (from Note 1.19)

Lecture 14 (0224)

  1. [Lemma] If \(X\) is compact, if \(K\) is closed in \(X\) then \(K\) is compact
  2. [Thm] Consider \(\mathscr{c}_0=\) vector space of sequences \(a\) which converge to 0, i.e., \(\lim_{n\to\infty} a_n=0\) if and only if \(\lim_{j\to\infty} \sup_{a\in K} \left\vert a_j \right\vert =0\)

Lecture 15 (0227)

  1. [Thm] A subset \(A\) of \(\mathscr{c}_0\) is totally bounded if the sequence \(n\to \sup_{a\in A} a_n\) converges to zero

  2. [Lemma] Suppose \(A\subset X\) and assume that for every \(\varepsilon > 0\), there is a finite number of balls \(B(z, \varepsilon)\), \(z\in Z\) (\(Z\): finite set in \(\mathscr{c}_0\)), \(\# z<\infty\). \(A\subset \cup_{z\in Z} B(z, \varepsilon)\), then \(A\) is totally bounded

  3. [Prop] Given \(\varepsilon>0\), find a finite collection of balls \(B(z, \frac{\varepsilon}{2})\), \(z\in Z\) (finite). Let \(Z'\subset Z\) be the collection of those \(z\in Z\) for which \(B(z, \frac{\varepsilon}{2})\cap A\)

    For \(z\in Z'\) there is an \(a_z\in A\cap B(z, \frac{\varepsilon}{2})\), \(B(a_z, \varepsilon)\supset B(a_z, \frac{\varepsilon}{2})\)

    If \(y\in B(z, \frac{\varepsilon}{2}), d(y, z)<\frac{\varepsilon}{2}, d(a_z, z)<\frac{\varepsilon}{2}\), then by triangle inequality \(d(y, a_z)<\varepsilon\)

  4. [Def] pointwise bounded functions: A class \(A\) of functions is pointwise bounded if for every \(x\) there is \(M_x<\infty\) such that \(\sup_{f\in A} \left\vert f(x) \right\vert \leq M_x\)

  5. [Def] uniformly bounded functions: \(A\) is uniformly bounded if \(\sup_{x\in X} \sup_{f\in A} \left\vert f(x) \right\vert \leq M_C\)

  6. [Rmk] If \(A\) is uniformly bounded, \(A\) may not be totally bounded

  7. [Fact] totally bounded \(\implies\) uniformly bounded (Linrong)

  8. [Def] equicontinuous: \(A\subset C(K)\) is equicontinuous if for every \(\varepsilon\) there exists a \(\delta\) such that \(\left\vert f(x)-f(y) \right\vert < \varepsilon\) for all \(f\in A\) and for all \(x, y\) such \(d(x, y)<\delta\)

  9. [Thm] Arzelà–Ascoli: \(A\subset C(K)\) is totally bounded if and only if \(A\) is pointwise bounded and equicontinuous

    Proof: \(w_{\delta}(f)=\sup_{d(x, y)< \delta} d(f(x), f(y))\), \(\sup_{f\in A} w_{\delta}(f)\to 0\) as \(\delta \to 0\)

  10. [Rmk] Arzelà–Ascoli theorem characterizes the totally bounded sets \(f: K\to \mathbb{R} \text{ or } \mathbb{C}\) in the sense \(C(K)\) (continuous functions on a compact set \(K\)), \(\left\| f \right\|=\sup_{x\in K} \left\vert f(x) \right\vert\)

  11. [Rmk] If \(A\subset C(K)\) is equicontinuous, then \(A\) is pointwise bounded if and only if it is uniformly bounded

Lecture 18 (0306)

  1. [Def] contraction: If \((X, d)\) is a metric space then \(T: X\to X\) is called a contraction if there exists \(\alpha<1\) such that \(d(T(x), T(y))\leq \alpha d(x, y)\)

  2. [Thm] Banach fixed point theorem (aka contraction principle): If \((X, d)\) is complete, then any contraction have a unique fixed point, i.e., a unique \(x\in X\) such that \(T(x)=x\)

    [Proof] method of successive approximation (construct a sequence, pick \(x_0\), then \(x_1=T(x_0), x_2=T(x_1), \dots, x_{n+1}=T(x_n)\)); \(d(x_{n+1}, x_n)=d(T(x_n), T(x_{n-1}))\leq \alpha d(x_n, x_{n-1})\leq \alpha \cdot \alpha^{n-1} d(x_1, x_0)\) \(\implies\) \(d(x_n, x_{n-1})\leq \alpha^{n-1} d(x_0, x_1)\)

Lecture 19 (0308)

  1. [Def] initial value problem for differential equation

    Pick a point \((x_0, y_0)\) to be initial point,

    • \(y'(x)=F(x, y(x))\)
    • \(y(x_0)=y_0\) (initial condition)

    For a function \((x, y)\mapsto F(x, y)\) continuous near \(x_0, y_0\), we define the initial value problem

  2. [Thm] Peano existence theorem: If \(F\) is continuous near \((x_0, y_0)\), then the initial value problem has a solution (based on compactness)

  3. [Thm] Picard–Lindelöf theorem: If in addition there is a condition \(\left\vert F(x, y)-F(x, \tilde{y}) \right\vert \leq C\left\vert y-\tilde{y} \right\vert\) for \((x, y)\) and \((x, \tilde{y})\) near \((x_0, y_0)\); then the solution is unique (based on the contraction principle)

  4. [Thm] Picard-Lindelöf theorem: Let \(D\subseteq \mathbb{R}\times \mathbb{R}^n\) be a closed rectangle with \((t_0, y_0)\in D\). Let \(f: D\to \mathbb{R}^n\) be a function that is continuous in \(t\) and Lipschitz continuous in \(y\). Then there exists some \(c>0\) such that the initial value problem has a unique solution \(y(t)\) on the interval \(\left[ x_0-\delta, x_0+\delta \right]\) (from Wikipedia)

  5. [Def] Lipschitz continuity: Given two metric spaces \((X, d_X)\) and \((Y, d_Y)\), a function \(f: X\to Y\) is called Lipschitz continuous if there exists a real constant \(K\geq 0\) such that, for all \(x_1\) and \(x_2\) in \(X\), \(d_Y(f(x_1), f(x_2))\leq Kd_X(x_1, x_2)\). Any such \(K\) is a Lipschitz constant. If \(0\leq K<1\) and \(f\) maps a metric space to itself, the function is called a contraction (from Wikipedia)

  6. [Def] Lipschitz condition: Function \(f(t, y)\) satisfies a Lipschitz condition in the variable \(y\) on a set \(D\subseteq \mathbb{R}^2\) if a constant \(L>0\) exists with \(\left\vert f(t, y_1)-f(t, y_2) \leq L \left\vert y_1-y_2 \right\vert \right\vert\) whenever \((t, y_1)\), \((t, y_2)\) are in \(D\). \(L\) is Lipschitz constant (from UCB Math)

  7. [Lemma] For a continuous function \(x\to y(x)\) defined on \(\left[ x_0-\delta, x_0+\delta \right]\) with values in \(\left[ y_0-b, y_0+b \right]\). The following two conditions are equivalent:

    1. \(y\) is of class \(C^1\) (\(y'\) is continuous) on \(\left[ x_0-\delta, x_0+\delta \right]\) and satisfies \(y'(x)=F(x, y(x))\), \(y(x_0)=y_0\)
    2. \(y\) satisfies the integral equation \(y(x)=y_0+\int_{x_0}^{x} F(t, y(t)) ~dt\)

Lecture 20 (0310)

  1. [Cor] Let \(R\) be the rectangle \(\left\{ (x, y): \left\vert x-x_0 \right\vert \leq a, \left\vert y-y_0 \right\vert \leq b \right\}\) and assume \(F\) is a continuous function on \(R\), \(M=\max{(x, y)\in R} \left\vert F(x, y) \right\vert\). For \(\delta\) in Picard-Lindelöf, \(\delta< \min\left\{ a, \frac{b}{M} \right\}\); weakly (without contraction), additionally \(\delta < \frac{1}{C}\)

Lecture 21 (0320)

  1. [Def] methods of successive approximation: including fixed-point iteration, Picard-Lindelöf theorem, Euler polynomial method
  2. [Def] Euler polynomial method:
    1. \(y'(y)=F(t, y(t))\), \(y'(t_0)=F(t, y_0)\)
    2. Given a partition of \([t_0, t_0+a_\star]\) s.t. \(t_0<t_1<\cdots<t_N=t_0+a_\star\)
    3. Define \(y(t)=\begin{cases} y_0 + F(t_0, y_0)(t-t_0) & t_0 \leq t \leq t_1 \\ y(t_1) + F(t_1, y(t_1))(t-t_1) & t_1 \leq t \leq t_2 \\ y(t_2) + F(t_2, y(t_2))(t-t_2) & t_2 \leq t \leq t_3 \\ \vdots \end{cases}\)

Lecture 22 (0322)

Lecture 23 (0324)

  1. [Def] \(\ell^p\) norms on \(\mathbb{R}^n\) or \(\mathbb{C}^n\): \(\left\| x \right\|_{p}=\left( \sum_{i=1}^{n} \left\vert x_i \right\vert ^p \right) ^{1/p}\) (\(p=2\): standard Euclidean norm), \(\left\| x \right\|_{\infty}=\max_{i=1, \dots, n} \left\vert x_i \right\vert\)
  2. [Thm] The function \(x\to \left\| x \right\|_{p}\) is a norm on \(\mathbb{R}^n\) if and only if \(p\geq 1\)
  3. [Thm] Minkowski’s inequality: if \(p\geq 1\) and \(f, g\in \ell^p\), \(\left\| f+g \right\|_{p}\leq \left\| f \right\|_{p}+\left\| g \right\|_{p}\) with equality for \(p>1\) iff \(f\) and \(g\) are positively linearly dependent (i.e., proportional), in particular: (Wikipedia, Wolfram MathWorld)
    1. \(\left[ \int_{a}^{b} \left\vert f(x)+g(x) \right\vert ^p dx \right] ^ {1/p}\leq \left[ \int_{a}^{b} \left\vert f(x) \right\vert ^p dx \right] ^ {1/p} + \left[ \int_{a}^{b} \left\vert g(x) \right\vert ^p dx \right] ^ {1/p}\)
    2. with \(a_k, b_k>0\), \(\left[ \sum_{k=1}^{n} \left\vert a_k+b_k \right\vert ^p \right] ^{1/p}\leq \left( \sum_{k=1}^{n} \left\vert a_k \right\vert ^p \right) ^{1/p} + \left( \sum_{k=1}^{n} \left\vert b_k \right\vert ^p \right) ^{1/p}\)
  4. [Thm] Cauchy-Schwarz inequality: \(\left\vert \left< u, v \right> \right\vert \leq \left< u, u \right> \cdot \left< v, v \right>\)
    1. in \(\mathbb{R}^n\): \(\left\vert \sum_i x_i y_i \right\vert \leq \left( \sum_i \left\vert x_i \right\vert ^2 \right) ^{1/2} \left( \sum_i \left\vert y_i \right\vert ^2 \right) ^{1/2}\)
  5. [Thm] Hölder’s inequality: for \(p, q\in [1, \infty]\) with \(\frac{1}{p}+\frac{1}{q}=1\), then \(\left\| fg \right\|_{1}\leq \left\| f \right\|_{p} \left\| g \right\|_{q}\), in particular, \(\left\vert \sum_{i=1}^{n} x_i y_i \right\vert \leq \left( \sum_{i=1}^{n} \left\vert x_i \right\vert ^p \right) ^{1/p} \left( \sum_{i=1}^{n} \left\vert y_i \right\vert ^q \right) ^{1/q}\); proved by generalized AM-GM inequality
  6. [Thm] original AM-GM inequality: \(\forall x, y>0\): \(\frac{x+y}{2}\geq \sqrt{xy}\) (arithmetic-geometric)
  7. [Thm] generalized AM-GM inequality: if \(a, b\geq 0\) and \(\theta\in (0, 1)\), then \(a^{1-\theta}b^{\theta}\leq (1-\theta) a + \theta b\) with \(a, b>0\), s.t., \(\left( \frac{b}{a} \right) ^{\theta} \leq (1-\theta)+\theta \cdot \frac{b}{a}\)

Lecture 24 (0327)

  1. [Def] equivalence of norms: Given a vector space \(\mathbb{V}\) and two norms on \(\mathbb{V}\) \(\left\| \cdot \right\|_{a}\), \(\left\| \cdot \right\|_{b}\). The two norms are equivalent if there exist constant \(c, C >0\) such that \(c\left\| v \right\|_{a}\leq \left\| v \right\|_{b}\leq C\left\| v \right\|_{a}\) for all \(v\in \mathbb{V}\)
  2. [Cor] equivalence of norms defines an equivalence relation \(\left\| v \right\|_{a}\sim \left\| v \right\|_{b}\), as \(\frac{1}{C}\left\| v \right\|_{b}\leq \left\| v \right\|_{a} \leq \frac{1}{c}\left\| v \right\|_{b}\)
  3. [Thm] on a finite dimensional vector space over \(\mathbb{R}\) or \(\mathbb{C}\), all norms are equivalent

Lecture 25 (0329)

  1. [Thm] if \(p_1\leq p_2\), \(\left\| x \right\|_{\ell^{p_2}} \leq \left\| x \right\|_{\ell^{p_1}}\)
  2. [Note] \(\ell^p(\mathbb{N})\) are complete normed space, i.e., Banach space
  3. [Def] linear operator/function: given \(V\), \(W\) vector spaces, \(T: V\to W\) is linear if \(T(\alpha v+\beta w)=\alpha T(v)+\beta T(w)\)
  4. [Note] sometimes we write \(Tv\) for \(T(v)\)
  5. [Thm] \(V\), \(W\) vector spaces both over \(\mathbb{R}\) or \(\mathbb{C}\), \(T: V\to W\) linear, then the following are equivalent
    1. \(T\) is continuous everywhere
    2. \(T\) is continuous at \(0\)
    3. there is a constant \(C\) such that \(\left\| Tv \right\|_{W}\leq C\left\| v \right\|_{V}\)
    Proof: 1 \(\implies\) 2 \(\implies\) 3 \(\implies\) 1

Lecture 26 (0331)

  1. [Thm] linear operator \(T: V\to W\), \(T\) is continuous \(\iff\) \(\left\| Tv \right\|_{W}=C\left\| v \right\|_{V}\), \(C\) is independent of \(v\)
  2. [Note] jargon: we refer to a continuous linear operator as a “bounded operator” because \(T\) maps bounded sets to bounded sets
  3. [Thm] if \(V\) is finite dimensional, then \(T: V\to W\) is always continuous
  4. [Def] operator norm \(\left\| \cdot \right\|_{\operatorname{op}}\): for a continuous linear operator \(T: V\to W\), the smallest possible constant in \(\left\| Tv \right\|_{W}\leq C\left\| v \right\|_{V}\) for all \(v\) is called the operator norm
  5. [Def] \(\mathcal{L}(V, W)\): set of all bounded linear operators \(T: V\to W\) s.t. \(S+T\) is the operator \(v\mapsto Sv+Tv\), \(v\mapsto cTv\) linear, \(c\in \mathbb{R}\) or \(\mathbb{C}\)
    1. [Prop] \(\left\| Sv \right\|_{W}=C_S \left\| v \right\|_{V}\), \(\left\| Tv \right\|_{W}=C_T \left\| v \right\|_{V}\) \(\implies\) \(\left\| Sv+Tv \right\|_{W}=(C_S+C_T) \left\| v \right\|_{V}\)
    2. [Prop] operator norm: \(\left\| T \right\|_{op}=\sup_{v\not ={0}}\frac{\left\| Tv \right\|_{W}}{\left\| v \right\|_{V}}=\sup_{v\not ={0}}\left\| Tv \right\|_{W}=\) maximum norm on the image of \(T\), when resticted to the unit sphere

Lecture 27 (0403)

  1. TODO: two-two norm
  2. [Def] Riemann derivative: \(f: (c, d)\to \mathbb{R}\), \(a\in (c, d)\), \(f\) is differentiable at \(a\) if the derivative \(\lim_{h\to 0}\frac{f(a+h)-f(a)}{h}=\lim_{x\to a}\frac{f(x)-f(a)}{x-a}=f'(a)\) exists

Lecture 28 (0405)

  1. [Def] Fréchet derivative: given normed spaces \(V\), \(W\), \(\Omega\subset V\) open set, \(a\in \Omega\); define \(f:\Omega\to W\) is Fréchet differentiable at \(a\), if there exists a bounded linear operator \(T: V\to W\) s.t. \(\lim_{h\to 0} \frac{\left\| f(a+h)-f(a)-T[h] \right\|_{W}}{\left\| h \right\|_{V}}=0\); in other words, with \(a+h\in \Omega\), \(f(a+h)=f(a)+T[h]+o(\left\| h \right\|_{V})\) s.t. \(\lim_{h\to 0}\frac{o(\left\| h \right\|_{V})}{\left\| h \right\|_{V}}=0\) in \(W\). This \(T\) is unique and is called the Fréchet derivative of \(f\) at \(a\), denoted \(Df_a\) or \(f'(a)\)
  2. [Example] \(V=\mathbb{R}^n\), \(W=\mathbb{R}^m\), \(T:V\to W\), if \(f:\Omega\to W\) is differentiable there is an \(m\times n\)-matrix \(A\) so \(Df_a[h]=Ah\), and the matrix is the derivative

Lecture 29 (0407)

Lecture 30 (0410)