# Summary

M1, M2, M3,
P1,
Q1, Q2, Q3, Q4,

# Lectures

Lecture 1: Slides.
Lecture 2: Slides.
Lecture 3: Slides.
Lecture 4: Slides.

Lecture 1 Part 1 (Admin, 2021): Link and Link
Lecture 1 Part 2 (Supervised learning): Link
Lecture 1 Part 3 (Perceptron learning): Link
Lecture 2 Part 1 (Loss functions): Link
Lecture 2 Part 2 (Logistic regression): Link
Lecture 2 Part 3 (Convexity): Link
Lecture 3 Part 1 (Neural Network): Link
Lecture 3 Part 2 (Backpropogation): Link
Lecture 3 Part 3 (Multi-Layer Network): Link
Lecture 4 Part 1 (Stochastic Gradient): Link
Lecture 4 Part 2 (Multi-Class Classification): Link
Lecture 4 Part 3 (Regularization): Link


Image by sandserifcomics via towards data science

# Other Materials

Which face is real? Link
Guess two-thirds of the average? Link
Gradient Descent. Link
Plot 3D functions: Link
Neural Network: Link
Neural Network Videos by Grant Sanderson: Playlist
MNIST Neural Network Visualization: Link
Stochastic Gradient Descent: Link
Overfitting: Link
Neural Network Snake: Video
Neural Network Car: Video
Neural Network Flappy Bird: Video
Neural Network Mario: Video

Why does the (batch) perceptron algorithm work? Link
Why cannot use linear regression for binary classification? Link
Why does gradient descent work? Link
How to derive logistic regression gradient descent step formula? Link
Example (Quiz): Perceptron update formula Link
Example (Quiz): Gradient descent for logistic activation with squared error Link
Example (Quiz): Computation of Hessian of quadratic form Link
Example (Quiz): Computation of eigenvalues Link
Example (Homework): Gradient descent for linear regression Link
How to construct XOR network? Link
How derive 2-layer neural network gradient descent step? Link
How derive multi-layer neural network gradient descent induction step? Link
Comparison between L1 and L2 regularization. Link
Example (Quiz): Cross validation accuracy Link

Checklist: Link, "math crib sheet": Link
Multivariate Calculus: Textbook, Chapter 16 and/or (Economics) Tutorials, Chapters 2 and 3.
Linear Algebra: Textbook, Chapters on Determinant and Eigenvalue.
Probability and Statistics: Textbook, Chapters 3, 4, 5.