CS760, Spring 2023
Department of Computer Sciences
University of Wisconsin–Madison
This schedule is tentative and subject to change. Please check back often. In particular, the deadlines for the homework sets/project may change; please see Canvas for the actual deadlines.
The reading is not required but strongly recommended for all students. Those explicitly noted as optional are for students interested in that specific topic. "A, B; C; D" means to read (A OR B) AND C AND D. Text in red means a link to the reading material. Abbreviations for textbooks:
| Date | Lecture | Readings | Homework Released | Homework Due |
|---|---|---|---|---|
| Wednesday Jan. 25 | Course Overview: Topics, Goals, Evaluation | Jordan and Mitchell, Science, 2015 | Hw1 | |
| Monday Jan. 30 | ML Overview: Supervised/Unsupervised/RL, Classification/Regression, General Approach | |||
| Wednesday Feb. 1 | Supervised Learning I: Setup, Examples. k-Nearest Neighbors, Decision Trees | Hw2 | Hw1 Due | |
| Monday Feb. 6 | Supervised Learning II: Decision Trees (cont'd) | Murphy Chapter 16.2 / Shalev-Shwartz and Ben-David Chapter 18 | ||
| Wednesday Feb. 8 | Evaluation: Bias, Cross-Validation, Precision/Recall, ROC Curves | |||
| Monday Feb. 13 | Regression I: Linear Regression, Logistic Regression, Normal equations, GD | Murphy 8.1-3 and 8.6; Maximum Likelihood, Logistic Regression, and Stochastic Gradient Training | ||
| Wednesday Feb. 15 | Regression II: Linear regression, Logistic regression, Gradient Descent Analysis | Convergence Theorems for Gradient Descent | Hw3 | Hw2 Due |
| Monday Feb. 20 | Naive Bayes: Generative vs Discriminative Models, ML vs MAP | Mitchell 6.1-6.10, Murphy 3 | ||
| Wednesday Feb. 22 | Neural Networks I: Perceptron, Training, MLPs | Mitchell chapter 4, Murphy 16.5 and 28, Bishop 5.1-5.3 LeCun et al., Nature, 2015 | ||
| Monday Feb. 27 | Neural Networks II: Training, Optimization, SGD, Backpropagation | Hw4 | Hw3 Due | |
| Wednesday Mar. 1 | Neural Networks III: Regularization, Data Augmentation | |||
| Monday Mar. 6 | Neural Networks IV: CNNs
Neural Networks (cont'd): Practical training tips |
Goodfellow-Bengio-Courville chapter 9; CNN papers 1. LeNet 2. AlexNet 3. ResNet | ||
| Wednesday Mar. 8 | Unsupervised Learning 1: Clustering, GMMs, and EM | Andrew Ng's notes on the EM Algorithm | ||
| Monday Mar. 13 | Spring Recess | |||
| Wednesday Mar. 15 | Spring Recess | |||
| Monday Mar. 20 | Unsupervised Learning II: Dimensionality Reduction, PCA | |||
| Wednesday Mar. 22 | Unsupervised Learning III: Generative NN models | Optional tutorial: Goodfellow's GAN tutorial; Optional papers: 1. Normalizing Flows for Probabilistic Modeling and Inference 2. Generative Adversarial Networks (GANs) | ||
| Monday Mar. 27 | Graphical Models I: Bayesian Networks, Undirected models, Training, Structure Learning | Mitchell chapter 6, Bishop chapter 8.1, Shalev-Shwartz and Ben-David chapter 24; Heckerman Tutorial, Wainwright and Jordan Chapters 2, 3 | ||
| Wednesday Mar. 29 | Graphical Models II | |||
| Monday Apr. 3 | Less-than-full Supervision: Semi-Supervised Learning, Weak Supervision, Self-Supervision | Weak Supervision Notes | ||
| Wednesday Apr. 5 | Kernels + SVMs:Margins, Support Vectors, Kernels | Andrew Ng's note on SVM, Ben-Hur and Weston's note; Mohri-Rostamizadeh-Talwalkar Appendix B (Convex Optimization), Bishop Appendix E (Lagrange Multipliers) | ||
| Monday Apr. 10 | Kernels + SVMs (cont'd) | Mitchell Chapter 13 | ||
| Wednesday Apr. 12 | Reinforcement Learning I: MDPs, Value Iteration, Policy Iteration, Q-Learning | |||
| Monday Apr. 17 | Reinforcement Learning II: Function Approximation, Policy Search, Reinforce | |||
| Wednesday Apr. 19 | Learning Theory: Bayes' optimal classifier, approximation error vs estimation error (Whiteboard lecture, no slides) | Mohri-Rostamizadeh-Talwalkar Chapters 2 and 3, Mitchell Chapter 7 | ||
| Monday Apr. 24 | Learning Theory II: Bounding estimation error, VC Dimension (Whiteboard lecture, no slides) | Mohri-Rostamizadeh-Talwalkar Chapters 2 and 3, Mitchell Chapter 7 | ||
| Wednesday Apr. 26 | Neural Networks V: RNNs | Goodfellow-Bengio-Courville chapter 10; Optional papers to read for part 4: 1. LSTM 2. GRU | ||
| Monday May. 1 | Large Language models: Word Embeddings, Transformers, Pretraining | On the Opportunities and Risks of Foundation Models | ||
| Wednesday May. 3 | AI Ethics: Fairness, bias, privacy, adversarial attacks |