CS760, Fall 2021
Department of Computer Sciences
University of Wisconsin–Madison
This schedule is tentative and subject to change. Please check back often. In particular, the deadlines for the homework sets/project may change; please see Canvas for the actual deadlines.
The reading is not required but strongly recommended for all students. Those explicitly noted as optional are for students interested in that specific topic. "A, B; C; D" means to read (A OR B) AND C AND D. Text in red means a link to the reading material. Abbreviations for textbooks:
| Date | Lecture | Readings | Homework Released | Homework Due |
|---|---|---|---|---|
| Thursday Sept. 9 | Course Overview: Topics, Goals, Evaluation [Slides] [Recording] | Jordan and Mitchell, Science, 2015 | Homework 1 (background test) | |
| Tuesday Sept. 14 | ML Overview: Supervised/Unsupervised/RL, Classification/Regression, General Approach [Slides] [Quiz] [Recording] | |||
| Thursday Sept. 16 | Supervised Learning I: Setup, Examples. Instance-Based Learning, Decision Trees [Slides] [Quiz] [Recording] | Homework 1 due | ||
| Tuesday Sept. 21 | Supervised Learning II: Setup + Examples. Decision Trees [Slides] [Quiz] [Recording] | Murphy Chapter 16.2 / Shalev-Shwartz and Ben-David Chapter 18 | ||
| Thursday Sept. 23 | Evaluation: Bias, Cross-Validation, Precision/Recall, ROC Curves [Slides] [Quiz] [Recording: Part I] [Recording: Part II] | Homework 2 out | ||
| Tuesday Sept. 28 | Regression I: Linear Regression, Logistic Regression, Normal equations, GD [Slides] [Quiz] [Recording] | Murphy 8.1-3 and 8.6; Maximum Likelihood, Logistic Regression, and Stochastic Gradient Training | ||
| Thursday Sept. 30 | Regression II: Logistic Regression, Gradient Descent Analysis, SGD [Slides] [Quiz] [Recording] | Convergence Theorems for Gradient Descent | Homework 2 due | |
| Tuesday Oct. 5 | Naive Bayes: Generative vs Discriminative Models, ML vs MAP [Slides] [Quiz] [Recording] | Mitchell 6.1-6.10, Murphy 3 | Homework 3 out | |
| Thursday Oct. 7 | Neural Networks I: Perceptron, Training, MLPs[Slides] [Quiz] [Recording] | Mitchell chapter 4, Murphy 16.5 and 28, Bishop 5.1-5.3 LeCun et al., Nature, 2015 | ||
| Tuesday Oct. 12 | Neural Networks II: Training, Optimization, SGD, Backpropagation[Slides] [Quiz] [Recording] | Homework 3 due | ||
| Thursday Oct. 14 | Neural Networks III: Regularization, Data Augmentation [Slides] [Quiz] [Recording] | Homework 4 out | ||
| Tuesday Oct. 19 | Neural Networks IV: CNNs [Slides] [Quiz] [Recording] | Goodfellow-Bengio-Courville chapter 9; CNN papers 1. LeNet 2. AlexNet 3. ResNet | ||
| Thursday Oct. 21 | Neural Networks IV: RNNs [Slides] [Quiz] [Recording] | Goodfellow-Bengio-Courville chapter 10; Optional papers to read for part 4: 1. LSTM 2. GRU | Homework 4 due (pushed to Monday) | |
| Tuesday Oct. 26 | Practical Aspects of Training + Review [Slides] [Recording] | |||
| Wednesday Oct. 27 | Midterm | |||
| Thursday Oct. 28 | Generative Models: Autoregressive, Flows, GANs [Slides] [Recording] | Optional tutorial: Goodfellow's GAN tutorial; Optional papers: 1. Normalizing Flows for Probabilistic Modeling and Inference 2. Generative Adversarial Networks (GANs) | ||
| Tuesday Nov. 2 | Kernels + SVMs:Margins, Support Vectors, Kernels [Slides] [Recording] [Quiz] | Andrew Ng's note on SVM, Ben-Hur and Weston's note; Mohri-Rostamizadeh-Talwalkar Appendix B (Convex Optimization), Bishop Appendix E (Lagrange Multipliers) | Homework 5 out | |
| Thursday Nov. 4 | Graphical Models I: Bayesian Networks, Training, Structure Learning [Slides] [Recording] [Quiz] | |||
| Tuesday Nov. 9 | Graphical Models II: Undirected Models, Markov Random Fields [Slides] [Recording] [Quiz] | Mitchell chapter 6, Bishop chapter 8.1, Shalev-Shwartz and Ben-David chapter 24; Heckerman Tutorial, Wainwright and Jordan Chapters 2, 3 | Homework 6 out | Homework 5 due |
| Thursday Nov. 11 | Less-than-full Supervision: Semi-Supervised Learning, Weak Supervision, Self-Supervision [Slides] [Recording] [Quiz] | Weak Supervision Notes | ||
| Tuesday Nov. 16 | Unsupervised Learning I: Clustering, GMM models, EM [Slides] [Quiz] | Homework 7 out | ||
| Thursday Nov. 18 | Unsupervised Learning II: Dimensionality Reduction, PCA [Slides] [Recording] [Quiz] | Homework 6 due | ||
| Tuesday Nov. 23 | Learning Theory: Generalization, PAC [Slides] [Recording] | Mohri-Rostamizadeh-Talwalkar Chapter 2, Mitchell Chapter 7 | Homework 7 due | |
| Tuesday Nov. 30 | Reinforcement Learning I: MDPs, Value Iteration, Policy Iteration [Slides] [Recording] [Quiz] | Mitchell Chapter 13 | ||
| Thursday Dec. 2 | Reinforcement Learning II: Q-learning, Approximation [Slides] [Recording] [Quiz] | Homework 8 Out | ||
| Tuesday Dec. 7 | Reinforcement Learning III: Function Approximation, Policy Search, Reinforce [Slides] [Recording] | |||
| Thursday Dec. 9 | Large Language Models: Word Embeddings, Transformers, Pretraining [Slides] [Recording] | On the Opportunities and Risks of Foundation Models | ||
| Tuesday Dec. 14 | Fairness & Ethics [Slides] [Recording] | Homework 8, Project due | ||
| Monday Dec. 20 (12:25PM - 2:25PM) | Final Exam |