CS 760, Fall 2023
Department of Computer Sciences
University of Wisconsin–Madison
This schedule is tentative and subject to change. Please check back often. In particular, the deadlines for the homework sets/project may change; please see Canvas for the actual deadlines.
The reading is not required but strongly recommended for all students. Those explicitly noted as optional are for students who want to go deeper in that specific topic. "A, B; C; D" means to read (A OR B) AND C AND D. Text in blue means a link to the reading material. Abbreviations for textbooks:
| Date | Lecture | Readings | Homework Released | Homework Due |
|---|---|---|---|---|
| Thursday, Sept 7 | Course Overview: Topics, Goals, Evaluation | Jordan and Mitchell, Science, 2015 | Hw 1 Released | |
| Tuesday, Sept 12 | ML Overview: Supervised/Unsupervised/RL, Classification/Regression, General Approach | |||
| Thursday, Sept 14 | Supervised Learning I: Setup, Examples. k-Nearest Neighbors, Decision Trees | |||
| Tuesday, Sept 19 | Supervised Learning II: Decision Trees (cont'd) | Murphy Chapter 16.2 / Shalev-Shwartz and Ben-David Chapter 18 | Hw 2 Released | Hw 1 Due |
| Thursday, Sept 21 | Evaluation: Bias, Cross-Validation, Precision/Recall, ROC Curves | |||
| Tuesday, Sept 26 | Regression I: Linear Regression, Logistic Regression, Normal equations, GD | Murphy 8.1-3 and 8.6; Maximum Likelihood, Logistic Regression, and Stochastic Gradient Training | ||
| Thursday, Sept 28 | Regression II: Linear regression, Logistic regression, Gradient Descent Analysis | Convergence Theorems for Gradient Descent | Hw 3 Released | Hw 2 Due |
| Tuesday, Oct 3 | Naive Bayes: Generative vs Discriminative Models, ML vs MAP | Mitchell 6.1-6.10, Murphy 3 | ||
| Thursday, Oct 5 | Neural Networks I: Perceptron, Training, MLPs | Mitchell chapter 4, Murphy 16.5 and 28, Bishop 5.1-5.3 LeCun et al., Nature, 2015 | ||
| Tuesday, Oct 10 | Neural Networks II: Training, Optimization, SGD, Backpropagation | Hw 4 Released | Hw 3 Due | |
| Thursday, Oct 12 | Neural Networks III: Regularization, Data Augmentation | |||
| Tuesday, Oct 17 | Neural Networks IV: CNNs
Neural Networks (cont'd): Practical training tips |
Goodfellow-Bengio-Courville chapter 9; CNN papers 1. LeNet 2. AlexNet 3. ResNet | ||
| Midterm: October 18th at 5:45-7:15pm in Noland Hall Room 132. | ||||
| Thursday, Oct 19 | Neural Networks V: RNNs | Goodfellow-Bengio-Courville chapter 10; Optional papers to read for part 4: 1. LSTM 2. GRU | ||
| Tuesday, Oct 24 | Large Language models: Word Embeddings, Transformers, Pretraining | On the Opportunities and Risks of Foundation Models | ||
| Thursday, Oct 26 | Unsupervised Learning I: Clustering, GMMs, and EM | Andrew Ng's notes on the EM Algorithm | ||
| Tuesday, Oct 31 | Unsupervised Learning II: Dimensionality Reduction, PCA | Hw 5 Released | Hw 4 Due | |
| Thursday, Nov 2 | Unsupervised Learning III: Generative NN models | Optional tutorial: Goodfellow's GAN tutorial; Optional papers: 1. Normalizing Flows for Probabilistic Modeling and Inference 2. Generative Adversarial Networks (GANs) | ||
| Tuesday, Nov 7 | Graphical Models I: Bayesian Networks, Undirected models, Training, Structure Learning | Mitchell chapter 6, Bishop chapter 8.1, Shalev-Shwartz and Ben-David chapter 24; Heckerman Tutorial, Wainwright and Jordan Chapters 2, 3 | ||
| Thursday, Nov 9 | Graphical Models II | Hw 6 Released | Hw 5 Due | |
| Tuesday, Nov 14 | Less-than-full Supervision: Semi-Supervised Learning, Weak Supervision, Self-Supervision | Weak Supervision Notes | ||
| Thursday, Nov 16 | Learning Theory: Motivation, PAC-learning, Mistake Bounds, VC Dimension | Mohri-Rostamizadeh-Talwalkar Chapters 2 and 3, Mitchell Chapter 7 | ||
| Tuesday, Nov 21 | Bonus Material: Kernels + SVMs | Mitchell Chapter 13 | Hw 7 Released | Hw 6 Due |
| Thursday, Nov 23 | No class -- happy Thanksgiving! | |||
| Tuesday, Nov 28 | Reinforcement Learning I: MDPs and Value Functions | Sutton and Barto, Chapter 3 | ||
| Thursday, Nov 30 | Reinforcement Learning II: Policy iteration, value iteration, q-learning | Sutton and Barto, Chapter 4 and 6 | ||
| Tuesday, Dec 5 | Reinforcement Learning III: Q-learning, function approximation, and deep q-learning | Sutton and Barto, Chapter 9 | ||
| Thursday, Dec 7 | Reinforcement Learning IV: Function Approximation (cont'd), policy gradient methods | Sutton and Barto, Chapter 13 | Hw 7 Due | |
| Tuesday, Dec 12 | Societal Implications: Fairness, bias, privacy, adversarial attacks | |||
| Final Exam: December 18 from 2:45 - 4:45pm (Social Sciences Building 5206) | ||||