CS760, Spring 2020
Department of Computer Sciences
University of Wisconsin–Madison
This schedule is tentative and subject to change. Please check back often. In particular, the deadlines for the homework sets/project are tentative, please see Canvas for the actual deadlines.
The homework problem sets can be found on the Coursework page.
The reading is not required but strongly recommended for all students. Those explicitly noted as optional are for students interested in that specific topic. "A, B; C; D" means to read (A OR B) AND C AND D. Text in red means a link to the reading material. Abbreviations for textbooks:
| Date | Lecture | Readings | Homework Released | Homework Due |
|---|---|---|---|---|
| Wed, Jan 22 | course overview [Slides] | Murphy chapter 1; Jordan and Mitchell, Science, 2015 |
Homework 1 (background test) | |
| Fri, Jan 24 | No class | |||
| Mon, Jan 27 | machine learning overview [Slides] | |||
| Wed, Jan 29 | decision tree learning part 1 [Slides] | Murphy chapter 16.2, Shalev-Shwartz and Ben-David chapter 18 | ||
| Fri, Jan 31 | No class | |||
| Mon, Feb 3 | decision tree learning part 2 [Slides] | Optional papers to read: 1. CART paper: Breiman, Leo; Friedman, J. H.; Olshen, R. A.; Stone, C. J. (1984). Classification and regression trees. Monterey, CA: Wadsworth & Brooks/Cole Advanced Books & Software. ISBN 978-0-412-04841-8. 2. ID3 paper: Quinlan, J. R. 1986. Induction of Decision Trees. Mach. Learn. 1, 1 (Mar. 1986), 81–106 3. C4.5 paper: Quinlan, J. R. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, 1993. |
||
| Wed, Feb 5 | instance-based methods [Slides] | Shalev-Shwartz and Ben-David chapter 19 | Homework 2 | Homework 1 due on Canvas |
| Fri, Feb 7 | No class | |||
| Mon, Feb 10 | No class | |||
| Wed, Feb 12 | instance-based methods continued | Homework 3 | Homework 2 due | |
| Fri, Feb 14 | evaluating learning algorithms part 1 [Slides] | Mitchell chapter 5, Murphy 5.7.2, 6.5.3; Manning et al., Sections 8.3-8.4 |
||
| Mon, Feb 17 | evaluating learning algorithms part 2 [Slides] | |||
| Wed, Feb 19 | Naive Bayes [Slides] | Mitchell 6.1-6.10, Murphy 3 | Homework 4 | Homework 3 due |
| Fri, Feb 21 | regression (linear and logistic) [Slides] | Murphy 8.1-3 and 8.6; Maximum Likelihood, Logistic Regression, and Stochastic Gradient Training |
||
| Mon, Feb 24 | regression continued | |||
| Wed, Feb 26 | neural networks part 1 [Slides] | Mitchell chapter 4, Murphy 16.5 and 28, Bishop 5.1-5.3, Goodfellow-Bengio-Courville 6; LeCun et al., Nature, 2015 |
Homework 4 due | |
| Fri, Feb 28 | No class | Homework 5 | ||
| Mon, Mar 2 | neural networks part 1 continue | Homework 6 | ||
| Wed, Mar 4 | neural networks part 2 [Slides] | Mohri-Rostamizadeh-Talwalkar Appendix B (Convex Optimization), Bishop Appendix E (Lagrange Multipliers); Goodfellow-Bengio-Courville chapter 7 and 8; Optional paper to read: 1. Bishop, Neural Computation, 1995. |
||
| Fri, Mar 6 | neural networks part 2 continue | |||
| Mon, Mar 9 | Midterm 1 | |||
| Mar 14 - Mar 22 | No class and Spring recess | Homework 5 due on Mar 13 | ||
| Mon, Mar 23 | neural networks part 3 [Slides] | Goodfellow-Bengio-Courville chapter 9; Optional papers to read for part 3: 1. LeNet 2. AlexNet 3. ResNet |
Project proposal | Homework 6 due |
| Wed, Mar 25 | neural networks part 3 continued | |||
| Fri, Mar 27 | neural networks part 4 [Slides] |
Goodfellow-Bengio-Courville chapter 10; Optional papers to read for part 4: 1. LSTM 2. GRU |
||
| Mon, Mar 30 | discriminative vs. generative learning [Slides] | Ng and Jordan, NIPS 2001 | ||
| Wed, Apr 1 | Bayesian networks part 1 [Slides] | Mitchell chapter 6, Bishop chapter 8.1, Shalev-Shwartz and Ben-David chapter 24; Heckerman Tutorial | Project proposal due | |
| Fri, Apr 3 | Bayesian networks part 2 [Slides] | |||
| Mon, Apr 6 | Bayesian networks part 2 continue | |||
| Wed, Apr 8 | support vector machines part 1 [Slides] | Andrew Ng's note on SVM, Ben-Hur and Weston's note; Mohri-Rostamizadeh-Talwalkar Appendix B (Convex Optimization), Bishop Appendix E (Lagrange Multipliers) |
Homework 7 | |
| Fri, Apr 10 | support vector machines part 2 [Slides] | Bishop chapter 6.1,6.2,7.1, or Shalev-Shwartz and Ben-David chapter 15, 16 | ||
| Mon, Apr 13 | introduction to learning theory [Slides] |
Mohri-Rostamizadeh-Talwalkar Chapter 2, Mitchell chapter 7; Optional paper to read: Geman et al., Neural Computation, 1992 (Sections 1-3) |
||
| Wed, Apr 15 | introduction to learning theory continue | Homework 8 | Homework 7 due | |
| Fri, Apr 17 | introduction to learning theory continue | |||
| Mon, Apr 20 | dimension reduction [Slides] | Bishop 12.1 and 12.3, or Shalev-Shwartz and Ben-David 22 and 23 | ||
| Wed, Apr 22 | reinforcement learning part 1 [Slides] | Mitchell Chapter 13 | ||
| Fri, Apr 24 | reinforcement learning part 2 [Slides] | Reinforcement Learning, A Survey by Kaelbling, et al | ||
| Mon, Apr 27 | Homework 8 due | |||
| Tue, Apr 28 | Midterm #2 | |||
| Thu, May 7 | Project report due |