Machine Learning

CS760, Spring 2021
Department of Computer Sciences
University of Wisconsin–Madison

Important Notes

This schedule is tentative and subject to change. Please check back often. In particular, the deadlines for the homework sets/project are tentative, please see Canvas for the actual deadlines.

How to interpret the column Readings in the Tentative Schedule

The reading is not required but strongly recommended for all students. Those explicitly noted as optional are for students interested in that specific topic. "A, B; C; D" means to read (A OR B) AND C AND D. Text in red means a link to the reading material. Abbreviations for textbooks:

Tentative Schedule

Please view the pdf slides using Adobe PDF reader. Some notations do not show up correctly in say Chrome.
Date Lecture Readings Homework Released Homework Due
Mon, Jan 25 course overview [Slides] [Quizzes] Murphy chapter 1;
Jordan and Mitchell, Science, 2015		 Homework 1 (background test)
Fri, Jan 29 machine learning overview [Slides] [Quizzes]
Mon, Feb 1 decision tree learning part 1 [Slides] [Quizzes] Murphy chapter 16.2, Shalev-Shwartz and Ben-David chapter 18 Homework 1 due
Fri, Feb 5 decision tree learning part 2 [Slides] [Quizzes] Optional papers to read:
1. CART paper: Breiman, Leo; Friedman, J. H.; Olshen, R. A.; Stone, C. J. (1984). Classification and regression trees. Monterey, CA: Wadsworth & Brooks/Cole Advanced Books & Software. ISBN 978-0-412-04841-8.
2. ID3 paper: Quinlan, J. R. 1986. Induction of Decision Trees. Mach. Learn. 1, 1 (Mar. 1986), 81–106
3. C4.5 paper: Quinlan, J. R. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, 1993.
Mon, Feb 8 instance-based methods [Slides] [Quizzes] Shalev-Shwartz and Ben-David chapter 19
Wed, Feb 10 evaluating learning algorithms part 1 [Slides] [Quizzes] Homework 2
Mon, Feb 15 evaluating learning algorithms part 2 [Slides] [Quizzes]
Fri, Feb 19 Naive Bayes [Slides] [Quizzes] Mitchell 6.1-6.10, Murphy 3
Mon, Feb 22 regression (linear and logistic) [Slides] [Quizzes] Murphy 8.1-3 and 8.6;
Maximum Likelihood, Logistic Regression, and Stochastic Gradient Training		 Homework 3 Homework 2 due
Fri, Feb 26 regression continued [Quizzes]
Mon, Mar 1 neural networks part 1 [Slides][Quizzes] Mitchell chapter 4, Murphy 16.5 and 28, Bishop 5.1-5.3, Goodfellow-Bengio-Courville 6;
LeCun et al., Nature, 2015 		Homework 4 Homework 3 due
Fri, Mar 5 neural networks part 1 continue [Quizzes]
Mon, Mar 8 neural networks part 2 [Slides] [Quizzes] Homework 5 Homework 4 due
Fri, Mar 12 Midterm
Mon, Mar 15 neural networks part 2 continue [Quizzes] Mohri-Rostamizadeh-Talwalkar Appendix B (Convex Optimization), Bishop Appendix E (Lagrange Multipliers);
Goodfellow-Bengio-Courville chapter 7 and 8;
Optional paper to read:
1. Bishop, Neural Computation, 1995.
Wed, Mar 17 neural networks part 3 [Slides] [Quizzes] Goodfellow-Bengio-Courville chapter 9;
Optional papers to read for part 3:
1. LeNet
2. AlexNet
3. ResNet
Mon, Mar 22 neural networks part 4 [Slides] [Quizzes] Goodfellow-Bengio-Courville chapter 10;
Optional papers to read for part 4:
1. LSTM
2. GRU
Homework 6 Homework 5 due
Wed, Mar 24 neural networks part 5 [Slides] [Quizzes] Optional tutorial:
Goodfellow's tutorial about GAN;
Optional papers:
1. Deep Boltzmann Machines (DBM)
2. Generative Adversarial Networks (GAN)
Mon, Mar 29 introduction to learning theory part 1 [Slides] [Quizzes]
Wed, Mar 31 introduction to learning theory part 2 [Slides] [Quizzes] Mohri-Rostamizadeh-Talwalkar Chapter 2, Mitchell chapter 7
Mon, Apr 5 Bayesian networks part 1 [Slides] [Quizzes] Mitchell chapter 6, Bishop chapter 8.1, Shalev-Shwartz and Ben-David chapter 24; Heckerman Tutorial Homework 6 due
Wed, Apr 7 Bayesian networks part 2 [Slides] [Quizzes] Homework 7
Mon, Apr 12 support vector machines part 1 [Slides] [Quizzes] Andrew Ng's note on SVM, Ben-Hur and Weston's note;
Mohri-Rostamizadeh-Talwalkar Appendix B (Convex Optimization), Bishop Appendix E (Lagrange Multipliers)
Fri, Apr 16 support vector machines part 2 [Slides] [Quizzes] Bishop chapter 6.1,6.2,7.1, or Shalev-Shwartz and Ben-David chapter 15, 16
Mon, Apr 19 dimension reduction [Slides] [Quizzes] Bishop 12.1 and 12.3, or Shalev-Shwartz and Ben-David 22 and 23 Homework 8 Homework 7 due
Wen, Apr 21 reinforcement learning part 1 [Slides] [Quizzes] Mitchell Chapter 13
Mon, Apr 26 reinforcement learning part 2 [Slides] [Quizzes] Reinforcement Learning, A Survey by Kaelbling, et al
Wed, Apr 28 ensemble methods [Slides] [Quizzes] Homework 8 due
Thu, May 6 Final exam