CS760, Spring 2021

Department of Computer Sciences

University of Wisconsinâ€“Madison

This schedule is **tentative** and subject to change. Please
check back often. In particular, the deadlines for the homework sets/project are tentative, please see Canvas for the actual deadlines.

The reading is not required but strongly recommended for all students. Those explicitly noted as optional are for students interested in that specific topic. "A, B; C; D" means to read (A OR B) AND C AND D. Text in red means a link to the reading material. Abbreviations for textbooks:

- Mitchell:
*Machine Learning*, Tom Mitchell. - Bishop:
*Pattern Recognition and Machine Learning*, Chris Bishop. - Murphy:
*Machine Learning: A Probabilistic Perspective*, Kevin Murphy. - Shalev-Shwartz and Ben-David:
*Understanding Machine Learning: From Theory to Algorithms*, Shalev-Shwartz, Ben-David. - Goodfellow-Bengio-Courville:
*Deep Learning*, Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Mohri-Rostamizadeh-Talwalkar:
*Foundations of Machine Learning*, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar.

Date | Lecture | Readings | Homework Released | Homework Due |
---|---|---|---|---|

Mon, Jan 25 | course overview [Slides] [Quizzes] | Murphy chapter 1; Jordan and Mitchell, Science, 2015 |
Homework 1 (background test) | |

Fri, Jan 29 | machine learning overview [Slides] [Quizzes] | |||

Mon, Feb 1 | decision tree learning part 1 [Slides] [Quizzes] | Murphy chapter 16.2, Shalev-Shwartz and Ben-David chapter 18 | Homework 1 due | |

Fri, Feb 5 | decision tree learning part 2 [Slides] [Quizzes] | Optional papers to read: 1. CART paper: Breiman, Leo; Friedman, J. H.; Olshen, R. A.; Stone, C. J. (1984). Classification and regression trees. Monterey, CA: Wadsworth & Brooks/Cole Advanced Books & Software. ISBN 978-0-412-04841-8. 2. ID3 paper: Quinlan, J. R. 1986. Induction of Decision Trees. Mach. Learn. 1, 1 (Mar. 1986), 81â€“106 3. C4.5 paper: Quinlan, J. R. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, 1993. |
||

Mon, Feb 8 | instance-based methods [Slides] [Quizzes] | Shalev-Shwartz and Ben-David chapter 19 | ||

Wed, Feb 10 | evaluating learning algorithms part 1 [Slides] [Quizzes] | Homework 2 | ||

Mon, Feb 15 | evaluating learning algorithms part 2 [Slides] [Quizzes] | |||

Fri, Feb 19 | Naive Bayes [Slides] [Quizzes] | Mitchell 6.1-6.10, Murphy 3 | ||

Mon, Feb 22 | regression (linear and logistic) [Slides] [Quizzes] | Murphy 8.1-3 and 8.6; Maximum Likelihood, Logistic Regression, and Stochastic Gradient Training |
Homework 3 | Homework 2 due |

Fri, Feb 26 | regression continued [Quizzes] | |||

Mon, Mar 1 | neural networks part 1 [Slides][Quizzes] | Mitchell chapter 4, Murphy 16.5 and 28, Bishop 5.1-5.3, Goodfellow-Bengio-Courville 6; LeCun et al., Nature, 2015 |
Homework 4 | Homework 3 due |

Fri, Mar 5 | neural networks part 1 continue [Quizzes] | |||

Mon, Mar 8 | neural networks part 2 [Slides] [Quizzes] | Homework 5 | Homework 4 due | |

Fri, Mar 12 | Midterm | |||

Mon, Mar 15 | neural networks part 2 continue [Quizzes] | Mohri-Rostamizadeh-Talwalkar Appendix B (Convex Optimization), Bishop Appendix E (Lagrange Multipliers); Goodfellow-Bengio-Courville chapter 7 and 8; Optional paper to read: 1. Bishop, Neural Computation, 1995. |
||

Wed, Mar 17 | neural networks part 3 [Slides] [Quizzes] | Goodfellow-Bengio-Courville chapter 9; Optional papers to read for part 3: 1. LeNet 2. AlexNet 3. ResNet |
||

Mon, Mar 22 | neural networks part 4 [Slides] [Quizzes] |
Goodfellow-Bengio-Courville chapter 10; Optional papers to read for part 4: 1. LSTM 2. GRU |
Homework 6 | Homework 5 due |

Wed, Mar 24 | neural networks part 5 [Slides] [Quizzes] |
Optional tutorial: Goodfellow's tutorial about GAN; Optional papers: 1. Deep Boltzmann Machines (DBM) 2. Generative Adversarial Networks (GAN) |
||

Mon, Mar 29 | introduction to learning theory part 1 [Slides] [Quizzes] | |||

Wed, Mar 31 | introduction to learning theory part 2 [Slides] [Quizzes] | Mohri-Rostamizadeh-Talwalkar Chapter 2, Mitchell chapter 7 | ||

Mon, Apr 5 | Bayesian networks part 1 [Slides] [Quizzes] | Mitchell chapter 6, Bishop chapter 8.1, Shalev-Shwartz and Ben-David chapter 24; Heckerman Tutorial | Homework 6 due | |

Wed, Apr 7 | Bayesian networks part 2 [Slides] [Quizzes] | Homework 7 | ||

Mon, Apr 12 | support vector machines part 1 [Slides] [Quizzes] | Andrew Ng's note on SVM, Ben-Hur and Weston's note; Mohri-Rostamizadeh-Talwalkar Appendix B (Convex Optimization), Bishop Appendix E (Lagrange Multipliers) |
||

Fri, Apr 16 | support vector machines part 2 [Slides] [Quizzes] | Bishop chapter 6.1,6.2,7.1, or Shalev-Shwartz and Ben-David chapter 15, 16 | ||

Mon, Apr 19 | dimension reduction [Slides] [Quizzes] | Bishop 12.1 and 12.3, or Shalev-Shwartz and Ben-David 22 and 23 | Homework 8 | Homework 7 due |

Wen, Apr 21 | reinforcement learning part 1 [Slides] [Quizzes] | Mitchell Chapter 13 | ||

Mon, Apr 26 | reinforcement learning part 2 [Slides] [Quizzes] | Reinforcement Learning, A Survey by Kaelbling, et al | ||

Wed, Apr 28 | ensemble methods [Slides] [Quizzes] | Homework 8 due | ||

Thu, May 6 | Final exam |