CS 760: Machine Learning (Spring 2017)

• Course Overview, Feature Vector Representation, Unsupervised Learning Overview (Mitchell Ch. 1)
• Brief Introduction to Probability (Mitchell Ch. 6, supplementary background notes on probability and Bayesian Networks: 1, 2, 3)
• Decision Trees (Mitchell Ch. 3, Skewing)
• Instance-Based Learning, k-Nearest Neighbor (Mitchell Ch. 8.1 and 8.2)
• Bayesian Network Learning including Naive Bayes and TAN (Heckerman Tutorial; Recommended: Friedman, Geiger & Goldszmidt, Machine Learning Journal 1997; Friedman, Nachman & Peer, UAI-99; Mitchell Ch. 6; additional lecture notes on Gibbs Sampling and MCMC theory [PDF])
• Machine Learning Methodology (Mitchell Ch. 5; Optional Supplements: The Case Against Accuracy Estimation for Comparing Induction Algorithms by F. Provost, T. Fawcett, and R. Kohavi, Proc. ICML-98; The Relationship Between Precision-Recall and ROC Curves by J. Davis and M. Goadrich, Proc. ICML-06)
• Neural Networks and Deep Learning (Yujia Bao's Guest Lecture on Deep Learning, Mitchell Ch. 4, Andrew Ng's Deep Learning Tutorial)
• Generative Adversarial Networks
• Computational Learning Theory [PDF](Mitchell Ch. 7)
• Regression (Linear and Logistic, including LASSO-penalized forms)
• Support Vector Machines (Ben-Hur and Weston, 2010, Alternative SVM Lecture by Gautam Kunapuli (optional), Chris Burges's tutorial (optional))
• SVM by Sequential Minimal Optimization (SMO)[PDF] (Platt's original SMO paper)
• Ensemble Methods [PDF] (Dietterich, 2002)
• Temporal Models (includes dynamic Bayesian networks, continuous-time Bayesian networks, piecewise-constant conditional intensity models, Hawkes processes)
• Reinforcement Learning (Mitchell Ch. 13)
• Review for Exam on April 10
• Exam on April 12
• Rule Learning and Relational Learning (Mitchell Ch. 10)
• Markov Networks. Try this tutorial on log-linear models by Frank Ferraro and Jason Eisner.
• Statistical Relational Learning
• The lectures below for ILP and SRL will not be used in class, but are left here for background.
• Background for Rule Learning and Inductive Logic Programming (Mitchell Ch. 10; for added background see De Raedt & Muggleton)
• Rule Learning and Inductive Logic Programming (Mitchell Ch. 10; for added background see De Raedt & Muggleton)
  • Logical Foundations
  • ILP as Refinement Graph Search (Lavrac & Dzeroski, Sections 3.2, 4.1, Chapter 7)
• Statistical Relational Learning
  • Markov Networks and Markov Logic Networks: see also Lecture by Pedro Domingos on Statistical Modeling of Relational Data (Domingos & Richardson)
  • Plate Models and Probabilistic Relational Models (Heckerman, Meek & Koller)
  • View Learning in SRL: With an Application to Mammography (Davis et al., 2007)
• Dimensionality-Reduction
• Remaining Topics: Active Learning, Causal Discovery, Multiple-Instance Learning (if time permits)

• Homework Assignments (5 anticipated): 40%
• Exam: 35%
• Project: 25%

Homework assignments are due at the start of class on the assigned due date, and late homeworks will be penalized 10 points (out of 100) for each day that passes after the assigned due date. Homeworks cannot be submitted more than one week late; the submission site will be locked at that time. At the start of the course every student will be given 5 "free" days, each of which may be used to offset a 10-point late penalty. Only 2 free days can be used for any given written assignment, so that solutions can be posted at next class period. Free days are non-transferable, and no credit will be given for unused free days. Nevertheless, please use them sparingly because the late penalty is strictly enforced. Please do not ask for special consideration for travel, exams in other classes, and other extenuating circumstances; this is what the free days are there for.

Homework Assignments