University of Wisconsin Computer Sciences Header Map (repeated with
textual links if page includes departmental footer)

CS 760 - Machine Learning

General Course Information

Instructor: Jude Shavlik
6393 CS Building
shavlik@cs.wisc.edu
Office Hr: Monday 1:30-2:30pm or by appointment
Teaching Assistant: Zhiting Xu
5387 CS Building
zhiting@cs.wisc.edu
Office Hr: Wednesday 10-10:50am, Thursday 10-11am, or by appointment (send email)
Prerequisite: CS 540 or equivalent
Course Textbook: Machine Learning by Tom Mitchell, McGraw-Hill (includes errata). You might want to check AbeBooks or Amazon for used copies.
We will also read parts of Sutton and Barto's on-line RL textbook (feel free to buy the hardcopy version). And we will read parts of a book on Markov Logic Networks and another on Semi-supervised Learning that are downloadable for free because the University of Wisconsin has a site license with the publisher (see Lecture 1 for the URL). Several journal articles and conference papers will also be assigned.
Meeting Times: MWF 11am-12:15pm
We will meet 30 times over the semester, mainly skipping Fridays and only meeting once a week the last few weeks. We will not meet Friday, February 19 because I will be traveling. Until then, we will meet every MWF unless otherwise noted.
Location: Room 1257 CS Building
Archive of Class Email - use your CS dept. login to access. Send mail to this list. Note that email will be sent to your "@wisc.edu" address.
Exam: MONDAY April 26, 2010, Room 1240 CS, 11am-12:30pm
One 8.5x11-inch sheet of notes and non-programmable calculators allowed.

Course Overview and Requirements

The intent of this course is to present a broad introduction to machine learning, including discussions of each of the major approaches currently being investigated. Class lectures will discuss general issues in machine learning, as well as present established algorithms. One secondary goal is to compare and contrast the various approaches, determining under which conditions each is most appropriate. Another is to relate these algorithms to human learning processes.

The work in the course will consist of four homework assignments (about one every two weeks during the first part of the class), one exam (very late in the semester), and a course project. Your solutions will be partially automatically graded, so they must be written to run in Java on the instructional computers.

The homework assignments will generally be programming assignments that involve experimenting with machine learning algorithms and experimental methodology. Lab reports insightfully analyzing your experimental results will be required. Occasionally there may also be "paper-and-pencil" homework problems.

The final project can be a more ambitious experiment or enhancement involving algorithms used in the homeworks or the implementation of an additional algorithm. In either case, the implementation should be accompanied by a short paper (7-10 pages with a large font) describing the project. Projects not involving programming are also possible.

Homeworks will count for 30% of the grade, the "midterm" exam for 40%, the project 25%, and quality class participation 5%. The course will be graded on the conventional (A-F) system.

Lecture Notes

Powerpoint files containing each lecture will be available as the course progresses.
(Access is limited to University of Wisconsin sites.)

Reading Assignments

Assigned April 28, 2010:

Read Chapter 10 of Mitchell's textbook.

Read Chapters 1 and 2, plus Sections 4.1 and 4.2 of Markov Logic: An Interface Layer for Articial Intelligence, by Domingos and Lowd, in the Morgan-Claypool Series or read Markov Logic Networks, Richardson & Domingos, MLj:62, 2006.

Assigned April 7, 2010:

Read Chapter 7 of Mitchell's textbook and Chapter 2 of Zhu and Goldberg's book on semi-supervised learning in the Morgan-Claypool Series.

Assigned March 17, 2010:

Read Chapter 13 of Mitchell's textbook.
Read Preface, Chapter 1 (just skim Section 1.2), and Sections 6.4 and 6.5 of Sutton and Barto's on-line RL textbook.

Assigned March 8, 2010:

Read Chapter 4 of Mitchell's textbook and the SVM tutorial from ICML-2001 given by Nello Cristianini.
(You can find other SVM tutorials at http://www.kernel-machines.org/tutorials; the one by Burgess is a good one to read if you wish additional details.)

Assigned February 24, 2010:

Pages 97-105 of 'Machine Learning Research: Four Current Directions' by T. Dietterich in the AI Magazine.

Random Forests, L. Breiman, Machine Learning, 45, pp. 5-32, 2001 (won't be on the exam, beyond what is covered in the lecture notes)

Model Selection: Beyond the Bayesian/Frequentist Divide , I. Guyon, A. Saffari, G. Dror, and G. Cawley, JMLR 11, pp. 61-87, 2010.

Assigned February 15, 2010:

Read Chapter 3 of Mitchell's textbook

Assigned February 10, 2010:

Read and 'Building the Case Against Accuracy Estimation for Comparing Induction Algorithms' by F. Provost, T. Fawcett, and R. Kohavi, Proc. ICML-98, Madison, pp. 445-453.

Read Sections 6.7-6.11 (skim 6.11.5) and 6.13 of Mitchell's textbook.

Assigned January 27, 2010:

Read Sections 2.1-2.3 and Chapter 5 of Mitchell's textbook.

Assigned January 20, 2010:

Read Preface, Chapter 1, Sections 6.1-6.2, and Sections 8.1-8.2 of Mitchell's textbook. Also read the on-line draft chapter Mitchell created on Naive Bayes and Logistic Regression

Homework Assignments

Homework 4: Reinforcement Learning
Due Monday, 4/19/09
Solution
Homework 3: Training Perceptrons and Using Kernels
Due Wednesday, 4/7/10
Testbeds used during grading
Homework 2: Decision Trees and Random Forests
Due Monday, 3/15/10
Testbeds used during grading
Homework 1: Experimental Methodology, Feature Selection, k-NN, and Naive Bayes
Due Wednesday, 2/17/10
Testbeds used during grading
Homework 0: Creating Your Personal Dataset
Due Monday, 2/1/10
Late policy on HWs:
- HWs are due at 4 pm. If you don't turn them in at the end of class, then put them in the TA's mailbox (5th floor of CS).
- Each student will have FIVE "free" late days for use over the semester. Once these are exhausted, there will be a penalty of 10 points per day (measured 4pm-to-4pm; weekends and university holidays are free). "Days" are calendar days, not class days.
- To make the TA's job tractable, no HWs will be accepted more than one week late.
Academic Misconduct
All examinations, programming assignments, and written homeworks must be done individually. Cheating and plagiarism will be dealt with in accordance with University procedures (see the Academic Misconduct Guide for Students). Hence, for example, code for programming assignments must not be developed in groups, nor should code be shared. You are encouraged to discuss with your peers, the TAs or the instructor ideas, approaches and techniques broadly, but not at a level of detail where specific implementation issues are described by anyone. If you have any questions on this, please ask the instructor before you act.