University of Wisconsin Computer Sciences Header Map (repeated with
textual links if page includes departmental footer)

CS 760 - Machine Learning

General Course Information

Instructor: Jude Shavlik
6393 CS Building
shavlik@cs.wisc.edu
Office Hr: Wednesday 2:30-3:30pm or by appointment
Teaching Assistant: Ting Chen
1308 CS Building
tchen@cs.wisc.edu
Office Hrs: Thursday 1-2pm or by appointment (send email)
Prerequisite: CS 540 or equivalent
Course Textbook: Machine Learning by Tom Mitchell, McGraw-Hill (includes errata).
We will also read parts of Sutton and Barto's on-line RL textbook (feel free to buy the hardcopy version). And parts of a book in Markov Logic Networks (when it is published, it will be downloadable for free because the University of Wisconsin has a site license with the publisher). Several journal articles and conference papers will also be assigned.
Meeting Times: MWF 9:30am-10:45am (We will meet 30 times over the semester.)
Location: Room 115 Psychology
Archive of Class Email - use your CS dept. login to access. Send mail to this list. Note that email will be sent to your "@wisc.edu" address.
Exam: Date TBA
One 8.5x11-inch sheet of notes and non-programmable calculators allowed.

Course Overview and Requirements

The intent of this course is to present a broad introduction to machine learning, including discussions of each of the major approaches currently being investigated. Class lectures will discuss general issues in machine learning, as well as present established algorithms. One secondary goal is to compare and contrast the various approaches, determining under which conditions each is most appropriate. Another is to relate these algorithms to human learning processes.

The work in the course will consist of four homework assignments (about one every two weeks during the first part of the class), one exam (very late in the semester), and a course project. Your solutions will be partially automatically graded, so they must be written to run in Java on the instructional computers.

The homework assignments will generally be programming assignments that involve experimenting with machine learning algorithms and experimental methodology. Lab reports insightfully analyzing your experimental results will be required. Occasionally there may also be "paper-and-pencil" homework problems.

The final project can be a more ambitious experiment or enhancement involving algorithms used in the homeworks or the implementation of an additional algorithm. In either case, the implementation should be accompanied by a short paper (7-10 pages with a large font) describing the project. Projects not involving programming are also possible.

Homeworks will count for 30% of the grade, the "midterm" exam for 40%, the project 25%, and quality class participation 5%. The course will be graded on the conventional (A-F) system.

Lecture Notes

Powerpoint files containing each lecture will be available as the course progresses.
(Access is limited to University of Wisconsin sites.)

Reading Assignments

Assigned May 6, 2009:: Read Chapter 12 of Mitchell's textbook.
Assigned April 22, 2009:: Read Chapter 10 of Mitchell's textbook.
Read Markov Logic Networks, Richardson & Domingos, MLj:62, 2006.
Assigned April 13, 2009:: Read Chapter 7 of Mitchell's textbook.
Assigned March 25, 2009:: Read Chapter 13 of Mitchell's textbook.
Read Preface, Chapter 1 (just skim Section 1.2), and Sections 6.4, 6.5, and 8.3.2 of Sutton and Barto's on-line RL textbook.
Assigned March 4, 2009:: Read Chapter 4 of Mitchell's textbook and the SVM tutorial from ICML-2001 given by Nello Cristianini.
(You can find other SVM tutorials at http://www.kernel-machines.org/tutorials; the one by Burgess is a good one to read if you wish additional details.)
Assigned February 27, 2009:: Pages 97-105 of 'Machine Learning Research: Four Current Directions' by T. Dietterich in the AI Magazine.
Assigned February 16, 2009:: Read Chapter 3 of Mitchell's textbook
Assigned February 13, 2009:: Read and 'Building the Case Against Accuracy Estimation for Comparing Induction Algorithms' by F. Provost, T. Fawcett, and R. Kohavi, Proc. ICML-98, Madison, pp. 445-453.
Assigned February 6, 2009:: Read Sections 6.7-6.11 (skim 6.11.5) of Mitchell's textbook.
Assigned January 30, 2009:: Read Sections 2.1-2.3 and Chapter 5 of Mitchell's textbook.
Assigned January 21, 2009:: Read Preface, Chapter 1, Sections 6.1-6.2, and Sections 8.1-8.2 of Mitchell's textbook. Also read the on-line draft chapter Mitchell created on Naive Bayes and Logistic Regression

Homework Assignments

Homework 4: Reinforcement Learning
Due Wednesday, 4/22/09
Solution
Homework 3: Training Perceptrons and Using Kernels
Due Monday, 4/10/09
Testbeds used during grading
Homework 2: Decision Trees and Random Forests
Due Wednesday, 3/11/09
Testbeds used during grading
Homework 1: Experimental Methodology, Feature Selection, k-NN, and Naive Bayes
Due Friday, 2/20/09
Testbeds used during grading
Homework 0: Creating Your Personal Dataset
Due Monday, 2/2/09
Late policy on HWs:
- HWs are due at 4 pm. If you don't turn them in at the end of class, then put them in the TA's mailbox (5th floor of CS).
- Each student will have FIVE "free" late days for use over the semester. Once these are exhausted, there will be a penalty of 10 points per day (measured 4pm-to-4pm; weekends and university holidays are free). "Days" are calendar days, not class days.
- To make the TA's job tractable, no HWs will be accepted more than one week late.
Academic Misconduct
All examinations, programming assignments, and written homeworks must be done individually. Cheating and plagiarism will be dealt with in accordance with University procedures (see the Academic Misconduct Guide for Students). Hence, for example, code for programming assignments must not be developed in groups, nor should code be shared. You are encouraged to discuss with your peers, the TAs or the instructor ideas, approaches and techniques broadly, but not at a level of detail where specific implementation issues are described by anyone. If you have any questions on this, please ask the instructor before you act.