CS 760 - Machine Learning
General Course Information
6393 CS Building
Office Hr: Monday 1:30-2:30pm or by appointment
- Teaching Assistant:
5387 CS Building
Office Hr: Wednesday 10-10:50am, Thursday 10-11am, or by appointment (send email)
- Prerequisite: CS 540 or equivalent
- Course Textbook:
Machine Learning by Tom Mitchell, McGraw-Hill
(includes errata). You might want to check AbeBooks or Amazon for used copies.
We will also read parts of
Sutton and Barto's on-line RL textbook (feel free to buy the hardcopy version).
And we will read parts of a book on Markov Logic Networks and another on Semi-supervised Learning that are
downloadable for free because the University of Wisconsin has a site license with the publisher
(see Lecture 1 for the URL).
Several journal articles and conference papers will also be assigned.
- Meeting Times: MWF 11am-12:15pm
We will meet 30 times over the semester, mainly skipping Fridays
and only meeting once a week the last few weeks.
We will not meet Friday, February 19 because I will be traveling.
Until then, we will meet every MWF unless otherwise noted.
- Location: Room 1257 CS Building
Archive of Class Email - use your CS dept. login to access.
Send mail to this list.
Note that email will be sent to your "@wisc.edu" address.
- Exam: MONDAY April 26, 2010, Room 1240 CS, 11am-12:30pm
One 8.5x11-inch sheet of notes and non-programmable calculators allowed.
Course Overview and Requirements
The intent of this course is to present a broad introduction to machine
learning, including discussions of each of the major approaches currently
Class lectures will discuss general issues in machine learning, as well as
present established algorithms. One secondary goal is to compare and contrast
the various approaches, determining under which conditions each is most
appropriate. Another is to relate these algorithms to human learning
The work in the course will consist of four homework assignments (about
one every two weeks during the first part of the class),
one exam (very late in the semester), and a course project.
Your solutions will be partially automatically graded, so they
must be written to run in Java on the instructional computers.
The homework assignments will generally be programming assignments that
involve experimenting with machine learning algorithms and experimental methodology.
Lab reports insightfully analyzing your experimental results will be required.
Occasionally there may also be "paper-and-pencil" homework problems.
The final project can be a more ambitious experiment or enhancement involving
algorithms used in the homeworks or the implementation of an additional algorithm.
In either case, the implementation should be accompanied by a short paper
(7-10 pages with a large font) describing the project. Projects not involving
programming are also possible.
Homeworks will count for 30% of the grade, the "midterm" exam
for 40%, the project 25%, and quality class participation 5%.
The course will be graded on the conventional (A-F) system.
Powerpoint files containing each lecture
will be available as the course progresses.
(Access is limited to University of Wisconsin sites.)
- Assigned April 28, 2010:
- Read Chapter 10 of Mitchell's textbook.
Read Chapters 1 and 2, plus Sections 4.1 and 4.2 of
Markov Logic: An Interface Layer for Articial Intelligence, by Domingos and Lowd,
in the Morgan-Claypool Series
Markov Logic Networks, Richardson & Domingos, MLj:62, 2006.
- Assigned April 7, 2010:
- Read Chapter 7 of Mitchell's textbook and Chapter 2 of
Zhu and Goldberg's book on semi-supervised learning
in the Morgan-Claypool Series.
- Assigned March 17, 2010:
- Read Chapter 13 of Mitchell's textbook.
Read Preface, Chapter 1 (just skim Section 1.2), and Sections 6.4 and 6.5 of
Sutton and Barto's on-line RL textbook.
- Assigned March 8, 2010:
- Read Chapter 4 of Mitchell's textbook and
the SVM tutorial
from ICML-2001 given by Nello Cristianini.
(You can find other SVM tutorials
the one by Burgess is a good one to read if you wish additional details.)
- Assigned February 24, 2010:
- Pages 97-105 of
'Machine Learning Research: Four Current Directions'
by T. Dietterich in the AI Magazine.
Random Forests, L. Breiman,
Machine Learning, 45, pp. 5-32, 2001
(won't be on the exam, beyond what is covered in the lecture notes)
Model Selection: Beyond the Bayesian/Frequentist Divide ,
I. Guyon, A. Saffari, G. Dror, and G. Cawley, JMLR 11, pp. 61-87, 2010.
- Assigned February 15, 2010:
- Read Chapter 3 of Mitchell's textbook
- Assigned February 10, 2010:
- Read and
'Building the Case Against Accuracy Estimation for
Comparing Induction Algorithms' by F. Provost,
T. Fawcett, and R. Kohavi, Proc. ICML-98, Madison, pp. 445-453.
Read Sections 6.7-6.11 (skim 6.11.5) and 6.13 of Mitchell's textbook.
- Assigned January 27, 2010:
- Read Sections 2.1-2.3 and Chapter 5 of Mitchell's textbook.
- Assigned January 20, 2010:
- Read Preface, Chapter 1, Sections 6.1-6.2,
and Sections 8.1-8.2 of Mitchell's textbook. Also read
the on-line draft chapter Mitchell created on Naive Bayes
and Logistic Regression
Homework 4: Reinforcement Learning
Due Monday, 4/19/09
Homework 3: Training Perceptrons and Using Kernels
Due Wednesday, 4/7/10
Testbeds used during grading
Homework 2: Decision Trees and Random Forests
Due Monday, 3/15/10
Testbeds used during grading
Homework 1: Experimental Methodology, Feature Selection, k-NN, and Naive Bayes
Due Wednesday, 2/17/10
Testbeds used during grading
Homework 0: Creating Your Personal Dataset
Due Monday, 2/1/10
- Late policy on HWs:
- HWs are due at 4 pm. If you don't turn
them in at the end of class, then put them in the TA's mailbox
(5th floor of CS).
- Each student will have FIVE "free" late days for use
over the semester. Once these are exhausted, there will be
a penalty of 10 points per day (measured 4pm-to-4pm; weekends
and university holidays are free).
"Days" are calendar days, not class days.
- To make the TA's job tractable, no HWs will be accepted more than
one week late.
- Academic Misconduct
All examinations, programming assignments, and written homeworks must
be done individually. Cheating and
plagiarism will be dealt with in accordance with University
procedures (see the
Academic Misconduct Guide for Students).
Hence, for example, code for programming assignments must not
be developed in groups, nor should code be shared. You are
encouraged to discuss with your peers, the TAs or
the instructor ideas, approaches and techniques broadly, but not at a level
of detail where specific implementation issues are described by anyone.
If you have any questions on this, please ask the instructor before you act.
Postscript until 1999 (converted from postscript to pdf using unix' ps2pdf); MS Word after.
Some ML-Related Links
Related Local Links
This page was created by
Computer Sciences Department
College of Letters and Science
University of Wisconsin - Madison
5355a Computer Sciences and Statistics ~ 1210 West Dayton Street, Madison,
email@example.com ~ voice: 608-262-1204 ~