CS760 Machine Learning
Fall 2019
This course provides an introduction to the theory and practical methods for machine learning,
and is designed to give a graduate-level student a thorough grounding in the methodologies,
mathematics and algorithms of machine learning. Topics covered include nearest neighbor method,
decision tree learning, Support Vector Machines, Bayesian networks, neural networks and deep
learning, unsupervised learning and reinforcement learning. The course covers theoretical concepts
such as inductive bias, the PAC learning framework, Bayesian learning methods, mistake bounds, etc.
Lectures TuTh 11:00AM - 12:15PM in EDUCATION L196, see calendar below
Professor:
Jerry Zhu, jerryzhu@cs.wisc.edu, Office hour Fridays 2:30-3:30pm, CS 6391
Teaching Assistants:
Swati Anand, sanand24@wisc.edu Office hour Thursdays 4pm, CS 4205
Ashwin Tayade, tayade@wisc.edu Office hour Tuesdays at 4pm, CS 4296
Graders:
Arpit Jain, ajain74@wisc.edu
Sushma Kudlur Nirvanappa, kudlurnirvan@wisc.edu
Prerequisites
Students entering the class are expected to have a background knowledge of probability,
linear algebra, and calculus, and have good programming experience.
Textbooks (optional)
Pattern Recognition and Machine Learning, Chris Bishop.
Machine Learning, Tom Mitchell.
Understanding Machine Learning: From Theory to Algorithms, Shalev-Shwartz, Ben-David.
Grading: Homeworks (60%), exams (40%)
Discussion forum:
The instructors and TAs will post announcements, clarifications, hints, etc. on Piazza. Hence you must
check the CS760 Piazza page frequently throughout the term. If you have a question, your best option
is to post a message on Piazza. The staff (instructors and TAs) will check the forum regularly, and if
you use the forum, other students will be able to help you too. When using the forum, please do not post
answers to homework questions before the homework is due.
If your question is personal or not of interest to other students, you may mark your question as private
on Piazza, so only the instructors will see it. If you wish to talk with one of us individually, you are
welcome to come to our office hours. Please reserve email for the questions you can't get answered in
office hours or through the forum.
CS760 Piazza
Homeworks
Please submit hw pdf via UW-Madison's Canvas system.
Homework assignments will include written problems and sometimes programming.
Accounts will be provided on the Computer Sciences Department's instructional
Unix workstations located in rooms 1350, 1351, and 1370. Frequently-asked
questions (FAQs) on homework assignments will be e-mailed to the class mailing
list. Homework is always due the minute before class starts on the due date.
Late submissions will not be accepted. Assignment grading questions must be
raised with the TAs within 72 hours after it is returned. Regrading request
for a part of a homework question may trigger the grader to regrade the entire
homework and could potentially take points off. Regrading will be done on the
original submitted work, no changes allowed.
We will drop one lowest homework score from your final homework average calculation. The drop is
meant for emergency. We do not provide additional drops, late days, or homework extensions.
We encourage you to use a study group for doing your homework. Students are expected to help
each other out, and if desired, form ad-hoc homework groups.
Exams:
There will be 2 exams. Topics: everything up to the time of the exam, including slides, notes, selected
readings (but not whole books) Makeup exams will not be scheduled. Please plan for
exams at these times and let us know about any exam conflicts during the first two weeks of the semester.
If an emergency arises that conflicts with the exam times, email us as soon as possible. Emergency exam
conflicts will be handled on a case-by-case basis. Exam conflicts originating from a lecture conflict
will be accommodated. Exam grading questions must be raised with the instructor within 72 hours after
it is returned. If a regrade request is submitted for a part of a question on the exam, the grader reserves
the right to regrade the entire exam and could potentially take points off.
- Exam 1: Wed Oct. 23 5:30-7pm, Noland 132.
- Exam 2: Mon Dec. 16, 2019 7:25-9:25pm, PSYCHOLOGY 105
All exams are closed book. Bring a calculator and copious amount of blank scratch paper.
For midterm, one 8.5x11 sheet of paper with notes on both sides allowed (handwritten or typed).
For final exam, two such sheets are allow (e.g. you can re-use the midterm sheet). Lectures
and readings on the syllabus page are required, with a few exceptions (to be posted before
the exam). You are responsible for topics covered in lecture even if there are no lecture
notes on the topic.
Academic Integrity:
You are encouraged to discuss with your peers, the TA or the instructors ideas, approaches
and techniques broadly. However, all examinations, programming assignments, and written
homeworks must be written up individually. For example, code for programming assignments
must not be developed in groups, nor should code be shared. Make sure you work through all
problems yourself, and that your final write-up is your own. If you feel your peer discussions
are too deep for comfort, declare it in the homework solution: "I discussed with X,Y,Z the following
specific ideas: A, B, C; therefore our solutions may have similarities on D, E, F..."
You may use books or legit online resources to help solve homework problems, but you must always credit
all such sources in your writeup and you must never copy material verbatim.
Cheating and plagiarism will be dealt with in accordance with University procedures
(see the UW-Madison Academic Misconduct Rules and Procedures).
Disability Information
The University of Wisconsin-Madison supports the right of all enrolled students to a full and equal
educational opportunity. The Americans with Disabilities Act (ADA), Wisconsin State Statute (36.12),
and UW-Madison policy (Faculty Document 1071) require that students with disabilities be reasonably
accommodated in instruction and campus life. Reasonable accommodations for students with disabilities
is a shared faculty and student responsibility. Students are expected to inform Professor Zhu of their
need for instructional accommodations by the end of the third week of the semester, or as soon as
possible after a disability has been incurred or recognized. Professor Zhu will work either directly
with the student or in coordination with the McBurney Center to identify and provide reasonable
instructional accommodations. Disability information, including instructional accommodations as part
of a student's educational record, is confidential and protected under FERPA.
Topics and slides
overview
decision trees
instance based learning [xing2002 | Suarez2018]
evaluation
naive Bayes
linear and logistic regression
neural networks and deep learning
parts: backpropagation, regularization, CNN, RNN, autoencoder and GAN
[Deep Learning, Goodfellow et al. 2016] [Andrew Ng NIPS16 DL tutorial]
learning theory [notes | additional reading: ch 1, 2; optionally 3,4,5,6]
reinforcement learning
probabilistic graphical models [tutorial]
support vector machines [notes | dual via Lagrange multipliers, Ch 5]
random forest and feature selection
dimension reduction (PCA) [notes]
active learning and semi-supervised learning (download from a UW IP address)
adversarial attacks and defenses [Papernot'16 | Wiyatno'19 | Zhu'18]