CS760 Machine Learning
Fall 2019


This course provides an introduction to the theory and practical methods for machine learning, and is designed to give a graduate-level student a thorough grounding in the methodologies, mathematics and algorithms of machine learning. Topics covered include nearest neighbor method, decision tree learning, Support Vector Machines, Bayesian networks, neural networks and deep learning, unsupervised learning and reinforcement learning. The course covers theoretical concepts such as inductive bias, the PAC learning framework, Bayesian learning methods, mistake bounds, etc. Lectures TuTh 11:00AM - 12:15PM in EDUCATION L196, see calendar below Professor: Jerry Zhu, jerryzhu@cs.wisc.edu, Office hour Fridays 2:30-3:30pm, CS 6391 Teaching Assistants: Swati Anand, sanand24@wisc.edu Office hour Thursdays 4pm, CS 4205 Ashwin Tayade, tayade@wisc.edu Office hour Tuesdays at 4pm, CS 4296 Graders: Arpit Jain, ajain74@wisc.edu Sushma Kudlur Nirvanappa, kudlurnirvan@wisc.edu Prerequisites Students entering the class are expected to have a background knowledge of probability, linear algebra, and calculus, and have good programming experience. Textbooks (optional) Pattern Recognition and Machine Learning, Chris Bishop. Machine Learning, Tom Mitchell. Understanding Machine Learning: From Theory to Algorithms, Shalev-Shwartz, Ben-David. Grading: Homeworks (60%), exams (40%) Discussion forum: The instructors and TAs will post announcements, clarifications, hints, etc. on Piazza. Hence you must check the CS760 Piazza page frequently throughout the term. If you have a question, your best option is to post a message on Piazza. The staff (instructors and TAs) will check the forum regularly, and if you use the forum, other students will be able to help you too. When using the forum, please do not post answers to homework questions before the homework is due. If your question is personal or not of interest to other students, you may mark your question as private on Piazza, so only the instructors will see it. If you wish to talk with one of us individually, you are welcome to come to our office hours. Please reserve email for the questions you can't get answered in office hours or through the forum. CS760 Piazza Homeworks Please submit hw pdf via UW-Madison's Canvas system. Homework assignments will include written problems and sometimes programming. Accounts will be provided on the Computer Sciences Department's instructional Unix workstations located in rooms 1350, 1351, and 1370. Frequently-asked questions (FAQs) on homework assignments will be e-mailed to the class mailing list. Homework is always due the minute before class starts on the due date. Late submissions will not be accepted. Assignment grading questions must be raised with the TAs within 72 hours after it is returned. Regrading request for a part of a homework question may trigger the grader to regrade the entire homework and could potentially take points off. Regrading will be done on the original submitted work, no changes allowed. We will drop one lowest homework score from your final homework average calculation. The drop is meant for emergency. We do not provide additional drops, late days, or homework extensions. We encourage you to use a study group for doing your homework. Students are expected to help each other out, and if desired, form ad-hoc homework groups. Exams: There will be 2 exams. Topics: everything up to the time of the exam, including slides, notes, selected readings (but not whole books) Makeup exams will not be scheduled. Please plan for exams at these times and let us know about any exam conflicts during the first two weeks of the semester. If an emergency arises that conflicts with the exam times, email us as soon as possible. Emergency exam conflicts will be handled on a case-by-case basis. Exam conflicts originating from a lecture conflict will be accommodated. Exam grading questions must be raised with the instructor within 72 hours after it is returned. If a regrade request is submitted for a part of a question on the exam, the grader reserves the right to regrade the entire exam and could potentially take points off. All exams are closed book. Bring a calculator and copious amount of blank scratch paper. For midterm, one 8.5x11 sheet of paper with notes on both sides allowed (handwritten or typed). For final exam, two such sheets are allow (e.g. you can re-use the midterm sheet). Lectures and readings on the syllabus page are required, with a few exceptions (to be posted before the exam). You are responsible for topics covered in lecture even if there are no lecture notes on the topic. Academic Integrity: You are encouraged to discuss with your peers, the TA or the instructors ideas, approaches and techniques broadly. However, all examinations, programming assignments, and written homeworks must be written up individually. For example, code for programming assignments must not be developed in groups, nor should code be shared. Make sure you work through all problems yourself, and that your final write-up is your own. If you feel your peer discussions are too deep for comfort, declare it in the homework solution: "I discussed with X,Y,Z the following specific ideas: A, B, C; therefore our solutions may have similarities on D, E, F..." You may use books or legit online resources to help solve homework problems, but you must always credit all such sources in your writeup and you must never copy material verbatim. Cheating and plagiarism will be dealt with in accordance with University procedures (see the UW-Madison Academic Misconduct Rules and Procedures). Disability Information The University of Wisconsin-Madison supports the right of all enrolled students to a full and equal educational opportunity. The Americans with Disabilities Act (ADA), Wisconsin State Statute (36.12), and UW-Madison policy (Faculty Document 1071) require that students with disabilities be reasonably accommodated in instruction and campus life. Reasonable accommodations for students with disabilities is a shared faculty and student responsibility. Students are expected to inform Professor Zhu of their need for instructional accommodations by the end of the third week of the semester, or as soon as possible after a disability has been incurred or recognized. Professor Zhu will work either directly with the student or in coordination with the McBurney Center to identify and provide reasonable instructional accommodations. Disability information, including instructional accommodations as part of a student's educational record, is confidential and protected under FERPA. Topics and slides overview decision trees instance based learning [xing2002 | Suarez2018] evaluation naive Bayes linear and logistic regression neural networks and deep learning parts: backpropagation, regularization, CNN, RNN, autoencoder and GAN [Deep Learning, Goodfellow et al. 2016] [Andrew Ng NIPS16 DL tutorial] learning theory [notes | additional reading: ch 1, 2; optionally 3,4,5,6] reinforcement learning probabilistic graphical models [tutorial] support vector machines [notes | dual via Lagrange multipliers, Ch 5] random forest and feature selection dimension reduction (PCA) [notes] active learning and semi-supervised learning (download from a UW IP address) adversarial attacks and defenses [Papernot'16 | Wiyatno'19 | Zhu'18]