Foundation Models & the Future of Machine Learning

CS 839, Fall 2023
Department of Computer Sciences
University of Wisconsin–Madison


Important Notes

This schedule is tentative and subject to change. Please check back often. In particular, the deadlines for the homework sets/project may change; please see Canvas for the actual deadlines.

Tentative Schedule

Date Lecture Readings Homework Released Homework Due
Thursday Sept. 7 Introduction and Course Overview
Tuesday Sept. 12 Machine Learning Mini-Review
Thursday Sept. 14 Transformers & Attention
Tuesday Sept. 19 Models (Encoder-Only, Encoder-Decoder, Decoder-Only) I
Thursday Sept. 21 Models (Encoder-Only, Encoder-Decoder, Decoder-Only) II
Tuesday Sept. 26 Prompting I HW 1 Released
Thursday Sept. 28 Prompting II
Tuesday Oct. 3 Reasoning & Chain-of-Thought
Thursday Oct. 5 In-Context Learning: Practice and Theory
Tuesday Oct. 10 Fine-Tuning, Specialization, Adaptation
Thursday Oct. 12 Training Homework 1 Due
Tuesday Oct. 17 Reinforcement Learning from Human Feedback
Thursday Oct. 19 Data
Tuesday Oct. 24 Multimodal and Specialized Foundation Models (CodeLMs and more)
Thursday Oct. 26 Knowledge
Tuesday Oct. 31 Scaling & Scaling Laws
Thursday Nov. 2 Project & Presentation Details Due: 1/2 page info on presentations
Tuesday Nov. 7 Security, Privacy, Toxicity + Future Areas
To Dec. 13 Student Presentations