CS 839: Advanced Topics in Reinforcement Learning
CS 839, Fall 2026, Section 001
Department of Computer Sciences
University of Wisconsin–Madison
| Date | Topic | Assigned Reading | Assignments |
|---|---|---|---|
| Thursday, Sept 3 | Welcome and Course Overview (Slides) | Chapter 1 of "Reinforcement Learning: An Introduction" | |
| Tuesday, Sept 8 | Bandits (Slides) | Chapters 2 and 3 of "Reinforcement Learning: An Introduction". You may skim 2.5 - 2.9. | |
| Thursday, Sept 10 | Markov Decision Processes (Slides) | ||
| Tuesday, Sept 15 | Dynamic Programming (Slides) | Chapters 4 and 5 of "Reinforcement Learning: An Introduction" | |
| Thursday, Sept 17 | Monte Carlo Methods (Slides) | ||
| Tuesday, Sept 22 | Temporal Difference Learning I (Slides) | Chapter 6 of "Reinforcement Learning: An Introduction" | |
| Thursday, Sept 24 | Temporal Difference Learning II (Slides) | ||
| Tuesday, Sept 29 | Models and Planning I (Slides) | Chapter 8 of "Reinforcement Learning: An Introduction" | |
| Thursday, Oct 1 | Models and Planning II (Slides) | Final project proposal due at 11:59 PM | |
| Tuesday, Oct 6 | Function Approximation I (Slides) | Chapter 9 of "Reinforcement Learning: An Introduction" (You may skim 9.5, 9.7, 9.9, and 9.10; 9.7 will be assigned later)
Chapter 11 of "Reinforcement Learning: An Introduction" (Only up to (and including) 11.3) |
|
| Thursday, Oct 8 | Function Approximation II (Slides) | Programming assignment due at 11:59 PM | |
| Tuesday, Oct 13 | Deep RL I (Slides) | Section 9.7 and 16.5 of "Reinforcement Learning: An Introduction" | |
| Thursday, Oct 15 | Deep RL II (Slides) | ||
| Tuesday, Oct 20 | Policy Gradients I (Slides) | Chapter 13 (skip 13.6) of "Reinforcement Learning: An Introduction" | |
| Thursday, Oct 22 | Policy Gradients II (Slides) | Literature survey due at 11:59 PM | |
| Tuesday, Oct 27 | Multi-agent RL (Slides) | Multi-agent Reinforcement Learning: Foundations and Modern Approaches (Chapter 1 and 5) | |
| Thursday, Oct 29 | Midterm Exam in Class | ||
| Tuesday, Nov 3 | Abstraction and Hierarchy I (Slides) | Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning | |
| Thursday, Nov 5 | Abstraction and Hierarchy II (Slides) | ||
| Tuesday, Nov 10 | Learning from Humans: RLHF, Inverse RL, Imitation Learning (Slides) | Deep Reinforcement Learning from Human Preferences | |
| Thursday, Nov 12 | RL + LLMs (Slides) | ||
| Tuesday, Nov 17 | Reproducibility and Evaluation (Slides) | Empirical Design in Reinforcement Learning Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (Read all of Sections 1, 2, 4, 6, and 7. Skim Section 5) |
|
| Thursday, Nov 19 | Offline RL (Slides) | ||
| Tuesday, Nov 24 | No class (Thanksgiving week) | ||
| Thursday, Nov 26 | Happy Thanksgiving! (No class) | ||
| Tuesday, Dec 1 | Applications (Slides) | Challenges of Real-World Reinforcement Learning | |
| Thursday, Dec 3 | World Models | ||
| Tuesday, Dec 8 | Project Lightning Talks | ||
| FINAL PROJECT REPORTS DUE: December 11 at 11:59 PM | |||
| Everything below here is tentative and subject to change. | |||