CS 839: Advanced Topics in Reinforcement Learning
CS 839, Fall 2025, Section 004
Department of Computer Sciences
University of Wisconsin–Madison
| Date | Topic | Assigned Reading | Assignments |
|---|---|---|---|
| Thursday, Sept 4 | Welcome and Course Overview (Slides) | Chapter 1 of "Reinforcement Learning: An Introduction" | |
| Tuesday, Sept 9 | Bandits (Slides) | Chapters 2 and 3 of "Reinforcement Learning: An Introduction". You may skim 2.5 - 2.9. | |
| Thursday, Sept 11 | Markov Decision Processes (Slides) | ||
| Tuesday, Sept 16 | Dynamic Programming I (Slides) | Chapter 4 of "Reinforcement Learning: An Introduction" | |
| Thursday, Sept 18 | Dynamic Programming II (Slides) | ||
| Tuesday, Sept 23 | Monte Carlo Methods I (Slides) | Chapter 5 of "Reinforcement Learning: An Introduction" | |
| Thursday, Sept 25 | Monte Carlo Methods II (Slides) | ||
| Tuesday, Sept 30 | Temporal Difference Learning I (Slides) | Chapter 6 of "Reinforcement Learning: An Introduction" | |
| Thursday, Oct 2 | Temporal Difference Learning II (Slides) | Final project proposal due | |
| Tuesday, Oct 7 | Models and Planning I (Slides) | Chapter 8 of "Reinforcement Learning: An Introduction" | |
| Thursday, Oct 9 | Models and Planning II (Slides) | ||
| Tuesday, Oct 14 | Function Approximation I (Slides) |
Chapter 9 of "Reinforcement Learning: An Introduction" (You may skim 9.5, 9.7, 9.9, and 9.10; 9.7 will be assigned later)
Chapter 11 of "Reinforcement Learning: An Introduction" (Only up to (and including) 11.3) |
|
| Thursday, Oct 16 | Function Approximation II (Slides) | Programming assignment due | |
| Tuesday, Oct 21 | Deep RL I (Slides) | Section 9.7 and 16.5 of "Reinforcement Learning: An Introduction" | |
| Thursday, Oct 23 | Deep RL II (Slides) | ||
| Everything below here is tentative and subject to change. | |||
| Tuesday, Oct 28 | Policy Gradients I (Slides) | Chapter 13 of "Reinforcement Learning: An Introduction" | |
| Thursday, Oct 30 | Policy Gradients II (Slides) | Literature survey due | |
| Tuesday, Nov 4 | Abstraction and Hierarchy I (Slides) | Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning | |
| Tentative Midterm Exam: November 5 | |||
| Thursday, Nov 6 | Abstraction and Hierarchy II (Slides) | ||
| Tuesday, Nov 11 | Multi-agent RL I (Slides) | Multi-agent Reinforcement Learning: Foundations and Modern Approaches (Chapter 1 and 5) | |
| Thursday, Nov 13 | Multi-agent RL II (Slides) | ||
| Tuesday, Nov 18 | Learning from Humans: RLHF, Inverse RL, Imitation Learning | ||
| Thursday, Nov 20 | Learning Cognition: Learning to Reason and Plan | ||
| Tuesday, Nov 25 | Reproducibility and Evaluation (Slides) | Empirical Design in Reinforcement Learning | |
| Thursday, Nov 27 | Happy Thanksgiving! (No class) | ||
| Tuesday, Dec 2 | Offline RL (Slides) | Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (Read all of Sections 1, 2, 4, 6, and 7. Skim Section 5) Challenges of Real-World Reinforcement Learning |
|
| Thursday, Dec 4 | Applications (Slides) | ||
| Tuesday, Dec 9 | Project Lightning Talks | ||
| FINAL PROJECT REPORTS DUE: December 12 | |||