CS 839 Advanced Topics in Reinforcement Learning
CS 839, Fall 2022
Department of Computer Sciences
University of Wisconsin–Madison
| Date | Topic | Assigned Reading | Assignments |
|---|---|---|---|
| Thursday, Sept 8 | Welcome and Course Overview (Slides) | Chapter 1 of "Reinforcement Learning: An Introduction" | |
| Tuesday, Sept 13 | Bandits (Slides) | Chapters 2 and 3 of "Reinforcement Learning: An Introduction". You may skim 2.5 - 2.9. | HW 1 Released |
| Thursday, Sept 15 | Markov Decision Processes (Slides) | ||
| Tuesday, Sept 20 | Dynamic Programming I (Slides) | Chapter 4 of "Reinforcement Learning: An Introduction" | |
| Thursday, Sept 22 | Dynamic Programming II (Slides) | ||
| Tuesday, Sept 27 | Monte Carlo Methods I (Slides) | Chapter 5 of "Reinforcement Learning: An Introduction" | |
| Thursday, Sept 29 | Monte Carlo Methods II (Slides) | HW 1 due by 9:29AM; HW 2 Released | |
| Tuesday, Oct 4 | Temporal Difference Learning I (Slides) | Chapter 6 of "Reinforcement Learning: An Introduction" | |
| Thursday, Oct 6 | Temporal Difference Learning II (Slides) | ||
| FINAL PROJECT PROPOSAL DUE: Thursday at Midnight Central Time | |||
| Tuesday, Oct 11 | Models and Planning I (Slides) | Chapter 8 of "Reinforcement Learning: An Introduction" | |
| Thursday, Oct 13 | Models and Planning II (Slides) | HW 2 due at 9:29AM; HW 3 Released | |
| Tuesday, Oct 18 | Function Approximation I (Slides) | Chapter 9 of "Reinforcement Learning: An Introduction" (You may skim 9.5, 9.7, 9.9, and 9.10 -- 9.7 will be assigned later) | |
| Thursday, Oct 20 | Function Approximation II (Slides) | ||
| Tuesday, Oct 25 | Function Approximation III (Slides) | Chapter 11 of "Reinforcement Learning: An Introduction" | |
| Thursday, Oct 27 | Function Approximation IV (Slides) | HW 3 due at 9:29AM; HW 4 Released | |
| Tuesday, Nov 1 | Deep RL I (Slides) | Section 9.7 and 16.5 of "Reinforcement Learning: An Introduction" | |
| LITERATURE SURVEY DUE: Thursday @ 11:59pm Central | |||
| Thursday, Nov 3 | Deep RL II (Slides) | ||
| Tuesday, Nov 8 | Policy Gradients and Average Reward I (Slides) | Chapter 13 and skim Chapter 10 of "Reinforcement Learning: An Introduction" | |
| Thursday, Nov 10 (Slides) | Policy Gradients and Average Reward I | ||
| Tuesday, Nov 15 | Abstraction and Hierarchy I (Slides) | Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning | |
| Thursday, Nov 17 | Abstraction and Hierarchy II (Slides) | HW 4 due at 9:29AM | |
| Tuesday, Nov 22 | Reproducibility and Evaluation (Slides) | Deep Reinforcement Learning that Matters | |
| Thursday, Nov 24 | Happy Thanksgiving! (No class) | ||
| Tuesday, Nov 29 | Multi-agent RL I (Slides) | Markov games as a framework for multi-agent reinforcement learning | |
| Thursday, Dec 1 | Multi-agent RL II (Slides) | ||
| Tuesday, Dec 6 | Offline RL I (Slides) | Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (Read all of Sections 1, 2, 4, 6, and 7. Skim Section 5) | |
| Thursday, Dec 8 | Offline RL II (Slides) | ||
| Tuesday, Dec 13 | Applications (Slides) | Challenges of Real-World Reinforcement Learning | |
| FINAL PROJECT REPORTS DUE: December 14 @ 11:59pm Central | |||
| Everything below here is tentative and subject to change. | |||
| Please help fill in the course evaluation (Dec 1 - Dec 15). | |||