# Lecture Note

Lecture 13: Slides
Lecture 14: Review
Annotated Lecture 13 Section 1: Slides
Annotated Lecture 14 Section 1: Slides, Piazza post.
Annotated Week 7 Section 2: Part I, Part II

How to compute value function given policy? Link
How to compute optimal value function? Link

Random Integer: Link
Multi Armed Bandit (don't play this): Link
Autonomous driving: Link
Stochastic growth model: Link
Q Learning: Link
Deep reinforcement learning: Link
Learning games: Link

# Written (Math) Problems

No math homework.

Stat

# Programming Problem

Any machine learning project of your choice. Two possibilities:
(1) Homework 1 to Homework 6 algorithms applied to a new data set.
(2) Algorithms covered in Lecture 1 to Lecture 13 but not implemented in the Homework applied to a Homework data set or a new data set.
Grading: 8 points (4 points goes to P7 and 4 points goes to P12), 4 for submission, 2 for something new, 2 for something interesting. You can only submit to one of P7 or P12, not both. See P12.
Deadline: August 18.

(1) report.pdf contains your question and your results, similar to the hint files for P1 to P6.
(2) comments.txt contains information on how to run your program, in particular, the names of the data files are required.
(3) Submit code but NOT data.

More details: see Piazza Post Link
Note: the original plan is a programming homework on reinforcement learning: solve the simple example of Stochastic Growth Model with log utility and square root production function. See Optimal Growth for the details about the model. It requires too much economics background, but if you are interested, you can implement the model for 8 points.