April, 2004: My recent paper When Do Differences Matter? On-Line Feature Extraction Through Cognitive Economy describes how cognitive economy may be worked out in a reinforcement learning system, and presents experimental studies based on the puck-on-a-hill task. Contributions of this paper include a description of how the idea of cognitive economy addresses the open problem of knowing how accurately we need to learn the action values, and a new methodology for testing and camparing representations by means of learning curves.

May 26, 2002: Until I rewrite this page, my dissertation gives the best summary of my recent work. The abstract page summarizes the dissertation and gives links to downloadable versions in PDF and compressed PostScript formats.

Some Reinforcement Learning Resources

My publications

Short subjects:

• What is reinforcement learning, and why is it hard?
• What is cognitive economy?

Simulation code for several control problems:

• Pole-cart problem, driver module---Just the driver; supply your own controller
• Sample Q-learning controller module---Suitable for use with the pole-cart driver module. (Currently, doesn't use probabilistic action selection).
• Barto-Sutton-Anderson pole-cart solution
• Chuck Anderson's public domain code for neural networks and reinforcement learning
• Suggestions on additional links?

Other RL resources:

• Jürgen Schmidhuber's pages on Reinforcement Learning and POMDPs and RL economies
• Proposed Standard for Reinforcement Learning Software, by Rich Sutton and Juan Carlos Santamaria
• NeuroProse Archive (Ohio State University)
• GMD Reinforcement Learning Archive
• Rich Sutton's home page and RL archive
• IDIAP Neural Network Home Page, including links to conferences