R. Maclin & J. Shavlik (1994).
Incorporating Advice into Agents that Learn from Reinforcements. Department of Computer Sciences, University of Wisconsin, UW TR 1227.
(A shorter version appears in AAAI-94.)

This publication is available in PDF and available in postscript.

Abstract:

Learning from reinforcements is a promising approach for creating intelligent agents. However, reinforcement learning usually requires a large number of training episodes. We present a system called RATLE that addresses this shortcoming by allowing a connectionist Q-learner to accept advice given, at any time and in a natural manner, by an external observer. In RATLE, the advice-giver watches the learner and occasionally makes suggestions, expressed as instructions in a simple programming language. Based on techniques from knowledge-based neural networks, RATLE inserts these programs directly into the agent's utility function. Subsequent reinforcement learning further integrates and refines the advice. We present empirical evidence that shows our approach leads to statistically-significant gains in expected reward. Importantly, the advice improves the expected reward regardless of the stage of training at which it is given.

Return to the publications of the Univ. of Wisconsin Machine Learning Research Group.