Richard Maclin (1995).
Learning from Instruction and Experience: Methods for Incorporating Procedural Domain Theories into Knowledge-Based Neural Networks. PhD thesis, Department of Computer Sciences, University of Wisconsin-Madison.
(Also appears as UW Technical Report CS-TR-95-1285)


This publication is contained in the following 2 postscript files
File 1, File 2
and contained in the following 2 PDF files
File 1, File 2.

Abstract:

This thesis defines and evaluates two systems that allow a teacher to provide instructions to a machine learner. My systems, FSKBANN and RATLE, expand the language that a teacher may use to provide advice to the learner. In particular, my techniques allow a teacher to give partially correct instructions about procedural tasks - tasks that are solved as sequences of steps. FSKBANN and RATLE allow a computer to learn both from instruction and from experience.
Experiments with these systems on several testbeds demonstrate that they produce learners that successfully use and refine the instructions they are given.
In my initial approach, FSKBANN, the teacher provides instructions as a set of propositional rules organized around one or more finite-state automata (FSAs). FSKBANN maps the knowledge in the rules and FSAs into a recurrent neural network. I used FSKBANN to refine the Chou-Fasman algorithm, a method for solving the secondary-structure prediction problem, a difficult task in molecular biology. FSKBANN produces a refined algorithm that outperforms the original (non-learning) Chou-Fasman algorithm, as well as a standard neural-network approach.
My second system, RATLE, allows a teacher to communicate advice, using statements in a simple programming language, to a connectionist, reinforcement-learning agent. The teacher indicates conditions of the environment and actions the agent should take under those conditions. RATLE allows the teacher to give advice continuously by translating the teacher's statements into additions to the agent's neural network. The RATLE language also includes novel (to the theory-refinement literature) features such as multi-step plans and looping constructs. In experiments with RATLE on two simulated testbeds involving multiple agents, I demonstrate that a RATLE agent receiving advice outperforms both an agent that does not receive advice and an agent that receives instruction, but does not refine it.
My methods provide an appealing approach for learning from both instruction and experience in procedural tasks. This work widens the ``information pipeline'' between humans and machine learners, without requiring that the human provide absolutely correct information to the learner.

Return to the publications of the Univ. of Wisconsin Machine Learning Research Group.