Geoffrey G. Towell (1991).
Symbolic Knowledge and Neural Networks: Insertion, Refinement and Extraction. PhD thesis, Department of Computer Sciences, University of Wisconsin-Madison.
(Also appears as UW Technical Report 1072)
Data.

This publication is contained in the following 4 postscript files
File 1, File 2, File 3, File 4
and contained in the following 4 PDF files
File 1, File 2, File 3, File 4.

The data associated with this publication is available online.

Abstract:

Explanation-based and empirical learning are two largely complementary methods of machine learning. These approaches to machine learning both have serious problems which preclude their being a general purpose learning method. However, a ``hybrid'' learning method that combines explanation-based with empirical learning may be able to use the strengths of one learning method to address the weaknesses of the other method. Hence, a system that effectively combines the two approaches to learning can be expected to be superior to either approach in isolation. This thesis describes a hybrid system called KBANN which is shown to be an effective combination of these two learning methods.
KBANN (Knowledge-Based Artificial Neural Networks) is a three-part hybrid learning system built on top of ``neural'' learning techniques. The first part uses a set of approximately-correct rules to determine the structure and initial link weights of an artificial neural network, thereby making the rules accessible for modification by neural learning. The second part of KBANN modifies the resulting network using essentially standard neural learning techniques. The third part of KBANN extracts refined rules from trained networks.
KBANN is evaluated by empirical tests in the domain of molecular biology. Networks created by KBANN are shown to be superior, in terms of their ability to correctly classify unseen examples, to a wide variety of learning systems as well as techniques proposed by experts in the problems investigated. In addition, empirical tests show that KBANN is robust to errors in the initial rules and insensitive to problems resulting from the presence of extraneous input features.
The third part of KBANN, which extracts rules from trained networks, addresses a significant problem in the use of neural networks -- understanding what a neural network learns. Empirical tests of the proposed rule-extraction method show that it simplifies understanding of trained networks by reducing the number of: consequents (hidden units), antecedents (weighted links), and possible antecedent weights. Surprisingly, the extracted rules are often more accurate at classifying examples not seen during training than the trained network from which they came.

Return to the publications of the Univ. of Wisconsin Machine Learning Research Group.