M. Craven & J. Shavlik (1995).
Extracting Comprehensible Concept Representations from Trained Neural Networks. Presented at the IJCAI Workshop on Comprehensibility in Machine Learning, Montreal, Quebec, Canada.

This publication is available in PDF and available in postscript.

Abstract:

Although they are applicable to a wide array of problems, and have demonstrated good performance on a number of difficult, real-world tasks, neural networks are not usually applied to problems in which comprehensibility of the acquired concepts is important. The concept representations formed by neural networks are hard to understand because they typically involve distributed, nonlinear relationships encoded by a large number of real-valued parameters. To address this limitation, we have been developing algorithms for extracting ``symbolic'' concept representations from trained neural networks. We first discuss why it is important to be able to understand the concept representations formed by neural networks. We then briefly describe our approach and discuss a number of issues pertaining to comprehensibility that have arisen in our work. Finally, we discuss choices that we have made in our research to date, and open research issues that we have not yet addressed.

Return to the publications of the Univ. of Wisconsin Machine Learning Research Group.