M. Craven & J. Shavlik (1996).
Extracting Tree-Structured Representations of Trained Networks. Advances in Neural Information Processing Systems, pp. 24-30, Denver, CO. MIT Press.
Code.

This publication is available in PDF and available in postscript.

The software associated with this publication is available online.

Abstract:

A significant limitation of neural networks is that the representations they learn are usually incomprehensible to humans. We present a novel algorithm, TREPAN, for extracting comprehensible, symbolic representations from trained neural networks. Our algorithm uses queries to induce a decision tree that approximates the concept represented by a given network. Our experiments demonstrate that TREPAN is able to produce decision trees that maintain a high level of fidelity to their respective networks, while being comprehensible and accurate. Unlike previous work in this area, our algorithm is both general in its applicability and scales well to large networks and problems with high-dimensional input spaces.

Return to the publications of the Univ. of Wisconsin Machine Learning Research Group.