The purpose of this thesis is to develop heuristics for employing Trepan, an algorithm for extracting decision trees from neural networks. Typically, several parameters need to be chosen to obtain a satisfactory performance of the algorithm. The current understanding of the various interactions between these is not well understood. By empirically evaluating the performance of the algorithm on a test set of databases chosen from benchmark machine learning and real world problems, several heuristics are proposed to explain and improve the performance of the algorithm. The experimentation is further validated by performance statistic measures. The algorithm is extended to work for multi-class regression problems and its ability to comprehend generalized feedforward networks is investigated. This work thus serves to provide improvements, an increased understanding of the behavior of the algorithm and heuristics to choose parameters for a better performance.