Characterization of Symbolic Rules Embedded in Deep DIMLP Networks: A Challenge to Transparency of Deep Learning

[1] M. Golea, On the complexity of rule extraction from neural networks and network querying, in: Rule Extraction From Trained Artificial Neural Networks Workshop, Society For the Study of Artificial Intelligence and Simulation of Behavior Workshop Series (AISB), 1996, pp. 51-59Search in Google Scholar

[2] T. Hailesilassie, Rule extraction algorithm for deep neural networks: A review, International Journal of Computer Science and Information Security 14, 7, 2016, 376Search in Google Scholar

[3] G. Bologna, Symbolic rule extraction from the dimlp neural network, in: Hybrid neural systems, Springer, 2000, pp. 240-25410.1007/10719871_17Search in Google Scholar

[4] G. Bologna, A study on rule extraction from several combined neural networks, International journal of neural systems 11, 03, 2001, 247-25510.1142/S0129065701000680Search in Google Scholar

[5] G. Bologn, Is it worth generating rules from neural network ensembles?, Journal of Applied Logic 2, 3, 2004, 325-34810.1016/j.jal.2004.03.004Search in Google Scholar

[6] A. A. Freitas, Comprehensible classification models: a position paper, ACM SIGKDD explorations newsletter 15, 1, 2014, 1-1010.1145/2594473.2594475Search in Google Scholar

[7] J. Chorowski, J. M. Zurada, Learning understandable neural networks with nonnegative weight constraints, Neural Networks and Learning Systems, IEEE Transactions on 26, 1, 2015, 62-6910.1109/TNNLS.2014.2310059Search in Google Scholar

[8] S. I. Gallant, Connectionist expert systems, Communications of the ACM 31 (2) (1988) 152-169.10.1145/42372.42377Search in Google Scholar

[9] R. Andrews, J. Diederich, A. B. Tickle, Survey and critique of techniques for extracting rules from trained artificial neural networks, Knowledgebased systems 8, 6, 1995, 373-38910.1016/0950-7051(96)81920-4Search in Google Scholar

[10] J. Diederich, Rule extraction from support vector machines, Vol. 80, Springer Science & Business Media, 200810.1007/978-3-540-75390-2Search in Google Scholar

[11] L. K. Hansen, P. Salamon, Neural network ensembles, IEEE transactions on pattern analysis and machine intelligence 12, 1990, 993-100110.1109/34.58871Search in Google Scholar

[12] Z.-H. Zhou, Y. Jiang, S.-F. Chen, Extracting symbolic rules from trained neural network ensembles, Artificial Intelligence Communications 16 , 1, 2003 3-16.Search in Google Scholar

[13] R. Setiono, B. Baesens, C. Mues, Recursive neural network rule extraction for data with mixed attributes, Neural Networks, IEEE Transactions on 19 , 2, 2008, 299-30710.1109/TNN.2007.90864118269960Search in Google Scholar

[14] A. Hara, Y. Hayashi, Ensemble neural network rule extraction using re-rx algorithm, in: Neural Networks (IJCNN), The 2012 International Joint Conference on, IEEE, 2012, pp. 1-610.1109/IJCNN.2012.6252446Search in Google Scholar

[15] Y. Hayashi, R. Sato, S. Mitra, A new approach to three ensemble neural network rule extraction using recursive-rule extraction algorithm, in: Neural Networks (IJCNN), The 2013 International Joint Conference on, IEEE, 2013, pp. 1-710.1109/IJCNN.2013.6706823Search in Google Scholar

[16] S. N. Tran, A. dAvila Garcez, Knowledge extraction from deep belief networks for images, in: IJCAI-2013Workshop on Neural-Symbolic Learning and Reasoning, 2013Search in Google Scholar

[17] J. Zilke, Extracting rules from deep neural networks, Master’s thesis, Computer Science Department, Technische Universitt Darmstadt, 201510.1007/978-3-319-46307-0_29Search in Google Scholar

[18] R. Setiono, W. K. Leow, Fernn: An algorithm for fast extraction of rules from neural networks, Applied Intelligence 12 , 1-2, 2000, 15-2510.1023/A:1008307919726Search in Google Scholar

[19] J. R. Quinlan, C4.5: Programs for machine learning. morgan kaufmann publishers, inc., 1993, Machine Learning 16, 3, 1994, 235-24010.1007/BF00993309Search in Google Scholar

[20] G. Bologna, C. Pellegrini, Constraining the mlp power of expression to facilitate symbolic rule extraction, in: Neural Networks Proceedings, 1998, IEEE World Congress on Computational Intelligence. The 1998 IEEE International Joint Conference on, Vol. 1, IEEE, 1998, pp. 146-151Search in Google Scholar

[21] G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, Extreme learning machine: theory and applications, Neurocomputing 70 , 1, 2006, 489-50110.1016/j.neucom.2005.12.126Search in Google Scholar

[22] L. Breiman, Bagging predictors, Machine learning 24, 2, 1996, 123-14010.1007/BF00058655Search in Google Scholar

[23] L. Breman, Bias, variance, and arcing classifiers (technical report 460), Statistics Department, University of CaliforniaSearch in Google Scholar

[24] P. Vincent, H. Larochelle, Y. Bengio, P.A. Manzagol, Extracting and composing robust features with denoising autoencoders, in: Proceedings of the 25th international conference on Machine learning, ACM, 2008, pp. 1096-110310.1145/1390156.1390294Search in Google Scholar

[25] M. Lichman, http://archive.ics.uci.edu/ml (UCI machine learning repository 2013)Search in Google Scholar

[26] Y. Hayashi, S. Nakano, S. Fujisawa, Use of the recursiverule extraction algorithm with continuous attributes to improve diagnostic accuracy in thyroid disease, Informatics in Medicine Unlocked 1, 2015, 1-810.1016/j.imu.2015.12.003Search in Google Scholar

[27] W. Duch, R. Adamczak, K. Grøbczewski, A new methodology of extraction, optimization and application of crisp and fuzzy logical rules, Neural Networks, IEEE Transactions on 12 , 2, 2001, 277-30610.1109/72.91452418244384Search in Google Scholar

[28] S. Abe, R. Thawonmas, M. Kayama, A fuzzy classifier with ellipsoidal regions for diagnosis problems, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 29 , 1, 1999, 140-14810.1109/5326.740676Search in Google Scholar

[29] J. Huysmans, R. Setiono, B. Baesens, J. Vanthienen, Minerva: Sequential covering for rule extraction, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 38 , 2, 2008, 299-30910.1109/TSMCB.2007.91207918348915Search in Google Scholar

[30] K. Odajima, Y. Hayashi, G. Tianxia, R. Setiono, Greedy rule generation from discrete data and its use in neural network rule extraction, Neural Networks 21 , 7, 2008, 1020-102810.1016/j.neunet.2008.01.00318442894Search in Google Scholar

[31] Y. Hayashi, S. Nakano, Use of a recursiverule extraction algorithm with j48graft to achieve highly accurate and concise rule extraction from a large breast cancer dataset, Informatics in Medicine Unlocked 1, 2015, 9-1610.1016/j.imu.2015.12.002Search in Google Scholar

[32] Y. LeCun, C. Cortes, C. Burges, The mnist database of handwritten digits, 1998, 2012, Available electronically at http://yann.lecun.com/exdb/mnistSearch in Google Scholar

[33] V. Cherkassky, S. Dhar, Interpretation of blackbox predictive models, in: Measures of Complexity, Springer, 2015, pp. 267-28610.1007/978-3-319-21852-6_19Search in Google Scholar

[34] W. Verbeke, D. Martens, C. Mues, B. Baesens, Building comprehensible customer churn prediction models with advanced rule induction techniques, Expert Systems with Applications 38 , 3, 2011, 2354-236410.1016/j.eswa.2010.08.023Search in Google Scholar

[35] G. Bologna, Y. Hayashi, Qsvm: A support vector machine for rule extraction, in: International WorkConference on Artificial Neural Networks, Springer, 2015, pp. 276-28910.1007/978-3-319-19222-2_23Search in Google Scholar