Emerging Modularity During the Evolution of Neural Networks

[1] S. Ahmadian, S. Jalali, S. Islam, A. Khosravi, E. Fazli, and S. Nahavandi. A novel deep neuroevolution-based image classification method to diagnose coronavirus disease (covid-19). Comput Biol Med., (139:104994), 2021.10.1016/j.compbiomed.2021.104994855814934749098 Search in Google Scholar

[2] A. Baldominos, Y. Saez, and P. Isasi. Evolutionary convolutional neural networks: an application to handwriting recognition. Neurocomputing, 283:38–52, 2018. Search in Google Scholar

[3] C.Y. Baldwin and K.B. Clark. Design Rules: The power of modularity. Chapter 3: What Is Modularity? MIT Press, 2018. Search in Google Scholar

[4] A. Billard and M. J. Mataric. Learning human movements by imitation: evaluation of a biologically inspired connectionist architecture. Robotics and Autonomous Systems, 941:1–16, 2001. Search in Google Scholar

[5] V. D. Blondel, J.-L. Guillaume, R. Lambiotte, and E. Lefebvre. Fast unfolding of communities in large networks. Journal of Statistical Mechanics Theory and Experiment, 10:10008, 2008.10.1088/1742-5468/2008/10/P10008 Search in Google Scholar

[6] R. Calabretta and J. Neirotti. Adaptive agents in changing environments, the role of modularity. Neural Process Lett, 42:257–274, 2015.10.1007/s11063-014-9355-8 Search in Google Scholar

[7] M. Carcenac. A modular neural network applied to image transformation and mental images. Neural Computing and Applications, 17:549–568, 2008.10.1007/s00521-007-0152-4 Search in Google Scholar

[8] J. Clune, B.E. Beckmann, P.K. McKinley, and C. Ofria. Investigating whether hyperneat produces modular neural networks. In Proceedings of the 12th annual conference on Genetic and evolutionary computation, pages 635–642, 2010.10.1145/1830483.1830598 Search in Google Scholar

[9] J. Clune, J-B. Mouret, and H. Lipson. The evolutionary origins of modularity. In Proceedings of the Royal Society B, 2013.10.1145/2464576.2464596 Search in Google Scholar

[10] [10] A.S. Cofino, J.M. Gutierrez, and M.L. Ivanissevich. Evolving modular networks with genetic algorithms: application to nonlinear time series. Expert Systems, 21(4):208–216, 2004. Search in Google Scholar

[11] Y. J. Cruz, M. Rivas, R. Quiza, A. Villalonga, R. E. Haber, and G. Beruvides. Ensemble of convolutional neural networks based on an evolutionary algorithm applied to an industrial welding process. Computers in Industry, 133:103530, 2021.10.1016/j.compind.2021.103530 Search in Google Scholar

[12] K. Deb, A. Pratap, S. Agarwal,, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA–II. IEEE Transactions on Evolutionary Computation, 6(2):182–197, 2002. Search in Google Scholar

[13] S. Doncieux and J. Meyer. Evolving modular neural networks to solve challenging control problems. In Proceedings of the Fourth International ICSC Symposium on Engineering of Intelligent Systems, 2004. Search in Google Scholar

[14] K. O. Ellefsen and J. Torresen. Evolving neural networks with multiple internal models. In Proceedings of the 14th European Conference on Artificial Life ECAL 2017, volume 14, pages 138–145, 2017.10.7551/ecal_a_025 Search in Google Scholar

[15] K.O. Ellefsen, J-B. Mouret, and J. Clune. Neural modularity helps organisms evolve to learn new skills without forgetting old skills. PLoS Computational Biology, 11(4):e1004128, 2015.10.1371/journal.pcbi.1004128438333525837826 Search in Google Scholar

[16] C. Espinosa-Soto and A. Wagner. Specialization can drive the evolution of modularity. PLoS Computational Biology, 6(3):e1000719, 2010.10.1371/journal.pcbi.1000719284794820360969 Search in Google Scholar

[17] C. Fernando, D. Banarse, M. Reynolds, F. Besse, D. Pfau, M. Jaderberg, M. Lanctot, and D.Wierstra. Convolution by evolution: differentiable pattern producing networks. In Proceedings of the 2016 Genetic and Evolutionary Computation Conference, pages 109–116, 2016.10.1145/2908812.2908890 Search in Google Scholar

[18] D. Filan, S. Hod, C. Wild, A. Critch, and S. Russell. Pruned neural networks are surprisingly modular. Technical Report arXiv:2003.04881 [cs.NE], ArXiV, 2020. Search in Google Scholar

[19] J. Gauci and K. Stanley. Generating large–scale neural networks through discovering geometric regularities. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 997–1004, 2007.10.1145/1276958.1277158 Search in Google Scholar

[20] S. Han and S. Oh. An optimized modular neural network controller based on environment classification and selective sensor usage for mobile robot reactive navigation. Neural Computation and Application, 17:161–173, 2008.10.1007/s00521-006-0079-1 Search in Google Scholar

[21] J. Huizinga, J.B. Mouret, and J. Clune. Evolving neural networks that are both modular and regular: Hyperneat plus the connection cost technique. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pages 697–704, 2014.10.1145/2576768.2598232 Search in Google Scholar

[22] M. Hulse, S. Wischmann, and F. Pasemann. Structure and function of evolved neuro– controllers for autonomous robots. Connection Science, 16(4):249–266, 2004.10.1080/09540090412331314795 Search in Google Scholar

[23] L. Kirsch, J. Kunze, and David Barber. Modular networks: Learning to decompose neural computation. Technical Report arXiv:1811.05249 [cs.LG], ArXiV, 2018. Search in Google Scholar

[24] J. Koutnik, J. Schmidhuber, and F. Gomez. Evolving deep unsupervised convolutional networks for vision–based reinforcement learning. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pages 541–548, 2014.10.1145/2576768.2598358 Search in Google Scholar

[25] V. Landassuri-Moreno and J. A. Bullinaria. Biasing the evolution of modular neural networks. In 2011 IEEE Congress of Evolutionary Computation, 2011.10.1109/CEC.2011.5949855 Search in Google Scholar

[26] J. Liang, E. Meyerson, and R. Miikkulainen. Evolutionary architecture search for deep multitask networks. In GECCO ’18: Proceedings of the Genetic and Evolutionary Computation Conference, pages 466–473, 2018.10.1145/3205455.3205489 Search in Google Scholar

[27] I. Loshchilov and F. Hutter. CMA–ES for hyper-parameter optimization of deep neural networks. Technical Report arXiv: abs/1604.07269 [cs.NE], ArXiV, 2016. Search in Google Scholar

[28] P. Melin, D. Bravo, and O. Castillo. Fingerprint recognition using the fuzzy sugeno integral for response integration in modular neural networks. International Journal of General Systems, 37(4):499–515, 2008.10.1080/03081070701321910 Search in Google Scholar

[29] R. Miikkulainen, J. Liang, E. Meyerson, A. Rawal, D. Fink, O. Francon, B. Raju, H. Shahrzad, A. Navruzyan, N. Duffy, and B. Hodjat. Evolving deep neural networks. Technical Report arXiv abs/1703.00548 [cs.NE], ArXiV, 2017. Search in Google Scholar

[30] J-B. Mouret and S. Doncieux. Evolving modular neural networks through exaptation. In 2009 IEEE Congress on Evolutionary Computation, pages 1570–1577, 2009.10.1109/CEC.2009.4983129 Search in Google Scholar

[31] H. Munn and M. Gallagher. Modularity in NEAT reinforcement learning networks, 2022. Search in Google Scholar

[32] N. NourAshrafoddin, A. R. Vahdat, and M. M. Ebadzadeh. Automatic design of modular neural networks using genetic programming. In Proceedings of the 17th International Conference on Artificial Neural Networks ICANN 2007 Part I, pages 788–798, 2007.10.1007/978-3-540-74690-4_80 Search in Google Scholar

[33] M. Potter. The Design and Analysis of a Computational Model of Cooperative Coevolution. PhD thesis, George Mason University, 1997. Search in Google Scholar

[34] M. A. Potter and K. A. De Jong. Cooperative coevolution: An architecture for evolving coadapted subcomponents. Evolutionary Computation, 8(1):1–29, 2000. Search in Google Scholar

[35] T. Praczyk. Probabilistic neural network application to warship radio stations identification. Computational Methods in Science and Technology, 13(1):53–58, 2007.10.12921/cmst.2007.13.01.53-57 Search in Google Scholar

[36] T. Praczyk. Using augmenting modular neural networks to evolve neuro–controllers for a team of underwater vehicles. Soft Computing, 18(12):2445–2460, 2014.10.1007/s00500-014-1221-0 Search in Google Scholar

[37] T. Praczyk. Cooperative co–evolutionary neural networks. Journal of Intelligent & Fuzzy Systems, 30(5):2843–2858, 2016.10.3233/IFS-162095 Search in Google Scholar

[38] T. Praczyk. Hill climb modular assembler encoding: Evolving modular neural networks of fixed modular architecture. Knowledge-Based Systems, 232:107493, nov 2021.10.1016/j.knosys.2021.107493 Search in Google Scholar

[39] K. Soltanian, A. Ebnenasir, and M. Afsharchi. Modular grammatical evolution for the generation of artificial neural networks. Evolutionary Computation, 30(2):291–327, 06 2022.10.1162/evco_a_0030234878521 Search in Google Scholar

[40] S. Sotirov, E. Sotirova, V. Atanassova, K. Atanassov, O. Castillo, P. Melin, T. Petkov, and S. Surchev. A hybrid approach for modular neural network design using intercriteria analysis and intuitionistic fuzzy logic. Complexity, 1:1–11, 2018.10.1155/2018/3927951 Search in Google Scholar

[41] K. O. Stanley and R. Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary Computation, 10:99–127, 2002.10.1162/10636560232016981112180173 Search in Google Scholar

[42] Y. Sun, B. Xue, M. Zhang, and G. G. Yen. Automatically designing CNN architectures using genetic algorithm for image classification. Technical Report arXiv:1808.03818 [cs.NE], ArXiV, 2018. Search in Google Scholar

[43] C. R. Tosh. Can computational efficiency alone drive the evolution of modularity in neural networks? Scientific Reports, 6:31982, 2016.10.1038/srep31982500415227573614 Search in Google Scholar

[44] C. R. Tosh and L. McNally. The relative efficiency of modular and non–modular networks of different size. In Proceedings of the Royal Society B: Biological Sciences, volume 282:20142568, 2015.10.1098/rspb.2014.2568434415225631996 Search in Google Scholar

[45] A. Turan, S. D. Hinchberger, and M. H. El Naggar. Predicting the dynamic properties of glyben using a modular neural network (MNN). Canadian Geotechnical Journal, 45:1629–1638, 2008.10.1139/T08-054 Search in Google Scholar

[46] V. K. Valsalam and R. Miikkulainen. Evolving symmetric and modular neural networks for distributed control. In Proceedings of the Genetic and Evolutionary Computation Conference, 2009.10.1145/1569901.1570002 Search in Google Scholar

[47] L. Xie and A. Yuille. Genetic CNN. Technical Report arXiv abs/1703.01513 [cs.NE], ArXiV, 2017. Search in Google Scholar

[48] X. Yao and Y. Liu. A new evolutionary system for evolving artificial neural networks. IEEE Transactions on Neural Networks, 8(3):694–713, 1997.10.1109/72.57210718255671 Search in Google Scholar

[49] S.R. Young, D.C. Rose, T.P. Karnowsky, S.H. Lim, and R.M. Patton. Optimizing deep learning hyper–parameters through an evolutionary algorithm. In Proceedings of the Workshop on Machine Learning in High–Performance Computing Environments, number 4, pages 1–5, 2015.10.1145/2834892.2834896 Search in Google Scholar

[50] Z. Zhu, S. Guo, and M. Liao. Deep neuroevolution: Evolving neural network for character locomotion controller. In 2021 2nd International Conference on Artificial Intelligence and Information Systems, ICAIIS 2021, New York, NY, USA, 2021. Association for Computing Machinery.10.1145/3469213.3470259 Search in Google Scholar