<abstract xmlns="http://www.w3.org/1999/xhtml"><p>This paper presents a local modification of the Levenberg-Marquardt algorithm (LM). First, the mathematical basics of the classic LM method are shown. The classic LM algorithm is very efficient for learning small neural networks. For bigger neural networks, whose computational complexity grows significantly, it makes this method practically inefficient. In order to overcome this limitation, local modification of the LM is introduced in this paper. The main goal of this paper is to develop a more complexity efficient modification of the LM method by using a local computation. The introduced modification has been tested on the following benchmarks: the function approximation and classification problems. The obtained results have been compared to the classic LM method performance. The paper shows that the local modification of the LM method significantly improves the algorithm’s performance for bigger networks. Several possible proposals for future works are suggested.</p></abstract>

This paper presents a local modification of the Levenberg-Marquardt algorithm (LM). First, the mathematical basics of the classic LM method are shown. The classic LM algorithm is very efficient for learning small neural networks. For bigger neural networks, whose computational complexity grows significantly, it makes this method practically inefficient. In order to overcome this limitation, local modification of the LM is introduced in this paper. The main goal of this paper is to develop a more complexity efficient modification of the LM method by using a local computation. The introduced modification has been tested on the following benchmarks: the function approximation and classification problems. The obtained results have been compared to the classic LM method performance. The paper shows that the local modification of the LM method significantly improves the algorithm’s performance for bigger networks. Several possible proposals for future works are suggested.

Local Levenberg-Marquardt Algorithm for Learning Feedforwad Neural Networks

Department Electrical and Computer Engineering

Journal of Artificial Intelligence and Soft Computing Research

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

{"article-title":"Local Levenberg-Marquardt Algorithm for Learning Feedforwad Neural Networks"}

This paper presents a local modification of the Levenberg-Marquardt algorithm (LM). First, the mathematical basics of the classic LM method are shown. The...

Local Levenberg-Marquardt Algorithm for Learning Feedforwad Neural Networks

Publié en ligne: 15 juin 2020

Pages: 299 - 316

Reçu: 21 oct. 2019

Accepté: 19 mai 2020

DOI: https://doi.org/10.2478/jaiscr-2020-0020

Mots clés
feed-forward neural network, neural network learning algorithm, optimization problem, Levenberg-Marquardt algorithm, QR decomposition, Givens rotation

© 2020 Jarosław Bilski et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Local Levenberg-Marquardt Algorithm for Learning Feedforwad Neural Networks

Publié en ligne: 15 juin 2020

Pages: 299 - 316

Reçu: 21 oct. 2019

Accepté: 19 mai 2020

DOI: https://doi.org/10.2478/jaiscr-2020-0020

Mots clésfeed-forward neural network, neural network learning algorithm, optimization problem, Levenberg-Marquardt algorithm, QR decomposition, Givens rotation

© 2020 Jarosław Bilski et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Mots clés
feed-forward neural network, neural network learning algorithm, optimization problem, Levenberg-Marquardt algorithm, QR decomposition, Givens rotation