<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>Residual deep neural networks (ResNets) are mathematically described as interacting particle systems. In the case of infinitely many layers the ResNet leads to a system of coupled system of ordinary differential equations known as neural differential equations. For large scale input data we derive a mean–field limit and show well–posedness of the resulting description. Further, we analyze the existence of solutions to the training process by using both a controllability and an optimal control point of view. Numerical investigations based on the solution of a formal optimality system illustrate the theoretical findings.</p>
</abstract>

Residual deep neural networks (ResNets) are mathematically described as interacting particle systems. In the case of infinitely many layers the ResNet leads to a system of coupled system of ordinary differential equations known as neural differential equations. For large scale input data we derive a mean–field limit and show well–posedness of the resulting description. Further, we analyze the existence of solutions to the training process by using both a controllability and an optimal control point of view. Numerical investigations based on the solution of a formal optimality system illustrate the theoretical findings.

Continuous limits of residual neural networks in case of large input data

“G. Castelnuovo” – Sapienza University of Rome

Communications in Applied and Industrial Mathematics

This work is licensed under the Creative Commons Attribution 4.0 International License.

{"article-title":"Continuous limits of residual neural networks in case of large input data"}

Residual deep neural networks (ResNets) are mathematically described as interacting particle systems. In the case of infinitely many layers the ResNet leads to...

Continuous limits of residual neural networks in case of large input data

Data publikacji: 24 gru 2022

Zakres stron: 96 - 120

Otrzymano: 11 lip 2022

Przyjęty: 12 lis 2022

DOI: https://doi.org/10.2478/caim-2022-0008

Słowa kluczowe
Neural networks, mean-field limit, well-posedness, optimal control, controllability

© 2022 Michael Herty et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Continuous limits of residual neural networks in case of large input data

Data publikacji: 24 gru 2022

Zakres stron: 96 - 120

Otrzymano: 11 lip 2022

Przyjęty: 12 lis 2022

DOI: https://doi.org/10.2478/caim-2022-0008

Słowa kluczoweNeural networks, mean-field limit, well-posedness, optimal control, controllability

© 2022 Michael Herty et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Słowa kluczowe
Neural networks, mean-field limit, well-posedness, optimal control, controllability