<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>Residual deep neural networks (ResNets) are mathematically described as interacting particle systems. In the case of infinitely many layers the ResNet leads to a system of coupled system of ordinary differential equations known as neural differential equations. For large scale input data we derive a mean–field limit and show well–posedness of the resulting description. Further, we analyze the existence of solutions to the training process by using both a controllability and an optimal control point of view. Numerical investigations based on the solution of a formal optimality system illustrate the theoretical findings.</p>
</abstract>

Residual deep neural networks (ResNets) are mathematically described as interacting particle systems. In the case of infinitely many layers the ResNet leads to a system of coupled system of ordinary differential equations known as neural differential equations. For large scale input data we derive a mean–field limit and show well–posedness of the resulting description. Further, we analyze the existence of solutions to the training process by using both a controllability and an optimal control point of view. Numerical investigations based on the solution of a formal optimality system illustrate the theoretical findings.

Continuous limits of residual neural networks in case of large input data

“G. Castelnuovo” – Sapienza University of Rome

Communications in Applied and Industrial Mathematics

This work is licensed under the Creative Commons Attribution 4.0 International License.

{"article-title":"Continuous limits of residual neural networks in case of large input data"}

Residual deep neural networks (ResNets) are mathematically described as interacting particle systems. In the case of infinitely many layers the ResNet leads to...

Continuous limits of residual neural networks in case of large input data

Online veröffentlicht: 24. Dez. 2022

Seitenbereich: 96 - 120

Eingereicht: 11. Juli 2022

Akzeptiert: 12. Nov. 2022

DOI: https://doi.org/10.2478/caim-2022-0008

Schlüsselwörter
Neural networks, mean-field limit, well-posedness, optimal control, controllability

© 2022 Michael Herty et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Continuous limits of residual neural networks in case of large input data

Online veröffentlicht: 24. Dez. 2022

Seitenbereich: 96 - 120

Eingereicht: 11. Juli 2022

Akzeptiert: 12. Nov. 2022

DOI: https://doi.org/10.2478/caim-2022-0008

SchlüsselwörterNeural networks, mean-field limit, well-posedness, optimal control, controllability

© 2022 Michael Herty et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Schlüsselwörter
Neural networks, mean-field limit, well-posedness, optimal control, controllability