<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>In order to give full play to the application of big data in film and television media and imaging in the cloud era, this study proposes a communication-efficient distributed deep neural network training method based on the DANE algorithm framework. The DANE algorithm is an approximate Newtonian method that has been widely used in communication-efficient distributed machine learning. It has the advantages of fast convergence and no need to calculate the inverse of the Hessian matrix, which can significantly reduce the communication and computational overhead in high-dimensional situations. In order to further improve the computational efficiency, it is necessary to study how to speed up the local optimization of DANE. It is a feasible method to choose to use the most popular adaptive gradient optimization algorithm Adam to replace the commonly used stochastic gradient descent method to solve the local single-machine suboptimization problem of DANE. Experiments show that Adam-based optimization can converge significantly faster than the original SGD-based implementation with little sacrifice in model generalization performance. With the increase of sampling rate, DANE-Adam significantly outperforms the DANE method in terms of convergence speed, and at the same time, the accuracy can be kept almost unchanged, which are 0.96, 0.88 and 0.75, respectively. This shows that Adam-based optimization can converge significantly faster than the original SGD-based implementation with little sacrifice in model generalization performance, with significant potential value.</p>
</abstract>

In order to give full play to the application of big data in film and television media and imaging in the cloud era, this study proposes a communication-efficient distributed deep neural network training method based on the DANE algorithm framework. The DANE algorithm is an approximate Newtonian method that has been widely used in communication-efficient distributed machine learning. It has the advantages of fast convergence and no need to calculate the inverse of the Hessian matrix, which can significantly reduce the communication and computational overhead in high-dimensional situations. In order to further improve the computational efficiency, it is necessary to study how to speed up the local optimization of DANE. It is a feasible method to choose to use the most popular adaptive gradient optimization algorithm Adam to replace the commonly used stochastic gradient descent method to solve the local single-machine suboptimization problem of DANE. Experiments show that Adam-based optimization can converge significantly faster than the original SGD-based implementation with little sacrifice in model generalization performance. With the increase of sampling rate, DANE-Adam significantly outperforms the DANE method in terms of convergence speed, and at the same time, the accuracy can be kept almost unchanged, which are 0.96, 0.88 and 0.75, respectively. This shows that Adam-based optimization can converge significantly faster than the original SGD-based implementation with little sacrifice in model generalization performance, with significant potential value.

Analysis of the application methods of film and television media and images in the era of big data cloud

Applied Mathematics and Nonlinear Sciences

This work is licensed under the Creative Commons Attribution 4.0 International License.

In order to give full play to the application of big data in film and television media and imaging in the cloud era, this study proposes a...

Analysis of the application methods of film and television media and images in the era of big data cloud

Data publikacji: 28 kwi 2023

Zakres stron: -

Otrzymano: 03 mar 2022

Przyjęty: 17 lip 2022

DOI: https://doi.org/10.2478/amns.2023.1.00063

Słowa kluczowe
Deep learning, DANE, Adam’s algorithm, Random sampling, Distributed optimization

© 2023 Qing Zhang, published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Analysis of the application methods of film and television media and images in the era of big data cloud

Data publikacji: 28 kwi 2023

Zakres stron: -

Otrzymano: 03 mar 2022

Przyjęty: 17 lip 2022

DOI: https://doi.org/10.2478/amns.2023.1.00063

Słowa kluczoweDeep learning, DANE, Adam’s algorithm, Random sampling, Distributed optimization

© 2023 Qing Zhang, published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Słowa kluczowe
Deep learning, DANE, Adam’s algorithm, Random sampling, Distributed optimization