<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>Transfer learning has surfaced as a compelling technique in machine learning, enabling the transfer of knowledge across networks. This study evaluates the efficacy of ImageNet pretrained state-of-the-art networks, including DenseNet, ResNet, and VGG, in implementing transfer learning for prepruned models on compact datasets, such as Fashion MNIST, CIFAR10, and CIFAR100. The primary objective is to reduce the number of neurons while preserving high-level features. To this end, local sensitivity analysis is employed alongside p-norms and various reduction levels. This investigation discovers that VGG16, a network rich in parameters, displays resilience to high-level feature pruning. Conversely, the ResNet architectures reveal an interesting pattern of increased volatility. These observations assist in identifying an optimal combination of the norm and the reduction level for each network architecture, thus offering valuable directions for model-specific optimization. This study marks a significant advance in understanding and implementing effective pruning strategies across diverse network architectures, paving the way for future research and applications.</p>
</abstract>

Transfer learning has surfaced as a compelling technique in machine learning, enabling the transfer of knowledge across networks. This study evaluates the efficacy of ImageNet pretrained state-of-the-art networks, including DenseNet, ResNet, and VGG, in implementing transfer learning for prepruned models on compact datasets, such as Fashion MNIST, CIFAR10, and CIFAR100. The primary objective is to reduce the number of neurons while preserving high-level features. To this end, local sensitivity analysis is employed alongside p-norms and various reduction levels. This investigation discovers that VGG16, a network rich in parameters, displays resilience to high-level feature pruning. Conversely, the ResNet architectures reveal an interesting pattern of increased volatility. These observations assist in identifying an optimal combination of the norm and the reduction level for each network architecture, thus offering valuable directions for model-specific optimization. This study marks a significant advance in understanding and implementing effective pruning strategies across diverse network architectures, paving the way for future research and applications.

Choice of the <italic>p</italic>-norm for High Level Classification Features Pruning in Modern Convolutional Neural Networks With Local Sensitivity Analysis

International Journal of Applied Mathematics and Computer Science

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Choice of the -norm for High Level Classification Features Pruning in Modern Convolutional Neural Networks With Local Sensitivity Analysis

Transfer learning has surfaced as a compelling technique in machine learning, enabling the transfer of knowledge across networks. This study evaluates the...

Choice of the p-norm for High Level Classification Features Pruning in Modern Convolutional Neural Networks With Local Sensitivity Analysis

Publié en ligne: 21 déc. 2023

Pages: 663 - 672

Reçu: 05 mai 2023

Accepté: 09 août 2023

DOI: https://doi.org/10.34768/amcs-2023-0047

Mots clés
convolutional neural network, pruning, sensitivity analysis, transfer learning, ImageNet

© 2023 Ernest Jeczmionek et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Choice of the p-norm for High Level Classification Features Pruning in Modern Convolutional Neural Networks With Local Sensitivity Analysis

Publié en ligne: 21 déc. 2023

Pages: 663 - 672

Reçu: 05 mai 2023

Accepté: 09 août 2023

DOI: https://doi.org/10.34768/amcs-2023-0047

Mots clésconvolutional neural network, pruning, sensitivity analysis, transfer learning, ImageNet

© 2023 Ernest Jeczmionek et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Mots clés
convolutional neural network, pruning, sensitivity analysis, transfer learning, ImageNet