<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>The analysis of teachers' and students' behaviors in physical education classrooms is an important way to improve the quality of physical education teaching and teaching methods, which helps teachers to check the gaps and improve the teaching level. In this paper, for the problems of data differences between multiple modalities and the conflict between feature extraction modules of different modalities, we designed a dual-stream framework HRformer algorithm based on Transformer, which unifies the skeletal modalities and video modalities in the algorithm. The relationship between skeletal and video modalities is modeled using the self-attention mechanism, and the matching and fusion of skeletal features and video data is performed to construct a behavior recognition model for teachers and students in the sports classroom based on multimodal data. Then, the model is compared with mainstream networks on the dataset to verify its performance. To conduct model application and example analysis, a university collects data on physical education classroom teachers and students for a semester. It is found that the multimodal model in this paper has a classification F1 value of 95.61%, 93.19%, and 93.74% for the three types of behavior recognition, namely, skill training (ST), game activity (GA), and rest, respectively, which are higher than the two methods of single skeletal modality and video modality. The model has the highest recognition accuracy of 97.12% and 98.15% for Game Activity (GA). Based on real physical education classroom data, the practical application of the model in physical education teaching classrooms in this paper is fruitful, and the results of behavioral recognition classification are in line with the design expectation. This study develops an effective method for classifying teacher and student behaviors in a physical education classroom. It provides a useful exploration for the integration and innovation of physical education teaching and information technology.</p>
</abstract>

The analysis of teachers' and students' behaviors in physical education classrooms is an important way to improve the quality of physical education teaching and teaching methods, which helps teachers to check the gaps and improve the teaching level. In this paper, for the problems of data differences between multiple modalities and the conflict between feature extraction modules of different modalities, we designed a dual-stream framework HRformer algorithm based on Transformer, which unifies the skeletal modalities and video modalities in the algorithm. The relationship between skeletal and video modalities is modeled using the self-attention mechanism, and the matching and fusion of skeletal features and video data is performed to construct a behavior recognition model for teachers and students in the sports classroom based on multimodal data. Then, the model is compared with mainstream networks on the dataset to verify its performance. To conduct model application and example analysis, a university collects data on physical education classroom teachers and students for a semester. It is found that the multimodal model in this paper has a classification F1 value of 95.61%, 93.19%, and 93.74% for the three types of behavior recognition, namely, skill training (ST), game activity (GA), and rest, respectively, which are higher than the two methods of single skeletal modality and video modality. The model has the highest recognition accuracy of 97.12% and 98.15% for Game Activity (GA). Based on real physical education classroom data, the practical application of the model in physical education teaching classrooms in this paper is fruitful, and the results of behavioral recognition classification are in line with the design expectation. This study develops an effective method for classifying teacher and student behaviors in a physical education classroom. It provides a useful exploration for the integration and innovation of physical education teaching and information technology.

Construction of a Classification Model for Teacher and Student Behavior in Physical Education Classrooms –Based on Multimodal Data

Anhui Business Vocational College, Anhui Business and Technology College

Applied Mathematics and Nonlinear Sciences

This work is licensed under the Creative Commons Attribution 4.0 International License.

The analysis of teachers' and students' behaviors in physical education classrooms is an important way to improve the quality of physical education teaching...

Construction of a Classification Model for Teacher and Student Behavior in Physical Education Classrooms –Based on Multimodal Data

Published Online: Jul 09, 2024

Page range: -

Received: Mar 08, 2024

Accepted: Jun 04, 2024

DOI: https://doi.org/10.2478/amns-2024-1818

Keywords
Transformer, Skeletal heat map flow, Multimodal data, Attention mechanism, Behavior classification

© 2024 Yan Zhao et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Construction of a Classification Model for Teacher and Student Behavior in Physical Education Classrooms –Based on Multimodal Data

Published Online: Jul 09, 2024

Page range: -

Received: Mar 08, 2024

Accepted: Jun 04, 2024

DOI: https://doi.org/10.2478/amns-2024-1818

KeywordsTransformer, Skeletal heat map flow, Multimodal data, Attention mechanism, Behavior classification

© 2024 Yan Zhao et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Keywords
Transformer, Skeletal heat map flow, Multimodal data, Attention mechanism, Behavior classification