<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>With the industrialization and urbanization of modern society as well as the diversified development of entertainment forms, the inheritance of shadow puppetry as a folk art is facing great challenges. Based on this, this paper uses modern digital technology to help traditional shadow play quickly realize digital transformation and adapt to the current era. The YOLOv5 model is used to recognize the body movements of a person manipulating shadow puppets in virtual space, monitor the interaction status between the person and the virtual shadow puppet characters in virtual space, and at the same time, use the loss function to calculate and categorize the data loss in the recognition process. In order to avoid the data loss caused by the action moving too fast or the light, this paper introduces the Kalman filter algorithm to recover the lost and incoherent actions. It thus constructs the interaction recognition model based on the Kalman filter to improve YOLO. After conducting experiments in this paper, it was found that the model built in this paper has an accuracy of 94% in recognition of unobstructed actions, and it takes 1.31 seconds to recognize them. For occluded actions, the recognition accuracy rate is 86%. Overall, the recognition interaction model constructed in this paper maintains a recognition accuracy rate of more than 85% for actions. In the experiment on processing virtual shadow play characters, the average generation time of the virtual models of any shadow play character is no more than 0.5 seconds. The model constructed in this paper has a superior performance in action recognition and virtual shadow play character processing.</p>
</abstract>

With the industrialization and urbanization of modern society as well as the diversified development of entertainment forms, the inheritance of shadow puppetry as a folk art is facing great challenges. Based on this, this paper uses modern digital technology to help traditional shadow play quickly realize digital transformation and adapt to the current era. The YOLOv5 model is used to recognize the body movements of a person manipulating shadow puppets in virtual space, monitor the interaction status between the person and the virtual shadow puppet characters in virtual space, and at the same time, use the loss function to calculate and categorize the data loss in the recognition process. In order to avoid the data loss caused by the action moving too fast or the light, this paper introduces the Kalman filter algorithm to recover the lost and incoherent actions. It thus constructs the interaction recognition model based on the Kalman filter to improve YOLO. After conducting experiments in this paper, it was found that the model built in this paper has an accuracy of 94% in recognition of unobstructed actions, and it takes 1.31 seconds to recognize them. For occluded actions, the recognition accuracy rate is 86%. Overall, the recognition interaction model constructed in this paper maintains a recognition accuracy rate of more than 85% for actions. In the experiment on processing virtual shadow play characters, the average generation time of the virtual models of any shadow play character is no more than 0.5 seconds. The model constructed in this paper has a superior performance in action recognition and virtual shadow play character processing.

Research on the design of virtual experience of shadow play based on digital technology and immersive interactive design theory

Applied Mathematics and Nonlinear Sciences

This work is licensed under the Creative Commons Attribution 4.0 International License.

With the industrialization and urbanization of modern society as well as the diversified development of entertainment forms, the inheritance of shadow puppetry...

Research on the design of virtual experience of shadow play based on digital technology and immersive interactive design theory

Published Online: May 30, 2024

Page range: -

Received: Feb 15, 2024

Accepted: Apr 27, 2024

DOI: https://doi.org/10.2478/amns-2024-1221

Keywords
YOLOv5 model, Loss function, Kalman filter, Interaction design, Shadow theater experience

© 2024 Enze Li et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Research on the design of virtual experience of shadow play based on digital technology and immersive interactive design theory

Published Online: May 30, 2024

Page range: -

Received: Feb 15, 2024

Accepted: Apr 27, 2024

DOI: https://doi.org/10.2478/amns-2024-1221

KeywordsYOLOv5 model, Loss function, Kalman filter, Interaction design, Shadow theater experience

© 2024 Enze Li et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Keywords
YOLOv5 model, Loss function, Kalman filter, Interaction design, Shadow theater experience