<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>In this study, a machine learning-based system, which recognises the Turkish sign language person-independent in real-time, was developed. A leap motion sensor was used to obtain raw data from individuals. Then, handcraft features were extracted by using Euclidean distance on the raw data. Handcraft features include finger-to-finger, finger -to-palm, finger -to-wrist bone, palm-to-palm and wrist-to-wrist distances. LR, <italic>k</italic>-NN, RF, DNN, ANN single classifiers were trained using the handcraft features. Cascade voting approach was applied with two-step voting. The first voting was applied for each classifier’s final prediction. Then, the second voting, which voted the prediction of all classifiers at the final decision stage, was applied to improve the performance of the proposed system. The proposed system was tested in real-time by an individual whose hand data were not involved in the training dataset. According to the results, the proposed system presents 100 % value of accuracy in the classification of one hand letters. Besides, the recognition accuracy ratio of the system is 100 % on the two hands letters, except “J” and “H” letters. The recognition accuracy rates were 80 % and 90 %, respectively for “J” and “H” letters. Overall, the cascade voting approach presented a high average classification performance with 98.97 % value of accuracy. The proposed system enables Turkish sign language recognition with high accuracy rates in real time.</p>
</abstract>

In this study, a machine learning-based system, which recognises the Turkish sign language person-independent in real-time, was developed. A leap motion sensor was used to obtain raw data from individuals. Then, handcraft features were extracted by using Euclidean distance on the raw data. Handcraft features include finger-to-finger, finger -to-palm, finger -to-wrist bone, palm-to-palm and wrist-to-wrist distances. LR, k-NN, RF, DNN, ANN single classifiers were trained using the handcraft features. Cascade voting approach was applied with two-step voting. The first voting was applied for each classifier’s final prediction. Then, the second voting, which voted the prediction of all classifiers at the final decision stage, was applied to improve the performance of the proposed system. The proposed system was tested in real-time by an individual whose hand data were not involved in the training dataset. According to the results, the proposed system presents 100 % value of accuracy in the classification of one hand letters. Besides, the recognition accuracy ratio of the system is 100 % on the two hands letters, except “J” and “H” letters. The recognition accuracy rates were 80 % and 90 %, respectively for “J” and “H” letters. Overall, the cascade voting approach presented a high average classification performance with 98.97 % value of accuracy. The proposed system enables Turkish sign language recognition with high accuracy rates in real time.

Real-Time Turkish Sign Language Recognition Using Cascade Voting Approach with Handcrafted Features

Faculty of Engineering and Architecture, Computer Engineering

Applied Computer Systems

This work is licensed under the Creative Commons Attribution 4.0 International License.

In this study, a machine learning-based system, which recognises the Turkish sign language person-independent in real-time, was developed. A leap motion sensor...

Real-Time Turkish Sign Language Recognition Using Cascade Voting Approach with Handcrafted Features

Data publikacji: 04 cze 2021

Zakres stron: 12 - 21

DOI: https://doi.org/10.2478/acss-2021-0002

Słowa kluczowe
Cascade voting, handcrafted features, machine learning, sign language recognition, Turkish sign language

© 2021 Abdulkadir Karacı et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Real-Time Turkish Sign Language Recognition Using Cascade Voting Approach with Handcrafted Features

Data publikacji: 04 cze 2021

Zakres stron: 12 - 21

DOI: https://doi.org/10.2478/acss-2021-0002

Słowa kluczoweCascade voting, handcrafted features, machine learning, sign language recognition, Turkish sign language

© 2021 Abdulkadir Karacı et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Słowa kluczowe
Cascade voting, handcrafted features, machine learning, sign language recognition, Turkish sign language