1. bookVolume 26 (2021): Issue 1 (May 2021)
Journal Details
License
Format
Journal
First Published
08 Nov 2012
Publication timeframe
2 times per year
Languages
English
access type Open Access

Real-Time Turkish Sign Language Recognition Using Cascade Voting Approach with Handcrafted Features

Published Online: 04 Jun 2021
Page range: 12 - 21
Journal Details
License
Format
Journal
First Published
08 Nov 2012
Publication timeframe
2 times per year
Languages
English
Abstract

In this study, a machine learning-based system, which recognises the Turkish sign language person-independent in real-time, was developed. A leap motion sensor was used to obtain raw data from individuals. Then, handcraft features were extracted by using Euclidean distance on the raw data. Handcraft features include finger-to-finger, finger -to-palm, finger -to-wrist bone, palm-to-palm and wrist-to-wrist distances. LR, k-NN, RF, DNN, ANN single classifiers were trained using the handcraft features. Cascade voting approach was applied with two-step voting. The first voting was applied for each classifier’s final prediction. Then, the second voting, which voted the prediction of all classifiers at the final decision stage, was applied to improve the performance of the proposed system. The proposed system was tested in real-time by an individual whose hand data were not involved in the training dataset. According to the results, the proposed system presents 100 % value of accuracy in the classification of one hand letters. Besides, the recognition accuracy ratio of the system is 100 % on the two hands letters, except “J” and “H” letters. The recognition accuracy rates were 80 % and 90 %, respectively for “J” and “H” letters. Overall, the cascade voting approach presented a high average classification performance with 98.97 % value of accuracy. The proposed system enables Turkish sign language recognition with high accuracy rates in real time.

Keywords

[1] B. Unutmaz, A. C. Karaca, and M. K. Gullu, “Turkish Sign Language Recognition Using Kinect Skeleton and Convolutional Neural Network,” in 2019 27th Signal Processing and Comm. Appl. Conference (SIU), Apr. 2019, pp. 1–4. https://doi.org/10.1109/siu.2019.8806380 Search in Google Scholar

[2] P. Kumar, P. P. Roy, and D. P. Dogra, “Independent Bayesian Classifier Combination Based Sign Language Recognition Using Facial Expression,” Information Sciences, vol. 428, pp. 30–48, Feb. 2018. https://doi.org/10.1016/j.ins.2017.10.046 Search in Google Scholar

[3] H. Haberdar and S. Albayrak, “Real Time Isolated Turkish Sign Language Recognition from Video Using Hidden Markov Models with Global Features,” Lecture Notes in Computer Science, pp. 677–687, 2005. https://doi.org/10.1007/11569596_70 Search in Google Scholar

[4] T. Kapuściński and D. Warchoł, “Hand Posture Recognition Using Skeletal Data and Distance Descriptor,” Applied Sciences, vol. 10, no. 6, p. 2132, Mar. 2020. https://doi.org/10.3390/app10062132 Search in Google Scholar

[5] P. Kumar, H. Gauba, P. Pratim Roy, and D. Prosad Dogra, “A Multimodal Framework for Sensor Based Sign Language Recognition,” Neurocomputing, vol. 259, pp. 21–38, Oct. 2017. https://doi.org/10.1016/j.neucom.2016.08.132 Search in Google Scholar

[6] B. Oktekin, “İşitme ve Konuşma Engelli Bireyler için İşaret Tanıma Sistemi Geliştirme,” Uluslararasi Kibris Universitesi Fen-Edebiyat Fakultesi, vol. 25, no. 97-1, pp. 593–609, Jan. 2019. https://doi.org/10.22559/folklor.969 Search in Google Scholar

[7] Y. Mori and M. Toyonaga, “Data-Glove for Japanese Sign Language Training System with Gyro-Sensor,” in 2018 Joint 10th International Conference on Soft Computing and Intelligent Systems (SCIS) and 19th International Symposium on Advanced Intelligent Systems (ISIS), Dec. 2018, pp. 1354–1357. https://doi.org/10.1109/scis-isis.2018.00211 Search in Google Scholar

[8] B. Demircioǧlu, G. Bülbül, and H. Köse, “Turkish Sign Language Recognition with Leap Motion [Leap Motion ile Türk Işaret Dili Tanima],” in 2016 24th Signal Processing and Communication Application Conference (SIU), May 2016. https://doi.org/10.1109/siu.2016.7495809 Search in Google Scholar

[9] K. Kakayev and S. Albayrak, “Turkish Sign Language Recognition Using Hidden Markov Model,” in Computer Science & Information Technology (CS & IT), Jun. 2016, pp. 11–18. https://doi.org/10.5121/csit.2016.60802 Search in Google Scholar

[10] A. Memiş and S. Albayrak, “A Kinect Based Sign Language Recognition System Using Spatio-Temporal Features,” in Sixth International Conference on Machine Vision (ICMV 2013), Dec. 2013. https://doi.org/10.1117/12.2051018 Search in Google Scholar

[11] O. Yalcinkaya, A. Atvar, and P. Duygulu, “Turkish Sign Language Recognition Application Using Motion History Image,” in 2016 24th Signal Processing and Communication Application Conference (SIU), May 2016, pp. 801–804. https://doi.org/10.1109/siu.2016.7495861 Search in Google Scholar

[12] E. Ezel, O. Baykan, “Vision-Based Turkish Sign Language Recognition Using Convolutional Neural Networks,” in International Conference on Theoretical and Applied Computer Science and Engineering (ICTACSE, 2017), Nov. 2017, p. 14. Search in Google Scholar

[13] M. Aktas, B. Gokberk, and L. Akarun, “Recognizing Non-Manual Signs in Turkish Sign Language,” in 2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA), Nov. 2019, pp. 1–6. https://doi.org/10.1109/ipta.2019.8936081 Search in Google Scholar

[14] S. G. Moreira Almeida, F. G. Guimarães, and J. Arturo Ramírez, “Feature Extraction in Brazilian Sign Language Recognition Based on Phonological Structure and Using RGB-D Sensors,” Expert Systems with Applications, vol. 41, no. 16, pp. 7259–7271, Nov. 2014. https://doi.org/10.1016/j.eswa.2014.05.024 Search in Google Scholar

[15] W. Tao, M. C. Leu, and Z. Yin, “American Sign Language Alphabet Recognition Using Convolutional Neural Networks with Multiview Augmentation and Inference Fusion,” Engineering Applications of Artificial Intelligence, vol. 76, pp. 202–213, Nov. 2018. https://doi.org/10.1016/j.engappai.2018.09.006 Search in Google Scholar

[16] A. addin I. Sidig, H. Luqman, and S. A. Mahmoud, “Transform-Based Arabic Sign Language Recognition,” Procedia Computer Science, vol. 117, pp. 2–9, 2017. https://doi.org/10.1016/j.procs.2017.10.087 Search in Google Scholar

[17] M. A. Almasre and H. Al-Nuaim, “A Comparison of Arabic Sign Language Dynamic Gesture Recognition Models,” Heliyon, vol. 6, no. 3, p. e03554, Mar. 2020. https://doi.org/10.1016/j.heliyon.2020.e03554 Search in Google Scholar

[18] M. Mohandes, S. Aliyu, and M. Deriche, “Arabic Sign Language Recognition Using the Leap Motion Controller,” in 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE), Jun. 2014, pp. 960–965. https://doi.org/10.1109/isie.2014.6864742 Search in Google Scholar

[19] A. Karacı, K. Akyol, and Y. Gültepe, “Turkish Sign Language Alphabet Recognition with Leap Motion,” in Proceedings of the International Conference on Advanced Technologies, Computer Engineering and Science (ICATCES’18)’, May 2018, pp. 189–192. Search in Google Scholar

[20] A. Vaitkevičius, M. Taroza, T. Blažauskas, R. Damaševičius, R. Maskeliūnas, and M. Woźniak, “Recognition of American Sign Language Gestures in a Virtual Reality Using Leap Motion,” Applied Sciences, vol. 9, no. 3, pp. 445, Jan. 2019. https://doi.org/10.3390/app9030445 Search in Google Scholar

[21] B. K. Dedeturk and B. Akay, “Spam Filtering Using a Logistic Regression Model Trained by an Artificial Bee Colony Algorithm,” Applied Soft Computing, vol. 91, p. 106229, Jun. 2020. https://doi.org/10.1016/j.asoc.2020.106229 Search in Google Scholar

[22] O. Sagi and L. Rokach, “Ensemble Learning: A Survey,” Wiley Interdiscip. Rev. Data Min. Knowl. Discov., vol. 8, no. 4, Feb. 2018. https://doi.org/10.1002/widm.1249 Search in Google Scholar

[23] A. Karacı and A. Kemal, “Classification of Alcohols Obtained From QCM Sensors Using Hybrid Ensemble Classifier’, in 2nd International Turkish World Engineering and Science Congress, Nov. 2019, pp. 159–163. Search in Google Scholar

[24] A. Karaci, A. Caglar, B. Aydinli, and S. Pekol, “The Pyrolysis Process Verification of Hydrogen Rich Gas (H–rG) Production by Artificial Neural Network (ANN),” International Journal of Hydrogen Energy, vol. 41, no. 8, pp. 4570–4578, Mar. 2016. https://doi.org/10.1016/j.ijhydene.2016.01.094 Search in Google Scholar

[25] A. Karacı, “Self-Care Problems Classification of Children with Physical and Motor Disability by Deep Neural Networks,” J. Polytech., vol. 23, no. 2, pp. 333–341, 2020. Search in Google Scholar

[26] A. Karaci, “Predicting Breast Cancer with Deep Neural Networks” in Lecture Notes on Data Engineering and Communications Technologies, D. Hemanth, U. Kose, Eds. 2020, pp. 996–1003. Search in Google Scholar

[27] S. Ozbay, and M. Safar, “Real-Time Sign Languages Recognition Based on Hausdorff Distance, Hu Invariants and Neural Network,” in 2017 International Conference on Engineering and Technology (ICET), Aug. 2017, pp. 1–8. https://doi.org/10.1109/icengtechnol.2017.8308204 Search in Google Scholar

[28] D. Naglot and M. Kulkarni, “Real Time Sign Language Recognition Using the Leap Motion Controller,” in 2016 International Conference on Inventive Computation Technologies (ICICT), Aug. 2016, pp. 1–5. https://doi.org/10.1109/inventive.2016.7830097 Search in Google Scholar

[29] P. Kumar, R. Saini, S. K. Behera, D. P. Dogra, and P. P. Roy, “Real-Time Recognition of Sign Language Gestures and Air-Writing Using Leap Motion,” in 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), May 2017, pp. 157–160. https://doi.org/10.23919/mva.2017.7986825 Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo