Static Pakistani Sign Language Classification using Support Vector Machine

Shaheer Mirza; Sheikh Muhammad   Munaf; Shahid Ali; Muhammad Asif

doi:10.33317/ssurj.436

Authors

Shaheer Mirza Department of Biomedical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
Sheikh Muhammad Munaf Department of Software Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
Shahid Ali Department of Speech Language and Hearing Sciences, Faculty of Health Sciences, Ziauddin University, Karachi, Pakistan
Muhammad Asif Department of Electrical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan

DOI:

https://doi.org/10.33317/ssurj.436

Keywords:

Pattern Recognition, Pakistani Sign Language, Machine Learning, Image Processing

Abstract

In this study, a system is proposed that uses the Support Vector Machine (SVM) technique with Bag-of-Words (BoW) and recognizes static Pakistani Sign Language (PSL) alphabets. The application of the BoW technique with SVM, on a PSL images' dataset, has not been performed previously. Similarly, no publicly available dataset for PSL is available and previous studies have achieved a maximum classification accuracy of 91.98%. For this study, a total of 511 images are collected for 36 static PSL alphabet signs from a native signer. The Sign Language (SL) recognition system uses the collected images as input and converts them to grayscale. To segment the images, the system uses the thresholding technique and Speeded Up Robust Feature (SURF) to extract the features. The system uses K-means clustering to cluster the extracted features. To form the BoW, the system computes the Euclidean distance among SURF descriptors and clustered data. The system then uses 5-fold cross-validation to divide the codebooks obtained from the BoW into training and testing. The developed system yields an overall accuracy of 97.87% for the classification of static PSL signs at 1,500×1,500 image dimensions and 500 Bags.

References

World Health Organization. 2021. Deafness and Hearing Loss. Retrieved from: https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss

Pakistan Association of the Deaf. 2021. Deaf Statistic. Retrieved from: http://www.padeaf.org/quick-links/deaf-statistics

Kausar, S., Javed, M. Y., Tehsin, S., & Anjum, A. (2016). A novel mathematical modeling and parameterization for sign language classification. International Journal of Pattern Recognition and Artificial Intelligence, 30(05), 1650009.

Ameen, S., & Vadera, S. (2017). A convolutional neural network to classify American Sign Language fingerspelling from depth and colour images. Expert Systems, 34(3), e12197.

Singha, J., Roy, A., & Laskar, R. H. (2018). Dynamic hand gesture recognition using vision-based approach for human–computer interaction. Neural Computing and Applications, 29(4), 1129-1141.

Nasri, S., Behrad, A., & Razzazi, F. (2015). Spatio-temporal 3D surface matching for hand gesture recognition using ICP algorithm. Signal, Image and Video Processing, 9(5), 1205-1220.

Hikawa, H., & Kaida, K. (2014). Novel FPGA implementation of hand sign recognition system with SOM–Hebb classifier. IEEE Transactions on Circuits and Systems for Video Technology, 25(1), 153-166.

Rastgoo, R., Kiani, K., & Escalera, S. (2018). Multi-modal deep hand sign language recognition in still images using restricted Boltzmann machine. Entropy, 20(11), 809.

Ibrahim, N. B., Selim, M. M., & Zayed, H. H. (2018). An automatic Arabic sign language recognition system (ArSLRS). Journal of King Saud University-Computer and Information Sciences, 30(4), 470-477.

Jiang, X., & Zhang, Y. D. (2019). Chinese sign language fingerspelling via six-layer convolutional neural network with leaky rectified linear units for therapy and rehabilitation. Journal of Medical Imaging and Health Informatics, 9(9), 2031-2090.

Cui, R., Liu, H., & Zhang, C. (2019). A deep neural framework for continuous sign language recognition by iterative training. IEEE Transactions on Multimedia, 21(7), 1880-1891.

Koller, O., Zargaran, S., Ney, H., & Bowden, R. (2018). Deep sign: Enabling robust statistical continuous sign language recognition via hybrid CNN-HMMs. International Journal of Computer Vision, 126(12), 1311-1325.

Athira, P. K., Sruthi, C. J., & Lijiya, A. (2019). A signer independent sign language recognition with co-articulation elimination from live videos: an indian scenario. Journal of King Saud University-Computer and Information Sciences.

Kelly, D., McDonald, J., & Markham, C. (2010). A person independent system for recognition of hand postures used in sign language. Pattern Recognition Letters, 31(11), 1359-1368.

Shah, S. M. S., Naqvi, H. A., Khan, J. I., Ramzan, M., & Khan, H. U. (2018). Shape based pakistan sign language categorization using statistical features and support vector machines. IEEE Access, 6, 59242-59252.

Ahmed, H., Gilani, S. O., Jamil, M., Ayaz, Y., & Shah, S. I. A. (2016). Monocular vision-based signer-independent Pakistani sign language recognition system using supervised learning. Indian J. Sci. Technol, 9(25), 1-16.

Khan, N. S., Shahzada, A., Ata, S., Abid, A., Farooq, M. S., Mushtaq, M. T., & Khan, I. (2014). A vision based approach for Pakistan sign language alphabets recognition. La Pensée, 76 (03).

Shah, F., Shah, M. S., Akram, W., Manzoor, A., Mahmoud, R. O., & Abdelminaam, D. S. (2021). Sign language recognition using multiple kernel learning: A case study of Pakistan sign language. IEEE Access, 9, 67548-67558.

Saqib, S., Ditta, A., Khan, M. A., Kazmi, S. A. R., & Alquhayz, H. (2021). Intelligent dynamic gesture recognition using CNN empowered by edit distance. Cmc-Computers Materials & Continua, 66(2), pp. 2061-2076.

Zadghorban, M., & Nahvi, M. (2018). An algorithm on sign words extraction and recognition of continuous Persian sign language based on motion and shape features of hands. Pattern Analysis and Applications, 21(2), 323-335.

Elakkiya, R., & Selvamani, K. (2019). Subunit sign modeling framework for continuous sign language recognition. Computers & Electrical Engineering, 74, 379-390.

Klomsae, A., Auephanwiriyakul, S., & Theera-Umpon, N. (2017). A novel string grammar unsupervised possibilistic C-medians algorithm for sign language translation systems. Symmetry, 9(12), 321.

Joshi, G., Vig, R., & Singh, S. (2018). DCA‐based unimodal feature‐level fusion of orthogonal moments for Indian sign language dataset. IET Computer Vision, 12(5), 570-577.