Main Article Content
Abstract
Difficulty in communication is an obstacle for deaf friends who cannot learn the language orally or acquire normal speech skills. The development of sign language gesture recognition technology is an important step to improve accessibility and social integration for the deaf community. The use of MediaPipe Holistic Keypoints and deep learning techniques provides significant potential in recognizing and understanding sign language gestures. The main objective of this study is to classify Indonesian Sign Language (BISINDO) gestures using MediaPipe Holistic Keypoints and a deep learning approach to identify basic words in sign language. By extracting features using mediapipe holistic and sending them to the LSTM 6 hidden layer model with 70:30 split train test and 250 epochs, an accuracy of 68% was produced. This is due to the limited number of datasets taken for the study.
Keywords
Article Details
References
- Akshit Tayade, Halder, A., 2021. Real-time Vernacular Sign Language Recognition using MediaPipe and Machine Learning. https://doi.org/10.13140/RG.2.2.32364.03203
- Aljabar, A., Suharjito, S., 2020. BISINDO (Bahasa Isyarat Indonesia) Sign Language Recognition Using CNN and LSTM. Adv. Sci. Technol. Eng. Syst. J. 5, 282–287. https://doi.org/10.25046/aj050535
- Al-qurishi, M., Khalid, T., Souissi, R., 2021. Deep Learning for Sign Language Recognition : Current Techniques , Benchmarks , and Open Issues. IEEE Access PP, 1. https://doi.org/10.1109/ACCESS.2021.3110912
- Badarinath, Shamitha, 2023. Sign Language Recognition utilizing LSTM and Mediapipe for Dynamic Gestures of ISL. Int. J. Multidiscip. Res. 5, 6868. https://doi.org/10.36948/ijfmr.2023.v05i05.6868
- Bazarevsky, V., Zhang, F., 2020. BlazePose : On-device Real-time Body Pose tracking.
- Bora, J., Dehingia, S., Boruah, A., Chetia, A.A., Gogoi, D., 2023. Real-time Assamese Sign Language Recognition using MediaPipe and Deep Learning. Procedia Comput. Sci. 218, 1384–1393. https://doi.org/10.1016/j.procs.2023.01.117
- Jiang, S., Sun, B., Wang, L., Bai, Y., Li, K., Fu, Y., 2021. Skeleton Aware Multi-modal Sign Language Recognition.
- Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.-L., Yong, M.G., Lee, J., Chang, W.-T., Hua, W., Georg, M., Grundmann, M., 2019. MediaPipe: A Framework for Building Perception Pipelines.
- Moetia Putri, H., Fuadi, W., 2020. Pendeteksian Bahasa Isyarat Indonesia Secara Real-time menggunakan Long Short Term Memory (LSTM). Tts 1, 1–13.
- Qiang Zhu, Mei-Chen Yeh, Kwang-Ting Cheng, Avidan, S., 2006. Fast Human Detection Using a Cascade of Histograms of Oriented Gradients, in: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR’06). Presented at the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR’06), IEEE, New York, NY, USA, pp. 1491–1498. https://doi.org/10.1109/CVPR.2006.119
- Shamitha S H, S.H., Badarinath K, K., 2023. Sign Language Recognition utilizing LSTM and Mediapipe for Dynamic Gestures of ISL. Int. J. Multidiscip. Res. 5, 6868. https://doi.org/10.36948/ijfmr.2023.v05i05.6868
References
Akshit Tayade, Halder, A., 2021. Real-time Vernacular Sign Language Recognition using MediaPipe and Machine Learning. https://doi.org/10.13140/RG.2.2.32364.03203
Aljabar, A., Suharjito, S., 2020. BISINDO (Bahasa Isyarat Indonesia) Sign Language Recognition Using CNN and LSTM. Adv. Sci. Technol. Eng. Syst. J. 5, 282–287. https://doi.org/10.25046/aj050535
Al-qurishi, M., Khalid, T., Souissi, R., 2021. Deep Learning for Sign Language Recognition : Current Techniques , Benchmarks , and Open Issues. IEEE Access PP, 1. https://doi.org/10.1109/ACCESS.2021.3110912
Badarinath, Shamitha, 2023. Sign Language Recognition utilizing LSTM and Mediapipe for Dynamic Gestures of ISL. Int. J. Multidiscip. Res. 5, 6868. https://doi.org/10.36948/ijfmr.2023.v05i05.6868
Bazarevsky, V., Zhang, F., 2020. BlazePose : On-device Real-time Body Pose tracking.
Bora, J., Dehingia, S., Boruah, A., Chetia, A.A., Gogoi, D., 2023. Real-time Assamese Sign Language Recognition using MediaPipe and Deep Learning. Procedia Comput. Sci. 218, 1384–1393. https://doi.org/10.1016/j.procs.2023.01.117
Jiang, S., Sun, B., Wang, L., Bai, Y., Li, K., Fu, Y., 2021. Skeleton Aware Multi-modal Sign Language Recognition.
Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.-L., Yong, M.G., Lee, J., Chang, W.-T., Hua, W., Georg, M., Grundmann, M., 2019. MediaPipe: A Framework for Building Perception Pipelines.
Moetia Putri, H., Fuadi, W., 2020. Pendeteksian Bahasa Isyarat Indonesia Secara Real-time menggunakan Long Short Term Memory (LSTM). Tts 1, 1–13.
Qiang Zhu, Mei-Chen Yeh, Kwang-Ting Cheng, Avidan, S., 2006. Fast Human Detection Using a Cascade of Histograms of Oriented Gradients, in: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR’06). Presented at the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR’06), IEEE, New York, NY, USA, pp. 1491–1498. https://doi.org/10.1109/CVPR.2006.119
Shamitha S H, S.H., Badarinath K, K., 2023. Sign Language Recognition utilizing LSTM and Mediapipe for Dynamic Gestures of ISL. Int. J. Multidiscip. Res. 5, 6868. https://doi.org/10.36948/ijfmr.2023.v05i05.6868