Continuous Sign Language Recognition for Quranic Recitation by Deaf People Using Deep Learning

Abstract

This study proposes a deep learning-based system for recognizing Quranic recitation in the sign language, aimed at enhancing accessibility for the Deaf Muslim community. A central contribution is the construction of a novel dataset comprising videos from three Deaf signers performing Surah Al-Fatihah and Surah Al-Ikhlas, guided by the 2022 official Quranic sign language standard introduced by Indonesia’s Ministry of Religious Affairs. The recognition task is framed as a continuous sign language recognition (CSLR) problem to handle unsegmented input sequences. Five pre-trained convolutional neural networks—EfficientNet, GoogleNet, MobileNetV2, ResNet18, and ShuffleNet—were evaluated as visual feature extractors. These were followed by a temporal encoder composed of 1D CNN and BiLSTM, with sequence alignment performed using the Connectionist Temporal Classification (CTC). The experimental results show that ResNet18 and MobileNetV2 achieved the best performance with Word Error Rates (WER) of 5.00% and 7.92% on the test set, respectively. A cross-participant evaluation was also conducted to assess model generalization, although the results revealed performance gaps likely due to signer variation and limited data. The study highlights the suitability of lightweight and residual architectures for CSLR tasks in religious contexts and provides a benchmark for future research on inclusive sign language technologies. In cross-participant evaluation, the model achieved a validation WER of 8.44% on seen signers and 50.46% on an unseen signer, reflecting generalization challenges commonly observed in low-resource CSLR settings. The proposed system lays the groundwork for AI-assisted Quranic education tools tailored to the Deaf Muslim population.