Melola Discovery

← Melola Discovery

Akses terbuka Jurnal SINTA 1

Deep Learning-Based Continuous Sign Language Recognition

Journal of Robotics and Control (JRC) · Vol. 6 No. 3 (2025) · hlm. 1106–1119

DOI
10.18196/jrc.v6i3.25881
ISSN
2715-5072, 2715-5056
Terbit
2025-05-08
Penerbit
Universitas Muhammadiyah Yogyakarta
Bahasa
eng
Lisensi
creativecommons.org/licenses/by-sa/4.0

Abstrak

This study focuses on the development of a continuous sign language recognition system based on deep neural network models. A new Kazakh Sign Language (QazSL) dataset is created. DL models for continuous KazSL are developed, their accuracy and robustness under different environmental conditions are analyzed, and an optimized model algorithm to improve sign recognition processes are proposed. The main goal is to improve gesture recognition accuracy, account for gesture variability and environmental conditions, and promote the development of adaptive technologies for low-resource languages. This paper proposes a QazSL recognition system using an YOLOv8n and optimized 2DCNN models to improve accessibility for the hearing impaired. The optimized 2DCNN method includes optimal data preprocessing techniques and new training architecture, followed by model training and testing with precision, recall, and accuracy metrics. The proposed systems were trained using an opencourse K-RSL dataset with 5 signers and a newly created QazSL dataset, recorded by 7 signers. The test accuracy of gesture recognition are 98.12% for Yolov8n and 98, 57% for 2DCNN, indicating the robustness and capability of the models for realtime application. Certain issues, such as background variation and gesture consistency, were found to affect recognition under different conditions. This research contributes to the development of AI-based assistive technology to facilitate social inclusion and access to communication for deaf and hard-of-hearing people. By addressing the challenges identified in gesture recognition, this study paves the way for more reliable interactions between users and technology. Future work will focus on optimizing the model further to enhance its performance in varied environments and to expand its applicability across different languages and sign systems.

Kata kunci

Jaringan kolaborasi para penulis

23 penulis · 78 tautan kepenulisan bersama · 49 lintas organisasi

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • L. N. Gumilyov Eurasian National University
  • Astana IT University
  • Shandong University
  • Astana International University
  • M.Auezov South Kazakhstan State University
  • Kazakh-British Technical University
  • South Kazakhstan Medical Academy
  • Organisasi lain
  • Afiliasi tidak diketahui
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama
Tamara Kokenovna ZhukabayevaL. N. Gumilyov Eurasian National University5
Dilaram BaumuratovaAstana International University3
Aigul AdamovaAstana IT University2
Nurdaulet KarabayevL. N. Gumilyov Eurasian National University2
Akbota YerzhanovaS.Seifullin Kazakh Agro Technical University1
Dinara KaibassovaAstana IT University1
Zhanat ManbetovaAfiliasi tidak diketahui1
Akmaral KassymovaAfiliasi tidak diketahui1
Zhuldyz TashenovaL. N. Gumilyov Eurasian National University1
Saduakas BaizhumanovAfiliasi tidak diketahui1
Kulaisha AikhynbayM.Auezov South Kazakhstan State University1
Yerik MardenovAstana International University1
Erik MardenovAfiliasi tidak diketahui1
Khu Ven–TsenM.Auezov South Kazakhstan State University1
Oliver DiasUniversitat de Barcelona1
Bauyrzhan BerlikozhaShandong University1
Azamat ZhamanovNorth American University1

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) https://journal.umy.ac.id/index.php/jrc/oai, pertama di-harvest 9 Nov 2026, terakhir terlihat 16 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en-US">Deep Learning-Based Continuous Sign Language Recognition</dc:title>
	<dc:creator>Zholshiyeva, Lazzat</dc:creator>
	<dc:creator>Zhukabayeba, Tamara</dc:creator>
	<dc:creator>Serek, Azamat </dc:creator>
	<dc:creator>Duisenbek, Ramazan</dc:creator>
	<dc:creator>Berdieva, Meruert</dc:creator>
	<dc:creator>Shapay, Nurshapagat</dc:creator>
	<dc:subject xml:lang="en-US">Computer Vision</dc:subject>
	<dc:subject xml:lang="en-US">Sign Language</dc:subject>
	<dc:subject xml:lang="en-US">Real-Time Recognition</dc:subject>
	<dc:subject xml:lang="en-US">Deep Learning</dc:subject>
	<dc:subject xml:lang="en-US">2DCNN</dc:subject>
	<dc:subject xml:lang="en-US">Yolo Optimization</dc:subject>
	<dc:description xml:lang="en-US">This study focuses on the development of a continuous sign language recognition system based on deep neural network models. A new Kazakh Sign Language (QazSL) dataset is created. DL models for continuous KazSL are developed, their accuracy and robustness under different environmental conditions are analyzed, and an optimized model algorithm to improve sign recognition processes are proposed. The main goal is to improve gesture recognition accuracy, account for gesture variability and environmental conditions, and promote the development of adaptive technologies for low-resource languages. This paper proposes a QazSL recognition system using an YOLOv8n and optimized 2DCNN models to improve accessibility for the hearing impaired. The optimized 2DCNN method includes optimal data preprocessing techniques and new training architecture, followed by model training and testing with precision, recall, and accuracy metrics. The proposed systems were trained using an opencourse K-RSL dataset with 5 signers and a newly created QazSL dataset, recorded by 7 signers. The test accuracy of gesture recognition are 98.12% for Yolov8n and 98, 57% for 2DCNN, indicating the robustness and capability of the models for realtime application. Certain issues, such as background variation and gesture consistency, were found to affect recognition under different conditions. This research contributes to the development of AI-based assistive technology to facilitate social inclusion and access to communication for deaf and hard-of-hearing people. By addressing the challenges identified in gesture recognition, this study paves the way for more reliable interactions between users and technology. Future work will focus on optimizing the model further to enhance its performance in varied environments and to expand its applicability across different languages and sign systems.</dc:description>
	<dc:publisher xml:lang="en-US">Universitas Muhammadiyah Yogyakarta</dc:publisher>
	<dc:date>2025-05-08</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://journal.umy.ac.id/index.php/jrc/article/view/25881</dc:identifier>
	<dc:identifier>10.18196/jrc.v6i3.25881</dc:identifier>
	<dc:source xml:lang="en-US">Journal of Robotics and Control (JRC); Vol. 6 No. 3 (2025); 1106-1119</dc:source>
	<dc:source>2715-5072</dc:source>
	<dc:source>2715-5056</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://journal.umy.ac.id/index.php/jrc/article/view/25881/11513</dc:relation>
	<dc:rights xml:lang="en-US">Copyright (c) 2025 Lazzat Zholshiyeva, Tamara Zhukabayeba, Azamat  Serek, Ramazan Duisenbek, Meruert Berdieva, Nurshapagat Shapay</dc:rights>
	<dc:rights xml:lang="en-US">https://creativecommons.org/licenses/by-sa/4.0</dc:rights>
</oai_dc:dc>