Melola Discovery

← Melola Discovery

Jurnal

Sign Language Detection Application to Facilitate Communication for Speech and Hearing Impaired Individuals Based on Computer Vision Technology Using Inception Resnetv2

Rekayasa · Vol. 20 No. 2 (2022) · hlm. 108–117

DOI
10.15294/rekayasa.v20i2.19433
ISSN
2527-6964, 0216-6631
Terbit
2022-12-31
Penerbit
Universitas Negeri Semarang
Bahasa
eng

Abstrak

Communication is a fundamental human need, yet not everyone possesses perfect communication abilities. People Communication is a fundamental human need, yet individuals with speech and hearing impairments face challenges due to the limited understanding of sign language among the general public. This study applies Artificial Intelligence and Computer Vision to enhance communication accessibility by detecting hand gestures and converting them into text. The lack of real-time sign language translation remains a barrier for individuals with disabilities. Existing systems often struggle with accuracy and device compatibility. This research develops and evaluates HARDISC, an Android-based application that recognizes letters A–Z through hand movement detection using a camera. The goal is to provide an effective and inclusive communication tool for the speech and hearing impaired. HARDISC utilizes Transfer Learning with Inception ResNetV2 and VGG16 for gesture classification. Image processing enables the camera to detect and translate hand movements into text. Model evaluation was based on accuracy, loss values, and device compatibility. Results show Inception ResNetV2 achieved 98.98% accuracy with a 0.0417 loss value, while VGG16 recorded 99.40% accuracy with a 0.0146 loss value, demonstrating high performance. HARDISC is compatible with Android KitKat 4 to Android 12, ensuring accessibility. This application provides an innovative, real-time solution to bridge communication gaps for individuals with speech and hearing impairments, improving their interaction with the general public.

Kata kunci

Jaringan kolaborasi para penulis

25 penulis · 62 tautan kepenulisan bersama

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • Diponegoro University
  • State University of Semarang
  • Afiliasi tidak diketahui
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama
Riana DM PutriAfiliasi tidak diketahui1
Isdiyarto IsdiyartoAfiliasi tidak diketahui1
Azizah MunawarohAfiliasi tidak diketahui1
Niko Dwi CahyoAfiliasi tidak diketahui1
Riana Defi Mahadji PutriState University of Semarang1
Rini Kartiko SariAfiliasi tidak diketahui1
Eko SupraptonoAfiliasi tidak diketahui1
Galang Wahid KurniawanAfiliasi tidak diketahui1
Mohamad IrfanAfiliasi tidak diketahui1
Sonhaji SonhajiAfiliasi tidak diketahui1
Ardhana Luthfi HidayattullahAfiliasi tidak diketahui1
Catur Rini WidyastutiAfiliasi tidak diketahui1
Radenrara Dewi Artanti PutriAfiliasi tidak diketahui1
Delvani Putri Faradilla HamidiAfiliasi tidak diketahui1
MohammadAfiliasi tidak diketahui1
Dwi Kharisma LuthfianiAfiliasi tidak diketahui1
Ade Rahmania YuningsihAfiliasi tidak diketahui1
Rahmat GernowoDiponegoro University1
R. Rizal IsnantoDiponegoro University1

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) https://journal.unnes.ac.id/journals/rekayasa/oai, pertama di-harvest 6 Nov 2026, terakhir terlihat 8 Nov 2026.
Metadata asli (oai_dc) 
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en">Sign Language Detection Application to Facilitate Communication for Speech and Hearing Impaired Individuals Based on Computer Vision Technology Using Inception Resnetv2</dc:title>
	<dc:creator xml:lang="en">Arfriandi, Arief</dc:creator>
	<dc:creator xml:lang="en">Prabowo, Yoga Agung</dc:creator>
	<dc:creator xml:lang="en">Najuda, Moh Dafi</dc:creator>
	<dc:creator xml:lang="en">Puspita, Tri Anggi Ratna</dc:creator>
	<dc:creator xml:lang="en">Virnanda, Shakira Wahyu</dc:creator>
	<dc:creator xml:lang="en">Sunardiyo, Said</dc:creator>
	<dc:subject xml:lang="en">Artificial Intelligence </dc:subject>
	<dc:subject xml:lang="en">Computer Vision</dc:subject>
	<dc:subject xml:lang="en">Disabilities </dc:subject>
	<dc:subject xml:lang="en">HARDISC</dc:subject>
	<dc:subject xml:lang="en">real-time</dc:subject>
	<dc:description xml:lang="en">Communication is a fundamental human need, yet not everyone possesses perfect communication abilities. People Communication is a fundamental human need, yet individuals with speech and hearing impairments face challenges due to the limited understanding of sign language among the general public. This study applies Artificial Intelligence and Computer Vision to enhance communication accessibility by detecting hand gestures and converting them into text. The lack of real-time sign language translation remains a barrier for individuals with disabilities. Existing systems often struggle with accuracy and device compatibility. This research develops and evaluates HARDISC, an Android-based application that recognizes letters A–Z through hand movement detection using a camera. The goal is to provide an effective and inclusive communication tool for the speech and hearing impaired. HARDISC utilizes Transfer Learning with Inception ResNetV2 and VGG16 for gesture classification. Image processing enables the camera to detect and translate hand movements into text. Model evaluation was based on accuracy, loss values, and device compatibility. Results show Inception ResNetV2 achieved 98.98% accuracy with a 0.0417 loss value, while VGG16 recorded 99.40% accuracy with a 0.0146 loss value, demonstrating high performance. HARDISC is compatible with Android KitKat 4 to Android 12, ensuring accessibility. This application provides an innovative, real-time solution to bridge communication gaps for individuals with speech and hearing impairments, improving their interaction with the general public.</dc:description>
	<dc:publisher xml:lang="en">Universitas Negeri Semarang</dc:publisher>
	<dc:date>2022-12-31</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en">Peer-reviewed Article</dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://journal.unnes.ac.id/journals/rekayasa/article/view/19433</dc:identifier>
	<dc:identifier>10.15294/rekayasa.v20i2.19433</dc:identifier>
	<dc:source xml:lang="en">Rekayasa; Vol. 20 No. 2 (2022); 108-117</dc:source>
	<dc:source>2527-6964</dc:source>
	<dc:source>0216-6631</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://journal.unnes.ac.id/journals/rekayasa/article/view/19433/3338</dc:relation>
	<dc:rights xml:lang="en">Copyright (c) 2022 Rekayasa : Jurnal Penerapan Teknologi dan Pembelajaran</dc:rights>
</oai_dc:dc>