Melola Discovery

← Melola Discovery

Jurnal SINTA 1

Bengali Word Detection from Lip Movements Using Mask RCNN and Generalized Linear Model

Indonesian Journal of Electical Engineering and Informatics (IJEEI) · Vol. 12 No. 2 (2024) · hlm. 373–383

DOI
10.52549/ijeei.v12i2.5088
ISSN
2089-3272, 2089-3272
Terbit
2024-06-28
Penerbit
IAES Indonesian Section
Bahasa
en

Abstrak

Speech processing with the help of lip detection and lip reading is an advancing field. For this, we need proper algorithms and techniques to detect lips and movements of lips perfectly. Lip detection and configuration are the most important parts of speech recognition. In this paper, we focus on detecting the lip segment properly. Mask R-CNN (Regional Convolutional Neural Network) performs object detection and instance segmentation per video frame to detect the lip segment. The process of mask R-CNN adds only a small overhead to Faster R-CNN and is quite simple to train, running at 5 frames per second. The Mask R-CNN involves keypoint detection which helps to extract the location of the lip landmarks pixel by pixel. Once the lip region is extracted and the landmarks are highlighted, we observe how the lip landmarks change as the object's lips move over time to each Bengali word. The keypoint changes that are observed during each millisecond are then the landmarks used to train the GLM (Generalized Linear Model). In addition, we compare the performance of GLM with Naive Bayes, Logistic Regression, and Decision Tree. The GLM has exhibited the highest 91.8% accuracy, whereas the Naive Bayes, Logistic Regression, and Decision Tree show the accuracy of 87.1%, 38.3%, and 82.2%, respectively.

Kata kunci

Jaringan kolaborasi para penulis

40 penulis · 91 tautan kepenulisan bersama · 57 lintas organisasi · menampilkan 40 yang paling banyak terhubung

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • BRAC University
  • University of Technology Malaysia
  • Chittagong University of Engineering & Technology
  • Woosong University
  • Hankuk University of Foreign Studies
  • Multimedia University
  • Bangladesh University of Business and Technology
  • Organisasi lain
  • Afiliasi tidak diketahui
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama
Md. Monirul IslamHankuk University of Foreign Studies4
Md Roman BhuiyanMultimedia University2
Faisal Bin Al AbidUniversity of Technology Malaysia2
Aryati Binti BakriUniversity of Technology Malaysia2
Shefayatuj Johara ChowdhuryBangladesh University of Business and Technology2
Rubel BiswasBRAC University1
Md Junayed HasanBRAC University1
Nibras Ar RakibBRAC University1
SM Zamshed FarhanAfiliasi tidak diketahui1
Md. Mashrur Bari SobhanBRAC University1
Arafat HabibBRAC University1
Rafid AhmedBRAC University1
Md. Sazzadul IslamBRAC University1
Fabian Parsia GeorgeBRAC University1
Istiaque Mannafee ShaikatBRAC University1
Prommy Sultana HossainBRAC University1
Mohammad Zavid ParvezBRAC University1
Amitabha ChakrabartyBRAC University1
Zannatul FerdoushBRAC University1
Booshra Nazifa MahmudBRAC University1
Mohammad Abul KashemUniversity of Dhaka1
Md. Hasan FurhadAfiliasi tidak diketahui1
Ripon Kumar ChakraborttyUNSW Canberra1
Michael J. RyanAfiliasi tidak diketahui1
Iqbal Hasan SarkerChittagong University of Engineering & Technology1
Mohammod Abul KashemDhaka University of Engineering & Technology1
Shabbir AhmadAfiliasi tidak diketahui1
Md. Golam Rabiul AlamBRAC University1
Tasnim Sakib AponBRAC University1
Md. Saiful IslamChittagong University of Engineering & Technology1
Mohammad MohiuddinChittagong University of Engineering & Technology1
Hasin Jawad AliIslamic University of Technology1
Darmawan SatyanandaUniversity of Technology Malaysia1
Ferdaus Anam JibonInternational University of Business Agriculture and Technology1
Kingkar Prosad GhoshR. P. Shaha University1
Shahina HaqueDaffodil International University1
Md. Suhag AliJahangirnagar University1
Shatabdi SarkerAmerican International University-Bangladesh1

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) http://section.iaesonline.com/index.php/IJEEI/oai, pertama di-harvest 8 Nov 2026, terakhir terlihat 8 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en">Bengali Word Detection from Lip Movements Using Mask  RCNN and Generalized Linear Model</dc:title>
	<dc:creator xml:lang="en">Bhuiyan, Abul Bashar</dc:creator>
	<dc:creator xml:lang="en">Uddin, Jia</dc:creator>
	<dc:subject xml:lang="en">Word Detection</dc:subject>
	<dc:subject xml:lang="en">Lip Movements</dc:subject>
	<dc:subject xml:lang="en">Machine learning</dc:subject>
	<dc:subject xml:lang="en">Image Segmentation</dc:subject>
	<dc:subject xml:lang="en">Accuracy</dc:subject>
	<dc:description xml:lang="en">Speech processing with the help of lip detection and lip reading is an advancing field. For this, we need proper algorithms and techniques to detect lips and movements of lips perfectly. Lip detection and configuration are the most important parts of speech recognition. In this paper, we focus on detecting the lip segment properly. Mask R-CNN (Regional Convolutional Neural Network) performs object detection and instance segmentation per video frame to detect the lip segment. The process of mask R-CNN adds only a small overhead to Faster R-CNN and is quite simple to train, running at 5 frames per second. The Mask R-CNN involves keypoint detection which helps to extract the location of the lip landmarks pixel by pixel. Once the lip region is extracted and the landmarks are highlighted, we observe how the lip landmarks change as the object's lips move over time to each Bengali word. The keypoint changes that are observed during each millisecond are then the landmarks used to train the GLM (Generalized Linear Model). In addition, we compare the performance of GLM with Naive Bayes, Logistic Regression, and Decision Tree. The GLM has exhibited the highest 91.8% accuracy, whereas the Naive Bayes, Logistic Regression, and Decision Tree show the accuracy of 87.1%, 38.3%, and 82.2%, respectively.</dc:description>
	<dc:publisher xml:lang="en">IAES Indonesian Section</dc:publisher>
	<dc:date>2024-06-28</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://section.iaesonline.com/index.php/IJEEI/article/view/5088</dc:identifier>
	<dc:identifier>10.52549/ijeei.v12i2.5088</dc:identifier>
	<dc:source xml:lang="en">Indonesian Journal of Electrical Engineering and Informatics (IJEEI); Vol. 12 No. 2: June 2024; 373-383</dc:source>
	<dc:source>2089-3272</dc:source>
	<dc:source>2089-3272</dc:source>
	<dc:language>en</dc:language>
	<dc:relation>https://section.iaesonline.com/index.php/IJEEI/article/view/5088/917</dc:relation>
</oai_dc:dc>