Melola Discovery

← Melola Discovery

Jurnal SINTA 1

Enhanced Emotion Recognition in Videos: A Convolutional Neural Network Strategy for Human Facial Expression Detection and Classification

Indonesian Journal of Electical Engineering and Informatics (IJEEI) · Vol. 11 No. 1 (2023) · hlm. 286–299

DOI
10.52549/ijeei.v11i1.4449
ISSN
2089-3272, 2089-3272
Terbit
2023-03-24
Penerbit
IAES Indonesian Section
Bahasa
en

Abstrak

The human face is essential in conveying emotions, as facial expressions serve as effective, natural, and universal indicators of emotional states. Automated emotion recognition has garnered increasing interest due to its potential applications in various fields, such as human-computer interaction, machine learning, robotic control, and driver emotional state monitoring. With artificial intelligence and computational power advancements, visual emotion recognition has become a prominent research area. Despite extensive research employing machine learning algorithms like convolutional neural networks (CNN), challenges remain concerning input data processing, emotion classification scope, data size, optimal CNN configurations, and performance evaluation. To address these issues, we propose a comprehensive CNN-based model for real-time detection and classification of five primary emotions: anger, happiness, neutrality, sadness, and surprise. We employ the Amsterdam Dynamic Facial Expression Set – Bath Intensity Variations (ADFES-BIV) video dataset, extracting image frames from the video samples. Image processing techniques such as histogram equalization, color conversion, cropping, and resizing are applied to the frames before labeling. The Viola-Jones algorithm is then used for face detection on the processed grayscale images. We develop and train a CNN on the processed image data, implementing dropout, batch normalization, and L2 regularization to reduce overfitting. The ideal hyperparameters are determined through trial and error, and the model's performance is evaluated. The proposed model achieves a recognition accuracy of 99.38%, with the confusion matrix, recall, precision, F1 score, and processing time further quantifying its performance characteristics. The model's generalization performance is assessed using images from the Warsaw Set of Emotional Facial Expression Pictures (WSEFEP) and Extended Cohn-Kanade Database (CK+) datasets. The results demonstrate the efficiency and usability of our proposed approach, contributing valuable insights into real-time visual emotion recognition.

Kata kunci

Jaringan kolaborasi para penulis

40 penulis · 106 tautan kepenulisan bersama · 30 lintas organisasi · menampilkan 40 yang paling banyak terhubung

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • International Islamic University Malaysia
  • Yogyakarta State University
  • Universitas Gadjah Mada
  • University of Kuala Lumpur
  • University of Indonesia
  • International Centre for Education in Islamic Finance
  • University of Sousse
  • Organisasi lain
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama
Aris NasuhaYogyakarta State University5
Anggun WinursitoYogyakarta State University5
Siti Hajar YusoffInternational Islamic University Malaysia5
Hasmah MansorInternational Islamic University Malaysia4
Malik Arman MorshidiInternational Islamic University Malaysia4
Ardy Seto PriambodoYogyakarta State University4
Huda Adibah Mohd RamliInternational Islamic University Malaysia3
Rashidah Funke OlanrewajuInternational Islamic University Malaysia3
Suriza Ahmad ZabidiInternational Islamic University Malaysia3
Md. Rafiqul IslamInternational Islamic University Malaysia2
Noreha Abdul MalikInternational Islamic University Malaysia2
Elsheikh Mohamed Ahmed ElsheikhInternational Islamic University Malaysia2
Mohammed H. A. ArafaInternational Islamic University Malaysia2
Kazi Istiaque AhmedInternational Islamic University Malaysia2
Diyah PutrianiUniversitas Gadjah Mada2
Gairuzazmi Mat GhaniInternational Islamic University Malaysia2
Amart SulongInternational Islamic University Malaysia2
Othman Omran KhalifaInternational Islamic University Malaysia2
Hassan DaoUniversity of Kuala Lumpur2
Abdul MutholibInternational Islamic University Malaysia2
Intan Rahmithul Husna YaldiInternational Islamic University Malaysia2
Yasmin Mahira Saiful MunirInternational Islamic University Malaysia2
Tika ArundinaUniversity of Indonesia2
Mohd Azmi OmarInternational Centre for Education in Islamic Finance2
Nur Atikah Muhamat SalehInternational Islamic University Malaysia2
Israth Jahan ChowdhuryInternational Islamic University Malaysia1
Nur Aishah ZainalInternational Islamic University Malaysia1
W. IdrisInternational Islamic University Malaysia1
Siti Noorjannah IbrahimInternational Islamic University Malaysia1
Islam Md. RafiqulInternational Islamic University Malaysia1
Ali Kadhim LwasInternational Islamic University Malaysia1
Md Moktarul AlamInternational Islamic University Malaysia1
Jalel ChebilUniversity of Sousse1
Jit Singh MandeepNational University of Malaysia1

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) http://section.iaesonline.com/index.php/IJEEI/oai, pertama di-harvest 8 Nov 2026, terakhir terlihat 8 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en">Enhanced Emotion Recognition in Videos: A Convolutional Neural Network Strategy for Human Facial Expression Detection and Classification</dc:title>
	<dc:creator xml:lang="en">Ashraf, Arselan</dc:creator>
	<dc:creator xml:lang="en">Gunawan, Teddy Surya</dc:creator>
	<dc:creator xml:lang="en">Arifin, Fatchul</dc:creator>
	<dc:creator xml:lang="en">Kartiwi, Mira</dc:creator>
	<dc:creator xml:lang="en">Sophian, Ali</dc:creator>
	<dc:creator xml:lang="en">Habaebi, Mohamed Hadi</dc:creator>
	<dc:subject xml:lang="en">artificial intelligence</dc:subject>
	<dc:subject xml:lang="en">convolutional neural networks</dc:subject>
	<dc:subject xml:lang="en">emotion recognition</dc:subject>
	<dc:subject xml:lang="en">human-computer interaction</dc:subject>
	<dc:subject xml:lang="en">machine learning</dc:subject>
	<dc:description xml:lang="en">The human face is essential in conveying emotions, as facial expressions serve as effective, natural, and universal indicators of emotional states. Automated emotion recognition has garnered increasing interest due to its potential applications in various fields, such as human-computer interaction, machine learning, robotic control, and driver emotional state monitoring. With artificial intelligence and computational power advancements, visual emotion recognition has become a prominent research area. Despite extensive research employing machine learning algorithms like convolutional neural networks (CNN), challenges remain concerning input data processing, emotion classification scope, data size, optimal CNN configurations, and performance evaluation. To address these issues, we propose a comprehensive CNN-based model for real-time detection and classification of five primary emotions: anger, happiness, neutrality, sadness, and surprise. We employ the Amsterdam Dynamic Facial Expression Set – Bath Intensity Variations (ADFES-BIV) video dataset, extracting image frames from the video samples. Image processing techniques such as histogram equalization, color conversion, cropping, and resizing are applied to the frames before labeling. The Viola-Jones algorithm is then used for face detection on the processed grayscale images. We develop and train a CNN on the processed image data, implementing dropout, batch normalization, and L2 regularization to reduce overfitting. The ideal hyperparameters are determined through trial and error, and the model's performance is evaluated. The proposed model achieves a recognition accuracy of 99.38%, with the confusion matrix, recall, precision, F1 score, and processing time further quantifying its performance characteristics. The model's generalization performance is assessed using images from the Warsaw Set of Emotional Facial Expression Pictures (WSEFEP) and Extended Cohn-Kanade Database (CK+) datasets. The results demonstrate the efficiency and usability of our proposed approach, contributing valuable insights into real-time visual emotion recognition.</dc:description>
	<dc:publisher xml:lang="en">IAES Indonesian Section</dc:publisher>
	<dc:date>2023-03-24</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://section.iaesonline.com/index.php/IJEEI/article/view/4449</dc:identifier>
	<dc:identifier>10.52549/ijeei.v11i1.4449</dc:identifier>
	<dc:source xml:lang="en">Indonesian Journal of Electrical Engineering and Informatics (IJEEI); Vol. 11 No. 1: March 2023; 286-299</dc:source>
	<dc:source>2089-3272</dc:source>
	<dc:source>2089-3272</dc:source>
	<dc:language>en</dc:language>
	<dc:relation>https://section.iaesonline.com/index.php/IJEEI/article/view/4449/822</dc:relation>
</oai_dc:dc>