Akses terbuka Jurnal SINTA 1
Multimodal music emotion recognition in Indonesian songs based on CNN-LSTM, XLNet transformers
Bulletin of Electrical Engineering and Informatics · Vol. 12 No. 1 (2022) · hlm. 355–364
- ISSN
- 2089-3191, 2302-9285
- Terbit
- 2022-11-07
- Penerbit
- Institute of Advanced Engineering and Science (IAES)
- Bahasa
- en
Abstrak
Music carries emotional information and allows the listener to feel the emotions contained in the music. This study proposes a multimodal music emotion recognition (MER) system using Indonesian song and lyrics data. In the proposed multimodal system, the audio data will use the mel spectrogram feature, and the lyrics feature will be extracted by going through the tokenizing process from XLNet. Convolutional long short term memory network (CNN-LSTM) performs the audio classification task, while XLNet transformers performs the lyrics classification task. The outputs of the two classification tasks are probability weight and actual prediction with the value of positive, neutral, and negative emotions, which are then combined using the stacking ensemble method. The combined output will be trained into an artificial neural network (ANN) model to get the best probability weight output. The multimodal system achieves the best performance with an accuracy of 80.56%. The results showed that the multimodal method of recognizing musical emotions gave better performance than the single modal method. In addition, hyperparameter tuning can affect the performance of multimodal systems.
Kata kunci
Jaringan kolaborasi para penulis
31 penulis · 50 tautan kepenulisan bersama · 12 lintas organisasi
Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.
- Binus University
- Universitas Muhammadiyah Kalimantan Timur
- Syarif Hidayatullah State Islamic University Jakarta
- Andalas University
- Afiliasi tidak diketahui
- Organisasi berbeda
- Organisasi sama
- Afiliasi tidak diketahui
- Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
| Rekan penulis | Organisasi | Artikel bersama |
|---|---|---|
| Muhammad Farhan Fadhil | Binus University | 2 |
| Maimunah Maskur | Binus University | 1 |
| Hendra Kurniawan | Binus University | 1 |
| Suma'inna Suma'inna | Afiliasi tidak diketahui | 1 |
| Muhammad Manaqib | Syarif Hidayatullah State Islamic University Jakarta | 1 |
| Bhakti Prabandyo Wicaksono | Binus University | 1 |
| Hendra Saputra | Andalas University | 1 |
| Faldi Faldi | Universitas Muhammadiyah Kalimantan Timur | 1 |
| Ferry Fadzlul Rahman | Universitas Muhammadiyah Kalimantan Timur | 1 |
| Sayekti Harits Suryawan | Universitas Muhammadiyah Kalimantan Timur | 1 |
| Wawan Joko Pranoto | Universitas Muhammadiyah Kalimantan Timur | 1 |
| Fathur Rahman | Universitas Muhammadiyah Kalimantan Timur | 1 |
| Satrio Sadewo | Binus University | 1 |
| Yasinta Roesmiatun Purnamadewi | Afiliasi tidak diketahui | 1 |
| Ahmad Iqbal Abdurrahman | Binus University | 1 |
| Panji Wijonarko | Binus University | 1 |
| Yoga Tanoko | Binus University | 1 |
| Jonathan Christian Setyono | Binus University | 1 |
| Gede Putra Kusuma | Binus University | 1 |
| Francisco Calvin Arnel Ferano | Binus University | 1 |
| Khasyi Al Mukarram | Binus University | 1 |
| M. Anang Mukhlas | Binus University | 1 |
| Stefanus Michael | Binus University | 1 |
| Ebenhaiser Jonathan Caprisiano | Binus University | 1 |
| Muhammad Hafizh Ramadhansyah | Binus University | 1 |
| Juan Hebert | Binus University | 1 |
| Michael J. Henry | Binus University | 1 |
| Wiryanata Chandra | Binus University | 1 |
| Mhd Taufik | Binus University | 1 |
Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.