Akses terbuka Jurnal SINTA 1
EmoVibe: AI-driven multimodal emotion analysis for mental health via social media dashboards
IAES International Journal of Artificial Intelligence · Vol. 14 No. 6 (2025) · hlm. 4565–4578
- ISSN
- 2252-8938, 2089-4872
- Terbit
- 2025-12-01
- Penerbit
- Institute of Advanced Engineering and Science
- Bahasa
- eng
Abstrak
Monitoring mental health via social media often utilizes unimodal approaches, such as sentiment analysis on text or single-staged image categorization, or executes early feature fusion. However, in real-world contexts where emotions are conveyed via text, emojis, and images, unimodal approach leads to obscured decision-making pathways and overall diminished performance. To overcome these limitations, we propose EmoVibe, a hybrid multimodal AI framework for emotive analysis. EmoVibe uses attention-based late fusion strategy, where text embeddings are generated from bidirectional encoder representations from transformers (BERT) and visual features are extracted by vision transformer. Subsequently, emoticon vectors linked to avatars are processed independently. Later, these independent data features are integrated at higher levels, enhancing interpretability and performance. In contrast to early fusion methods and integrated multimodal large language models (LLMs) like CLIP, Flamingo, GPT-4V, MentaLLaMA, and domain-adapted models like EmoBERTa, EmoVibe preserves modality-specific contexts without premature fusion. This architecture saves processing cost, allowing for clearer, unambiguous rationalization and explanations. EmoVibe outperforms unimodal baselines and early fusion models, obtaining 89.7% accuracy on GoEmotions, FER, and AffectNet, compared to BERT's 87.4% and ResNet-50's 84.2%, respectively. Furthermore, a customizable, real time, privacy-aware dashboard is created, supporting physicians and end users. This technology enables scalable and proactive intervention options and fosters user self-awareness of mental health.
Kata kunci
Jaringan kolaborasi para penulis
9 penulis · 21 tautan kepenulisan bersama · 4 lintas organisasi
Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.
- Symbiosis International University
- Bharati Vidyapeeth (Deemed to be University)
- Afiliasi tidak diketahui
- Organisasi berbeda
- Organisasi sama
- Afiliasi tidak diketahui
- Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
| Rekan penulis | Organisasi | Artikel bersama |
|---|---|---|
| Ketan V. Kotecha | Symbiosis International University | 1 |
| Nikita Chaudhari | Symbiosis International University | 1 |
| Payal S. Kadam | Bharati Vidyapeeth (Deemed to be University) | 1 |
| Vaishali D. Khairnar | Afiliasi tidak diketahui | 1 |
| Shruti Patil | Symbiosis International University | 1 |
Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.