Melola Discovery

← Melola Discovery

Akses terbuka Jurnal SINTA 1

WVisionBERT-VL: a multimodal model architecture for toxicity classification on social media platforms using large language models

IAES International Journal of Artificial Intelligence · Vol. 15 No. 4 (2026) · hlm. 3365–3375

DOI
10.11591/ijai.v15.i4.pp3365-3375
ISSN
2252-8938, 2089-4872
Terbit
2026-08-01
Penerbit
Institute of Advanced Engineering and Science
Bahasa
eng
Lisensi
creativecommons.org/licenses/by-sa/4.0

Abstrak

The increasing prevalence of toxic content on social media, conveyed through text, images, and videos, poses significant challenges for automated content moderation systems. Although prior studies have reported promising results in unimodal and bimodal settings, they often fail to capture implicit and contextual toxicity emerging from interactions across multiple modalities, particularly in non-English environments. This paper proposes WVisionBERT-VL, an end-to-end multimodal framework for toxicity detection that integrates text, image, and video modalities within a unified architecture. The proposed model incorporates modality-specific encoders, bidirectional multi-head cross attention (BMHCA) for cross-modal synchronization, and an adaptive fusion gate to dynamically balance modality contributions. A balanced multimodal dataset is constructed from social media platforms, including X, Instagram, and TikTok, and refined using a model-based labeling strategy with limited human-in-the-loop validation. Experimental results on a custom Indonesian dataset demonstrate strong in-domain performance, achieving an accuracy of 94.12%, a macro-F1 of 0.9407, and a receiver operating characteristic - area under the curve (ROC-AUC) of 0.9721, with robustness further validated through five-fold cross-validation. Cross-dataset evaluation highlights challenges related to domain shift, underscoring the need for future research on robust and domain-adaptive multimodal toxicity detection.

Kata kunci

Jaringan kolaborasi para penulis

30 penulis · 68 tautan kepenulisan bersama · 13 lintas organisasi

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • Gunadarma University
  • Universitas Raharja
  • Jamia Millia Islamia
  • Saarland University
  • Universitas Esa Unggul
  • Afiliasi tidak diketahui
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama
Rina RefiantiGunadarma University2
Reni Diah KusumawatiAfiliasi tidak diketahui1
Irfan HumainiAfiliasi tidak diketahui1
Agung WibowoGunadarma University1
Himanshu DuttJamia Millia Islamia1
Radityo Hendratmojo Jati NotonegoroAfiliasi tidak diketahui1
Dewi Agushinta RahayuAfiliasi tidak diketahui1
Rifki KosasihAfiliasi tidak diketahui1
Asep JuarnaGunadarma University1
Adang SuhendraGunadarma University1
Tubagus Maulana KusumaGunadarma University1
Rifiana AriefGunadarma University1
Hustinawaty HustinawatyGunadarma University1
Untung RahardjaUniversitas Raharja1
Prihandoko PrihandokoGunadarma University1
Nur AzizahUniversitas Raharja1
Po Abas SunaryaUniversitas Raharja1
Charlotte PashaAfiliasi tidak diketahui1
I Made WiryanaGunadarma University1
Souza Nurafrianto Windiartono PutraSaarland University1
Nuryani NuryaniAfiliasi tidak diketahui1
Detty PurnamasariGunadarma University1
Riya WidayantiUniversitas Esa Unggul1
Avinanta TariganGunadarma University1
Dewi Agushinta RGunadarma University1
Rifki KosasihGunadarma University1
Radityo Hendratmojo Jati NotonegoroGunadarma University1

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) http://ijai.iaescore.com/index.php/IJAI/oai, pertama di-harvest 8 Nov 2026, terakhir terlihat 16 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en-US">WVisionBERT-VL: a multimodal model architecture for toxicity classification on social media platforms using large language models</dc:title>
	<dc:creator>Ningrum, Witta Listiya</dc:creator>
	<dc:creator>Mutiara, Achmad Benny</dc:creator>
	<dc:creator>Ikasari, Diana</dc:creator>
	<dc:subject xml:lang="en-US"></dc:subject>
	<dc:subject xml:lang="en-US">Content moderation; Cross-modal attention; Multimodal toxicity detection; Social media analysis; Transformer-based fusio</dc:subject>
	<dc:description xml:lang="en-US">The increasing prevalence of toxic content on social media, conveyed through text, images, and videos, poses significant challenges for automated content moderation systems. Although prior studies have reported promising results in unimodal and bimodal settings, they often fail to capture implicit and contextual toxicity emerging from interactions across multiple modalities, particularly in non-English environments. This paper proposes WVisionBERT-VL, an end-to-end multimodal framework for toxicity detection that integrates text, image, and video modalities within a unified architecture. The proposed model incorporates modality-specific encoders, bidirectional multi-head cross attention (BMHCA) for cross-modal synchronization, and an adaptive fusion gate to dynamically balance modality contributions. A balanced multimodal dataset is constructed from social media platforms, including X, Instagram, and TikTok, and refined using a model-based labeling strategy with limited human-in-the-loop validation. Experimental results on a custom Indonesian dataset demonstrate strong in-domain performance, achieving an accuracy of 94.12%, a macro-F1 of 0.9407, and a receiver operating characteristic - area under the curve (ROC-AUC) of 0.9721, with robustness further validated through five-fold cross-validation. Cross-dataset evaluation highlights challenges related to domain shift, underscoring the need for future research on robust and domain-adaptive multimodal toxicity detection.</dc:description>
	<dc:publisher xml:lang="en-US">Institute of Advanced Engineering and Science</dc:publisher>
	<dc:contributor xml:lang="en-US">Gunadarma University</dc:contributor>
	<dc:date>2026-08-01</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en-US"></dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://ijai.iaescore.com/index.php/IJAI/article/view/28019</dc:identifier>
	<dc:identifier>10.11591/ijai.v15.i4.pp3365-3375</dc:identifier>
	<dc:source xml:lang="en-US">IAES International Journal of Artificial Intelligence (IJ-AI); Vol 15, No 4: August 2026; 3365-3375</dc:source>
	<dc:source>2252-8938</dc:source>
	<dc:source>2089-4872</dc:source>
	<dc:source>10.11591/ijai.v15.i4</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://ijai.iaescore.com/index.php/IJAI/article/view/28019/15297</dc:relation>
	<dc:rights xml:lang="en-US">Copyright (c) 2026 Witta Listiya Ningrum, Achmad Benny Mutiara, Diana Ikasari</dc:rights>
	<dc:rights xml:lang="en-US">http://creativecommons.org/licenses/by-sa/4.0</dc:rights>
</oai_dc:dc>