Classifying Indonesian Hoax News Titles with SVM, XGBoost, and BiLSTM

Trisna, I Nyoman Prayana; Putra, I Made Wiraharja Jaya; Vihikan, Wayan Oger

doi:10.22146/ijccs.106608

Open access Jurnal SINTA 2

Classifying Indonesian Hoax News Titles with SVM, XGBoost, and BiLSTM

IJCCS (Indonesian Journal of Computing and Cybernetics Systems) · Vol. 19 No. 4 (2025) · pp. 361–370

DOI: 10.22146/ijccs.106608
ISSN: 2460-7258, 1978-1520
Published: 2025-10-31
Publisher: IndoCEISS in colaboration with Universitas Gadjah Mada, Indonesia.
Language: eng
License: creativecommons.org/licenses/by-sa/4.0

Abstract

This study investigates the automated detection of hoaxes related to President Jokowi in Indonesian news by analyzing only news titles, aiming for efficient detection and reduced traffic to harmful websites. We compared the performance of traditional (SVM, XGBoost) and deep learning (BiLSTM) algorithms, with and without Synthetic Minority Over-sampling Technique (SMOTE) to address class imbalance in a dataset scraped from trusted news sources (CNN Indonesia, Detik News) and a fact-checking platform (turnbackhoax.id). The results indicate that BiLSTM generally outperformed SVM and XGBoost, demonstrating the potential of deep learning for this task. However, applying SMOTE negatively impacted BiLSTM's performance, suggesting overfitting. Notably, precision consistently exceeded recall across all models, indicating high reliability in identifying hoaxes but a potential for missing a significant number of actual hoaxes. This highlights a trade-off between avoiding false positives and ensuring comprehensive detection. The findings also suggest that language-specific characteristics influence algorithm effectiveness. This research contributes to developing efficient and accurate tools for combating misinformation in the Indonesian online environment, emphasizing the importance of title-based analysis and careful consideration on data balancing.

Keywords

Collaboration network of the authors

12 authors · 21 co-authorship links · 6 across different organisations

Hover or tap a name to highlight its co-authors. Double-click to open the author's page.

Universitas Gadjah Mada
Udayana University
Affiliation unknown

Different organisations
Same organisation
Affiliation unknown
Thicker line = more joint articles · bigger circle = more articles

Show as table

Co-author	Organisation	Joint articles
I Made Agus Dwi Suarjaya	Udayana University	2
Komang Sri Yanisa Putri	Affiliation unknown	1
Nyoman Tri Rahaditya Kusuma	Affiliation unknown	1
Guntur Budi Herwanto	Universitas Gadjah Mada	1
Annisa Maulida Ningtyas	Universitas Gadjah Mada	1
I Gede Mujiyatna	Universitas Gadjah Mada	1
Kurniawan Eka Nugraha	Universitas Gadjah Mada	1
Afiahayati Afiahayati	Universitas Gadjah Mada	1
Muhammad Auzan	Universitas Gadjah Mada	1

Affiliations come from OpenAlex; journal OAI-PMH metadata carries none. Authors without an OpenAlex ID are matched by name, which can merge or split people.

Metadata source: OAI-PMH (metadata jurnal) https://journal.ugm.ac.id/ijccs/oai, first harvested 26 Okt 2026, last seen 26 Okt 2026.

Original metadata (oai_dc)

<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en-US">Classifying Indonesian Hoax News Titles with SVM, XGBoost, and BiLSTM</dc:title>
	<dc:creator>Trisna, I Nyoman Prayana</dc:creator>
	<dc:creator>Putra, I Made Wiraharja Jaya</dc:creator>
	<dc:creator>Vihikan, Wayan Oger</dc:creator>
	<dc:subject xml:lang="en-US"></dc:subject>
	<dc:subject xml:lang="en-US">Hoax detection; SVM; XGBoost; BiLSTM; SMOTE</dc:subject>
	<dc:description xml:lang="en-US">This study investigates the automated detection of hoaxes related to President Jokowi in Indonesian news by analyzing only news titles, aiming for efficient detection and reduced traffic to harmful websites. We compared the performance of traditional (SVM, XGBoost) and deep learning (BiLSTM) algorithms, with and without Synthetic Minority Over-sampling Technique (SMOTE) to address class imbalance in a dataset scraped from trusted news sources (CNN Indonesia, Detik News) and a fact-checking platform (turnbackhoax.id). The results indicate that BiLSTM generally outperformed SVM and XGBoost, demonstrating the potential of deep learning for this task. However, applying SMOTE negatively impacted BiLSTM's performance, suggesting overfitting. Notably, precision consistently exceeded recall across all models, indicating high reliability in identifying hoaxes but a potential for missing a significant number of actual hoaxes. This highlights a trade-off between avoiding false positives and ensuring comprehensive detection. The findings also suggest that language-specific characteristics influence algorithm effectiveness. This research contributes to developing efficient and accurate tools for combating misinformation in the Indonesian online environment, emphasizing the importance of title-based analysis and careful consideration on data balancing.</dc:description>
	<dc:publisher xml:lang="en-US">IndoCEISS in colaboration with Universitas Gadjah Mada, Indonesia.</dc:publisher>
	<dc:contributor xml:lang="en-US"></dc:contributor>
	<dc:date>2025-10-31</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en-US"></dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://journal.ugm.ac.id/ijccs/article/view/106608</dc:identifier>
	<dc:identifier>10.22146/ijccs.106608</dc:identifier>
	<dc:source xml:lang="en-US">IJCCS (Indonesian Journal of Computing and Cybernetics Systems); Vol 19, No 4 (2025): October; 361-370</dc:source>
	<dc:source>2460-7258</dc:source>
	<dc:source>1978-1520</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://journal.ugm.ac.id/ijccs/article/view/106608/42786</dc:relation>
	<dc:rights xml:lang="en-US">Copyright (c) 2025 IJCCS (Indonesian Journal of Computing and Cybernetics Systems)</dc:rights>
	<dc:rights xml:lang="en-US">http://creativecommons.org/licenses/by-sa/4.0</dc:rights>
</oai_dc:dc>