Melola Discovery

← Melola Discovery

Jurnal Tanpa peringkat SINTA

Optimal Preprocessing and Feature Engineering Combination for DANA App Sentiment Classification: A Full Factorial Study

Scientific Journal of Informatics · Vol. 13 No. 4 (2026) · hlm. 767–782

DOI
10.15294/sji.v13i4.58535
ISSN
2407-7658, 2460-0040
Terbit
2026-10-26
Penerbit
Universitas Negeri Semarang
Bahasa
eng

Abstrak

Purpose: Selecting optimal text-processing methods for sentiment analysis of digital financial application reviews, especially in Indonesian, is challenging. Previous studies have typically evaluated preprocessing techniques, feature engineering methods, and classification models in isolation, leaving the question of their optimal combination unresolved. This study aims to systematically determine the best pipeline for sentiment classification of DANA application reviews on the Google Play Store. Methods: A full factorial experimental design was employed, generating 24 combinations across four preprocessing configurations (no preprocessing, stopword removal, stemming, and stopword removal combined with stemming), three feature extraction methods (Bag of Words, TF-IDF, and N-gram), and two classifiers (Naive Bayes and Logistic Regression). A dataset of 57,115 Indonesian reviews, comprising 34,435 positive (60.3%) and 22,680 negative (39.7%) instances, was used, with labels based on user star ratings. Result: From the experimental results, it was found that the best setting is one that does not involve any pre-processing, the use of N-grams, and Logistic Regression, giving an F1-Score of 0.8772 and surpassing the baseline F1-Score by 1.82 percent. Logistic Regression has been found to be significantly better than Naive Bayes using the Mann-Whitney U test (p=0.0001). Preprocessing and feature extraction independently had significant effects on model performance, as indicated by the Kruskal-Wallis test (p=0.0274 for preprocessing and p=0.0269 for feature extraction). N-gram was proven superior to TF-IDF in classifying the negative class based on the Wilcoxon Signed-Rank test (p=0.0039). However, preprocessing and feature extraction did not jointly exert a significant effect on model performance based on the Scheirer-Ray-Hare test (H=6.0117, p=0.3051), and no significant interaction effect was found between both factors (H=0.3683, p=0.9991), implying that both factors can be independently optimized. Novelty: This study contributes a systematic full factorial evaluation framework for Indonesian NLP pipelines in the fintech domain. Unlike previous studies that tested variables in isolation, this approach enables a fair and comprehensive comparison across method combinations, offering practitioners evidence-based guidance for sentiment analysis method selection. Future research may explore deep learning or large language model-based approaches to further improve classification performance.

Kata kunci

Jaringan kolaborasi para penulis

2 penulis · 1 tautan kepenulisan bersama

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • Universitas Islam Indragiri
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) https://journal.unnes.ac.id/journals/sji/oai, pertama di-harvest 6 Nov 2026, terakhir terlihat 8 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en">Optimal Preprocessing and Feature Engineering Combination for DANA App Sentiment Classification: A Full Factorial Study</dc:title>
	<dc:creator xml:lang="en">Samsudin, Samsudin</dc:creator>
	<dc:creator xml:lang="en">Putriani, Linda</dc:creator>
	<dc:subject xml:lang="en">Sentiment Analysis</dc:subject>
	<dc:subject xml:lang="en">Full Factorial Experiment</dc:subject>
	<dc:subject xml:lang="en">Feature Engineering</dc:subject>
	<dc:subject xml:lang="en">Text preprocessing</dc:subject>
	<dc:subject xml:lang="en">Fintech App Reviews</dc:subject>
	<dc:description xml:lang="en">Purpose: Selecting optimal text-processing methods for sentiment analysis of digital financial application reviews, especially in Indonesian, is challenging. Previous studies have typically evaluated preprocessing techniques, feature engineering methods, and classification models in isolation, leaving the question of their optimal combination unresolved. This study aims to systematically determine the best pipeline for sentiment classification of DANA application reviews on the Google Play Store.
Methods: A full factorial experimental design was employed, generating 24 combinations across four preprocessing configurations (no preprocessing, stopword removal, stemming, and stopword removal combined with stemming), three feature extraction methods (Bag of Words, TF-IDF, and N-gram), and two classifiers (Naive Bayes and Logistic Regression). A dataset of 57,115 Indonesian reviews, comprising 34,435 positive (60.3%) and 22,680 negative (39.7%) instances, was used, with labels based on user star ratings.
Result: From the experimental results, it was found that the best setting is one that does not involve any pre-processing, the use of N-grams, and Logistic Regression, giving an F1-Score of 0.8772 and surpassing the baseline F1-Score by 1.82 percent. Logistic Regression has been found to be significantly better than Naive Bayes using the Mann-Whitney U test (p=0.0001). Preprocessing and feature extraction independently had significant effects on model performance, as indicated by the Kruskal-Wallis test (p=0.0274 for preprocessing and p=0.0269 for feature extraction). N-gram was proven superior to TF-IDF in classifying the negative class based on the Wilcoxon Signed-Rank test (p=0.0039). However, preprocessing and feature extraction did not jointly exert a significant effect on model performance based on the Scheirer-Ray-Hare test (H=6.0117, p=0.3051), and no significant interaction effect was found between both factors (H=0.3683, p=0.9991), implying that both factors can be independently optimized.
Novelty: This study contributes a systematic full factorial evaluation framework for Indonesian NLP pipelines in the fintech domain. Unlike previous studies that tested variables in isolation, this approach enables a fair and comprehensive comparison across method combinations, offering practitioners evidence-based guidance for sentiment analysis method selection. Future research may explore deep learning or large language model-based approaches to further improve classification performance.</dc:description>
	<dc:publisher xml:lang="en">Universitas Negeri Semarang</dc:publisher>
	<dc:date>2026-10-26</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en">Peer-reviewed Article</dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://journal.unnes.ac.id/journals/sji/article/view/58535</dc:identifier>
	<dc:identifier>10.15294/sji.v13i4.58535</dc:identifier>
	<dc:source xml:lang="en">Scientific Journal of Informatics; Vol. 13 No. 4: Regular Issue; 767-782</dc:source>
	<dc:source>2407-7658</dc:source>
	<dc:source>2460-0040</dc:source>
	<dc:source>10.15294/sji.v13i4</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://journal.unnes.ac.id/journals/sji/article/view/58535/11695</dc:relation>
	<dc:rights xml:lang="en">Copyright (c) 2026 Scientific Journal of Informatics</dc:rights>
</oai_dc:dc>