Melola Discovery

← Melola Discovery

Jurnal Tanpa peringkat SINTA

Performance Analysis of Machine Learning Models using RFE Feature Selection and Bayesian Optimization in Imbalanced Data Classification with Shap-Based Explanations

Scientific Journal of Informatics · Vol. 12 No. 3 (2025) · hlm. 539–554

DOI
10.15294/sji.v12i3.31459
ISSN
2407-7658, 2460-0040
Terbit
2025-10-09
Penerbit
Universitas Negeri Semarang
Bahasa
eng

Abstrak

Purpose: This research aims to evaluates the performance of Random Forest (RF) and Light Gradient Boosting Machine (LightGBM) models integrated with Recursive Feature Elimination (RFE) for feature selection, Bayesian Optimization (BO) for hyperparameter tuning, and three imbalanced data handling techniques Random Undersampling (RUS), Random Oversampling (ROS), and SMOTENC. Identifying key determinants of household food insecurity in Papua using SHAP for transparent feature interpretation. Methods: The research used 2022 SUSENAS data from Papua Province. Exploring data composition and variable characteristics, and aggregating individual data into household data. Data were split using random sampling (80% training, 20% testing). Eighteen experimental scenarios were created by combining feature selection or no feature selection, three imbalance handling methods, and default or hyperparameter tuning. RF and LightGBM were evaluated over 50 iterations using accuracy, sensitivity, specificity, and G-Mean, with SHAP applied to the best-performing models for interpretability. Result: LightGBM achieved the highest accuracy and stability, particularly when combined with SMOTENC and RFE+BO. RF showed better performance in maintaining G-Mean when paired with RUS, with the highest G-Mean (0.756) obtained by RF + BO + RUS. Three-way ANOVA proved that model type, imbalance handling, feature selection, and their interaction significantly affected the G-Mean value. SHAP analysis shows that health, financial, and educational limitations can increase the risk of food insecurity. Novelty: This research offers a new integration between feature selection, hyperparameter tuning, and imbalanced data handling within an interpretable machine learning framework, thereby providing a robust solution for food vulnerability classification on imbalanced datasets.

Kata kunci

Jaringan kolaborasi para penulis

40 penulis · 95 tautan kepenulisan bersama · 7 lintas organisasi · menampilkan 40 yang paling banyak terhubung

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • IPB University
  • National Research and Innovation Agency
  • Ministry of Agriculture
  • Bandung Islamic University
  • Afiliasi tidak diketahui
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama
I Made SumertajayaIPB University3
Anwar FitriantoIPB University3
Bagus SartonoIPB University2
Aji Hamim WigenaIPB University2
Oktaviyani DaswatiIPB University2
Novi PuspitaIPB University1
Agus Mohamad SolehIPB University1
Kusnaeni KusnaeniIPB University1
Muhammad Edy RizalIPB University1
Abdullah PannuIPB University1
Budi SusetyoIPB University1
Nunung NurjanahNational Research and Innovation Agency1
Fathu NurrahmanIPB University1
Muhammad Nur AidiIPB University1
Zetil HikmahIPB University1
Budi SusetyoIPB University1
Wa Ode Rahmalia SafitriIPB University1
Nur HikmahMinistry of Agriculture1
I'lmisukma RisnawatiIPB University1
Kristuisno Martsuyanto KapilukaIPB University1
Mohamad Solehudin ZaenalIPB University1
INDAHWATI INDAHWATIIPB University1
Siti HafsahIPB University1
Ahmad SyauqiIPB University1
Nur Azizah Komara RifaiBandung Islamic University1
Latifah Kosim DarusmanIPB University1
Dewi Anggraini SeptaningsihIPB University1
Rudi HeryantoIPB University1
Mimin AminahIPB University1
Widiatmaka WidiatmakaIPB University1
Program Studi Pengelolaan Sumberdaya Alam dan Lingkungan Institut Pertanian BogorIPB University1
Musa HubeisIPB University1
Fakultas Ekonomi dan Manajemen, Institut Pertanian Bogor Departemen ManajemenAfiliasi tidak diketahui1
Institut Pertanian Bogor Fakultas Matematika dan Ilmu AlamAfiliasi tidak diketahui1
Akhmad FauziIPB University1
Ririn SetyowatiIPB University1
Eko Sugeng PribadiIPB University1

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) https://journal.unnes.ac.id/journals/sji/oai, pertama di-harvest 6 Nov 2026, terakhir terlihat 8 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en">Performance Analysis of Machine Learning Models using RFE Feature Selection and Bayesian Optimization in Imbalanced Data Classification with Shap-Based Explanations</dc:title>
	<dc:creator xml:lang="en">Aqmar, Nurzatil</dc:creator>
	<dc:creator xml:lang="en">Wijayanto, Hari</dc:creator>
	<dc:creator xml:lang="en">Mochamad Afendi, Farit</dc:creator>
	<dc:subject xml:lang="en">Random forest</dc:subject>
	<dc:subject xml:lang="en">Light gradient boosting machine</dc:subject>
	<dc:subject xml:lang="en">Recursive feature elimination</dc:subject>
	<dc:subject xml:lang="en">Bayesian optimization</dc:subject>
	<dc:subject xml:lang="en">Shapley additive explanations </dc:subject>
	<dc:description xml:lang="en">Purpose: This research aims to evaluates the performance of Random Forest (RF) and Light Gradient Boosting Machine (LightGBM) models integrated with Recursive Feature Elimination (RFE) for feature selection, Bayesian Optimization (BO) for hyperparameter tuning, and three imbalanced data handling techniques Random Undersampling (RUS), Random Oversampling (ROS), and SMOTENC. Identifying key determinants of household food insecurity in Papua using SHAP for transparent feature interpretation.
Methods: The research used 2022 SUSENAS data from Papua Province. Exploring data composition and variable characteristics, and aggregating individual data into household data. Data were split using random sampling (80% training, 20% testing). Eighteen experimental scenarios were created by combining feature selection or no feature selection, three imbalance handling methods, and default or hyperparameter tuning. RF and LightGBM were evaluated over 50 iterations using accuracy, sensitivity, specificity, and G-Mean, with SHAP applied to the best-performing models for interpretability.
Result: LightGBM achieved the highest accuracy and stability, particularly when combined with SMOTENC and RFE+BO. RF showed better performance in maintaining G-Mean when paired with RUS, with the highest G-Mean (0.756) obtained by RF + BO + RUS. Three-way ANOVA proved that model type, imbalance handling, feature selection, and their interaction significantly affected the G-Mean value. SHAP analysis shows that health, financial, and educational limitations can increase the risk of food insecurity.
Novelty: This research offers a new integration between feature selection, hyperparameter tuning, and imbalanced data handling within an interpretable machine learning framework, thereby providing a robust solution for food vulnerability classification on imbalanced datasets.</dc:description>
	<dc:publisher xml:lang="en">Universitas Negeri Semarang</dc:publisher>
	<dc:date>2025-10-09</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en">Peer-reviewed Article</dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://journal.unnes.ac.id/journals/sji/article/view/31459</dc:identifier>
	<dc:identifier>10.15294/sji.v12i3.31459</dc:identifier>
	<dc:source xml:lang="en">Scientific Journal of Informatics; Vol. 12 No. 3: Special Issue; 539-554</dc:source>
	<dc:source>2407-7658</dc:source>
	<dc:source>2460-0040</dc:source>
	<dc:source>10.15294/sji.v12i3</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://journal.unnes.ac.id/journals/sji/article/view/31459/6341</dc:relation>
	<dc:rights xml:lang="en">Copyright (c) 2025 Scientific Journal of Informatics</dc:rights>
</oai_dc:dc>