Melola Discovery

← Melola Discovery

Jurnal Tanpa peringkat SINTA

Comparative Analysis of High School Student and AI-Generated Essays Using IndoBERT and Linguistic Features

Scientific Journal of Informatics · Vol. 12 No. 3 (2025) · hlm. 525–538

DOI
10.15294/sji.v12i3.27732
ISSN
2407-7658, 2460-0040
Terbit
2025-10-04
Penerbit
Universitas Negeri Semarang
Bahasa
eng

Abstrak

Purpose: The purpose of this study is to address the growing challenge of distinguishing between essays written by humans and essays generated by AI, particularly in the context of high school education in Indonesia. This study aims to analyze the semantic and linguistic differences between student-written and ChatGPT-generated in Indonesian language. Methods: The study employs an IndoBERT-based semantic model trained with triplet loss to generate paragraph-level embeddings, allowing the measurement of semantic similarity within and between essay classes. Additionally, linguistic features such as lexical diversity, word count, modal usage, and stopword ratio were extracted to capture stylistic and structural differences. These three key features are combined and used as input to a neural network classifier. Result: The IndoBERT-based semantic model successfully grouped student-written and ChatGPT-generated essays into distinct clusters. The similarity scores within student essays ranged from 0.7 to 0.9, while the similarity between classes was mostly negative with a few outliers, reflecting the cosine similarity metric used in this study, which has a range of -1 to 1. The classification model showed a 90.55% accuracy and an AUC of 0.9999 when evaluated on the independent test set defined in the Data Preparation stage. These results suggest that student-written and ChatGPT-generated essays form distinct semantic clusters. Students’ essays show more linguistic diversity, while ChatGPT essays show consistency in the coherence and formality aspects of the essays. Novelty: This study provides empirical insights of semantic similarities and linguistic features to differentiate between human and AI-generated essays in the Indonesian language. It contributes to supporting academic integrity efforts and highlighting the need for further research across different writing models and contexts.

Kata kunci

Jaringan kolaborasi para penulis

12 penulis · 22 tautan kepenulisan bersama · 3 lintas organisasi

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • Telkom University
  • Institut Teknologi Sepuluh Nopember
  • University of Pembangunan Nasional Veteran Jawa Timur
  • Afiliasi tidak diketahui
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama
Agus Zainal ArifinInstitut Teknologi Sepuluh Nopember1
Bagus Satria WigunaInstitut Teknologi Sepuluh Nopember1
Cinthia Vairra HudiyantiInstitut Teknologi Sepuluh Nopember1
Aprilia Dewi ArdiyantiUniversity of Pembangunan Nasional Veteran Jawa Timur1
Allif Rosyidy HilmiAfiliasi tidak diketahui1
Yofinda Eka SetiawanAfiliasi tidak diketahui1
Aslam Chitami Priawan SiregarAfiliasi tidak diketahui1
Pima Hani SafitriTelkom University1
Daud MuhajirTelkom University1

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) https://journal.unnes.ac.id/journals/sji/oai, pertama di-harvest 6 Nov 2026, terakhir terlihat 8 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en">Comparative Analysis of High School Student and AI-Generated Essays Using IndoBERT and Linguistic Features</dc:title>
	<dc:creator xml:lang="en">Adani, Muhammad Harits Shofwan</dc:creator>
	<dc:creator xml:lang="en">Rausanfita, Alqis</dc:creator>
	<dc:creator xml:lang="en">Mustaqim, Tanzilal </dc:creator>
	<dc:subject xml:lang="en">ChatGPT</dc:subject>
	<dc:subject xml:lang="en">Essay</dc:subject>
	<dc:subject xml:lang="en">IndoBERT</dc:subject>
	<dc:subject xml:lang="en">Linguistic features</dc:subject>
	<dc:subject xml:lang="en">Semantic similarity</dc:subject>
	<dc:subject xml:lang="en">Text classification </dc:subject>
	<dc:description xml:lang="en">Purpose: The purpose of this study is to address the growing challenge of distinguishing between essays written by humans and essays generated by AI, particularly in the context of high school education in Indonesia. This study aims to analyze the semantic and linguistic differences between student-written and ChatGPT-generated in Indonesian language.
Methods: The study employs an IndoBERT-based semantic model trained with triplet loss to generate paragraph-level embeddings, allowing the measurement of semantic similarity within and between essay classes. Additionally, linguistic features such as lexical diversity, word count, modal usage, and stopword ratio were extracted to capture stylistic and structural differences. These three key features are combined and used as input to a neural network classifier.
Result: The IndoBERT-based semantic model successfully grouped student-written and ChatGPT-generated essays into distinct clusters. The similarity scores within student essays ranged from 0.7 to 0.9, while the similarity between classes was mostly negative with a few outliers, reflecting the cosine similarity metric used in this study, which has a range of -1 to 1. The classification model showed a 90.55% accuracy and an AUC of 0.9999 when evaluated on the independent test set defined in the Data Preparation stage. These results suggest that student-written and ChatGPT-generated essays form distinct semantic clusters. Students’ essays show more linguistic diversity, while ChatGPT essays show consistency in the coherence and formality aspects of the essays.
Novelty: This study provides empirical insights of semantic similarities and linguistic features to differentiate between human and AI-generated essays in the Indonesian language. It contributes to supporting academic integrity efforts and highlighting the need for further research across different writing models and contexts.</dc:description>
	<dc:publisher xml:lang="en">Universitas Negeri Semarang</dc:publisher>
	<dc:date>2025-10-04</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en">Peer-reviewed Article</dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://journal.unnes.ac.id/journals/sji/article/view/27732</dc:identifier>
	<dc:identifier>10.15294/sji.v12i3.27732</dc:identifier>
	<dc:source xml:lang="en">Scientific Journal of Informatics; Vol. 12 No. 3: Special Issue; 525-538</dc:source>
	<dc:source>2407-7658</dc:source>
	<dc:source>2460-0040</dc:source>
	<dc:source>10.15294/sji.v12i3</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://journal.unnes.ac.id/journals/sji/article/view/27732/6304</dc:relation>
	<dc:rights xml:lang="en">Copyright (c) 2025 Scientific Journal of Informatics</dc:rights>
</oai_dc:dc>