Melola Discovery

← Melola Discovery

Akses terbuka Jurnal SINTA 1

Generative artificial intelligence as an evaluator and feedback tool in distance learning: a case study on law implementation

IAES International Journal of Artificial Intelligence · Vol. 14 No. 3 (2025) · hlm. 2490–2505

DOI
10.11591/ijai.v14.i3.pp2490-2505
ISSN
2252-8938, 2089-4872
Terbit
2025-06-01
Penerbit
Institute of Advanced Engineering and Science
Bahasa
eng
Lisensi
creativecommons.org/licenses/by-sa/4.0

Abstrak

The development of generative artificial intelligence (GAI) has impacted various fields, including higher education. This research examines the use of GAI as an evaluator and feedback provider in distance legal education. This study tested five GAI models: ChatGPT, Perplexity, Gemini, Bing, and You, using a sample of 20 students and evaluations from legal experts. Descriptive statistical analysis and non-parametric tests, including Wilcoxon, intraclass correlation coefficient (ICC), Kappa, and Kendall's W, were used to assess accuracy, feedback quality, and usability. The results showed that ChatGPT was the most effective GAI, with the highest mean scores of 4.22 from experts and 4.12 from students, followed by Gemini with scores of 4.15 and 4.07. In terms of binary judgement accuracy, Gemini scored 80%, ChatGPT 60%, while Perplexity, Bing, and You had lower scores. Statistical analysis showed moderate agreement (ICC=0.439) and low alignment (Kappa=-0.058) between the GAIs and expert evaluations, with a Kendall's W value of 0.576 indicating moderate consistency in judgements. These findings emphasize the importance of selecting effective GAIs such as ChatGPT and Gemini to improve academic evaluation and learning in legal education, and pave the way for further innovations in the use of AI.

Kata kunci

Jaringan kolaborasi para penulis

26 penulis · 75 tautan kepenulisan bersama · 18 lintas organisasi

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • Yogyakarta State University
  • Indonesia Open University
  • Padjadjaran University
  • State University of Jakarta
  • Sunan Gunung Djati State Islamic University Bandung
  • Afiliasi tidak diketahui
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama
Yanuar Agung FadlullahYogyakarta State University2
Dwi Astuti AprijaniAfiliasi tidak diketahui2
Fitria AmastiniAfiliasi tidak diketahui2
Moh. Rizky Putra Aji UtamaAfiliasi tidak diketahui2
Sri Maulidia PermatasariIndonesia Open University1
Arya YusufYogyakarta State University1
Siswantoyo SiswantoyoYogyakarta State University1
Ria DymyatiYogyakarta State University1
Putu SudiraYogyakarta State University1
Yoga SyahriaYogyakarta State University1
Hazar KusmayantiPadjadjaran University1
Dede KaniaSunan Gunung Djati State Islamic University Bandung1
Rafan DarodjatPadjadjaran University1
Asma’ Khoirunnisa’Yogyakarta State University1
Yosef BudimanYogyakarta State University1
Gilang Adi PrasetyoYogyakarta State University1
Hanifah Mar’atush ShalihahYogyakarta State University1
Sugiri SugiriYogyakarta State University1
Ivan HanafiState University of Jakarta1
Nurul FathiaState University of Jakarta1

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) http://ijai.iaescore.com/index.php/IJAI/oai, pertama di-harvest 8 Nov 2026, terakhir terlihat 8 Nov 2026.
Metadata asli (oai_dc) 
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en-US">Generative artificial intelligence as an evaluator and feedback tool in distance learning: a case study on law implementation</dc:title>
	<dc:creator>Nurdiana, Dian</dc:creator>
	<dc:creator>Maulana, Muhamad Riyan</dc:creator>
	<dc:creator>Hasanah, Siti Hadijah</dc:creator>
	<dc:creator>Chairunnisa, Madiha Dzakiyyah</dc:creator>
	<dc:creator>Komuna, Avelyn Pingkan</dc:creator>
	<dc:creator>Rif'an, Muhammad</dc:creator>
	<dc:subject xml:lang="en-US">General AI Application, Artificial Intelligence, Evaluation on AI</dc:subject>
	<dc:subject xml:lang="en-US">AI evaluation; Assignment feedback; Distance learning; Generative artificial intelligence; Law science;</dc:subject>
	<dc:description xml:lang="en-US">The development of generative artificial intelligence (GAI) has impacted various fields, including higher education. This research examines the use of GAI as an evaluator and feedback provider in distance legal education. This study tested five GAI models: ChatGPT, Perplexity, Gemini, Bing, and You, using a sample of 20 students and evaluations from legal experts. Descriptive statistical analysis and non-parametric tests, including Wilcoxon, intraclass correlation coefficient (ICC), Kappa, and Kendall's W, were used to assess accuracy, feedback quality, and usability. The results showed that ChatGPT was the most effective GAI, with the highest mean scores of 4.22 from experts and 4.12 from students, followed by Gemini with scores of 4.15 and 4.07. In terms of binary judgement accuracy, Gemini scored 80%, ChatGPT 60%, while Perplexity, Bing, and You had lower scores. Statistical analysis showed moderate agreement (ICC=0.439) and low alignment (Kappa=-0.058) between the GAIs and expert evaluations, with a Kendall's W value of 0.576 indicating moderate consistency in judgements. These findings emphasize the importance of selecting effective GAIs such as ChatGPT and Gemini to improve academic evaluation and learning in legal education, and pave the way for further innovations in the use of AI.</dc:description>
	<dc:publisher xml:lang="en-US">Institute of Advanced Engineering and Science</dc:publisher>
	<dc:contributor xml:lang="en-US">Universitas Terbuka</dc:contributor>
	<dc:date>2025-06-01</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en-US"></dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://ijai.iaescore.com/index.php/IJAI/article/view/26571</dc:identifier>
	<dc:identifier>10.11591/ijai.v14.i3.pp2490-2505</dc:identifier>
	<dc:source xml:lang="en-US">IAES International Journal of Artificial Intelligence (IJ-AI); Vol 14, No 3: June 2025; 2490-2505</dc:source>
	<dc:source>2252-8938</dc:source>
	<dc:source>2089-4872</dc:source>
	<dc:source>10.11591/ijai.v14.i3</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://ijai.iaescore.com/index.php/IJAI/article/view/26571/14585</dc:relation>
	<dc:rights xml:lang="en-US">Copyright (c) 2025 Institute of Advanced Engineering and Science</dc:rights>
	<dc:rights xml:lang="en-US">http://creativecommons.org/licenses/by-sa/4.0</dc:rights>
</oai_dc:dc>