Melola Discovery

← Melola Discovery

Akses terbuka Jurnal Tanpa peringkat SINTA

The Limits of ChatGPT’s Pragmatic Interpretation: An Experimental Study of Conversational Implicature in Indonesian Language Conversations

Rainbow: Journal of Literature, Linguistics and Culture Studies · Vol. 15 No. 3 (2026) · hlm. 299–312

DOI
10.15294/rainbow.v15i3.65667
ISSN
2721-4540, 2252-6323
Terbit
2026-10-10
Penerbit
Universitas Negeri Semarang
Bahasa
eng
Lisensi
creativecommons.org/licenses/by/4.0

Abstrak

This study evaluates ChatGPT’s interpretative accuracy and contextual sensitivity in understanding conversational implicatures in Indonesian. A computational experimental design employed 80 minimal stimulus pairs, yielding 160 responses across five communication themes and four pragmatic phenomena. Each pair contained identical utterances presented in different situational contexts. Data were collected using GPT-6 at a ‘High’ reasoning level via Temporary Chat (unpersonalised). Two pragmatics experts independently assessed the responses using a 0–2 ordinal rubric. The analysis comprised Interpretation Accuracy (IA), Pairwise Joint Accuracy (PJA), descriptive comparisons between conditions, and qualitative examination of pragmatic inferences. IA reached 98.13% for Rater 1 and 91.25% for Rater 2, while PJA reached 96.25% and 82.50%, respectively. Exact agreement was 93.13%, with a weighted Cohen’s kappa of 0.332, indicating limitations in scoring consistency. According to Rater 2, pairwise accuracy was highest for indirect criticism or evaluation (95.00%) and lowest for indirect refusal (65.00%). Accuracy was descriptively higher in Condition A (96.25%) than in Condition B (86.25%). Qualitative analysis identified instances of inferences exceeding contextual support and insufficient use of relevant contextual information. These findings highlight the importance of evaluating both interpretative accuracy and contextual justification, while acknowledging the sensitivity of performance estimates to rater judgments.

Kata kunci

Jaringan kolaborasi para penulis

40 penulis · 79 tautan kepenulisan bersama · 6 lintas organisasi · menampilkan 40 yang paling banyak terhubung

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • Universitas Halu Oleo
  • Institut Agama Islam Negeri Kendari
  • Universitas Airlangga
  • STKIP Arrahmaniyah
  • Afiliasi tidak diketahui
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama
Amri TanduklangiUniversitas Halu Oleo2
Hilaluddin HanafiUniversitas Halu Oleo2
Zalili SailanUniversitas Halu Oleo2
Sahlan SahlanUniversitas Halu Oleo2
Muhammad YazidUniversitas Halu Oleo2
Amirudin RahimAfiliasi tidak diketahui2
La Ode SahidinUniversitas Halu Oleo2
La Ode BalawaUniversitas Halu Oleo2
Muhammad Kurniawan RachmanInstitut Agama Islam Negeri Kendari1
Hanna HannaUniversitas Halu Oleo1
Faridh HidayatAfiliasi tidak diketahui1
Lisa IndrajayaAfiliasi tidak diketahui1
Mursalim MursalimAfiliasi tidak diketahui1
Tambunan TambunanAfiliasi tidak diketahui1
Fadlan SalimAfiliasi tidak diketahui1
Dahlia SurahmawatiAfiliasi tidak diketahui1
Asrun LioAfiliasi tidak diketahui1
Ayu AstrianiAfiliasi tidak diketahui1
Darni DarniUniversitas Airlangga1
Sahlan SahlanAfiliasi tidak diketahui1
Ihsan ZulhiadUniversitas Halu Oleo1
I Dewa Putu TriyasaAfiliasi tidak diketahui1
Nana SumarnaAfiliasi tidak diketahui1
Najib HusainUniversitas Halu Oleo1
Ramli AkhmadAfiliasi tidak diketahui1
Nia Asriana HasiniUniversitas Halu Oleo1
Susi SusiSTKIP Arrahmaniyah1
Asran LaksanaAfiliasi tidak diketahui1
Irianto IbrahimAfiliasi tidak diketahui1
Nasruddin NasruddinAfiliasi tidak diketahui1
Rahman RahmanAfiliasi tidak diketahui1
Sahlan SahlanAfiliasi tidak diketahui1
La Muhammad RahmatAfiliasi tidak diketahui1
Haerun AnaAfiliasi tidak diketahui1
Muhammad KaldumAfiliasi tidak diketahui1
Yunra Nofian NitaAfiliasi tidak diketahui1

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) https://journal.unnes.ac.id/journals/rainbow/oai, pertama di-harvest 6 Nov 2026, terakhir terlihat 8 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en">The Limits of ChatGPT’s Pragmatic Interpretation: An Experimental Study of Conversational Implicature in Indonesian Language Conversations</dc:title>
	<dc:creator xml:lang="en">Badara, Aris</dc:creator>
	<dc:creator xml:lang="en">Rifa'ie, Muhammad</dc:creator>
	<dc:creator xml:lang="en">Yusoh, Ruslan </dc:creator>
	<dc:creator xml:lang="en">Salama, Ibrahim</dc:creator>
	<dc:subject xml:lang="en">ChatGPT</dc:subject>
	<dc:subject xml:lang="en">conversational implicatures</dc:subject>
	<dc:subject xml:lang="en">contextual sensitivity</dc:subject>
	<dc:subject xml:lang="en">pragmatics</dc:subject>
	<dc:subject xml:lang="en">minimal stimulus pairs</dc:subject>
	<dc:description xml:lang="en">This study evaluates ChatGPT’s interpretative accuracy and contextual sensitivity in understanding conversational implicatures in Indonesian. A computational experimental design employed 80 minimal stimulus pairs, yielding 160 responses across five communication themes and four pragmatic phenomena. Each pair contained identical utterances presented in different situational contexts. Data were collected using GPT-6 at a ‘High’ reasoning level via Temporary Chat (unpersonalised). Two pragmatics experts independently assessed the responses using a 0–2 ordinal rubric. The analysis comprised Interpretation Accuracy (IA), Pairwise Joint Accuracy (PJA), descriptive comparisons between conditions, and qualitative examination of pragmatic inferences. IA reached 98.13% for Rater 1 and 91.25% for Rater 2, while PJA reached 96.25% and 82.50%, respectively. Exact agreement was 93.13%, with a weighted Cohen’s kappa of 0.332, indicating limitations in scoring consistency. According to Rater 2, pairwise accuracy was highest for indirect criticism or evaluation (95.00%) and lowest for indirect refusal (65.00%). Accuracy was descriptively higher in Condition A (96.25%) than in Condition B (86.25%). Qualitative analysis identified instances of inferences exceeding contextual support and insufficient use of relevant contextual information. These findings highlight the importance of evaluating both interpretative accuracy and contextual justification, while acknowledging the sensitivity of performance estimates to rater judgments.        </dc:description>
	<dc:publisher xml:lang="en">Universitas Negeri Semarang</dc:publisher>
	<dc:date>2026-10-10</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en">Peer-reviewed Article</dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://journal.unnes.ac.id/journals/rainbow/article/view/65667</dc:identifier>
	<dc:identifier>10.15294/rainbow.v15i3.65667</dc:identifier>
	<dc:source xml:lang="en">Rainbow: Journal of Literature, Linguistics and Culture Studies; Vol. 15 No. 3 (2026): October 2026; 299-312</dc:source>
	<dc:source>2721-4540</dc:source>
	<dc:source>2252-6323</dc:source>
	<dc:source>10.15294/rainbow.v15i3</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://journal.unnes.ac.id/journals/rainbow/article/view/65667/12064</dc:relation>
	<dc:rights xml:lang="en">Copyright (c) 2026 Aris Badara, Rifa'ie, Yusoh, Salama (Author)</dc:rights>
	<dc:rights xml:lang="en">https://creativecommons.org/licenses/by/4.0</dc:rights>
</oai_dc:dc>