Melola Discovery

← Melola Discovery

Open access Journal

Indonesian–Arabic Academic Translation Quality: A Comparative Content Analysis of ChatGPT and Google Translate

Ta'lim al-'Arabiyyah: Jurnal Pendidikan Bahasa Arab & Kebahasaaraban · Vol. 9 No. 2 (2025) · pp. 207–223

DOI
10.15575/jta.v9i2.51510
ISSN
2721-5237, 2549-208X
Published
2025-12-25
Publisher
UIN Sunan Gunung Djati Bandung
Language
eng
License
creativecommons.org/licenses/by-sa/4.0

Abstract

This study compares the quality of Indonesian–Arabic academic translations produced by Google Translate and ChatGPT (GPT-4), a topic rarely examined despite widespread MT use in Arabic Language Education programs. Using a qualitative descriptive design with content analysis, the data comprised ten purposively selected undergraduate thesis titles from Indonesian university repositories, including UIN Sunan Kalijaga, UIN Suska Riau, UIN Imam Bonjol Padang, and Universitas Muhammadiyah Makassar. Titles were selected for their academic rigor and need for conceptual precision and formal Arabic register. Translations were analyzed at the phrase level using a Hybrid MQM–Nababan–Baker rubric encompassing seven dimensions: accuracy, acceptability, readability, lexical equivalence, grammatical equivalence, cohesion and coherence, and academic fluency, each rated on a three-point scale. Validity was ensured through alignment with Arabic translation theory, equivalence frameworks, and MQM standards, while iterative consistency checks supported reliability. Results reveal clear differences. ChatGPT achieved an average score of 20.0 out of 21 points (93%), which is classified as Very Good and indicates strong suitability for academic publication. Google Translate scored 13.3 out of 21 points (63%), classified as Good, but requiring post-editing. ChatGPT excelled in contextual meaning, syntactic restructuring, accurate idhafah, case governance, consistent terminology, and scholarly style, whereas Google Translate showed literal transfer. This study enriches AI-assisted translation discourse by grounding evaluation in Arabic translation theory and pedagogy, emphasizing generative AI’s pedagogical potential as a complementary tool while reaffirming the indispensable role of human expertise in maintaining linguistic accuracy, rigor, and academic standards.

Keywords

Collaboration network of the authors

34 authors · 73 co-authorship links · 1 across different organisations

Hover or tap a name to highlight its co-authors. Double-click to open the author's page.

  • Hasanuddin University
  • Universitas Negeri Padang
  • Muhammadiyah University of Surakarta
  • Affiliation unknown
  • Different organisations
  • Same organisation
  • Affiliation unknown
  • Thicker line = more joint articles · bigger circle = more articles
Show as table
Co-authorOrganisationJoint articles
Elsa YuniartiUniversitas Negeri Padang1
Neviyarni NeviyarniAffiliation unknown1
Mahalul AzamAffiliation unknown1
Netrawati NetrawatiAffiliation unknown1
Rihaliza RihalizaAffiliation unknown1
Firman FirmanHasanuddin University1
Harun Joko PrayitnoMuhammadiyah University of Surakarta1
Sonny ZulhudaAffiliation unknown1
Miftachul HudaAffiliation unknown1
Mohamad AliAffiliation unknown1
SUCI RAHMADANIHasanuddin University1
Mirza HazakiAffiliation unknown1
Arif FirmansyahAffiliation unknown1
Desi RoyantiAffiliation unknown1
Muza Agustina Muza AgustinaAffiliation unknown1
Dian MurdianaAffiliation unknown1
Maya Sari Maya SariAffiliation unknown1
Mursyid DjawasAffiliation unknown1
Cut Putrau UjongAffiliation unknown1
Dian Eka RahmawatiAffiliation unknown1
Candra Ayu HanggaratriAffiliation unknown1
Fajar Aswad AsruddinAffiliation unknown1
Mahyudin RitongaAffiliation unknown1
Adam MudinillahAffiliation unknown1
Eva ArdinalAffiliation unknown1
Tauhid TauhidAffiliation unknown1
Naqiyah NaqiyahAffiliation unknown1
Nita TrianaAffiliation unknown1
NaqiyahAffiliation unknown1
Laily LiddiniAffiliation unknown1

Affiliations come from OpenAlex; journal OAI-PMH metadata carries none. Authors without an OpenAlex ID are matched by name, which can merge or split people.

Metadata source: OAI-PMH (metadata jurnal) https://journal.uinsgd.ac.id/index.php/Talim/oai, first harvested 7 Nov 2026, last seen 7 Nov 2026.
Original metadata (oai_dc) 
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en-US">Indonesian–Arabic Academic Translation Quality: A Comparative Content Analysis of ChatGPT and Google Translate</dc:title>
	<dc:creator>Alfitri, Alfitri</dc:creator>
	<dc:creator>Zakaria, Gamal Abdul Nasir</dc:creator>
	<dc:creator>Misran, Misran</dc:creator>
	<dc:creator>Nurdianto, Talqis</dc:creator>
	<dc:subject xml:lang="en-US">Arabic language</dc:subject>
	<dc:subject xml:lang="en-US">ChatGPT</dc:subject>
	<dc:subject xml:lang="en-US">Google Translate</dc:subject>
	<dc:subject xml:lang="en-US">Indonesian-Arabic Translation</dc:subject>
	<dc:subject xml:lang="en-US">Machine translation</dc:subject>
	<dc:subject xml:lang="en-US">Translation quality evaluation</dc:subject>
	<dc:description xml:lang="en-US">This study compares the quality of Indonesian–Arabic academic translations produced by Google Translate and ChatGPT (GPT-4), a topic rarely examined despite widespread MT use in Arabic Language Education programs. Using a qualitative descriptive design with content analysis, the data comprised ten purposively selected undergraduate thesis titles from Indonesian university repositories, including UIN Sunan Kalijaga, UIN Suska Riau, UIN Imam Bonjol Padang, and Universitas Muhammadiyah Makassar. Titles were selected for their academic rigor and need for conceptual precision and formal Arabic register. Translations were analyzed at the phrase level using a Hybrid MQM–Nababan–Baker rubric encompassing seven dimensions: accuracy, acceptability, readability, lexical equivalence, grammatical equivalence, cohesion and coherence, and academic fluency, each rated on a three-point scale. Validity was ensured through alignment with Arabic translation theory, equivalence frameworks, and MQM standards, while iterative consistency checks supported reliability. Results reveal clear differences. ChatGPT achieved an average score of 20.0 out of 21 points (93%), which is classified as Very Good and indicates strong suitability for academic publication. Google Translate scored 13.3 out of 21 points (63%), classified as Good, but requiring post-editing. ChatGPT excelled in contextual meaning, syntactic restructuring, accurate idhafah, case governance, consistent terminology, and scholarly style, whereas Google Translate showed literal transfer. This study enriches AI-assisted translation discourse by grounding evaluation in Arabic translation theory and pedagogy, emphasizing generative AI’s pedagogical potential as a complementary tool while reaffirming the indispensable role of human expertise in maintaining linguistic accuracy, rigor, and academic standards.</dc:description>
	<dc:publisher xml:lang="en-US">UIN Sunan Gunung Djati Bandung</dc:publisher>
	<dc:date>2025-12-25</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://journal.uinsgd.ac.id/index.php/Talim/article/view/51510</dc:identifier>
	<dc:identifier>10.15575/jta.v9i2.51510</dc:identifier>
	<dc:source xml:lang="en-US">Ta'lim al-'Arabiyyah: Jurnal Pendidikan Bahasa Arab &amp; Kebahasaaraban; Vol. 9 No. 2 (2025): Ta'lim al-'Arabiyyah: Jurnal Pendidikan Bahasa Arab dan Kebahasaaraban; 207-223</dc:source>
	<dc:source>2721-5237</dc:source>
	<dc:source>2549-208X</dc:source>
	<dc:source>10.15575/jta.v9i2</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://journal.uinsgd.ac.id/index.php/Talim/article/view/51510/15745</dc:relation>
	<dc:relation>https://journal.uinsgd.ac.id/index.php/Talim/article/view/51510/15744</dc:relation>
	<dc:rights xml:lang="en-US">Copyright (c) 2025 Alfitri Alfitri, Gamal Abdul Nasir Zakaria, Misran Misran, Talqis Nurdianto</dc:rights>
	<dc:rights xml:lang="en-US">https://creativecommons.org/licenses/by-sa/4.0</dc:rights>
</oai_dc:dc>