Jurnal Tanpa peringkat SINTA
Combining Multiple Text Representations for Improved Automatic Evaluation of Indonesian Essay Answers
Scientific Journal of Informatics · Vol. 11 No. 3 (2024) · hlm. 681–692
- ISSN
- 2407-7658, 2460-0040
- Terbit
- 2024-08-30
- Penerbit
- Universitas Negeri Semarang
- Bahasa
- eng
Abstrak
multiple-choices, regarding students’ learning achievement. When the number of students in a class is huge, however, examinations using essay questions become hard to conduct and take a long evaluation time. Automatic essay evaluation has, therefore, become a potential approach in this situation. Various methods have been proposed, however, optimal solutions for such evaluation in the Indonesian language are less known. Furthermore, with the rapid development of machine learning approaches, in particular deep learning approaches, the investigation of such optimal solutions becomes more necessary. Method: To address the aforementioned issue, this study proposed the investigation of text representation approaches for optimal automatic evaluation of Indonesian essay answers. The investigation compared pre-trained word embedding methods such as Word2vec, GloVe, FastText, and RoBERTa, as well as compared text encoding methods such as long short-term memories (LSTMs) and transformers. LSTMs are able to capture temporal semantics by employing state variables, while transformers are able to capture long-term dependency between parts of their input sequences. Additionally, we investigated classification-based and similarity-based training to build text encoders. We expected that these training approaches allowed encoders to extract different views of information. We compared classification results produced by different text encoders and combinations of text encoders. Result: We evaluated various text representation approaches using the UKARA dataset. Our experiments showed that the FastText word embedding method outperformed the Word2vec, GloVe, and RoBERTa methods. The FastText method achieved an F1-score of 75.43% on validation sets, while the Word2vec, GloVe, and RoBERTa methods achieved F1-scores of 69.56%, 74.53%, and 72.87%, respectively. In addition, the experiments showed that combinations of text encoders outperformed individual encoders. The combination of the LSTM encoder, the transformer encoder, and the TF-IDF encoder obtained an F1-score of 77.22% in the best case, which is better than the best F1-scores of the individual LSTM encoders (75.35%), the best combination of transformer encoders (71.49%), and the individual TF-IDF encoder (76.69%). We observed that LSTM encoders produced better performance when they were built using classification-based training. Meanwhile, the transformer encoders obtained better performance when built using similarity-based training. Novelty: The novelty proposed in this research is the optimal combination of text encoders specifically constructed for the evaluation of essay answers in the Indonesian language. Our experiments showed that the combination of three encoders - namely the LSTM encoder built using classification-based training, the transformer encoder built using classification-based and similarity-based training, and the TF-IDF encoder - obtained the best classification performance.
Kata kunci
Jaringan kolaborasi para penulis
39 penulis · 56 tautan kepenulisan bersama · 6 lintas organisasi
Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.
- Universitas Gadjah Mada
- Institut Teknologi Del
- Universitas Teknokrat Indonesia
- Jenderal Soedirman University
- Afiliasi tidak diketahui
- Organisasi berbeda
- Organisasi sama
- Afiliasi tidak diketahui
- Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
| Rekan penulis | Organisasi | Artikel bersama |
|---|---|---|
| Sigit Priyanta | Universitas Gadjah Mada | 2 |
| Retantyo Wardoyo | Universitas Gadjah Mada | 2 |
| Anifudin Azis | Universitas Gadjah Mada | 1 |
| Praretno Wibowo | Afiliasi tidak diketahui | 1 |
| Juwita Maharanti | Afiliasi tidak diketahui | 1 |
| Iqnatius Dimas Nugroho | Universitas Gadjah Mada | 1 |
| Amelia Nursanti | Universitas Gadjah Mada | 1 |
| Tri Kuntoro Priyambodo | Universitas Gadjah Mada | 1 |
| Zaied Shouran | Universitas Gadjah Mada | 1 |
| Nur Azizul Haqimi | Universitas Gadjah Mada | 1 |
| Friska Ayu Listya | Universitas Gadjah Mada | 1 |
| Wahyono Wahyono | Universitas Gadjah Mada | 1 |
| Riska Amalia Praptiwi | Universitas Gadjah Mada | 1 |
| Dayan Ramly Ramadhan | Universitas Gadjah Mada | 1 |
| Hendri Kurniawan Prakosa | Universitas Gadjah Mada | 1 |
| Anindhita Nisitasari | Universitas Gadjah Mada | 1 |
| Raymond Luis | Universitas Gadjah Mada | 1 |
| Muhammad Zha'farudin Pudya Wardana | Universitas Gadjah Mada | 1 |
| Muhammad Anshar | Universitas Gadjah Mada | 1 |
| Raden Sumiharto | Universitas Gadjah Mada | 1 |
| Sarah Rosdiana Tambunan | Institut Teknologi Del | 1 |
| Luthfia Nurma Hapsari | Universitas Gadjah Mada | 1 |
| Hizbullah Hamda | Universitas Gadjah Mada | 1 |
| Arie Kurniawan | Universitas Gadjah Mada | 1 |
| Sri Mulyana | Universitas Gadjah Mada | 1 |
| Reka Alamsyah | Universitas Gadjah Mada | 1 |
| Doni Setyawan | Universitas Gadjah Mada | 1 |
| E. Elsa Herdiana Murhandarwati | Universitas Gadjah Mada | 1 |
| Debby Alita | Universitas Teknokrat Indonesia | 1 |
| Muslikhah Muslikhah | Afiliasi tidak diketahui | 1 |
| Elih Sudiapermana | Afiliasi tidak diketahui | 1 |
| Arief Kelik Nugroho | Jenderal Soedirman University | 1 |
| Hardyanto Soebono | Universitas Gadjah Mada | 1 |
| Dina Fitriana Rosyada | Universitas Gadjah Mada | 1 |
| Krida Tri Wahyuli | Universitas Gadjah Mada | 1 |
| Muhammad Hasani | Universitas Gadjah Mada | 1 |
Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.