Akses terbuka
Jurnal
SINTA 1
Indonesian-English Textual Similarity Detection Using Universal Sentence Encoder (USE) and Facebook AI Similarity Search (FAISS)
CommIT (Communication and Information Technology) Journal · Vol. 18 No. 2 (2024) · hlm. 183–195
ISSN 2460-7010, 1979-2484
Terbit 2024-09-10
Penerbit Bina Nusantara University
Bahasa eng
Abstrak
The tremendous development in Natural Language Processing (NLP) has enabled the detection of bilingual and multilingual textual similarity. One of the main challenges of the Textual Similarity Detection (TSD) system lies in learning effective text representation. The research focuses on identifying similar texts between Indonesian and English across a broad range of semantic similarity spectrums. The primary challenge is generating English and Indonesian dense vector representation, a.k.a. embeddings that share a single vector space. Through trial and error, the research proposes using the Universal Sentence Encoder (USE) model to construct bilingual embeddings and FAISS to index the bilingual dataset. The comparison between query vectors and index vectors is done using two approaches: the heuristic comparison with Euclidian distance and a clustering algorithm, Approximate Nearest Neighbors (ANN). The system is tested with four different semantic granularities, two text granularities, and evaluation metrics with a cutoff value of k={2,10}. Four semantic granularities used are highly similar or near duplicate, Semantic Entailment (SE), Topically Related (TR), and Out of Topic (OOT), while the text granularities take on the sentence and paragraph levels. The experimental results demonstrate that the proposed system successfully ranks similar texts in different languages within the top ten. It has been proven by the highest F1@2 score of 0.96 for the near duplicate category on the sentence level. Unlike the near-duplicate category, the highest F1 scores of 0.77 and 0.89 are shown by the SE and TR categories, respectively. The experiment results also show a high correlation between text and semantic granularity.
Jaringan kolaborasi para penulis
25 penulis ·
53 tautan kepenulisan bersama
· 15 lintas organisasi
Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.
Multimedia University Duta Wacana Christian University Telkom University SRM Institute of Science and Technology Jining Normal University Afiliasi tidak diketahui
Organisasi berbeda
Organisasi sama
Afiliasi tidak diketahui
Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.
Sumber metadata: OAI-PMH (metadata jurnal) https://journal.binus.ac.id/index.php/commit/oai, pertama di-harvest 8 Nov 2026, terakhir terlihat 16 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
<dc:title xml:lang="en-US">Indonesian-English Textual Similarity Detection Using Universal Sentence Encoder (USE) and Facebook AI Similarity Search (FAISS)</dc:title>
<dc:creator>Krisnawati, Lucia D.</dc:creator>
<dc:creator>Mahastama, Aditya W.</dc:creator>
<dc:creator>Haw, Su-Cheng</dc:creator>
<dc:creator>Ng, Kok-Why</dc:creator>
<dc:creator>Naveen, Palanichamy</dc:creator>
<dc:subject xml:lang="en-US">Textual Similarity Detection</dc:subject>
<dc:subject xml:lang="en-US">Universal Sentence Encoder (USE)</dc:subject>
<dc:subject xml:lang="en-US">Facebook AI Similarity Search (FAISS)</dc:subject>
<dc:description xml:lang="en-US">The tremendous development in Natural Language Processing (NLP) has enabled the detection of bilingual and multilingual textual similarity. One of the main challenges of the Textual Similarity Detection (TSD) system lies in learning effective text representation. The research focuses on identifying similar texts between Indonesian and English across a broad range of semantic similarity spectrums. The primary challenge is generating English and Indonesian dense vector representation, a.k.a. embeddings that share a single vector space. Through trial and error, the research proposes using the Universal Sentence Encoder (USE) model to construct bilingual embeddings and FAISS to index the bilingual dataset. The comparison between query vectors and index vectors is done using two approaches: the heuristic comparison with Euclidian distance and a clustering algorithm, Approximate Nearest Neighbors (ANN). The system is tested with four different semantic granularities, two text granularities, and evaluation metrics with a cutoff value of k={2,10}. Four semantic granularities used are highly similar or near duplicate, Semantic Entailment (SE), Topically Related (TR), and Out of Topic (OOT), while the text granularities take on the sentence and paragraph levels. The experimental results demonstrate that the proposed system successfully ranks similar texts in different languages within the top ten. It has been proven by the highest F1@2 score of 0.96 for the near duplicate category on the sentence level. Unlike the near-duplicate category, the highest F1 scores of 0.77 and 0.89 are shown by the SE and TR categories, respectively. The experiment results also show a high correlation between text and semantic granularity.</dc:description>
<dc:publisher xml:lang="en-US">Bina Nusantara University</dc:publisher>
<dc:date>2024-09-10</dc:date>
<dc:type>info:eu-repo/semantics/article</dc:type>
<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
<dc:type xml:lang="en-US">Peer-reviewed Article</dc:type>
<dc:format>application/pdf</dc:format>
<dc:identifier>https://journal.binus.ac.id/index.php/commit/article/view/11274</dc:identifier>
<dc:identifier>10.21512/commit.v18i2.11274</dc:identifier>
<dc:source xml:lang="en-US">CommIT (Communication and Information Technology) Journal; Vol. 18 No. 2 (2024): CommIT Journal ; 183-195</dc:source>
<dc:source>2460-7010</dc:source>
<dc:source>1979-2484</dc:source>
<dc:language>eng</dc:language>
<dc:relation>https://journal.binus.ac.id/index.php/commit/article/view/11274/5179</dc:relation>
<dc:rights xml:lang="en-US">Copyright (c) 2024 Lucia D. Krisnawati, Aditya W. Mahastama, Su-Cheng Haw, Kok-Why Ng, Palanichamy Naveen</dc:rights>
<dc:rights xml:lang="en-US">https://creativecommons.org/licenses/by-sa/4.0</dc:rights>
</oai_dc:dc>
Sitasi dan pengindeksan
Jumlah menurut tiap indeks pada 13 Nov 2026; setiap indeks menghitung sitasi dengan cara berbeda.
OpenAlex juga mencatat karya ini di: crossref, doaj.
Sitasi artikel ini
APA 7
Krisnawati, L. D., Mahastama, A. W., Haw, S. C., Ng, K. W., & Naveen, P. (2024). Indonesian-English Textual Similarity Detection Using Universal Sentence Encoder (USE) and Facebook AI Similarity Search (FAISS). CommIT (Communication and Information Technology) Journal, 18(2), 183–195. https://doi.org/10.21512/commit.v18i2.11274
Salin
IEEE
L. D. Krisnawati, A. W. Mahastama, S. C. Haw, K. W. Ng, and P. Naveen, “Indonesian-English Textual Similarity Detection Using Universal Sentence Encoder (USE) and Facebook AI Similarity Search (FAISS),” CommIT (Communication and Information Technology) Journal, vol. 18, no. 2, pp. 183–195, 2024, doi: 10.21512/commit.v18i2.11274.
Salin
Dibuat dari metadata jurnal; periksa nama penulis sebelum dipakai.