Melola Discovery

← Melola Discovery

Akses terbuka Jurnal SINTA 1

MedicalBERT: enhancing biomedical natural language processing using pretrained BERT-based model

IAES International Journal of Artificial Intelligence · Vol. 14 No. 3 (2025) · hlm. 2367–2378

DOI
10.11591/ijai.v14.i3.pp2367-2378
ISSN
2252-8938, 2089-4872
Terbit
2025-06-01
Penerbit
Institute of Advanced Engineering and Science
Bahasa
eng
Lisensi
creativecommons.org/licenses/by-sa/4.0

Abstrak

Recent advances in natural language processing (NLP) have been driven by pretrained language models like BERT, RoBERTa, T5, and GPT. These models excel at understanding complex texts, but biomedical literature, with its domain-specific terminology, poses challenges that models like Word2Vec and bidirectional long short-term memory (Bi-LSTM) can't fully address. GPT and T5, despite capturing context, fall short in tasks needing bidirectional understanding, unlike BERT. Addressing this, we proposed MedicalBERT, a pretrained BERT model trained on a large biomedical dataset and equipped with domain-specific vocabulary that enhances the comprehension of biomedical terminology. MedicalBERT model is further optimized and fine-tuned to address diverse tasks, including named entity recognition, relation extraction, question answering, sentence similarity, and document classification. Performance metrics such as the F1-score, accuracy, and Pearson correlation are employed to showcase the efficiency of our model in comparison to other BERT-based models such as BioBERT, SciBERT, and ClinicalBERT. MedicalBERT outperforms these models on most of the benchmarks, and surpasses the general-purpose BERT model by 5.67% on average across all the tasks evaluated respectively. This work also underscores the potential of leveraging pretrained BERT models for medical NLP tasks, demonstrating the effectiveness of transfer learning techniques in capturing domain-specific information.

Kata kunci

Jaringan kolaborasi para penulis

7 penulis · 16 tautan kepenulisan bersama

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • R.V. College of Engineering
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama
Praveena T.R.V. College of Engineering1

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) http://ijai.iaescore.com/index.php/IJAI/oai, pertama di-harvest 8 Nov 2026, terakhir terlihat 16 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en-US">MedicalBERT: enhancing biomedical natural language processing using pretrained BERT-based model</dc:title>
	<dc:creator>Reddy, K. Sahit</dc:creator>
	<dc:creator>Ragavenderan, N.</dc:creator>
	<dc:creator>K., Vasanth</dc:creator>
	<dc:creator>N. Naik, Ganesh</dc:creator>
	<dc:creator>H, Vishalakshi Prabhu</dc:creator>
	<dc:creator>G. S., Nagaraja</dc:creator>
	<dc:subject xml:lang="en-US">Natural Language Processing; Transformers; Neural Networks</dc:subject>
	<dc:subject xml:lang="en-US">BERT; Named entity recognition; Natural language processing; Pre-trained models; Relation extraction; Transformers</dc:subject>
	<dc:description xml:lang="en-US">Recent advances in natural language processing (NLP) have been driven by pretrained language models like BERT, RoBERTa, T5, and GPT. These models excel at understanding complex texts, but biomedical literature, with its domain-specific terminology, poses challenges that models like Word2Vec and bidirectional long short-term memory (Bi-LSTM) can't fully address. GPT and T5, despite capturing context, fall short in tasks needing bidirectional understanding, unlike BERT. Addressing this, we proposed MedicalBERT, a pretrained BERT model trained on a large biomedical dataset and equipped with domain-specific vocabulary that enhances the comprehension of biomedical terminology. MedicalBERT model is further optimized and fine-tuned to address diverse tasks, including named entity recognition, relation extraction, question answering, sentence similarity, and document classification. Performance metrics such as the F1-score, accuracy, and Pearson correlation are employed to showcase the efficiency of our model in comparison to other BERT-based models such as BioBERT, SciBERT, and ClinicalBERT. MedicalBERT outperforms these models on most of the benchmarks, and surpasses the general-purpose BERT model by 5.67% on average across all the tasks evaluated respectively. This work also underscores the potential of leveraging pretrained BERT models for medical NLP tasks, demonstrating the effectiveness of transfer learning techniques in capturing domain-specific information.</dc:description>
	<dc:publisher xml:lang="en-US">Institute of Advanced Engineering and Science</dc:publisher>
	<dc:contributor xml:lang="en-US"></dc:contributor>
	<dc:date>2025-06-01</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en-US"></dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://ijai.iaescore.com/index.php/IJAI/article/view/25454</dc:identifier>
	<dc:identifier>10.11591/ijai.v14.i3.pp2367-2378</dc:identifier>
	<dc:source xml:lang="en-US">IAES International Journal of Artificial Intelligence (IJ-AI); Vol 14, No 3: June 2025; 2367-2378</dc:source>
	<dc:source>2252-8938</dc:source>
	<dc:source>2089-4872</dc:source>
	<dc:source>10.11591/ijai.v14.i3</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://ijai.iaescore.com/index.php/IJAI/article/view/25454/14578</dc:relation>
	<dc:rights xml:lang="en-US">Copyright (c) 2025 Institute of Advanced Engineering and Science</dc:rights>
	<dc:rights xml:lang="en-US">http://creativecommons.org/licenses/by-sa/4.0</dc:rights>
</oai_dc:dc>