Melola Discovery

← Melola Discovery

Jurnal SINTA 2

Development of Focused Crawlers for Building Large Punjabi News Corpus

Journal of ICT Research and Applications (JICTRA) · Vol. 15 No. 3 (2021) · hlm. 205–215

DOI
10.5614/itbj.ict.res.appl.2021.15.3.1
ISSN
2338-5499, 2337-5787
Terbit
2021-12-28
Penerbit
DRI - ITB
Bahasa
eng

Abstrak

Web crawlers are as old as the Internet and are most commonly used by search engines to visit websites and index them into repositories. They are not limited to search engines but are also widely utilized to build corpora in different domains and languages. This study developed a focused set of web crawlers for three Punjabi news websites. The web crawlers were developed to extract quality text articles and add them to a local repository to be used in further research. The crawlers were implemented using the Python programming language and were utilized to construct a corpus of more than 134,000 news articles in nine different news genres. The crawler code and extracted corpora were made publicly available to the scientific community for research purposes.

Kata kunci

Jaringan kolaborasi para penulis

2 penulis · 1 tautan kepenulisan bersama

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • Punjabi University
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) https://journals.itb.ac.id/index.php/jictra/oai, pertama di-harvest 26 Okt 2026, terakhir terlihat 15 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en-US">Development of Focused Crawlers for Building Large Punjabi News Corpus</dc:title>
	<dc:creator>Mahi, Gurjot Singh</dc:creator>
	<dc:creator>Verma, Amandeep</dc:creator>
	<dc:subject xml:lang="en-US">corpus</dc:subject>
	<dc:subject xml:lang="en-US">crawler</dc:subject>
	<dc:subject xml:lang="en-US">NLP</dc:subject>
	<dc:subject xml:lang="en-US">Punjabi language</dc:subject>
	<dc:subject xml:lang="en-US">scraper</dc:subject>
	<dc:subject xml:lang="en-US">text extraction</dc:subject>
	<dc:subject xml:lang="en-US">text processing</dc:subject>
	<dc:description xml:lang="en-US"> 
Web crawlers are as old as the Internet and are most commonly used by search engines to visit websites and index them into repositories. They are not limited to search engines but are also widely utilized to build corpora in different domains and languages. This study developed a focused set of web crawlers for three Punjabi news websites. The web crawlers were developed to extract quality text articles and add them to a local repository to be used in further research. The crawlers were implemented using the Python programming language and were utilized to construct a corpus of more than 134,000 news articles in nine different news genres. The crawler code and extracted corpora were made publicly available to the scientific community for research purposes.</dc:description>
	<dc:publisher xml:lang="en-US">DRI - ITB</dc:publisher>
	<dc:date>2021-12-28</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en-US">Peer-reviewed Article</dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://journals.itb.ac.id/index.php/jictra/article/view/16160</dc:identifier>
	<dc:identifier>10.5614/itbj.ict.res.appl.2021.15.3.1</dc:identifier>
	<dc:source xml:lang="en-US">Journal of ICT Research and Applications; Vol. 15 No. 3 (2021); 205-215</dc:source>
	<dc:source>2338-5499</dc:source>
	<dc:source>2337-5787</dc:source>
	<dc:source>10.5614/itbj.ict.res.appl.2021.15.3</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://journals.itb.ac.id/index.php/jictra/article/view/16160/5558</dc:relation>
	<dc:rights xml:lang="en-US">Copyright (c) 2021 Journal of ICT Research and Applications</dc:rights>
</oai_dc:dc>