Melola Discovery

← Melola Discovery

Jurnal Tanpa peringkat SINTA

Semantic Specificity Degradation in Zero-Shot Generative Vision-Language Models

Edu Komputika Journal · Vol. 13 No. 1 (2026) · hlm. 87–96

DOI
10.15294/edukom.v13i1.44852
ISSN
2599-297X, 2252-6811
Terbit
2026-08-31
Penerbit
Universitas Negeri Semarang
Bahasa
eng

Abstrak

Vision-Language Models (VLMs) have shown strong zero-shot performance in generating free-form image descriptions. However, most evaluations focus on hallucinated content, while less attention is given to whether models preserve specific object identities. This study investigates whether zero-shot generative VLMs retain fine-grained semantic information or tend to produce broader category labels. We introduce the Generic Downgrade Rate (GDR), a metric that measures predictions that are correct at a general category level but incorrect at the specific class level. A controlled zero-shot experiment was conducted using BLIP-2 (OPT-2.7B) on a balanced subset of Food-101 containing 101 classes and 10 images per class (N = 1,010). Model outputs were evaluated using Fine Accuracy, Coarse Accuracy, and GDR. Fine Accuracy was 0.0099, while Coarse Accuracy reached 0.3069. The resulting GDR of 0.2970 indicates that many predictions captured the general food category but failed to preserve the specific class identity. Bootstrap confidence intervals showed that the difference between fine- and coarse-level accuracy was stable across resampled data. Overall, the results indicate a clear loss of semantic specificity in zero-shot generative outputs. By distinguishing general semantic correctness from exact class identification, this evaluation provides a more detailed view of how VLMs preserve different levels of semantic information.

Kata kunci

Jaringan kolaborasi para penulis

9 penulis · 19 tautan kepenulisan bersama · 5 lintas organisasi

Arahkan kursor atau ketuk sebuah nama untuk menyorot rekan penulisnya. Klik dua kali untuk membuka halaman penulis.

  • University of Palangka Raya
  • Swinburne University of Technology
  • Organisasi berbeda
  • Organisasi sama
  • Afiliasi tidak diketahui
  • Garis lebih tebal = lebih banyak artikel bersama · lingkaran lebih besar = lebih banyak artikel
Tampilkan sebagai tabel
Rekan penulisOrganisasiArtikel bersama
Rony TeguhUniversity of Palangka Raya1
Adventus PandaUniversity of Palangka Raya1
Devi KarolitaUniversity of Palangka Raya1
Indra Fiqi RipaniUniversity of Palangka Raya1
Ariesta LestariUniversity of Palangka Raya1
Angeline Novia ToemonUniversity of Palangka Raya1
Misita AnwarSwinburne University of Technology1

Afiliasi berasal dari OpenAlex; metadata OAI-PMH jurnal tidak memuatnya. Penulis tanpa ID OpenAlex dicocokkan berdasarkan nama, sehingga dua orang bisa tergabung atau satu orang bisa terpisah.

Sumber metadata: OAI-PMH (metadata jurnal) https://journal.unnes.ac.id/journals/edukom/oai, pertama di-harvest 5 Nov 2026, terakhir terlihat 8 Nov 2026.
Metadata asli (oai_dc)
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en">Semantic Specificity Degradation in Zero-Shot Generative Vision-Language Models</dc:title>
	<dc:creator xml:lang="en">Nugrahaningsih, Nahumi</dc:creator>
	<dc:creator xml:lang="en">Sylviana, Felicia</dc:creator>
	<dc:subject xml:lang="en">Hierarchical Semantics</dc:subject>
	<dc:subject xml:lang="en">Multimodal Evaluation</dc:subject>
	<dc:subject xml:lang="en">Semantic Specificity</dc:subject>
	<dc:subject xml:lang="en">Vision-Language Models</dc:subject>
	<dc:subject xml:lang="en">Zero-Shot Learning </dc:subject>
	<dc:description xml:lang="en">Vision-Language Models (VLMs) have shown strong zero-shot performance in generating free-form image descriptions. However, most evaluations focus on hallucinated content, while less attention is given to whether models preserve specific object identities. This study investigates whether zero-shot generative VLMs retain fine-grained semantic information or tend to produce broader category labels. We introduce the Generic Downgrade Rate (GDR), a metric that measures predictions that are correct at a general category level but incorrect at the specific class level. A controlled zero-shot experiment was conducted using BLIP-2 (OPT-2.7B) on a balanced subset of Food-101 containing 101 classes and 10 images per class (N = 1,010). Model outputs were evaluated using Fine Accuracy, Coarse Accuracy, and GDR. Fine Accuracy was 0.0099, while Coarse Accuracy reached 0.3069. The resulting GDR of 0.2970 indicates that many predictions captured the general food category but failed to preserve the specific class identity. Bootstrap confidence intervals showed that the difference between fine- and coarse-level accuracy was stable across resampled data. Overall, the results indicate a clear loss of semantic specificity in zero-shot generative outputs. By distinguishing general semantic correctness from exact class identification, this evaluation provides a more detailed view of how VLMs preserve different levels of semantic information.</dc:description>
	<dc:publisher xml:lang="en">Universitas Negeri Semarang</dc:publisher>
	<dc:date>2026-08-31</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en">Peer-reviewed Article</dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://journal.unnes.ac.id/journals/edukom/article/view/44852</dc:identifier>
	<dc:identifier>10.15294/edukom.v13i1.44852</dc:identifier>
	<dc:source xml:lang="en">Edu Komputika Journal; Vol. 13 No. 1 (2026): Edu Komputika Journal; 87-96</dc:source>
	<dc:source>2599-297X</dc:source>
	<dc:source>2252-6811</dc:source>
	<dc:source>10.15294/edukom.v13i1</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://journal.unnes.ac.id/journals/edukom/article/view/44852/11468</dc:relation>
	<dc:rights xml:lang="en">Copyright (c) 2026 Edu Komputika Journal</dc:rights>
</oai_dc:dc>