Melola Discovery

← Melola Discovery

Open access Jurnal SINTA 2

Audio-Visual CNN using Transfer Learning for TV Commercial Break Detection

IJCCS (Indonesian Journal of Computing and Cybernetics Systems) · Vol. 17 No. 3 (2023) · pp. 291–300

DOI
10.22146/ijccs.76058
ISSN
2460-7258, 1978-1520
Published
2023-07-31
Publisher
IndoCEISS in colaboration with Universitas Gadjah Mada, Indonesia.
Language
eng
License
creativecommons.org/licenses/by-sa/4.0

Abstract

The TV commercial detection problem is a hard challenge due to the variety of programs and TV channels. The usage of deep learning methods to solve this problem has shown good results. However, it takes a long time with many training epochs to get high accuracy. This research uses transfer learning techniques to reduce training time and limits the number of training epochs to 20. From video data, the audio feature is extracted with Mel-spectrogram representation, and the visual features are picked from a video frame. The datasets were gathered by recording programs from various TV channels in Indonesia. Pre-trained CNN models such as MobileNetV2, InceptionV3, and DenseNet169 are re-trained and are used to detect commercials at the shot level. We do post-processing to cluster the shots into segments of commercials and non-commercials. The best result is shown by Audio-Visual CNN using transfer learning with an accuracy of 93.26% with only 20 training epochs. It is faster and better than the CNN model without using transfer learning with an accuracy of 88.17% and 77 training epochs. The result by adding post-processing increases the accuracy of Audio-Visual CNN using transfer learning to 96.42%.

Keywords

Collaboration network of the authors

10 authors · 14 co-authorship links

Hover or tap a name to highlight its co-authors. Double-click to open the author's page.

  • Universitas Gadjah Mada
  • Different organisations
  • Same organisation
  • Affiliation unknown
  • Thicker line = more joint articles · bigger circle = more articles
Show as table
Co-authorOrganisationJoint articles
Muhammad AnsharUniversitas Gadjah Mada1
Raden SumihartoUniversitas Gadjah Mada1
Hizbullah HamdaUniversitas Gadjah Mada1
Sri MulyanaUniversitas Gadjah Mada1
Arie KurniawanUniversitas Gadjah Mada1
Retantyo WardoyoUniversitas Gadjah Mada1
Doni SetyawanUniversitas Gadjah Mada1
E. Elsa Herdiana MurhandarwatiUniversitas Gadjah Mada1

Affiliations come from OpenAlex; journal OAI-PMH metadata carries none. Authors without an OpenAlex ID are matched by name, which can merge or split people.

Metadata source: OAI-PMH (metadata jurnal) https://journal.ugm.ac.id/ijccs/oai, first harvested 26 Okt 2026, last seen 26 Okt 2026.
Original metadata (oai_dc) 
<oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
	<dc:title xml:lang="en-US">Audio-Visual CNN using Transfer Learning for TV Commercial Break Detection</dc:title>
	<dc:creator>Pudya Wardana, Muhammad Zha'farudin</dc:creator>
	<dc:creator>Wibowo, Moh. Edi</dc:creator>
	<dc:subject xml:lang="en-US"></dc:subject>
	<dc:subject xml:lang="en-US">Commercial, TV, CNN, Transfer Learning, InceptionV3, MobileNetV2, DenseNet169, Video</dc:subject>
	<dc:description xml:lang="en-US">The TV commercial detection problem is a hard challenge due to the variety of programs and TV channels. The usage of deep learning methods to solve this problem has shown good results. However, it takes a long time with many training epochs to get high accuracy.    This research uses transfer learning techniques to reduce training time and limits the number of training epochs to 20. From video data, the audio feature is extracted with Mel-spectrogram representation, and the visual features are picked from a video frame. The datasets were gathered by recording programs from various TV channels in Indonesia. Pre-trained CNN models such as MobileNetV2, InceptionV3, and DenseNet169 are re-trained and are used to detect commercials at the shot level. We do post-processing to cluster the shots into segments of commercials and non-commercials.    The best result is shown by Audio-Visual CNN using transfer learning with an accuracy of 93.26% with only 20 training epochs. It is faster and better than the CNN model without using transfer learning with an accuracy of 88.17% and 77 training epochs. The result by adding post-processing increases the accuracy of Audio-Visual CNN using transfer learning to 96.42%.</dc:description>
	<dc:publisher xml:lang="en-US">IndoCEISS in colaboration with Universitas Gadjah Mada, Indonesia.</dc:publisher>
	<dc:contributor xml:lang="en-US"></dc:contributor>
	<dc:date>2023-07-31</dc:date>
	<dc:type>info:eu-repo/semantics/article</dc:type>
	<dc:type>info:eu-repo/semantics/publishedVersion</dc:type>
	<dc:type xml:lang="en-US"></dc:type>
	<dc:format>application/pdf</dc:format>
	<dc:identifier>https://journal.ugm.ac.id/ijccs/article/view/76058</dc:identifier>
	<dc:identifier>10.22146/ijccs.76058</dc:identifier>
	<dc:source xml:lang="en-US">IJCCS (Indonesian Journal of Computing and Cybernetics Systems); Vol 17, No 3 (2023): July; 291-300</dc:source>
	<dc:source>2460-7258</dc:source>
	<dc:source>1978-1520</dc:source>
	<dc:language>eng</dc:language>
	<dc:relation>https://journal.ugm.ac.id/ijccs/article/view/76058/36987</dc:relation>
	<dc:rights xml:lang="en-US">Copyright (c) 2023 IJCCS (Indonesian Journal of Computing and Cybernetics Systems)</dc:rights>
	<dc:rights xml:lang="en-US">http://creativecommons.org/licenses/by-sa/4.0</dc:rights>
</oai_dc:dc>