Melola Discovery

← Melola Discovery

Open access Journal SINTA 1

SeeAround: an offline mobile live support system for the visually impaired

Bulletin of Electrical Engineering and Informatics · Vol. 14 No. 1 (2024) · pp. 485–504

DOI
10.11591/eei.v14i1.7904
ISSN
2089-3191, 2302-9285
Published
2024-11-22
Publisher
Institute of Advanced Engineering and Science (IAES)
Language
en
License
creativecommons.org/licenses/by-sa/4.0/

Abstract

The inability of blind or partially-sighted people to understand visual content and real-life situations reduces their standard of living, especially in a world mainly tailored for sighted individuals. Despite the progress made by certain devices to assist them in using touch, sound, or other senses, these solutions often fall short of bridging the comprehension gap. Our work proposes an intuitive, user-friendly mobile-based framework named "SeeAround" that is capable of automatically providing real-time audio descriptions of the user's immediate visual surroundings. Our solution addresses this challenge by leveraging key point detection, image captioning, text-to-speech (TTS), optical character recognition (OCR), and translation algorithms to offer comprehensive support for visually impaired individuals. Our system architecture relies on convolutional neural networks (CNNs) such as Inception-V3, Inception-V4, and ResNet152-V2 to extract detailed features from images and employs a multi-gated recurrent unit (GRU) decoder to generate word-by-word natural language descriptions. Our framework was integrated into mobile applications and optimized with TensorFlow lite pre-trained models for easy integration on the Android platform.

Keywords

Collaboration network of the authors

6 authors · 9 co-authorship links · 2 across different organisations

Hover or tap a name to highlight its co-authors. Double-click to open the author's page.

  • Sidi Mohamed Ben Abdellah University
  • Pôle Léonard de Vinci
  • Different organisations
  • Same organisation
  • Affiliation unknown
  • Thicker line = more joint articles · bigger circle = more articles
Show as table
Co-authorOrganisationJoint articles
Mohamed Taj BennaniSidi Mohamed Ben Abdellah University1
Mohamed El MahjoubySidi Mohamed Ben Abdellah University1
Mohamed El FarSidi Mohamed Ben Abdellah University1

Affiliations come from OpenAlex; journal OAI-PMH metadata carries none. Authors without an OpenAlex ID are matched by name, which can merge or split people.

Metadata source: OpenAlex API (jurnal SINTA, ADR 0048) https://openalex.org/S2765016208, first harvested 19 Nov 2026, last seen 19 Nov 2026.
Original metadata (oai_dc)
<oai_dc:dc xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/"><dc:title xml:lang="en">SeeAround: an offline mobile live support system for the visually impaired</dc:title><dc:creator>Othmane Sebban</dc:creator><dc:creator>Ahmed Azough</dc:creator><dc:creator>Mohamed Lamrini</dc:creator><dc:subject>visually impaired people</dc:subject><dc:subject>audio description</dc:subject><dc:subject>mobile assistive technology</dc:subject><dc:subject>image captioning</dc:subject><dc:subject>keypoint detection</dc:subject><dc:subject>optical character recognition</dc:subject><dc:subject>text-to-speech</dc:subject><dc:subject>translation</dc:subject><dc:subject>convolutional neural networks</dc:subject><dc:subject>TensorFlow Lite</dc:subject><dc:subject>Android application</dc:subject><dc:description xml:lang="en">The inability of blind or partially-sighted people to understand visual content and real-life situations reduces their standard of living, especially in a world mainly tailored for sighted individuals. Despite the progress made by certain devices to assist them in using touch, sound, or other senses, these solutions often fall short of bridging the comprehension gap. Our work proposes an intuitive, user-friendly mobile-based framework named "SeeAround" that is capable of automatically providing real-time audio descriptions of the user's immediate visual surroundings. Our solution addresses this challenge by leveraging key point detection, image captioning, text-to-speech (TTS), optical character recognition (OCR), and translation algorithms to offer comprehensive support for visually impaired individuals. Our system architecture relies on convolutional neural networks (CNNs) such as Inception-V3, Inception-V4, and ResNet152-V2 to extract detailed features from images and employs a multi-gated recurrent unit (GRU) decoder to generate word-by-word natural language descriptions. Our framework was integrated into mobile applications and optimized with TensorFlow lite pre-trained models for easy integration on the Android platform.</dc:description><dc:publisher>Institute of Advanced Engineering and Science (IAES)</dc:publisher><dc:date>2024-11-22</dc:date><dc:type>info:eu-repo/semantics/article</dc:type><dc:format>application/pdf</dc:format><dc:identifier>https://doi.org/10.11591/eei.v14i1.7904</dc:identifier><dc:identifier>10.11591/eei.v14i1.7904</dc:identifier><dc:source>Bulletin of Electrical Engineering and Informatics; Vol 14 No 1 (2024); 485-504</dc:source><dc:source>2089-3191</dc:source><dc:source>2302-9285</dc:source><dc:language>en</dc:language><dc:relation>https://beei.org/index.php/EEI/article/download/7904/4055</dc:relation><dc:rights>https://creativecommons.org/licenses/by-sa/4.0/</dc:rights></oai_dc:dc>