SIGAN: a generative adversarial network architecture for sketch to photo synthesis

Abstract

Of late, with the rise of artificial intelligence (AI) and deep learning (DL) models, image translation has become a very important phenomena which could produce realistic photographic results. Synthesizing new images is widely used in different applications including the ones used by investigation agencies. Image generation from hand-drawn sketch to realistic photos and vice versa is required in different computer vision applications. Generative adversarial network (GAN) architecture is extensively employed for generating images. However, there is need for investigating further on improvising GAN architecture and the underlying loss functions towards leveraging performance. In this paper, we put forth a GAN architecture known as sketch-image GAN (SIGAN) for synthesizing realistic photos from hand-drawn sketches. Both generator (G) and discriminator (D) components are designed based on DL models following a non-cooperative game theory towards improving image generation performance. SIGAN exploits improvised image representation and learning of data distribution. The algorithm we have proposed is known as learning-based sketch-image generation (LbSIG). This algorithm exploits SIGAN architecture for efficiently generating realistic photo from given hand-drawn sketch. SIGAN is assessed using a benchmark dataset called CUHK face sketch database (CUFS). From the empirical study, it is observed that the proposed SIGAN architecture with underlying deep learning models could outperform existing GAN models in terms of Fréchet inception distance (FID) with 38.2346%.