End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning

Abstract

Sketch-based face recognition is an interesting task in vision and multimedia research, yet it is quite challenging due to the great difference between face photos and sketches. In this paper, we propose a novel approach for photo-sketch generation, aiming to automatically transform face photos into detail-preserving personal sketches. Unlike the traditional models synthesizing sketches based on a dictionary of exemplars, we develop a fully convolutional network to learn the end-to-end photo-sketch mapping. Our approach takes whole face photos as inputs and directly generates the corresponding sketch images with efficient inference and learning, in which the architecture are stacked by only convolutional kernels of very small sizes. To well capture the person identity during the photo-sketch transformation, we define our optimization objective in the form of joint generative-discriminative minimization. In particular, a discriminative regularization term is incorporated into the photo-sketch generation, enhancing the discriminability of the generated person sketches against other individuals. Extensive experiments on several standard benchmarks suggest that our approach outperforms other state-of-the-art methods in both photo-sketch generation and face sketch verification.

8 pages, 6 figures. Proceeding in ACM International Conference on Multimedia Retrieval (ICMR), 2015

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Fully Convolutional Networks for Semantic Segmentation
• Deep Joint Task Learning for Generic Object Extraction

Cited by in corpus (9)

• Identity-Aware CycleGAN for Face Photo-Sketch Synthesis and Recognition
• CSGAN: Cyclic-Synthesized Generative Adversarial Networks for Image-to-Image Transformation
• Freehand Sketch Generation from Mechanical Components
• Sparse Graphical Representation based Discriminant Analysis for Heterogeneous Face Recognition
• HoneyFaces: Increasing the Security and Privacy of Authentication Using Synthetic Facial Images
• Emerging Biometrics: Deep Inference and Other Computational Intelligence
• Enabling Robots to Draw and Tell: Towards Visually Grounded Multimodal Description Generation
• Face Sketch Synthesis via Semantic-Driven Generative Adversarial Network
• Face Sketch Synthesis with Style Transfer using Pyramid Column Feature