End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning

Abstract

Sketch-based face recognition is an interesting task in vision and multimedia research, yet it is quite challenging due to the great difference between face photos and sketches. In this paper, we propose a novel approach for photo-sketch generation, aiming to automatically transform face photos into detail-preserving personal sketches. Unlike the traditional models synthesizing sketches based on a dictionary of exemplars, we develop a fully convolutional network to learn the end-to-end photo-sketch mapping. Our approach takes whole face photos as inputs and directly generates the corresponding sketch images with efficient inference and learning, in which the architecture are stacked by only convolutional kernels of very small sizes. To well capture the person identity during the photo-sketch transformation, we define our optimization objective in the form of joint generative-discriminative minimization. In particular, a discriminative regularization term is incorporated into the photo-sketch generation, enhancing the discriminability of the generated person sketches against other individuals. Extensive experiments on several standard benchmarks suggest that our approach outperforms other state-of-the-art methods in both photo-sketch generation and face sketch verification.

arXiv comment8 pages, 6 figures. Proceeding in ACM International Conference on Multimedia Retrieval (ICMR), 2015

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Fully Convolutional Networks for Semantic Segmentation
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• Deep Joint Task Learning for Generic Object Extraction
• Scalable Object Detection using Deep Neural Networks

Cited by in corpus (19)

• Deep Face Recognition: A Survey
• Identity-Aware CycleGAN for Face Photo-Sketch Synthesis and Recognition
• Convolutional Sketch Inversion
• Facial-Sketch Synthesis: A New Challenge
• Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs
• CSGAN: Cyclic-Synthesized Generative Adversarial Networks for Image-to-Image Transformation
• Freehand Sketch Generation from Mechanical Components
• High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks
• Face Sketch Synthesis Style Similarity:A New Structure Co-occurrence Texture Measure
• Random Sampling for Fast Face Sketch Synthesis
• Sparse Graphical Representation based Discriminant Analysis for Heterogeneous Face Recognition
• SketchEmbedNet: Learning Novel Concepts by Imitating Drawings
• HoneyFaces: Increasing the Security and Privacy of Authentication Using Synthetic Facial Images
• Scoot: A Perceptual Metric for Facial Sketches
• Face Sketch Synthesis with Style Transfer using Pyramid Column Feature
• Enabling Robots to Draw and Tell: Towards Visually Grounded Multimodal Description Generation
• Face Sketch Synthesis via Semantic-Driven Generative Adversarial Network
• Emerging Biometrics: Deep Inference and Other Computational Intelligence
• Semi-Supervised Learning for Face Sketch Synthesis in the Wild