On the Reconstruction of Face Images from Deep Face Templates

Abstract

State-of-the-art face recognition systems are based on deep (convolutional) neural networks. Therefore, it is imperative to determine to what extent face templates derived from deep networks can be inverted to obtain the original face image. In this paper, we study the vulnerabilities of a state-of-the-art face recognition system based on template reconstruction attack. We propose a neighborly de-convolutional neural network (\textit{NbNet}) to reconstruct face images from their deep templates. In our experiments, we assumed that no knowledge about the target subject and the deep network are available. To train the \textit{NbNet} reconstruction models, we augmented two benchmark face datasets (VGG-Face and Multi-PIE) with a large collection of images synthesized using a face generator. The proposed reconstruction was evaluated using type-I (comparing the reconstructed images against the original face images used to generate the deep template) and type-II (comparing the reconstructed images against a different face image of the same subject) attacks. Given the images reconstructed from \textit{NbNets}, we show that for verification, we achieve TAR of 95.20\% (58.05\%) on LFW under type-I (type-II) attacks @ FAR of 0.1\%. Besides, 96.58\% (92.84\%) of the images reconstruction from templates of partition \textit{fa} (\textit{fb}) can be identified from partition \textit{fa} in color FERET. Our study demonstrates the need to secure deep templates in face recognition systems.

To appear in TPAMI, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Learning Face Representation from Scratch
• NIPS 2016 Tutorial: Generative Adversarial Networks
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Wasserstein GAN
• Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
• Inverting face embeddings with convolutional neural networks
• Pixel Deconvolutional Networks

Cited by in corpus (15)

• Deep Face Recognition: A Survey
• Reversing the Irreversible: A Survey on Inverse Biometrics
• Biometric Template Protection for Neural-Network-based Face Recognition Systems: A Survey of Methods and Evaluation Techniques
• Towards Protecting Face Embeddings in Mobile Face Verification Scenarios
• A novel classification-selection approach for the self updating of template-based face recognition systems
• Secure Face Matching Using Fully Homomorphic Encryption
• Black-Box Face Recovery from Identity Features
• Cancelable Biometric Template Generation Using Random Feature Vector Transformations
• Approximating Optimal Morphing Attacks using Template Inversion
• Darker than Black-Box: Face Reconstruction from Similarity Queries
• Fuzzy Commitments Offer Insufficient Protection to Biometric Templates Produced by Deep Learning
• A Genetic Algorithm Enabled Similarity-Based Attack on Cancellable Biometrics
• IronMask: Modular Architecture for Protecting Deep Face Template
• Scalable Facial Image Compression with Deep Feature Reconstruction
• Revealing Unintentional Information Leakage in Low-Dimensional Facial Portrait Representations