Deep Convolutional Neural Networks in the Face of Caricature: Identity and Image Revealed

Abstract

Real-world face recognition requires an ability to perceive the unique features of an individual face across multiple, variable images. The primate visual system solves the problem of image invariance using cascades of neurons that convert images of faces into categorical representations of facial identity. Deep convolutional neural networks (DCNNs) also create generalizable face representations, but with cascades of simulated neurons. DCNN representations can be examined in a multidimensional "face space", with identities and image parameters quantified via their projections onto the axes that define the space. We examined the organization of viewpoint, illumination, gender, and identity in this space. We show that the network creates a highly organized, hierarchically nested, face similarity structure in which information about face identity and imaging characteristics coexist. Natural image variation is accommodated in this hierarchy, with face identity nested under gender, illumination nested under identity, and viewpoint nested under illumination. To examine identity, we caricatured faces and found that network identification accuracy increased with caricature level, and--mimicking human perception--a caricatured distortion of a face "resembled" its veridical counterpart. Caricatures improved performance by moving the identity away from other identities in the face space and minimizing the effects of illumination and viewpoint. Deep networks produce face representations that solve long-standing computational problems in generalized face recognition. They also provide a unitary theoretical framework for reconciling decades of behavioral and neural results that emphasized either the image or the object/face in representations, without understanding how a neural code could seamlessly accommodate both.

8 pages, 5 figures

References in corpus (5)

• FaceNet: A Unified Embedding for Face Recognition and Clustering
• Intriguing properties of neural networks
• Triplet Probabilistic Embedding for Face Verification and Clustering
• Deep Convolutional Neural Network Features and the Original Image
• How Deep is the Feature Analysis underlying Rapid Visual Categorization?

Cited by in corpus (4)

• 3D-CariGAN: An End-to-End Solution to 3D Caricature Generation from Face Photos
• Single Unit Status in Deep Convolutional Neural Network Codes for Face Identification: Sparseness Redefined
• Towards Gender-Neutral Face Descriptors for Mitigating Bias in Face Recognition
• A solution to generalized learning from small training sets found in infants repeated visual experiences of individual objects