ICE-GAN: Identity-aware and Capsule-Enhanced GAN with Graph-based Reasoning for Micro-Expression Recognition and Synthesis

Abstract

Micro-expressions are reflections of people's true feelings and motives, which attract an increasing number of researchers into the study of automatic facial micro-expression recognition. The short detection window, the subtle facial muscle movements, and the limited training samples make micro-expression recognition challenging. To this end, we propose a novel Identity-aware and Capsule-Enhanced Generative Adversarial Network with graph-based reasoning (ICE-GAN), introducing micro-expression synthesis as an auxiliary task to assist recognition. The generator produces synthetic faces with controllable micro-expressions and identity-aware features, whose long-ranged dependencies are captured through the graph reasoning module (GRM), and the discriminator detects the image authenticity and expression classes. Our ICE-GAN was evaluated on Micro-Expression Grand Challenge 2019 (MEGC2019) with a significant improvement (12.9%) over the winner and surpassed other state-of-the-art methods.

arXiv comment9 pages, 5 figures

References in corpus (7)

• Conditional Generative Adversarial Nets
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Capsules for Object Segmentation
• A Review on Facial Micro-Expressions Analysis: Datasets, Features and Metrics
• MER-GCN: Micro Expression Recognition Based on Relation Modeling with Graph Convolutional Network
• Single Neuron Segmentation using Graph-based Global Reasoning with Auxiliary Skeleton Loss from 3D Optical Microscope Images