Permutation-invariant Feature Restructuring for Correlation-aware Image Set-based Recognition

Abstract

We consider the problem of comparing the similarity of image sets with variable-quantity, quality and un-ordered heterogeneous images. We use feature restructuring to exploit the correlations of both inner$\&$inter-set images. Specifically, the residual self-attention can effectively restructure the features using the other features within a set to emphasize the discriminative images and eliminate the redundancy. Then, a sparse/collaborative learning-based dependency-guided representation scheme reconstructs the probe features conditional to the gallery features in order to adaptively align the two sets. This enables our framework to be compatible with both verification and open-set identification. We show that the parametric self-attention network and non-parametric dictionary learning can be trained end-to-end by a unified alternative optimization scheme, and that the full framework is permutation-invariant. In the numerical experiments we conducted, our method achieves top performance on competitive image set/video-based face recognition and person re-identification benchmarks.

Accepted to ICCV 2019

Cited by in corpus (8)

• On Learning Sets of Symmetric Elements
• Conservative Wasserstein Training for Pose Estimation
• Reinforced Wasserstein Training for Severity-Aware Semantic Segmentation in Autonomous Driving
• Symmetric-Constrained Irregular Structure Inpainting for Brain MRI Registration with Tumor Pathology
• Disentanglement for Discriminative Visual Recognition
• Discriminability Distillation in Group Representation Learning
• Energy-constrained Self-training for Unsupervised Domain Adaptation
• Importance-Aware Semantic Segmentation in Self-Driving with Discrete Wasserstein Training