Loss Function Search for Face Recognition

Abstract

In face recognition, designing margin-based (e.g., angular, additive, additive angular margins) softmax loss functions plays an important role in learning discriminative features. However, these hand-crafted heuristic methods are sub-optimal because they require much effort to explore the large design space. Recently, an AutoML for loss function search method AM-LFS has been derived, which leverages reinforcement learning to search loss functions during the training process. But its search space is complex and unstable that hindering its superiority. In this paper, we first analyze that the key to enhance the feature discrimination is actually \textbf{how to reduce the softmax probability}. We then design a unified formulation for the current margin-based softmax losses. Accordingly, we define a novel search space and develop a reward-guided search method to automatically obtain the best candidate. Experimental results on a variety of face recognition benchmarks have demonstrated the effectiveness of our method over the state-of-the-art alternatives.

Accepted by ICML2020. arXiv admin note: substantial text overlap with arXiv:1912.00833; text overlap with arXiv:1905.07375 by other authors

Cited by in corpus (9)

• Automated Self-Supervised Learning for Graphs
• Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search
• Auto Seg-Loss: Searching Metric Surrogates for Semantic Segmentation
• Prototype Memory for Large-scale Face Representation Learning
• AutoLoss: Automated Loss Function Search in Recommendations
• AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks
• HLIC: Harmonizing Optimization Metrics in Learned Image Compression by Reinforcement Learning
• An Efficient Training Approach for Very Large Scale Face Recognition
• Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search