AM-LFS: AutoML for Loss Function Search
arXiv:1905.07375
Abstract
Designing an effective loss function plays an important role in visual analysis. Most existing loss function designs rely on hand-crafted heuristics that require domain experts to explore the large design space, which is usually sub-optimal and time-consuming. In this paper, we propose AutoML for Loss Function Search (AM-LFS) which leverages REINFORCE to search loss functions during the training process. The key contribution of this work is the design of search space which can guarantee the generalization and transferability on different vision tasks by including a bunch of existing prevailing loss functions in a unified formulation. We also propose an efficient optimization framework which can dynamically optimize the parameters of loss function's distribution during training. Extensive experimental results on four benchmark datasets show that, without any tricks, our method outperforms existing hand-crafted loss functions in various computer vision tasks.
References in corpus (12)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Neural Architecture Search with Reinforcement Learning
- Learning Face Representation from Scratch
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- Efficient Neural Architecture Search via Parameter Sharing
- Neural Architecture Optimization
- ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
- Convolutional Neural Fabrics
- Learning to Teach with Dynamic Loss Functions
- Gradient Harmonized Single-stage Detector
- Learning to Teach
- IRLAS: Inverse Reinforcement Learning for Architecture Search
Cited by in corpus (6)
- Improving One-shot NAS by Suppressing the Posterior Fading
- Powering One-shot Topological NAS with Stabilized Share-parameter Proxy
- Joint Search of Data Augmentation Policies and Network Architectures
- PV-NAS: Practical Neural Architecture Search for Video Recognition
- Auto-weighting for Breast Cancer Classification in Multimodal Ultrasound
- Ada-Segment: Automated Multi-loss Adaptation for Panoptic Segmentation