Generalized Coarse-to-Fine Visual Recognition with Progressive Training
arXiv:1811.12047
Abstract
Computer vision is difficult, partly because the desired mathematical function connecting input and output data is often complex, fuzzy and thus hard to learn. Coarse-to-fine (C2F) learning is a promising direction, but it remains unclear how it is applied to a wide range of vision problems. This paper presents a generalized C2F framework by making two technical contributions. First, we provide a unified way of C2F propagation, in which the coarse prediction (a class vector, a detected box, a segmentation mask, etc.) is encoded into a dense (pixel-level) matrix and concatenated to the original input, so that the fine model takes the same design of the coarse model but sees additional information. Second, we present a progressive training strategy which starts with feeding the ground-truth instead of the coarse output into the fine model, and gradually increases the fraction of coarse output, so that at the end of training the fine model is ready for testing. We also relate our approach to curriculum learning by showing that data difficulty keeps increasing during the training process. We apply our framework to three vision tasks including image classification, object localization and semantic segmentation, and demonstrate consistent accuracy gain compared to the baseline training strategy.
References in corpus (10)
- Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
- Matching Networks for One Shot Learning
- Deep Reinforcement Learning for Dialogue Generation
- Convolutional Pose Machines
- Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks
- Reinforcement Learning Neural Turing Machines - Revised
- Dynamic Few-Shot Visual Learning without Forgetting
- ScaleNet: Guiding Object Proposal Generation in Supermarkets and Beyond
- Improved Training with Curriculum GANs
- SampleAhead: Online Classifier-Sampler Communication for Learning from Synthesized Data