SOSELETO: A Unified Approach to Transfer Learning and Training with Noisy Labels

Abstract

We present SOSELETO (SOurce SELEction for Target Optimization), a new method for exploiting a source dataset to solve a classification problem on a target dataset. SOSELETO is based on the following simple intuition: some source examples are more informative than others for the target problem. To capture this intuition, source samples are each given weights; these weights are solved for jointly with the source and target classification problems via a bilevel optimization scheme. The target therefore gets to choose the source samples which are most informative for its own classification task. Furthermore, the bilevel nature of the optimization acts as a kind of regularization on the target, mitigating overfitting. SOSELETO may be applied to both classic transfer learning, as well as the problem of training on datasets with noisy labels; we show state of the art results on both of these problems.

arXiv commentICLR workshop on Learning from Limited Labeled Data (LLD) -- Best Paper Award

References in corpus (9)

• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Classification with Noisy Labels by Importance Reweighting
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Training Convolutional Networks with Noisy Labels
• Deep Learning is Robust to Massive Label Noise
• Transfer Learning with Label Noise
• Shape from Shading through Shape Evolution

Cited by in corpus (4)

• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labels
• Label Noise Types and Their Effects on Deep Learning
• HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks