Unsupervised Learning via Meta-Learning

Abstract

A central goal of unsupervised learning is to acquire representations from unlabeled data or experience that can be used for more effective learning of downstream tasks from modest amounts of labeled data. Many prior unsupervised learning works aim to do so by developing proxy objectives based on reconstruction, disentanglement, prediction, and other metrics. Instead, we develop an unsupervised meta-learning method that explicitly optimizes for the ability to learn a variety of tasks from small amounts of data. To do so, we construct tasks from unlabeled data in an automatic way and run meta-learning over the constructed tasks. Surprisingly, we find that, when integrated with meta-learning, relatively simple task construction mechanisms, such as clustering embeddings, lead to good performance on a variety of downstream, human-specified tasks. Our experiments across four image datasets indicate that our unsupervised meta-learning approach acquires a learning algorithm without any labeled data that is applicable to a wide range of downstream classification tasks, improving upon the embedding learned by four prior unsupervised learning methods.

arXiv commentICLR 2019 camera-ready. 24 pages, 2 figures, links to code available at https://sites.google.com/view/unsupervised-via-meta

References in corpus (17)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Representation Learning with Contrastive Predictive Coding
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Semi-Supervised Learning with Deep Generative Models
• Adversarially Learned Inference
• Semi-Supervised Learning with Ladder Networks
• Semi-supervised Sequence Learning
• Disentangling factors of variation in deep representations using adversarial training
• Unsupervised Learning by Predicting Noise
• Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
• Data-dependent Initializations of Convolutional Neural Networks
• Discovering Hidden Factors of Variation in Deep Networks
• Loss is its own Reward: Self-Supervision for Reinforcement Learning
• Unsupervised Meta-Learning for Reinforcement Learning
• Auto-Meta: Automated Gradient Based Meta Learner Search
• Supervising Unsupervised Learning

Cited by in corpus (38)

• Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
• Meta-Learning Update Rules for Unsupervised Representation Learning
• Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions
• Multi-level Second-order Few-shot Learning
• Unsupervised Meta-Learning for Reinforcement Learning
• Unsupervised Deep Representation Learning and Few-Shot Classification of PolSAR Images
• A Comprehensive Overview and Survey of Recent Advances in Meta-Learning
• Assume, Augment and Learn: Unsupervised Few-Shot Meta-Learning via Random Labels and Data Augmentation
• Diversity Helps: Unsupervised Few-shot Learning via Distribution Shift-based Data Augmentation
• How to Train Your MAML to Excel in Few-Shot Classification
• Revisiting Meta-Learning as Supervised Learning
• Towards Cross-Granularity Few-Shot Learning: Coarse-to-Fine Pseudo-Labeling with Visual-Semantic Meta-Embedding
• Joint Noise-Tolerant Learning and Meta Camera Shift Adaptation for Unsupervised Person Re-Identification
• Transforming task representations to perform novel tasks
• Unsupervised Few-shot Learning via Self-supervised Training
• MT3: Meta Test-Time Training for Self-Supervised Test-Time Adaption
• Using learned optimizers to make models robust to input noise
• Are Few-Shot Learning Benchmarks too Simple ? Solving them without Task Supervision at Test-Time
• FSD-10: A Dataset for Competitive Sports Content Analysis
• Learning to Impute: A General Framework for Semi-supervised Learning
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback
• Machine learning with limited data
• Learning to Transfer: Unsupervised Meta Domain Translation
• Rethinking Class Relations: Absolute-relative Supervised and Unsupervised Few-shot Learning
• Meta-Learning to Cluster
• Few-Shot Unsupervised Continual Learning through Meta-Examples
• SketchEmbedNet: Learning Novel Concepts by Imitating Drawings
• An Information-Geometric Distance on the Space of Tasks
• Don't miss the Mismatch: Investigating the Objective Function Mismatch for Unsupervised Representation Learning
• Uniform Priors for Data-Efficient Transfer
• Zero-shot task adaptation by homoiconic meta-mapping
• Variable-Shot Adaptation for Online Meta-Learning
• Empirical Perspectives on One-Shot Semi-supervised Learning
• Characterizing Policy Divergence for Personalized Meta-Reinforcement Learning
• Divergent Search for Few-Shot Image Classification
• Online Unsupervised Learning of Visual Representations and Categories
• Constructing Multiple Tasks for Augmentation: Improving Neural Image Classification With K-means Features
• When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey