Towards a Neural Statistician

Abstract

An efficient learner is one who reuses what they already know to tackle a new problem. For a machine learner, this means understanding the similarities amongst datasets. In order to do this, one must take seriously the idea of working with datasets, rather than datapoints, as the key objects to model. Towards this goal, we demonstrate an extension of a variational autoencoder that can learn a method for computing representations, or statistics, of datasets in an unsupervised fashion. The network is trained to produce statistics that encapsulate a generative model for each dataset. Hence the network enables efficient learning from new datasets for both unsupervised and supervised tasks. We show that we are able to learn statistics that can be used for: clustering datasets, transferring generative models to new datasets, selecting representative samples of datasets and classifying previously unseen classes. We refer to our model as a neural statistician, and by this we mean a neural network that can learn to compute summary statistics of datasets without supervision.

Updated to camera ready version for ICLR 2017

References in corpus (2)

• Matching Networks for One Shot Learning
• Composing graphical models with neural networks for structured representations and fast inference

Cited by in corpus (83)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Transformers in Vision: A Survey
• Data Augmentation Generative Adversarial Networks
• Few-Shot Learning with Graph Neural Networks
• Predicting materials properties without crystal structure: Deep representation learning from stoichiometry
• One-Shot Imitation Learning
• TADAM: Task dependent adaptive metric for improved few-shot learning
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data
• Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments
• Bilevel Programming for Hyperparameter Optimization and Meta-Learning
• Generative Adversarial Residual Pairwise Networks for One Shot Learning
• How to train your MAML
• Meta-Learning without Memorization
• Adversarial Attack and Defense on Point Sets
• Low-Shot Learning from Imaginary Data
• Gaussian Prototypical Networks for Few-Shot Learning on Omniglot
• Small Sample Learning in Big Data Era
• On Learning Sets of Symmetric Elements
• Conditional Neural Processes
• A Comprehensive Overview and Survey of Recent Advances in Meta-Learning
• Simple and Effective VAE Training with Calibrated Decoders
• Toward Understanding Catastrophic Forgetting in Continual Learning
• Modular meta-learning
• The Variational Homoencoder: Learning to learn high capacity generative models from few examples
• Meta-trained agents implement Bayes-optimal agents
• Learning to Warm-Start Bayesian Hyperparameter Optimization
• Review of Mathematical frameworks for Fairness in Machine Learning
• Variational Memory Addressing in Generative Models
• Experience-Embedded Visual Foresight
• Few-shot Learning for Time-series Forecasting
• Recurrent Neural Processes
• Meta-learning autoencoders for few-shot prediction
• 3D Shape Synthesis for Conceptual Design and Optimization Using Variational Autoencoders
• Approximation capability of neural networks on spaces of probability measures and tree-structured domains
• The Kanerva Machine: A Generative Distributed Memory
• Are Few-Shot Learning Benchmarks too Simple ? Solving them without Task Supervision at Test-Time
• L2AE-D: Learning to Aggregate Embeddings for Few-shot Learning with Meta-level Dropout
• Fast Adaptation in Generative Models with Generative Matching Networks
• Meta-Learning surrogate models for sequential decision making
• Uncertainty in Multitask Transfer Learning
• Meta-Amortized Variational Inference and Learning
• Generative One-Shot Face Recognition
• ChartPointFlow for Topology-Aware 3D Point Cloud Generation
• Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design
• Neural Density Estimation and Likelihood-free Inference
• AgileNet: Lightweight Dictionary-based Few-shot Learning
• Neural Clustering Processes
• Learning Functions over Sets via Permutation Adversarial Networks
• Nested Multiple Instance Learning in Modelling of HTTP network traffic
• A Simple Framework for Uncertainty in Contrastive Learning
• Domain2Vec: Deep Domain Generalization
• Learning to Support: Exploiting Structure Information in Support Sets for One-Shot Learning
• Intelligence, physics and information -- the tradeoff between accuracy and simplicity in machine learning
• Augmentation-Interpolative AutoEncoders for Unsupervised Few-Shot Image Generation
• What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data
• Decoder Choice Network for Meta-Learning
• Amortized Bayesian inference for clustering models
• Fairness Through Causal Awareness: Learning Latent-Variable Models for Biased Data
• Learning to generate classifiers
• Improving Context-Based Meta-Reinforcement Learning with Self-Supervised Trajectory Contrastive Learning
• Energy-Based Processes for Exchangeable Data
• Finding the Homology of Decision Boundaries with Active Learning
• Learning a Hierarchical Latent-Variable Model of 3D Shapes
• Few-shot Learning for Spatial Regression
• You Never Cluster Alone
• No Representation without Transformation
• Meta-learning One-class Classifiers with Eigenvalue Solvers for Supervised Anomaly Detection
• Rapid Structural Pruning of Neural Networks with Set-based Task-Adaptive Meta-Pruning
• Conditional Graph Neural Processes: A Functional Autoencoder Approach
• Few-shot Learning for Topic Modeling
• Meta Learning in the Continuous Time Limit
• Disentangled Dynamic Representations from Unordered Data
• Zero-Shot Recognition via Optimal Transport
• Meta-Active Learning for Node Response Prediction in Graphs
• Mapping the Internet: Modelling Entity Interactions in Complex Heterogeneous Networks
• OR-Net: Pointwise Relational Inference for Data Completion under Partial Observation
• Meta-learning for Matrix Factorization without Shared Rows or Columns
• Recursively Conditional Gaussian for Ordinal Unsupervised Domain Adaptation
• Lightweight Data Fusion with Conjugate Mappings
• Local Contrast Learning
• Partially Observed Exchangeable Modeling
• How Sensitive are Meta-Learners to Dataset Imbalance?