Zero-shot task adaptation by homoiconic meta-mapping

Abstract

How can deep learning systems flexibly reuse their knowledge? Toward this goal, we propose a new class of challenges, and a class of architectures that can solve them. The challenges are meta-mappings, which involve systematically transforming task behaviors to adapt to new tasks zero-shot. The key to achieving these challenges is representing the task being performed in such a way that this task representation is itself transformable. We therefore draw inspiration from functional programming and recent work in meta-learning to propose a class of Homoiconic Meta-Mapping (HoMM) approaches that represent data points and tasks in a shared latent space, and learn to infer transformations of that space. HoMM approaches can be applied to any type of machine learning task. We demonstrate the utility of this perspective by exhibiting zero-shot remapping of behavior to adapt to new tasks.

arXiv comment27 pages

References in corpus (21)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• VQA: Visual Question Answering
• Continual Learning Through Synaptic Intelligence
• Zero-Shot Learning Through Cross-Modal Transfer
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Deep Learning: A Critical Appraisal
• Learning to reinforcement learn
• Unsupervised Learning via Meta-Learning
• Using Fast Weights to Attend to the Recent Past
• Grounded Language Learning in a Simulated 3D World
• A Deep Hierarchical Approach to Lifelong Learning in Minecraft
• Hierarchically Structured Meta-learning
• Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
• Selfless Sequential Learning
• Some Considerations on Learning to Explore via Meta-Reinforcement Learning
• An analytic theory of generalization dynamics and transfer learning in deep linear networks
• LGM-Net: Learning to Generate Matching Networks for Few-Shot Learning
• Automatically Composing Representation Transformations as a Means for Generalization
• Learnable Embedding Space for Efficient Neural Architecture Compression
• Contextual Parameter Generation for Universal Neural Machine Translation
• One-shot and few-shot learning of word embeddings