Differentiable Programs with Neural Libraries
arXiv:1611.02109
Abstract
We develop a framework for combining differentiable programming languages with neural networks. Using this framework we create end-to-end trainable systems that learn to write interpretable algorithms with perceptual components. We explore the benefits of inductive biases for strong generalization and modularity that come from the program-like structure of our models. In particular, modularity allows us to learn a library of (neural) functions which grows and improves as more tasks are solved. Empirically, we show that this leads to lifelong learning systems that transfer knowledge to new tasks more effectively than baselines.
Cited by in corpus (19)
- The Scattering Compositional Learner: Discovering Objects, Attributes, Relationships in Analogical Reasoning
- Systematic Generalization: What Is Required and Can It Be Learned?
- From explanation to synthesis: Compositional program induction for learning from demonstration
- Lifelong Learning of Compositional Structures
- Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networks
- Why Build an Assistant in Minecraft?
- Learning to Synthesize Programs as Interpretable and Generalizable Policies
- Robot Program Parameter Inference via Differentiable Shadow Program Inversion
- Synthesizing Action Sequences for Modifying Model Decisions
- Learning Differentiable Programs with Admissible Neural Heuristics
- Abductive Knowledge Induction From Raw Data
- World Programs for Model-Based Learning and Planning in Compositional State and Action Spaces
- Biology and Compositionality: Empirical Considerations for Emergent-Communication Protocols
- Neural Abstract Reasoner
- Web Question Answering with Neurosymbolic Program Synthesis
- Deep Algorithmic Question Answering: Towards a Compositionally Hybrid AI for Algorithmic Reasoning
- Synthesizing Machine Learning Programs with PAC Guarantees via Statistical Sketching
- Understanding in Artificial Intelligence
- Sequential Coordination of Deep Models for Learning Visual Arithmetic