Programming with a Differentiable Forth Interpreter

Abstract

Given that in practice training data is scarce for all but a small set of problems, a core question is how to incorporate prior knowledge into a model. In this paper, we consider the case of prior procedural knowledge for neural networks, such as knowing how a program should traverse a sequence, but not what local actions should be performed at each step. To this end, we present an end-to-end differentiable interpreter for the programming language Forth which enables programmers to write program sketches with slots that can be filled with behaviour trained from program input-output data. We can optimise this behaviour directly through gradient descent techniques on user-specified objectives, and also integrate the program into any larger neural computation graph. We show empirically that our interpreter is able to effectively leverage different levels of prior program structure and learn complex behaviours such as sequence sorting and addition. When connected to outputs of an LSTM and trained jointly, our interpreter achieves state-of-the-art accuracy for end-to-end reasoning about quantities expressed in natural language stories.

34th International Conference on Machine Learning (ICML 2017)

References in corpus (6)

• Sequence to Sequence Learning with Neural Networks
• Gradient-based Hyperparameter Optimization through Reversible Learning
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
• Neural Turing Machines
• TerpreT: A Probabilistic Programming Language for Program Induction
• Adaptive Neural Compilation

Cited by in corpus (23)

• Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions
• End-to-End Differentiable Proving
• TerpreT: A Probabilistic Programming Language for Program Induction
• Learning to Infer and Execute 3D Shape Programs
• A Semantic Loss Function for Deep Learning with Symbolic Knowledge
• Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples
• From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood
• Programs as Black-Box Explanations
• Neural Program Synthesis with Priority Queue Training
• Learning Compositional Neural Programs with Recursive Tree Search and Planning
• Lifelong Learning of Compositional Structures
• Learning to Synthesize Programs as Interpretable and Generalizable Policies
• Why Build an Assistant in Minecraft?
• Robot Program Parameter Inference via Differentiable Shadow Program Inversion
• Synthesizing Action Sequences for Modifying Model Decisions
• Genetic algorithms with DNN-based trainable crossover as an example of partial specialization of general search
• A Survey on Neural-symbolic Learning Systems
• World Programs for Model-Based Learning and Planning in Compositional State and Action Spaces
• Reconciling the Discrete-Continuous Divide: Towards a Mathematical Theory of Sparse Communication
• Towards Dynamic Computation Graphs via Sparse Latent Structure
• Sequential Coordination of Deep Models for Learning Visual Arithmetic
• Reinforcement Learning of Implicit and Explicit Control Flow in Instructions
• Type-driven Neural Programming by Example