Learning Continuous Semantic Representations of Symbolic Expressions

Abstract

Combining abstract, symbolic reasoning with continuous neural reasoning is a grand challenge of representation learning. As a step in this direction, we propose a new architecture, called neural equivalence networks, for the problem of learning continuous semantic representations of algebraic and logical expressions. These networks are trained to represent semantic equivalence, even of expressions that are syntactically very different. The challenge is that semantic representations must be computed in a syntax-directed manner, because semantics is compositional, but at the same time, small changes in syntax can lead to very large changes in semantics, which can be difficult for continuous neural architectures. We perform an exhaustive evaluation on the task of checking equivalence on a highly diverse class of symbolic algebraic and boolean expression types, showing that our model significantly outperforms existing architectures.

Accepted to ICML 2017

References in corpus (10)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Neural Programmer-Interpreters
• Neural GPUs Learn Algorithms
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
• A Convolutional Attention Network for Extreme Summarization of Source Code
• Learning Program Embeddings to Propagate Feedback on Student Code
• Neural Programmer: Inducing Latent Programs with Gradient Descent
• Learning to Discover Efficient Mathematical Identities

Cited by in corpus (17)

• Learning to Perform Local Rewriting for Combinatorial Optimization
• A Literature Study of Embeddings on Source Code
• Deep Learning for Symbolic Mathematics
• Property Invariant Embedding for Automated Reasoning
• Efficient Generator of Mathematical Expressions for Symbolic Regression
• Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization
• Relational Neural Machines
• Neuro-Symbolic Artificial Intelligence: Current Trends
• Neural Logic Rule Layers
• Compositional Generalization with Tree Stack Memory Units
• Embedding Symbolic Knowledge into Deep Networks
• Neural Program Synthesis with a Differentiable Fixer
• Improving the Robustness to Data Inconsistency between Training and Testing for Code Completion by Hierarchical Language Model
• Improve Language Modelling for Code Completion through Statement Level Language Model based on Statement Embedding Generated by BiLSTM
• Embedding Symbolic Temporal Knowledge into Deep Sequential Models
• Pretrained Language Models are Symbolic Mathematics Solvers too!
• Generating Symbolic Reasoning Problems with Transformer GANs