Learning Neurosymbolic Generative Models via Program Synthesis

Abstract

Significant strides have been made toward designing better generative models in recent years. Despite this progress, however, state-of-the-art approaches are still largely unable to capture complex global structure in data. For example, images of buildings typically contain spatial patterns such as windows repeating at regular intervals; state-of-the-art generative methods can't easily reproduce these structures. We propose to address this problem by incorporating programs representing global structure into the generative model---e.g., a 2D for-loop may represent a configuration of windows. Furthermore, we propose a framework for learning these models by leveraging program synthesis to generate training data. On both synthetic and real-world data, we demonstrate that our approach is substantially better than the state-of-the-art at both generating and completing images that contain global structure.

References in corpus (8)

• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• A Neural Representation of Sketch Drawings
• Towards Deeper Understanding of Variational Autoencoding Models
• The challenge of realistic music generation: modelling raw audio at scale
• Texture Synthesis with Spatial Generative Adversarial Networks
• HOUDINI: Lifelong Learning as Program Synthesis
• Selecting Representative Examples for Program Synthesis

Cited by in corpus (2)

• Web Question Answering with Neurosymbolic Program Synthesis
• Synthesizing Machine Learning Programs with PAC Guarantees via Statistical Sketching