Synthetic Datasets for Neural Program Synthesis

Abstract

The goal of program synthesis is to automatically generate programs in a particular language from corresponding specifications, e.g. input-output behavior. Many current approaches achieve impressive results after training on randomly generated I/O examples in limited domain-specific languages (DSLs), as with string transformations in RobustFill. However, we empirically discover that applying test input generation techniques for languages with control flow and rich input space causes deep networks to generalize poorly to certain data distributions; to correct this, we propose a new methodology for controlling and evaluating the bias of synthetic data distributions over both programs and specifications. We demonstrate, using the Karel DSL and a small Calculator DSL, that training deep networks on these distributions leads to improved cross-distribution generalization performance.

ICLR 2019

Cited by in corpus (11)

• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Contrastive Code Representation Learning
• TF-Coder: Program Synthesis for Tensor Manipulations
• Synthesize, Execute and Debug: Learning to Repair for Neural Program Synthesis
• BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Exploration
• Data Generation for Neural Programming by Example
• Program Synthesis with Best-First Bottom-Up Search
• Unsupervised Program Synthesis for Images By Sampling Without Replacement
• Latent Execution for Neural Program Synthesis
• BF++: a language for general-purpose program synthesis
• Grammar Filtering For Syntax-Guided Synthesis