JuICe: A Large Scale Distantly Supervised Dataset for Open Domain Context-based Code Generation

Abstract

Interactive programming with interleaved code snippet cells and natural language markdown is recently gaining popularity in the form of Jupyter notebooks, which accelerate prototyping and collaboration. To study code generation conditioned on a long context history, we present JuICe, a corpus of 1.5 million examples with a curated test set of 3.7K instances based on online programming assignments. Compared with existing contextual code generation datasets, JuICe provides refined human-curated data, open-domain code, and an order of magnitude more training data. Using JuICe, we train models for two tasks: (1) generation of the API call sequence in a code cell, and (2) full code cell generation, both conditioned on the NL-Code history up to a particular code cell. Experiments using current baseline code generation models show that both context and distant supervision aid in generation, and that the dataset is challenging for current systems.

References in corpus (5)

• Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars
• Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
• Mining Idioms from Source Code
• Latent Predictor Networks for Code Generation
• Simpler Context-Dependent Logical Forms via Model Projections

Cited by in corpus (4)

• Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebooks
• Code to Comment "Translation": Data, Metrics, Baselining & Evaluation
• PyTorrent: A Python Library Corpus for Large-scale Language Models
• Adversarial Training for Code Retrieval with Question-Description Relevance Regularization