NeurIPS 2020 NLC2CMD Competition: Translating Natural Language to Bash Commands
arXiv:2103.02523
Abstract
The NLC2CMD Competition hosted at NeurIPS 2020 aimed to bring the power of natural language processing to the command line. Participants were tasked with building models that can transform descriptions of command line tasks in English to their Bash syntax. This is a report on the competition with details of the task, metrics, data, attempted solutions, and lessons learned.
arXiv commentAppears in PMLR Volume 133: NeurIPS 2020 Competition and Demonstration Track. Competition URL: http://ibm.biz/nlc2cmd
References in corpus (14)
- Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
- Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning
- Measuring Coding Challenge Competence With APPS
- Quantifying the Carbon Emissions of Machine Learning
- A Syntactic Neural Model for General-Purpose Code Generation
- Unsupervised Translation of Programming Languages
- Energy Usage Reports: Environmental awareness as part of algorithmic accountability
- EvalAI: Towards Better Evaluation Systems for AI Agents
- From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood
- Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures
- Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task
- Semantic Evaluation for Text-to-SQL with Distilled Test Suites
- Photon: A Robust Cross-Domain Text-to-SQL System
- Project CLAI: Instrumenting the Command Line as a New Environment for AI Agents