822 citations · 1.1k across the 7 of their papers we have counts for
4 papers · 1 filter
NATURAL PLAN: Benchmarking LLMs on Natural Language Planning
Huaixiu Steven Zheng, Swaroop Mishra, Hugh Zhang +8
We introduce NATURAL PLAN, a realistic planning benchmark in natural language containing 3 key tasks: Trip Planning, Meeting Planning, and Calendar Scheduling. We focus our evaluat…
Larger language models do in-context learning differently
Jerry Wei, Jason Wei, Yi Tay +8
We study how in-context learning (ICL) in language models is affected by semantic priors versus input-label mappings. We investigate two setups-ICL with flipped labels and ICL with…
Large Language Models Can Be Easily Distracted by Irrelevant Context
Freda Shi, Xinyun Chen, Kanishka Misra +5
Large language models have achieved impressive performance on various natural language processing tasks. However, so far they have been evaluated primarily on benchmarks where all…
Latent Attention For If-Then Program Synthesis
Xinyun Chen, Chang Liu, Richard Shin +2
Automatic translation from natural language descriptions into programs is a longstanding challenging problem. In this work, we consider a simple yet important sub-problem: translat…