PBE Meets LLM: When Few Examples Aren't Few-Shot Enough

Shuning Zhang, Yongjoo Park

Large language models (LLMs) can generate code from natural language descriptions. Their performance is typically evaluated using programming benchmarks that simulate real-world ta…