67 citations · 137 across the 9 of their papers we have counts for
9 papers
The Best Programming Language for Tokenmaxxing: An Investigation of Coding Agent Behavior Across Programming Languages
Zixuan Wu, Carolyn Jane Anderson, Arjun Guha
Although coding agents are now very effective in a variety of programming languages, this paper first shows that the cost (in tokens) can very significantly by programming language…
Agnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environment
Aleksander Boruch-Gruszecki, Yangtian Zi, Zixuan Wu +4
Large language models (LLMs) already excel at writing code in high-resource languages such as Python and JavaScript, yet stumble on low-resource languages that remain essential to…
"I Would Have Written My Code Differently'': Beginners Struggle to Understand LLM-Generated Code
Yangtian Zi, Luisa Li, Arjun Guha +2
Large language models (LLMs) are being increasingly adopted for programming work. Prior work shows that while LLMs accelerate task completion for professional programmers, beginnin…
ReasoningWeekly: A General Knowledge and Verbal Reasoning Challenge for Large Language Models
Zixuan Wu, Francesca Lucchetti, Aleksander Boruch-Gruszecki +5
Existing benchmarks for frontier models often test specialized, "PhD-level" knowledge that is difficult for non-experts to grasp. In contrast, we present a benchmark with 613 probl…
Substance Beats Style: Why Beginning Students Fail to Code with LLMs
Francesca Lucchetti, Zixuan Wu, Arjun Guha +2
Although LLMs are increasing the productivity of professional programmers, existing work shows that beginners struggle to prompt LLMs to solve text-to-code tasks. Why is this the c…
GlyphPattern: An Abstract Pattern Recognition Benchmark for Vision-Language Models
Zixuan Wu, Yoolim Kim, Carolyn Jane Anderson
Vision-Language Models (VLMs) building upon the foundation of powerful large language models have made rapid progress in reasoning across visual and textual data. While VLMs perfor…
StarCoder 2 and The Stack v2: The Next Generation
Anton Lozhkov, Raymond Li, Loubna Ben Allal +63
The BigCode project, an open-scientific collaboration focused on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder2. In partnership wi…
How Beginning Programmers and Code LLMs (Mis)read Each Other
Sydney Nguyen, Hannah McLean Babe, Yangtian Zi +3
Generative AI models, specifically large language models (LLMs), have made strides towards the long-standing goal of text-to-code generation. This progress has invited numerous stu…
Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions
Federico Cassano, Luisa Li, Akul Sethi +8
A significant amount of research is focused on developing and evaluating large language models for a variety of code synthesis tasks. These include synthesizing code from natural l…