6 citations · 12 across the 4 of their papers we have counts for
4 papers
Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
Ruisheng Cao, Fangyu Lei, Haoyuan Wu +20
Data science and engineering workflows often span multiple stages, from warehousing to orchestration, using tools like BigQuery, dbt, and Airbyte. As vision language models (VLMs)…
NExT: Teaching Large Language Models to Reason about Code Execution
Ansong Ni, Miltiadis Allamanis, Arman Cohan +4
A fundamental skill among human developers is the ability to understand and reason about program execution. As an example, a programmer can mentally simulate code execution in natu…
Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models
Martin Riddell, Ansong Ni, Arman Cohan
While large language models have achieved remarkable performance on various code generation benchmarks, there have been growing concerns regarding potential contamination of these…
L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models
Ansong Ni, Pengcheng Yin, Yilun Zhao +11
Recently, large language models (LLMs), especially those that are pretrained on code, have demonstrated strong capabilities in generating programs from natural language inputs in a…