6 citations · 6 across the 8 of their papers we have counts for
4 papers · 1 filter
Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?
Wang Bill Zhu, Miaosen Chai, Shangshang Wang +5
Unlike code completion, debugging requires localizing faults and applying targeted edits. We observe that frontier LLMs often regenerate correct but over-edited solutions during de…
SWE-Bench Mobile: Can Large Language Model Agents Develop Industry-Level Mobile Applications?
Muxin Tian, Zhe Wang, Blair Yang +7
Can large language model agents develop industry-level mobile applications? We introduce \textbf{SWE-Bench Mobile}, a benchmark for evaluating coding agents on realistic software e…
TypyBench: Evaluating LLM Type Inference for Untyped Python Repositories
Honghua Dong, Jiacheng Yang, Xun Deng +4
Type inference for dynamic languages like Python is a persistent challenge in software engineering. While large language models (LLMs) have shown promise in code understanding, the…
Assessing Code Generation with Intermediate Languages
Xun Deng, Sicheng Zhong, Honghua Dong +4
Intermediate step methodologies like chain of thoughts (COT) have demonstrated effectiveness in enhancing the performance of Large Language Models (LLMs) on code generation. This s…