65 citations · 136 across the 30 of their papers we have counts for
14 papers · 1 filter
NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
Jingzhe Ding, Shengda Long, Changxin Pu +46
Recent advances in coding agents suggest rapid progress toward autonomous software development, yet existing benchmarks fail to rigorously evaluate the long-horizon capabilities re…
Seed-Coder: Let the Code Model Curate Data for Itself
ByteDance Seed, Yuyu Zhang, Jing Su +24
Code data in large language model (LLM) pretraining is recognized crucial not only for code-related tasks but also for enhancing general intelligence of LLMs. Current open-source L…
Towards a Unified View of Preference Learning for Large Language Models: A Survey
Bofei Gao, Feifan Song, Yibo Miao +22
Large Language Models (LLMs) exhibit remarkably powerful capabilities. One of the crucial factors to achieve success is aligning the LLM's output with human preferences. This align…
Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models
Bofei Gao, Feifan Song, Zhe Yang +17
Recent advancements in large language models (LLMs) have led to significant breakthroughs in mathematical reasoning capabilities. However, existing benchmarks like GSM8K or MATH ar…
Elephant in the Room: Unveiling the Impact of Reward Model Quality in Alignment
Yan Liu, Xiaoyuan Yi, Xiaokang Chen +6
The demand for regulating potentially risky behaviors of large language models (LLMs) has ignited research on alignment methods. Since LLM alignment heavily relies on reward models…
The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models
Yan Liu, Yu Liu, Xiaokang Chen +4
Pre-trained Language models (PLMs) have been acknowledged to contain harmful information, such as social biases, which may cause negative social impacts or even bring catastrophic…
CodeR: Issue Resolving with Multi-Agent and Task Graphs
Dong Chen, Shaoxin Lin, Muhan Zeng +14
GitHub issue resolving recently has attracted significant attention from academia and industry. SWE-bench is proposed to measure the performance in resolving issues. In this paper,…
CodeS: Natural Language to Code Repository via Multi-Layer Sketch
Daoguang Zan, Ailun Yu, Wei Liu +11
The impressive performance of large language models (LLMs) on code-related tasks has shown the potential of fully automated software development. In light of this, we introduce a n…
Improving Natural Language Capability of Code Large Language Model
Wei Li, Daoguang Zan, Bei Guan +3
Code large language models (Code LLMs) have demonstrated remarkable performance in code generation. Nonetheless, most existing works focus on boosting code LLMs from the perspectiv…
Can Programming Languages Boost Each Other via Instruction Tuning?
Daoguang Zan, Ailun Yu, Bo Shen +8
When human programmers have mastered a programming language, it would be easier when they learn a new programming language. In this report, we focus on exploring whether programmin…
PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback
Bo Shen, Jiaxin Zhang, Taihong Chen +9
Large Language Models for Code (Code LLM) are flourishing. New and powerful models are released on a weekly basis, demonstrating remarkable performance on the code generation task.…
Uncovering and Quantifying Social Biases in Code Generation
Yan Liu, Xiaokang Chen, Yan Gao +6
With the popularity of automatic code generation tools, such as Copilot, the study of the potential hazards of these tools is gaining importance. In this work, we explore the socia…
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Fengji Zhang, Bei Chen, Yue Zhang +6
The task of repository-level code completion is to continue writing the unfinished code based on a broader context of the repository. While for automated code completion tools, it…
CodeT: Code Generation with Generated Tests
Bei Chen, Fengji Zhang, Anh Nguyen +4
The task of generating code solutions for a given programming problem can benefit from the use of pre-trained language models such as Codex, which can produce multiple diverse samp…