Publications (17)
Delulu: A Verified Multi-Lingual Benchmark for Code Hallucination Detection in Fill-in-the-Middle Tasks
Mahdi Erfanian, Nelson Daniel Troncoso, Aashna Garg +4
Large Language Models for code generation frequently produce hallucinations in Fill-in-the-Middle (FIM) tasks -- plausible but incorrect completions such as invented API methods, i…
SemRep: Generative Code Representation Learning with Code Transformations
Weichen Li, Jiamin Song, Bogdan Alexandru Stoica +4
Code transformation is a foundational capability in the software development process, where its effectiveness relies on constructing a high-quality code representation to character…
Automating Code Review Activities by Large-Scale Pre-training
Zhiyu Li, Shuai Lu, Daya Guo +8
Code review is an essential part to software development lifecycle since it aims at guaranteeing the quality of codes. Modern code review activities necessitate developers viewing,…
First Identification of New X-Ray Spectra of Mo39+, Mo40+, W43+, W44+ and W45+ on EAST
Fudi Wang, Dian Lu, Mingfeng Gu +27
New high-resolution x-ray spectra of Mo39+, Mo40+, W43+, W44+ and W45+ have been carefully confirmed for the first time by use of the x-ray imaging crystal spectrometer (XCS) in Ex…
GraphCodeBERT: Pre-training Code Representations with Data Flow
Daya Guo, Shuo Ren, Shuai Lu +15
Pre-trained models for programming language have achieved dramatic empirical improvements on a variety of code-related tasks such as code search, code completion, code summarizatio…
FastContext: Training Efficient Repository Explorer for Coding Agents
Shaoqiu Zhang, Maoquan Wang, Yuling Shi +12
Large Language Model (LLM) coding agents have achieved strong results on software engineering tasks, yet repository exploration remains a major bottleneck: locating relevant code c…
Pythia: AI-assisted Code Completion System
Alexey Svyatkovskiy, Ying Zhao, Shengyu Fu +1
In this paper, we propose a novel end-to-end approach for AI-assisted code completion called Pythia. It generates ranked lists of method and API recommendations which can be used b…
SWE-Edit: Rethinking Code Editing for Efficient SWE-Agent
Yikai Zhang, Jiaxin Pei, Kenan Li +9
Large language model agents have made strong progress on software engineering, yet current systems suffer from a context coupling problem: the standard code editing interface confl…
DevBench: A Realistic, Developer-Informed Benchmark for Code Generation Models
Adarsh Kumarappan, Pareesa Ameneh Golnari, Wen Wen +5
DevBench is a telemetry-driven benchmark designed to evaluate Large Language Models (LLMs) on realistic code completion tasks. It includes 1,800 evaluation instances across six pro…
Lita: Light Agent Uncovers the Agentic Coding Capabilities of LLMs
Hankun Dai, Maoquan Wang, Mengnan Qi +6
Large language models (LLMs) are increasingly being applied to programming tasks, ranging from single-turn code completion to autonomous agents. Current code agent designs frequent…
Code Execution with Pre-trained Language Models
Chenxiao Liu, Shuai Lu, Weizhu Chen +5
Code execution is a fundamental aspect of programming language semantics that reflects the exact behavior of the code. However, most pre-trained models for code intelligence ignore…
Sphinx: Benchmarking and Modeling for LLM-Driven Pull Request Review
Daoan Zhang, Shuo Zhang, Zijian Jin +3
Pull request (PR) review is essential for ensuring software quality, yet automating this task remains challenging due to noisy supervision, limited contextual understanding, and in…
HyDRA: Hybrid Dynamic Routing Architecture for Heterogeneous LLM Pools
Aashna Garg, Siddharth Singha Roy, Jinu Jang +2
Production LLM deployments increasingly maintain heterogeneous model pools spanning order-of-magnitude cost differences. Existing routers make binary strong-vs-weak decisions and c…
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Shuai Lu, Daya Guo, Shuo Ren +19
Benchmark datasets have a significant impact on accelerating research in programming language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster machine le…
Synthetic Hallucinations, Real Gains: Hard Negatives from Frontier Models for FIM Hallucination Mitigation
Mahdi Erfanian, Nelson Daniel Troncoso, Aashna Garg +4
Small open-source code models that power IDE autocomplete still emit hallucinated Fill-in-the-Middle (FIM) completions: syntactically natural calls to methods, parameters, variable…
IntelliCode Compose: Code Generation Using Transformer
Alexey Svyatkovskiy, Shao Kun Deng, Shengyu Fu +1
In software development through integrated development environments (IDEs), code completion is one of the most widely used features. Nevertheless, majority of integrated developmen…
SWE-bench Goes Live!
Linghao Zhang, Shilin He, Chaoyun Zhang +12
The issue-resolving task, where a model generates patches to fix real-world bugs, has emerged as a critical benchmark for evaluating the capabilities of large language models (LLMs…