7 papers
When Elo Lies: Hidden Biases in Codeforces-Based Evaluation of Large Language Models
Shenyu Zheng, Ximing Dong, Xiaoshuang Liu +6
As Large Language Models (LLMs) achieve breakthroughs in complex reasoning, Codeforces-based Elo ratings have emerged as a prominent metric for evaluating competitive programming c…
SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints
Zhiyu Fan, Kirill Vasilevski, Dayi Lin +6
The advancement of large language models (LLMs) and code agents has demonstrated significant potential to assist software engineering (SWE) tasks, such as autonomous issue resoluti…
RepoForge: Training a SOTA Fast-thinking SWE Agent with an End-to-End Data Curation Pipeline Synergizing SFT and RL at Scale
Zhilong Chen, Chengzong Zhao, Boyuan Chen +9
Training software engineering (SWE) LLMs is bottlenecked by expensive infrastructure, inefficient evaluation pipelines, scarce training data, and costly quality control. We present…
SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort Estimation
Gustavo A. Oliva, Gopi Krishnan Rajbahadur, Aaditya Bhatia +7
High-quality labeled datasets are crucial for training and evaluating foundation models in software engineering, but creating them is often prohibitively expensive and labor-intens…
Adaptive Request Scheduling for CodeLLM Serving with SLA Guarantees
Shi Chang, Boyuan Chen, Kishanthan Thangarajah +2
Code Large Language Models (CodeLLMs) are increasingly integrated into modern software development workflows, yet efficiently serving them in resource-constrained, self-hosted envi…
Context-Aware CodeLLM Eviction for AI-assisted Coding
Kishanthan Thangarajah, Boyuan Chen, Shi Chang +1
AI-assisted coding tools powered by Code Large Language Models (CodeLLMs) are increasingly integrated into modern software development workflows. To address concerns around privacy…