52 citations · 60 across the 17 of their papers we have counts for
18 papers
EarlyEval: Cheaper Agent Evaluation via Early Outcome Prediction
Yuling Shi, Zhensu Sun, Junsen Dong +3
Evaluating LLM agents is essential for guiding their development, yet it has grown prohibitively expensive: a single pass of a frontier model over an agentic benchmark can cost hun…
Schrödinger's Code Repository: Have LLMs Learned SWE-bench or Memorized It?
Silin Chen, Yufei Yang, Xiaodong Gu +3
Repository-level coding benchmarks have become the standard for evaluating coding agents, yet they inherently suffer from data leakage because they are built upon popular open-sour…
SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring
Yuling Shi, Jinghan Xu, Kelin Fu +12
As AI coding agents take on increasingly complex, long-horizon software engineering tasks, existing benchmarks are rapidly saturating and their evaluation quality has come under se…
SWE-Pruner Pro: The Coder LLM Already Knows What to Prune
Yuhang Wang, Yuling Shi, Shaoqiu Zhang +6
Pruning long context for coding agents has been a vital technology for efficient context management. While existing context pruning methods such as SWE-Pruner realize this by attac…
Dockerless: Environment-Free Program Verifier for Coding Agents
Wenhao Zeng, Yuling Shi, Xiaodong Gu +10
Program verifiers play a central role in training coding agents, including selecting trajectories for supervised fine-tuning (SFT) and providing rewards for reinforcement learning…
Automated jailbreak attack targeting multiple defense strategies
Qi Wang, Chengcheng Wan, Weijia He +4
Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks. However, their safety remains a critical concern due to their susceptibility to…
ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation
Yeheng Chen, Chaoxiang Xie, Yuling Shi +4
LLMs have achieved strong results on both function-level code synthesis and repository-level code modification, yet a capability that falls between these two extremes -- compositio…
EffiSkill: Agent Skill Based Automated Code Efficiency Optimization
Zimu Wang, Yuling Shi, Mengfan Li +4
Code efficiency is a fundamental aspect of software quality, yet how to harness large language models (LLMs) to optimize programs remains challenging. Prior approaches have sought…
Seeing is Coding: On the Effectiveness of Vision Language Models in Code Understanding
Yuling Shi, Chaoxiang Xie, Zhensu Sun +7
Large Language Models (LLMs) have achieved remarkable success in source code understanding, yet as software systems grow in scale, computational efficiency has become a critical bo…
LibRec: Benchmarking Retrieval-Augmented LLMs for Library Migration Recommendations
Junxiao Han, Yarong Wang, Xiaodong Gu +5
In this paper, we propose LibRec, a novel framework that integrates the capabilities of LLMs with retrieval-augmented generation(RAG) techniques to automate the recommendation of a…
Pruning the Unsurprising: Efficient LLM Reasoning via First-Token Surprisal
Wenhao Zeng, Yaoning Wang, Chao Hu +4
Large Reasoning Models (LRMs) have demonstrated remarkable capabilities by scaling up the length of Chain-of-Thought (CoT). However, excessively long reasoning traces pose substant…
EvoC2Rust: A Skeleton-guided Framework for Project-Level C-to-Rust Translation
Chaofan Wang, Tingrui Yu, Beijun Shen +6
Translating legacy C codebases to Rust is increasingly demanded for building safety-critical systems. While various approaches have emerged for this task, they face inherent trade-…
LastingBench: Defend Benchmarks Against Knowledge Leakage
Yixiong Fang, Tianran Sun, Yuling Shi +2
The increasing complexity of large language models (LLMs) raises concerns about their ability to "cheat" on standard Question Answering (QA) benchmarks by memorizing task-specific…
AttentionRAG: Attention-Guided Context Pruning in Retrieval-Augmented Generation
Yixiong Fang, Tianran Sun, Yuling Shi +1
While RAG demonstrates remarkable capabilities in LLM applications, its effectiveness is hindered by the ever-increasing length of retrieved contexts, which introduces information…
CodeCipher: Learning to Obfuscate Source Code Against LLMs
Yalan Lin, Chengcheng Wan, Yixiong Fang +1
While large code language models have made significant strides in AI-assisted coding tasks, there are growing concerns about privacy challenges. The user code is transparent to the…
From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging
Yuling Shi, Songsong Wang, Chengcheng Wan +2
While large language models have made significant strides in code generation, the pass rate of the generated code is bottlenecked on subtle errors, often requiring human interventi…
Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers
Yuling Shi, Hongyu Zhang, Chengcheng Wan +1
Large language models have catalyzed an unprecedented wave in code generation. While achieving significant advances, they blur the distinctions between machine- and human-authored…
On the Effectiveness of Large Language Models in Domain-Specific Code Generation
Xiaodong Gu, Meng Chen, Yalan Lin +6
Large language models (LLMs) such as ChatGPT have shown remarkable capabilities in code generation. Despite significant achievements, they rely on enormous training data to acquire…