2 citations · 11 across the 51 of their papers we have counts for
14 papers · 1 filter
EarlyEval: Cheaper Agent Evaluation via Early Outcome Prediction
Yuling Shi, Zhensu Sun, Junsen Dong +3
Evaluating LLM agents is essential for guiding their development, yet it has grown prohibitively expensive: a single pass of a frontier model over an agentic benchmark can cost hun…
Membership Inference in Fine-tuned Diffusion Language Models via Token-level Memorization Asymmetry
Shengfang Zhai, Leo Marchyok, Yuling Shi +4
Diffusion language models (DLMs) have recently emerged as an alternative modeling paradigm to autoregressive LMs, offering advantages such as parallel generation and bidirectional…
SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring
Yuling Shi, Jinghan Xu, Kelin Fu +12
As AI coding agents take on increasingly complex, long-horizon software engineering tasks, existing benchmarks are rapidly saturating and their evaluation quality has come under se…
SWE-Pruner Pro: The Coder LLM Already Knows What to Prune
Yuhang Wang, Yuling Shi, Shaoqiu Zhang +6
Pruning long context for coding agents has been a vital technology for efficient context management. While existing context pruning methods such as SWE-Pruner realize this by attac…
Code Is More Than Text: Uncertainty Estimation for Code Generation
Yuling Shi, Caiqi Zhang, Yuexian Li +4
Large language models (LLMs) are increasingly deployed as code generators, where silently wrong programs pose real safety and reliability risks. Reliable uncertainty estimation (UE…
HEART-Bench: Do LLMs Make Persona-Consistent Decisions?
Weihan Peng, Chenxu Zhang, Qianao Wang +7
LLM agents in interactive entertainment and social applications, including companions, role-play characters, and persona clones, need decisions grounded in personality and past exp…
Seeing is Coding: On the Effectiveness of Vision Language Models in Code Understanding
Yuling Shi, Chaoxiang Xie, Zhensu Sun +7
Large Language Models (LLMs) have achieved remarkable success in source code understanding, yet as software systems grow in scale, computational efficiency has become a critical bo…
Reasoning in Trees: Improving Retrieval-Augmented Generation for Multi-Hop Question Answering
Yuling Shi, Maolin Sun, Zijun Liu +4
Retrieval-Augmented Generation (RAG) has demonstrated significant effectiveness in enhancing large language models (LLMs) for complex multi-hop question answering (QA). For multi-h…
Robust Preference Alignment via Directional Neighborhood Consensus
Ruochen Mao, Yuling Shi, Xiaodong Gu +1
Aligning large language models with human preferences is critical for creating reliable and controllable AI systems. A human preference can be visualized as a high-dimensional vect…
LongCodeZip: Compress Long Context for Code Language Models
Yuling Shi, Yichun Qian, Hongyu Zhang +2
Code generation under long contexts is becoming increasingly critical as Large Language Models (LLMs) are required to reason over extensive information in the codebase. While recen…
SWE-QA: Can Language Models Answer Repository-level Code Questions?
Weihan Peng, Yuling Shi, Yuhang Wang +3
Understanding and reasoning about entire software repositories is an essential capability for intelligent software engineering tools. While existing benchmarks such as CoSQA and Co…
LastingBench: Defend Benchmarks Against Knowledge Leakage
Yixiong Fang, Tianran Sun, Yuling Shi +2
The increasing complexity of large language models (LLMs) raises concerns about their ability to "cheat" on standard Question Answering (QA) benchmarks by memorizing task-specific…
AttentionRAG: Attention-Guided Context Pruning in Retrieval-Augmented Generation
Yixiong Fang, Tianran Sun, Yuling Shi +1
While RAG demonstrates remarkable capabilities in LLM applications, its effectiveness is hindered by the ever-increasing length of retrieved contexts, which introduces information…
From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging
Yuling Shi, Songsong Wang, Chengcheng Wan +2
While large language models have made significant strides in code generation, the pass rate of the generated code is bottlenecked on subtle errors, often requiring human interventi…