8 papers
SWE-Test: Benchmarking LLM Vulnerability Discovery via Input Prediction
Yuanxiang Shi, Jiayi Lin, Xuanyong Lin +8
Vulnerability discovery is becoming an important ability of large language model (LLM) agents: agents that silently miss real defects leave critical software exposed. Rigorously me…
MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome
Fangda Ye, Yuxin Hu, Pengxiang Zhu +19
Recent progress in deep research systems has been impressive, but evaluation still lags behind real user needs. Existing benchmarks predominantly assess final reports using fixed r…
AgentSwing: Adaptive Parallel Context Management Routing for Long-Horizon Web Agents
Zhaopeng Feng, Liangcai Su, Zhen Zhang +16
As large language models (LLMs) evolve into autonomous agents for long-horizon information-seeking, managing finite context capacity has become a critical bottleneck. Existing cont…
GMP: A Benchmark for Content Moderation under Co-occurring Violations and Dynamic Rules
Houde Dong, Yifei She, Kai Ye +3
Online content moderation is essential for maintaining a healthy digital environment, and reliance on AI for this task continues to grow. Consider a user comment using national ste…
R1-Fuzz: Specializing Language Models for Textual Fuzzing via Reinforcement Learning
Jiayi Lin, Liangcai Su, Junzhe Li +1
Fuzzing is effective for vulnerability discovery but struggles with complex targets such as compilers, interpreters, and database engines, which accept textual input that must sati…
How Far Are We from True Unlearnability?
Kai Ye, Liangcai Su, Chenxiong Qian
High-quality data plays an indispensable role in the era of large models, but the use of unauthorized data for model training greatly damages the interests of data owners. To overc…
ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation
Kai Ye, Liangcai Su, Chenxiong Qian
Code generation has emerged as a pivotal capability of Large Language Models(LLMs), revolutionizing development efficiency for programmers of all skill levels. However, the complex…
CONGRA: Benchmarking Automatic Conflict Resolution
Qingyu Zhang, Liangcai Su, Kai Ye +1
Resolving conflicts from merging different software versions is a challenging task. To reduce the overhead of manual merging, researchers develop various program analysis-based too…