11 papers
Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
Liting Lin, Boxi Yu, Yuzhong Zhang +3
Conversational LLM agents can cause real-world harm when their internal workflows fail, such as completing a transaction without confirmation. Testing these state-dependent failure…
Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility
Jialun Cao, Yuk-Kit Chan, Zixuan Ling +12
Code-related benchmarks play a critical role in evaluating large language models (LLMs), yet their quality fundamentally shapes how the community interprets model capabilities. In…
OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models
Shijie Cao, Qingyu Zhang, Boxi Yu +6
Omni modal large language models (OmniLLMs) have attracted wide attention for their ability to jointly process audio and video, but they generate large token sequences under audio-…
BeSpec: Behavior-Level Specification Alignment for Code Generation
Qinghua Xu, Guancheng Wang, Boxi Yu +1
LLMs have made substantial progress on automated code generation from natural-language descriptions of desired behavior (intent). Most existing methods improve generated programs t…
OpenRCA 2.0: From Outcome Labels to Causal Process Supervision
Aoyang Fang, Yifan Yang, Jin'ao Shang +7
Root cause analysis (RCA) poses a holistic test of LLM agentic capabilities, such as long-context understanding, multi-step reasoning, and tool use. However, existing datasets suff…
TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment
Changyue Li, Jiaming He, Youliang Yuan +4
Fine-Tuning-as-a-Service (FTaaS) platforms let users train large language models (LLMs) on customized tasks, but this pipeline could erode models' safety alignment. In practice, se…