18 papers
ScienceArena: Benchmarking LLMs on Latest Scientific Olympiad Competitions
Guangxiang Zhao, Qilong Shi, Xusen Xiao +13
Benchmark saturation and data contamination increasingly obscure genuine scientific reasoning in frontier LLMs. We introduce \textsc{ScienceArena}, an olympiad-style benchmark from…
ArborMem: Navigating Interaction States with Memory Forests
Zongwei Lv, Yuemeng Xu, Yilun Yao +8
Large language models increasingly serve as persistent conversational assistants, requiring memory that preserves relevant experience and maintains continuity across interactions.…
When Do Prompt-Side Agent Playbooks Transfer? Accuracy, Cost, and Runtime Shift in Agent Deployment
Weihong Lin, Lin Sun, Xiangzheng Zhang
Prompt-side playbooks can improve tool-using language agents without retraining, but their portability beyond the source setting is unclear. We study frozen playbook transfer under…
SkillSentry: Adaptive Honey Worlds for Dynamic Safety Testing of Agent Skills
Nizhang Li, Zonghao Ying, Xiangfan Wu +7
External skills extend the capabilities of large language model agents, but also introduce an execution-time attack surface: a skill that appears benign under inspection may reveal…
SafeFlow: Semantic Information-Flow Control for Blocking Malicious Propagation in Multi-Agent Systems
Haowen Dai, Zonghao Ying, Wenfeng Li +10
Multi-agent systems improve capability through task decomposition and role specialization, but these same mechanisms introduce an important safety blind spot: a harmful objective c…
Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models
Dongdong Yang, Deyue Zhang, Zhao Liu +5
Text-to-Image (T2I) generative models have achieved remarkable progress in synthesizing high-quality visual content, yet they remain vulnerable to adversarial misuse, particularly…
ElephantAgent: Contextual State Continuity in Agentic Systems
Jiankai Jin, Xiangzheng Zhang, Zhao Liu +4
Agentic systems enhance their capabilities by invoking external tools and maintaining persistent memory. However, these external dependencies introduce novel attack surfaces. Recen…
External Experience Serving in Production LLM Systems: A Deployment-Oriented Study of Quality-Cost Trade-offs
Lin Sun, Heming Zhang, Xiangzheng Zhang
Production LLM systems accumulate reusable operational experience, but the practical deployment issue is not merely whether such experience can help. It is how different serving st…
RealClawBench: Live OpenClaw Benchmarks from Real Developer-Agent Sessions
Zongwei Lv, Yaoming Li, Zhewen Tan +7
Agent benchmarks should reflect what users actually ask deployed agents to do, yet existing benchmarks often miss key realism properties of real developer-agent sessions. We introd…
A Primer in Post-Training Reasoning Data: What We Know About How It Works
Yaoming Li, Guangxiang Zhao, Qilong Shi +3
Post-training has become a primary driver of recent progress in large reasoning models, and reasoning data are often the key variable determining whether this stage succeeds. Work…
Harness-Bench: Evaluating Agent Harnesses across Models and Realistic Workflows
Yilun Yao, Xinyu Tan, Chao-Hsuan Liu +10
LLM agents are increasingly deployed as executable systems that use tools, modify workspaces, and produce concrete artifacts. In such workflows, performance depends not only on the…
MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection
Zhewen Tan, Yilun Yao, Huiyan Jin +9
Large language model agents increasingly rely on persistent memory to store past interactions, retrieve relevant demonstrations, and improve long-horizon task execution. However, t…
DMN: A Compositional Framework for Jailbreaking Multimodal LLMs with Multi-Image Inputs
Wenzhuo Xu, Zhipeng Wei, Zonghao Ying +4
Multimodal Large Language Models (MLLMs) are vulnerable to jailbreak attacks, which can elicit harmful responses from MLLMs. Many MLLMs support multi-image inputs, inadvertently in…
Demystifying and Detecting Agentic Workflow Injection Vulnerabilities in GitHub Actions
Shenao Wang, Xinyi Hou, Zhao Liu +5
GitHub Actions is increasingly used to deploy LLM-based agents for repository-centric tasks such as issue triage, pull-request review, code modification, and release assistance. Th…
SafeHarbor: Defining Precise Decision Boundaries via Hierarchical Memory-Augmented Guardrail for LLM Agent Safety
Zhe Liu, Zonghao Ying, Wenxin Zhang +5
Recent advances in foundation models have transformed LLMs from passive conversational systems into autonomous agents capable of reasoning and tool execution. While these capabilit…
BEAR: Budgeted Evidence Allocation for Multi-Document Reasoning
Lin Sun, Linglin Zhang, Jingang Huang +3
We argue that multi-document reasoning is constrained not only by how much text a model can read, but also by how limited query-time evidence budget is allocated across documents a…
Robust Privacy: Inference-Stage Privacy through Certified Robustness
Jiankai Jin, Xiangzheng Zhang, Zhao Liu +4
An adversary observing a model's released prediction can infer sensitive attributes of the queried input, or even reconstruct representatives of the model's training data. The infe…
Mask-GCG: Are All Tokens in Adversarial Suffixes Necessary for Jailbreak Attacks?
Junjie Mu, Zonghao Ying, Zhekui Fan +6
Jailbreak attacks on Large Language Models (LLMs) have demonstrated various successful methods whereby attackers manipulate models into generating harmful responses that they are d…