18 papers
CyberPersistBench: Evaluating LLM-Based Cyber Attackers on Installation and Persistence
Sujin Chen, Lijun Li, Xuhong Wang +1
While LLM-based attackers exhibit growing proficiency in vulnerability exploitation, most existing cybersecurity benchmarks suffer from single-stage truncation, prematurely termina…
Defense-as-Skill: Evolving Runtime Guard Skill for Skill-Augmented Agents
Xiaofang Yang, Ziqi Miao, Chen Qian +3
Skill-augmented agents load reusable skills as persistent runtime context, improving task performance but also giving malicious skills a durable channel for steering future actions…
MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps
Sujin Chen, Lijun Li, Tianyi Du +1
LLM-powered GUI agents that autonomously operate smartphones are rapidly transitioning from research prototypes to early real-world deployment. However, because these agents routin…
Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinations
Xinshun Feng, Ziqi Miao, Lijun Li +1
Large language model (LLM) agents are increasingly deployed in scientific research, where reliability is critical and the underlying knowledge is densely interconnected. In such se…
Is Deep Research Reliable? Misleading Knowledge Induces False Conclusions
Pengyu Zhu, Lijun Li, Longju Yang +2
Deep Research agents conduct long-horizon investigations by iteratively planning, retrieving evidence, and generating reports. However, it remains unclear whether they can resist a…
DeepSafe-SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring
Chunxiao Li, Yuan Xiong, Lijun Li +4
Large language models (LLMs) increasingly support science, but they can also convert hazardous scientific knowledge into actionable misuse guidance. Existing benchmarks often rely…
UniACE: A Unified Framework for Evaluating LLM Agentic Capabilities
Pengyu Zhu, Lijun Li, Yaxing Lyu +8
Agent benchmarks are increasingly used to compare large language models (LLMs) across domains, yet a reported score reflects a complete model--harness--environment configuration ra…
SEARL: Joint Optimization of Policy and Tool Graph Memory for Self-Evolving Agents
Xinshun Feng, Xinhao Song, Lijun Li +2
Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have demonstrated significant potential in single-turn reasoning tasks. With the paradigm shift toward self…
Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning
Ziqi Miao, Haonan Jia, Lijun Li +4
Reinforcement learning with verifiable rewards (RLVR) has substantially enhanced the reasoning capabilities of multimodal large language models (MLLMs). However, existing RLVR appr…
TreeTeaming: Autonomous Red-Teaming of Vision-Language Models via Hierarchical Strategy Exploration
Chunxiao Li, Lijun Li, Jing Shao
The rapid advancement of Vision-Language Models (VLMs) has brought their safety vulnerabilities into sharp focus. However, existing red teaming methods are fundamentally constraine…
Contextual Image Attack: How Visual Context Exposes Multimodal Safety Vulnerabilities
Yuan Xiong, Ziqi Miao, Lijun Li +3
While Multimodal Large Language Models (MLLMs) show remarkable capabilities, their safety alignments are susceptible to jailbreak attacks. Existing attack methods typically focus o…
Collaborative Shadows: Distributed Backdoor Attacks in LLM-Based Multi-Agent Systems
Pengyu Zhu, Lijun Li, Yaxing Lyu +3
LLM-based multi-agent systems (MAS) demonstrate increasing integration into next-generation applications, but their safety in backdoor attacks remains largely underexplored. Howeve…
HarmRLVR: Weaponizing Verifiable Rewards for Harmful LLM Alignment
Yuexiao Liu, Lijun Li, Xingjun Wang +1
Recent advancements in Reinforcement Learning with Verifiable Rewards (RLVR) have gained significant attention due to their objective and verifiable reward signals, demonstrating s…
The Plot Twist: Jailbreaking Unified Multimodal Models with a Three-Act NarrativeAttack
Shaoxiong Guo, Tianyi Du, Lijun Li +3
Unified Multimodal Understanding and Generation Models (UMMs) increasingly combine visual understanding and image generation within a single interactive workflow, making generated…
Self-adaptive Dataset Construction for Real-World Multimodal Safety Scenarios
Jingen Qu, Lijun Li, Bo Zhang +2
Multimodal large language models (MLLMs) are rapidly evolving, presenting increasingly complex safety challenges. However, current dataset construction methods, which are risk-orie…
Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Models
Ziqi Miao, Lijun Li, Yuan Xiong +3
Contextual priming, where earlier stimuli covertly bias later judgments, offers an unexplored attack surface for large language models (LLMs). We uncover a contextual priming vulne…
Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injection
Ziqi Miao, Yi Ding, Lijun Li +1
With the emergence of strong vision language capabilities, multimodal large language models (MLLMs) have demonstrated tremendous potential for real-world applications. However, the…
Evolutionary Guided Decoding: Iterative Value Refinement for LLMs
Zhenhua Liu, Lijun Li, Ruizhe Chen +5
While guided decoding, especially value-guided methods, has emerged as a cost-effective alternative for controlling language model outputs without re-training models, its effective…