11 papers
ClawSentry: A Progressive Multi-Tier Security Monitor for Safeguarding Autonomous LLM Agents
Kai Wang, Zeming Wei, BiaoJie Zeng +7
As large language model (LLM) agents move from conversation to executing code, reading local files, and orchestrating external tools, a single agent hijacked by a malicious third-p…
Hyperball May Not Be a Free Lunch
Yihao Xiao, Jialong Sun, Zitian Gao +5
For scale-invariant deep networks, Hyperball-style optimizers have shown strong performance in large-scale training by fixing the norms of matrix-valued parameters and normalizing…
The Salami Slicing Threat: Exploiting Cumulative Risks in LLM Systems
Yihao Zhang, Kai Wang, Jiangrong Wu +7
Large Language Models (LLMs) face prominent security risks from jailbreaking, a practice that manipulates models to bypass built-in security constraints and generate unethical or u…
TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systems
Kai Wang, Biaojie Zeng, Zeming Wei +7
With the rapid development of LLM-based multi-agent systems (MAS), their significant safety and security concerns have emerged, which introduce novel risks going beyond single agen…
AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystems
Yihao Zhang, Zeming Wei, Xiaokun Luan +7
Autonomous LLM-based agents increasingly operate as long-running processes forming densely interconnected multi-agent ecosystems, whose security properties remain largely unexplore…
RAPO: Risk-Aware Preference Optimization for Generalizable Safe Reasoning
Zeming Wei, Qiaosheng Zhang, Xia Hu +1
Large Reasoning Models (LRMs) have achieved tremendous success with their chain-of-thought (CoT) reasoning, yet also face safety issues similar to those of basic language models. I…
SMI: Statistical Membership Inference for Reliable Unlearned Model Auditing
Jialong Sun, Zeming Wei, Jiaxuan Zou +6
Machine unlearning (MU) is essential for enforcing the right to be forgotten in machine learning systems. A key challenge of MU is how to reliably audit whether a model has truly f…
False Sense of Security: Why Probing-based Malicious Input Detection Fails to Generalize
Cheng Wang, Zeming Wei, Qin Liu +1
Large Language Models (LLMs) can comply with harmful instructions, raising serious safety concerns despite their impressive capabilities. Recent work has leveraged probing-based ap…
Reliable Unlearning Harmful Information in LLMs with Metamorphosis Representation Projection
Chengcan Wu, Zeming Wei, Huanran Chen +2
While Large Language Models (LLMs) have demonstrated impressive performance in various domains and tasks, concerns about their safety are becoming increasingly severe. In particula…
Unveiling the Basin-Like Loss Landscape in Large Language Models
Huanran Chen, Yinpeng Dong, Zeming Wei +4
We discover the emergence of \textit{basins} in the loss landscape of large language models. As model scale increases, LLMs become progressively more resilient to random perturbati…
Towards the Worst-case Robustness of Large Language Models
Huanran Chen, Yinpeng Dong, Zeming Wei +2
Recent studies have revealed the vulnerability of large language models to adversarial attacks, where adversaries craft specific input sequences to induce harmful, violent, private…