5 papers
StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing
Zhijie Zheng, Yu Li, Chen Qian +5
LLM-based agents can interact with external environments through tool invocation, but this capability also introduces security risks such as file modification, information leakage,…
DeepSight: An All-in-One LM Safety Toolkit
Bo Zhang, Jiaxuan Guo, Lijun Li +17
As the development of Large Models (LMs) progresses rapidly, their safety is also a priority. In current Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) s…
Geometrically-Constrained Agent for Spatial Reasoning
Zeren Chen, Xiaoya Lu, Zhijie Zheng +6
Vision Language Models (VLMs) exhibit a fundamental semantic-to-geometric gap in spatial reasoning: they excel at qualitative semantic inference but their reasoning operates within…
When AI Agents Collude Online: Financial Fraud Risks by Collaborative LLM Agents on Social Platforms
Qibing Ren, Zhijie Zheng, Jiaxuan Guo +3
In this work, we study the risks of collective financial fraud in large-scale multi-agent systems powered by large language model (LLM) agents. We investigate whether agents can co…
Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report
Shanghai AI Lab, :, Xiaoyang Chen +35
To understand and identify the unprecedented risks posed by rapidly advancing artificial intelligence (AI) models, this report presents a comprehensive assessment of their frontier…