24 papers
Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Models
Shilinlu Yan, Bowen Chen, Yuechen Zhang +3
Visual-token compression improves the efficiency of large vision-language models, but can expose failures that full-token evaluation misses. We study adversarial images that preser…
RESCUE: Repairing Language Model Errors to Sparse Circuits via Reinforcement Learning
Chuanpu Liu, Miao Yu, Yikai Cai +5
Large language models (LLMs) exhibit strong general capabilities that mechanistic interpretability has attributed to sparse computational circuits. However, existing circuit studie…
Still There, No Longer Seen: Exposing Compression-Induced Risk in Large Vision-Language Models
Qiankun Li, Yuechen Zhang, Bowen Chen +4
Visual token compression reduces the inference cost of Large Vision-Language Models (LVLMs). However, aggregate robustness measures do not reveal whether a particular adversarial f…
In-Place Instruction Following in Diffusion Language Models
Zheng Nie, Zherui Li, Jiaming Zhang +3
Diffusion Large Language Models (dLLMs) generate text via bidirectional iterative denoising, naturally supporting user-specified constraints anchored at arbitrary output positions,…
From Inaudible Inputs to Model Failures: Low-Frequency Safety Risks in LALMs
Yuanhe Zhang, Weiliu Wang, Jie Ren +7
Large audio-language models (LALMs) have demonstrated strong capabilities in understanding diverse audio inputs. This diversity includes low-frequency signals that are inaudible to…
Structure-Guided Visual Perturbation Neutralization for LVLMs
Yuanhe Zhang, Xueting Wang, YanBin Ren +6
Image inputs enable Large Vision Language Models (LVLMs) to perceive fine-grained visual information, but also introduce a pixel-level attack surface through which adversarial pert…
Explaining and Breaking the Safety-Helpfulness Ceiling via Preference Dimensional Expansion
ShiYing Huang, Liang Lin, Yuer Li +6
In the realm of multi-objective alignment for large language models, balancing disparate human preferences often manifests as a zero-sum conflict. Specifically, the intrinsic tensi…
EchoDistill: Robust Large Audio Language Models via Noisy-to-Clean Self-Distillation
Liang Lin, Chunxi Luo, Kaiwen Luo +12
Large Audio Language Models (LALMs) remain vulnerable to acoustic noise, which can obscure task-relevant evidence and produce unreliable responses. We propose EchoDistill, a noisy-…
A Survey of Large Audio Language Models: Generalization, Trustworthiness, and Outlook
Kaiwen Luo, Zhenhong Zhou, Leyan Wang +32
Advances in Large Language Models (LLMs) have paved the way for Multimodal Large Language Models (MLLMs). Among these, Large Audio Language Models (LALMs) are essential for realizi…
SafeSeek: Universal Attribution of Safety Circuits in Language Models
Miao Yu, Siyuan Fu, Moayad Aloqaily +6
Mechanistic interpretability reveals that safety-critical behaviors (e.g., alignment, jailbreak, backdoor) in Large Language Models (LLMs) are grounded in specialized functional co…
LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Models
Wei Zhang, Lintong Du, Yuanhe Zhang +4
Despite the strong performance of Large Language Models (LLMs) on complex instruction-following tasks, precise control of output length remains a persistent challenge. Existing met…
MCPShield: A Security Cognition Layer for Adaptive Trust Calibration in Model Context Protocol Agents
Zhenhong Zhou, Yuanhe Zhang, Hongwei Cai +6
The Model Context Protocol (MCP) standardizes tool use for LLM-based agents and enable third-party servers. This openness introduces a security misalignment: agents implicitly trus…
HearSay Benchmark: Do Audio LLMs Leak What They Hear?
Jin Wang, Liang Lin, Kaiwen Luo +8
While Audio Large Language Models (ALLMs) have achieved remarkable progress in understanding and generation, their potential privacy implications remain largely unexplored. This pa…
CSSBench: Evaluating the Safety of Lightweight LLMs against Chinese-Specific Adversarial Patterns
Zhenhong Zhou, Shilinlu Yan, Chuanpu Liu +3
Large language models (LLMs) are increasingly deployed in cost-sensitive and on-device scenarios, and safety guardrails have advanced mainly in English. However, real-world Chinese…
RSA-Bench: Benchmarking Audio Large Models in Real-World Acoustic Scenarios
Yibo Zhang, Liang Lin, Kaiwen Luo +8
While Audio Large Models (ALMs) have achieved remarkable proficiency, their robustness remains brittle in real-world deployment. Existing evaluations largely rely on synthetic Gaus…
ChronosAudio: A Comprehensive Long-Audio Benchmark for Evaluating Audio-Large Language Models
Kaiwen Luo, Liang Lin, Yibo Zhang +8
Although Audio Large Language Models (ALLMs) have witnessed substantial advancements, their long audio understanding capabilities remain unexplored. A plethora of benchmarks have b…
Backdoor Collapse: Eliminating Unknown Threats via Known Backdoor Aggregation in Language Models
Liang Lin, Miao Yu, Moayad Aloqaily +5
Backdoor attacks are a significant threat to large language models (LLMs), often embedded via public checkpoints, yet existing defenses rely on impractical assumptions about trigge…
Backdoor Attribution: Elucidating and Controlling Backdoor in Language Models
Miao Yu, Zhenhong Zhou, Moayad Aloqaily +5
Fine-tuned Large Language Models (LLMs) are vulnerable to backdoor attacks through data poisoning, yet the internal mechanisms governing these attacks remain a black box. Previous…
DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models
Zherui Li, Zheng Nie, Zhenhong Zhou +7
The rapid advancement of Diffusion Large Language Models (dLLMs) introduces unprecedented vulnerabilities that are fundamentally distinct from Autoregressive LLMs, stemming from th…
Hidden in the Noise: Unveiling Backdoors in Audio LLMs Alignment through Latent Acoustic Pattern Triggers
Liang Lin, Miao Yu, Kaiwen Luo +9
As Audio Large Language Models (ALLMs) emerge as powerful tools for speech processing, their safety implications demand urgent attention. While considerable research has explored t…
Jailbreaking Large Language Diffusion Models: Revealing Hidden Safety Flaws in Diffusion-Based Text Generation
Yuanhe Zhang, Fangzhou Xie, Zhenhong Zhou +4
Large Language Diffusion Models (LLDMs) exhibit comparable performance to LLMs while offering distinct advantages in inference speed and mathematical reasoning tasks.The precise an…
Resource Consumption Red-Teaming for Large Vision-Language Models
Haoran Gao, Yuanhe Zhang, Zhenhong Zhou +7
Resource Consumption Attacks (RCAs) have emerged as a significant threat to the deployment of Large Language Models (LLMs). With the integration of vision modalities, additional at…
Goal-Aware Identification and Rectification of Misinformation in Multi-Agent Systems
Zherui Li, Yan Mi, Zhenhong Zhou +4
Large Language Model-based Multi-Agent Systems (MASs) have demonstrated strong advantages in addressing complex real-world tasks. However, due to the introduction of additional att…
Reinforced Lifelong Editing for Language Models
Zherui Li, Houcheng Jiang, Hao Chen +5
Large language models (LLMs) acquire information from pre-training corpora, but their stored knowledge can become inaccurate or outdated over time. Model editing addresses this cha…