12 papers
LatCom: Cross-Agent Latent Compression for Efficient Multi-Agent Collaboration
Shinan Zhang, Tao Zhang, Qihui Zhu +7
LLM-based multi-agent systems (MAS) increasingly use latent collaboration to avoid the information loss and repeated encoding-decoding overhead of natural-language communication. H…
HiRoute: Hierarchical Routed Prompt Tuning for Safety Alignment of Large Language Models
Fangzhou Chen, Shiji Zhao, Mengyang Wang +4
Large language models (LLMs) remain vulnerable to harmful requests and jailbreak attacks. Parameter-efficient safety alignment methods based on prompt tuning typically rely on a si…
GSTEP: Global Spatio-Temporal Density-Driven Visual Token Pruning for Efficient Video Large Language Models
Mengjie Zhang, Qihui Zhu, Tao Zhang +10
Video large language models (VideoLLMs) achieve strong video understanding performance, but their inference remains expensive due to the large number of redundant spatio-temporal v…
SepPrune:A Separator-based Pruning Framework for Efficient Multimodal Large Language Models
Yuchen Wang, Qihui Zhu, Yang Liu +2
Recent multimodal large language models (MLLMs), such as Qwen2.5-VL and InternVL3, generate large numbers of vision tokens for high-resolution inputs, leading to substantial comput…
RP-OPSD: Resolution-Privileged On-Policy Self-Distillation for Multimodal Large Language Models
Qihui Zhu, Yuchen Wang, Zijian Wen +7
On-Policy Self-Distillation (OPSD) uses privileged information available only to the teacher to provide dense token-level supervision on trajectories generated by the student. Howe…
AlloSpatial: Agentic Harness Framework for Spatial Reasoning in Foundation Models
Shouwei Ruan, Bin Wang, Zhenyu Wu +5
Multimodal Foundation Models (MFMs) have made substantial progress, yet remain fragile in spatial reasoning over the physical world. A key bottleneck lies in their inability to tra…
HAWK: Head Importance-Aware Visual Token Pruning in Multimodal Models
Qihui Zhu, Tao Zhang, Yuchen Wang +9
In multimodal large language models (MLLMs), the surge of visual tokens significantly increases the inference time and computational overhead, making them impractical for real-time…
Improving Safety Alignment via Balanced Direct Preference Optimization
Shiji Zhao, Mengyang Wang, Shukun Xiong +7
With the rapid development and widespread application of Large Language Models (LLMs), their potential safety risks have attracted widespread attention. Reinforcement Learning from…
Mind over Space: Can Multimodal Large Language Models Mentally Navigate?
Qihui Zhu, Shouwei Ruan, Xiao Yang +6
Despite the widespread adoption of MLLMs in embodied agents, their capabilities remain largely confined to reactive planning from immediate observations, consistently failing in sp…
World2Mind: Cognition Toolkit for Allocentric Spatial Reasoning in Foundation Models
Shouwei Ruan, Bin Wang, Zhenyu Wu +4
Achieving robust spatial reasoning remains a fundamental challenge for current Multimodal Foundation Models (MFMs). Existing methods either overfit statistical shortcuts via 3D gro…
From reactive to cognitive: brain-inspired spatial intelligence for embodied agents
Shouwei Ruan, Liyuan Wang, Caixin Kang +4
Spatial cognition enables adaptive goal-directed behavior by constructing internal models of space. Robust biological systems consolidate spatial knowledge into three interconnecte…
MoAPT: Mixture of Adversarial Prompt Tuning for Vision-Language Models
Shiji Zhao, Qihui Zhu, Shukun Xiong +7
Large pre-trained Vision Language Models (VLMs) demonstrate excellent generalization capabilities but remain highly susceptible to adversarial examples, posing potential security r…