3 citations · 5 across the 9 of their papers we have counts for
6 papers · 1 filter
The Evolution of Attention in Large Language Models: Mechanisms, Trade-offs, and Emerging Trends
Zhentao Tan, Jingyi Shen, Yanbo Li +3
Self-attention gives LLMs fine-grained, query-dependent access to context, but dense token interactions incur quadratic prefill cost and a key--value cache growing with context len…
SPIDER: Multi-Layer Semantic Token Pruning and Adaptive Sub-Layer Skipping in Multimodal Large Language Models
Tianxiang Chen, Zhentao Tan, Zi Ye +9
Multimodal Large Language Models face significant efficiency challenges that stem from two distinct yet coupled sources: data redundancy and computational redundancy. While most me…
From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMs
Zhentao Tan, Chang Liu, Yao Liu +2
As large language models (LLMs) scale rapidly, dense full-parameter adaptation becomes increasingly expensive, motivating sparse and modular architectures such as Mixture-of-Expert…
HydraHead: From Head-Level Functional Heterogeneity to Specialized Attention Hybridization
Zhentao Tan, Wei Chen, Jingyi Shen +4
The quadratic complexity of attention poses a critical bottleneck for long-context processing, spurring interest in hybrid attention designs. Most open-source hybrid models adopt a…
SAFE-Pruner: Semantic Attention-Guided Future-Aware Token Pruning for Efficient Vision-Language-Action Manipulation
Shilin Ma, Chubin Zhang, Changyuan Wang +6
Real-time inference of vision-language-action (VLA) models is essential for robotic control. While visual token pruning has shown strong potential for accelerating inference, most…
Learning to Focus and Precise Cropping: A Reinforcement Learning Framework with Information Gaps and Grounding Loss for MLLMs
Xuanpu Zhao, Zhentao Tan, Dianmo Sheng +6
To enhance the perception and reasoning capabilities of multimodal large language models in complex visual scenes, recent research has introduced agent-based workflows. In these wo…