3 citations · 5 across the 9 of their papers we have counts for
12 papers
The Evolution of Attention in Large Language Models: Mechanisms, Trade-offs, and Emerging Trends
Zhentao Tan, Jingyi Shen, Yanbo Li +3
Self-attention gives LLMs fine-grained, query-dependent access to context, but dense token interactions incur quadratic prefill cost and a key--value cache growing with context len…
SPIDER: Multi-Layer Semantic Token Pruning and Adaptive Sub-Layer Skipping in Multimodal Large Language Models
Tianxiang Chen, Zhentao Tan, Zi Ye +9
Multimodal Large Language Models face significant efficiency challenges that stem from two distinct yet coupled sources: data redundancy and computational redundancy. While most me…
From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMs
Zhentao Tan, Chang Liu, Yao Liu +2
As large language models (LLMs) scale rapidly, dense full-parameter adaptation becomes increasingly expensive, motivating sparse and modular architectures such as Mixture-of-Expert…
HydraHead: From Head-Level Functional Heterogeneity to Specialized Attention Hybridization
Zhentao Tan, Wei Chen, Jingyi Shen +4
The quadratic complexity of attention poses a critical bottleneck for long-context processing, spurring interest in hybrid attention designs. Most open-source hybrid models adopt a…
SAFE-Pruner: Semantic Attention-Guided Future-Aware Token Pruning for Efficient Vision-Language-Action Manipulation
Shilin Ma, Chubin Zhang, Changyuan Wang +6
Real-time inference of vision-language-action (VLA) models is essential for robotic control. While visual token pruning has shown strong potential for accelerating inference, most…
Learning to Focus and Precise Cropping: A Reinforcement Learning Framework with Information Gaps and Grounding Loss for MLLMs
Xuanpu Zhao, Zhentao Tan, Dianmo Sheng +6
To enhance the perception and reasoning capabilities of multimodal large language models in complex visual scenes, recent research has introduced agent-based workflows. In these wo…
Flora: Effortless Context Construction to Arbitrary Length and Scale
Tianxiang Chen, Zhentao Tan, Xiaofan Bo +5
Effectively handling long contexts is challenging for Large Language Models (LLMs) due to the rarity of long texts, high computational demands, and substantial forgetting of short-…
Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection
Tianxiang Chen, Zhentao Tan, Tao Gong +5
As a manner to augment pre-trained large language models (LLM), knowledge injection is critical to develop vertical domain large models and has been widely studied. Although most c…
MiM-ISTD: Mamba-in-Mamba for Efficient Infrared Small Target Detection
Tianxiang Chen, Zi Ye, Zhentao Tan +6
Recently, infrared small target detection (ISTD) has made significant progress, thanks to the development of basic models. Specifically, the models combining CNNs with transformers…
Bootstrapping Audio-Visual Segmentation by Strengthening Audio Cues
Tianxiang Chen, Zhentao Tan, Tao Gong +6
How to effectively interact audio with vision has garnered considerable interest within the multi-modality research field. Recently, a novel audio-visual segmentation (AVS) task ha…
TCI-Former: Thermal Conduction-Inspired Transformer for Infrared Small Target Detection
Tianxiang Chen, Zhentao Tan, Qi Chu +3
Infrared small target detection (ISTD) is critical to national security and has been extensively applied in military areas. ISTD aims to segment small target pixels from background…
SimAC: A Simple Anti-Customization Method for Protecting Face Privacy against Text-to-Image Synthesis of Diffusion Models
Feifei Wang, Zhentao Tan, Tianyi Wei +2
Despite the success of diffusion-based customization methods on visual content creation, increasing concerns have been raised about such techniques from both privacy and political…