6 papers
Bifocal: Training-Free Mixed Granularity Visual Token Pruning for Multimodal LLMs
Peiran Yin, Hao Wu, Zihao Yang +6
Visual token pruning for multimodal large language models (MLLMs) appears highly effective, with existing methods retaining over 95% of unpruned performance on common VQA benchmark…
ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents
Yichu Fang, Sitong Wei, Haozhe Hu +1
Agentic language models repeatedly encode tool and skill schemas that recur across requests in different combinations and orders, preventing standard prefix caching from reusing th…
WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
Haozhe Hu, Hao Wu, Peiran Yin +3
Pruning is a promising approach for improving the efficiency of large language models (LLMs). Static structured pruning is hardware-friendly, but its input-agnostic computation all…
Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models
Chao Han, Haozhe Hu, Xiaoyu Shen
Large language models (LLMs) are often compressed through static parameter pruning or dynamic token-level computation, yet aggressive sparsification can trigger rapid performance d…
UniRank: Unified Rank Allocation for Low-Rank LLM Compression
Chao Han, Haozhe Hu, Yongjie Du +5
Low-rank decomposition is a promising compression paradigm for large language models (LLMs), yet its effectiveness hinges on rank budget allocation across weight matrices: uniform…
Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy
Haozhe Hu, Hao Wu, Anhao Zhao +4
Pruning has emerged as a dominant paradigm for accelerating large language model (LLM) inference, spanning a broad spectrum of methods that remove computation across tokens, layers…