153 citations · 333 across the 85 of their papers we have counts for
13 papers · 1 filter
CommitKV: Lifecycle-Aware KV Cache Compression via Commit Transitions for Multi-Turn Agents
Weizhong Huang, Jinchao Zhang, Xiawu Zheng
Multi-turn Reasoning-and-Acting (ReAct) agents accumulate growing trajectories of reasoning, tool calls, and observations. Their key-value (KV) caches grow accordingly, increasing…
AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
Wanqi Yang, Yuexiao Ma, Alexander Conzelmann +4
Mixture-of-Experts (MoE) architectures scale model capacity through sparse expert activation, but their deployment remains memory-bound because all expert weights must reside in me…
Polybasic Speculative Decoding Through a Theoretical Perspective
Ruilin Wang, Huixia Li, Yuexiao Ma +4
Inference latency stands as a critical bottleneck in the large-scale deployment of Large Language Models (LLMs). Speculative decoding methods have recently shown promise in acceler…
Dynamic Low-Rank Sparse Adaptation for Large Language Models
Weizhong Huang, Yuxin Zhang, Xiawu Zheng +4
Despite the efficacy of network sparsity in alleviating the deployment strain of Large Language Models (LLMs), it endures significant performance degradation. Applying Low-Rank Ada…
Determining Layer-wise Sparsity for Large Language Models Through a Theoretical Perspective
Weizhong Huang, Yuxin Zhang, Xiawu Zheng +2
In this paper, we address the challenge of determining the layer-wise sparsity rates of large language models (LLMs) through a theoretical perspective. Specifically, we identify a…
Towards Efficient Automatic Self-Pruning of Large Language Models
Weizhong Huang, Yuxin Zhang, Xiawu Zheng +2
Despite exceptional capabilities, Large Language Models (LLMs) still face deployment challenges due to their enormous size. Post-training structured pruning is a promising solution…
AffineQuant: Affine Transformation Quantization for Large Language Models
Yuexiao Ma, Huixia Li, Xiawu Zheng +6
The significant resource requirements associated with Large-scale Language Models (LLMs) have generated considerable interest in the development of techniques aimed at compressing…
EBFT: Effective and Block-Wise Fine-Tuning for Sparse LLMs
Song Guo, Fan Wu, Lei Zhang +5
Existing methods for fine-tuning sparse LLMs often suffer from resource-intensive requirements and high retraining costs. Additionally, many fine-tuning methods often rely on appro…
Learning Sparse Neural Networks with Identity Layers
Mingjian Ni, Guangyao Chen, Xiawu Zheng +3
The sparsity of Deep Neural Networks is well investigated to maximize the performance and reduce the size of overparameterized networks as possible. Existing methods focus on pruni…
Probabilistic Modeling: Proving the Lottery Ticket Hypothesis in Spiking Neural Network
Man Yao, Yuhong Chou, Guangshe Zhao +4
The Lottery Ticket Hypothesis (LTH) states that a randomly-initialized large neural network contains a small sub-network (i.e., winning tickets) which, when trained in isolation, c…
A Unified Framework for Soft Threshold Pruning
Yanqi Chen, Zhengyu Ma, Wei Fang +3
Soft threshold pruning is among the cutting-edge pruning methods with state-of-the-art performance. However, previous methods either perform aimless searching on the threshold sche…
OMPQ: Orthogonal Mixed Precision Quantization
Yuexiao Ma, Taisong Jin, Xiawu Zheng +6
To bridge the ever increasing gap between deep neural networks' complexity and hardware capability, network quantization has attracted more and more research attention. The latest…
Multinomial Distribution Learning for Effective Neural Architecture Search
Xiawu Zheng, Rongrong Ji, Lang Tang +3
Architectures obtained by Neural Architecture Search (NAS) have achieved highly competitive performance in various computer vision tasks. However, the prohibitive computation deman…