3 citations · 6 across the 18 of their papers we have counts for
4 papers · 1 filter
Training Long-Context LLMs Efficiently via Chunk-wise Optimization
Wenhao Li, Yuxin Zhang, Gen Luo +2
While long-context large language models (LLMs) exhibit remarkable document processing capabilities, their prohibitively high training costs often hinder customized applications. T…
Dynamic Low-Rank Sparse Adaptation for Large Language Models
Weizhong Huang, Yuxin Zhang, Xiawu Zheng +4
Despite the efficacy of network sparsity in alleviating the deployment strain of Large Language Models (LLMs), it endures significant performance degradation. Applying Low-Rank Ada…
Determining Layer-wise Sparsity for Large Language Models Through a Theoretical Perspective
Weizhong Huang, Yuxin Zhang, Xiawu Zheng +2
In this paper, we address the challenge of determining the layer-wise sparsity rates of large language models (LLMs) through a theoretical perspective. Specifically, we identify a…
Towards Efficient Automatic Self-Pruning of Large Language Models
Weizhong Huang, Yuxin Zhang, Xiawu Zheng +2
Despite exceptional capabilities, Large Language Models (LLMs) still face deployment challenges due to their enormous size. Post-training structured pruning is a promising solution…