6 papers
Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting
Zipeng Gao, Zhi Zheng, Qingrong Xia +5
Speculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory-bound bottlenecks. However, traditional speculative decoding typically…
Entropy-KL Divergence-based Token Masking: A Novel Approach for Selective Fine-tuning of Large Language Models
Qi Liu, Mingdi Sun, Yongyi He +5
Supervised fine-tuning (SFT) followed by reinforcement learning (RL) has become a standard post-training paradigm for large language models. This paradigm provides a cold-start for…
Discovering Decoupled Functional Modules in Large Language Models
Yanke Yu, Jin Li, Ying Sun +3
Understanding the internal functional organization of Large Language Models (LLMs) is crucial for improving their trustworthiness and performance. However, how LLMs organize differ…
Adacc: An Adaptive Framework Unifying Compression and Activation Recomputation for LLM Training
Ping Chen, Zhuohong Deng, Ping Li +6
Training large language models (LLMs) is often constrained by GPU memory limitations. To alleviate memory pressure, activation recomputation and data compression have been proposed…
Optimizing Large Model Training through Overlapped Activation Recomputation
Ping Chen, Wenjie Zhang, Shuibing He +10
Large model training often uses recomputation to alleviate memory pressure and pipelines to exploit the parallelism of data, tensors, and devices. However, existing recomputation a…
FASP: Fast and Accurate Structured Pruning of Large Language Models
Hanyu Hu, Pengxiang Zhao, Ping Li +3
The rapid increase in the size of large language models (LLMs) has significantly escalated their computational and memory demands, posing challenges for efficient deployment, espec…