4 papers
Decode-Branch Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation
Liming Liu, Mingze Wang, Tuo Zhao
As large language models serve ever more requests, cumulative inference cost is growing relative to the one-time cost of training. In typical serving, prompt prefill runs in parall…
ARMOR: High-Performance Semi-Structured Pruning via Adaptive Matrix Factorization
Lawrence Liu, Alexander Liu, Mengdi Wang +2
Large language models (LLMs) present significant deployment challenges due to their immense computational and memory requirements. While semi-structured pruning, particularly 2:4 s…
NoWag: A Unified Framework for Shape Preserving Compression of Large Language Models
Lawrence Liu, Inesh Chakrabarti, Yixiao Li +3
Large language models (LLMs) exhibit remarkable performance across various natural language processing tasks but suffer from immense computational and memory demands, limiting thei…
IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining
Yixiao Li, Xianzhi Du, Ajay Jaiswal +4
Recent advancements in large language models have intensified the need for efficient and deployable models within limited inference budgets. Structured pruning pipelines have shown…