IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining

Yixiao Li, Xianzhi Du, Ajay Jaiswal +4

Recent advancements in large language models have intensified the need for efficient and deployable models within limited inference budgets. Structured pruning pipelines have shown…