From the 1 of 11 linked papers with an AI index.
11 papers
DarwinLM: Evolutionary Structured Pruning of Large Language Models
Shengkun Tang, Oliver Sieberling, Eldar Kurtic +2
The paper introduces DarwinLM, an evolutionary search method for training-aware structured pruning of large language models that integrates lightweight post‑pruning training to fin…
AIMER: Calibration-Free Task-Agnostic MoE Expert Pruning
Zongfang Liu, Guangyi Chen, Shengkun Tang +3
Mixture-of-Experts (MoE) language models increase parameter capacity without proportional per-token computation, yet deployment still requires storing the full expert pool, making…
SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
Shengkun Tang, Zekun Wang, Bo Zheng +7
Structured pruning and knowledge distillation (KD) are typical techniques for compressing large language models, but it remains unclear how they should be applied at pretraining sc…
EvoESAP: Non-Uniform Expert Pruning for Sparse MoE
Zongfang Liu, Shengkun Tang, Boyang Sun +2
Sparse Mixture-of-Experts (SMoE) language models achieve strong capability at low per-token compute, yet deployment remains constrained by memory footprint and throughput because t…
Diff-ES: Stage-wise Structural Diffusion Pruning via Evolutionary Search
Zongfang Liu, Shengkun Tang, Zongliang Wu +2
Diffusion models have achieved remarkable success in high-fidelity image generation but remain computationally demanding due to their multi-step denoising process and large model s…
Sink-Aware Pruning for Diffusion Language Models
Aidar Myrzakhan, Tianyi Li, Bowei Guo +2
Diffusion Language Models (DLMs) incur high inference cost due to iterative denoising, motivating efficient pruning. Existing pruning heuristics largely inherited from autoregressi…