1 citations · 1 across the 10 of their papers we have counts for
8 papers · 1 filter
GA-SVD: Generation-aware Singular Value Decomposition for LLM Compression
Shengkun Tang, Cong Zeng, Zongfang Liu +2
Large language models (LLMs) demonstrate strong generalization across a broad range of reasoning and generation tasks, but their large parameter counts impose substantial memory an…
WUSPruner: Second-Moment-Optimal Transforms for Structured Pruning
Shengkun Tang, Dan Alistarh, Zhiqiang Shen
Structured pruning is typically formulated as identifying which model dimensions to remove, often within the model's original representation. However, the effect of coordinate dele…
SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
Shengkun Tang, Zekun Wang, Bo Zheng +7
Structured pruning and knowledge distillation (KD) are typical techniques for compressing large language models, but it remains unclear how they should be applied at pretraining sc…
EvoESAP: Non-Uniform Expert Pruning for Sparse MoE
Zongfang Liu, Shengkun Tang, Boyang Sun +2
Sparse Mixture-of-Experts (SMoE) language models achieve strong capability at low per-token compute, yet deployment remains constrained by memory footprint and throughput because t…
Diff-ES: Stage-wise Structural Diffusion Pruning via Evolutionary Search
Zongfang Liu, Shengkun Tang, Zongliang Wu +2
Diffusion models have achieved remarkable success in high-fidelity image generation but remain computationally demanding due to their multi-step denoising process and large model s…
AIMER: Calibration-Free Task-Agnostic MoE Expert Pruning
Zongfang Liu, Guangyi Chen, Shengkun Tang +3
Mixture-of-Experts (MoE) language models increase parameter capacity without proportional per-token computation, yet deployment still requires storing the full expert pool, making…
Sink-Aware Pruning for Diffusion Language Models
Aidar Myrzakhan, Tianyi Li, Bowei Guo +2
Diffusion Language Models (DLMs) incur high inference cost due to iterative denoising, motivating efficient pruning. Existing pruning heuristics largely inherited from autoregressi…
Canzona: A Unified, Asynchronous, and Load-Balanced Framework for Distributed Matrix-based Optimizers
Liangyu Wang, Siqi Zhang, Junjie Wang +7
The scaling of Large Language Models (LLMs) drives interest in matrix-based optimizers (e.g., Shampoo, Muon, SOAP) for their convergence efficiency; yet their requirement for holis…