1 citations · 1 across the 3 of their papers we have counts for
3 papers
cs.LG2025
Discovering Transformer Circuits via a Hybrid Attribution and Pruning Framework
Hao Gu, Vibhas Nair, Amrithaa Ashok Kumar +2
Interpreting language models often involves circuit analysis, which aims to identify sparse subnetworks, or circuits, that accomplish specific tasks. Existing circuit discovery alg…
cs.LG2025
Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
Lujun Li, Zhu Qiyuan, Jiacheng Wang +4
Mixture of Experts (MoE) LLMs face significant obstacles due to their massive parameter scale, which imposes memory, storage, and deployment challenges. Although recent expert merg…
cs.LG2025★ 1 cited
Delta Decompression for MoE-based LLMs Compression
Hao Gu, Wei Li, Lujun Li +5
Mixture-of-Experts (MoE) architectures in large language models (LLMs) achieve exceptional performance, but face prohibitive storage and memory requirements. To address these chall…