MAR: Efficient Large Language Models via Module-aware Architecture Refinement

Junhong Cai, Guiqin Wang, Kejie Zhao +6

Large Language Models (LLMs) excel across diverse domains but suffer from high energy costs due to quadratic attention and dense Feed-Forward Network (FFN) operations. To address t…