3 papers
cs.CL2026
Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning
Minsik Choi, Hyegang Son, Changhoon Kim +1
Transformer-based models have achieved remarkable performance in NLP tasks. However, their structural characteristics-multiple layers and attention heads-introduce efficiency chall…
cs.CV2025
CorGi: Contribution-Guided Block-Wise Interval Caching for Training-Free Acceleration of Diffusion Transformers
Yonglak Son, Suhyeok Kim, Seungryong Kim +1
Diffusion transformer (DiT) achieves remarkable performance in visual generation, but its iterative denoising process combined with larger capacity leads to a high inference cost.…
cs.LG2025
First Attentions Last: Better Exploiting First Attentions for Efficient Transformer Training
Gyudong Kim, Hyukju Na, Jin Hyeon Kim +6
As training billion-scale transformers becomes increasingly common, employing multiple distributed GPUs along with parallel training methods has become a standard practice. However…