Dependency-Aware Semi-Structured Sparsity of GLU Variants in Large Language Models

Zhiyu Guo, Hidetaka Kamigaito, Taro Wanatnabe

The rapid advancement in Large Language Models (LLMs) has markedly enhanced the capabilities of language understanding and generation. However, the substantial model size poses har…