ESPACE: Dimensionality Reduction of Activations for Model Compression

Charbel Sakr, Brucek Khailany

We propose ESPACE, an LLM compression technique based on dimensionality reduction of activations. Unlike prior works on weight-centric tensor decomposition, ESPACE projects activat…