A Tensor-Train Decomposition based Compression of LLMs on Group Vector Systolic Accelerator

Sixiao Huang, Tintin Wang, Ang Li +5

Large language models (LLMs) are both storage-intensive and computation-intensive, posing significant challenges when deployed on resource-constrained hardware. As linear layers in…