Efficient In-Memory Acceleration of Sparse Block Diagonal LLMs

João Paulo Cardoso de Lima, Marc Dietrich, Jeronimo Castrillon +1

Structured sparsity enables deploying large language models (LLMs) on resource-constrained systems. Approaches like dense-to-sparse fine-tuning are particularly compelling, achievi…