Lizard: An Efficient Linearization Framework for Large Language Models

Chien Van Nguyen, Huy Nguyen, Ruiyi Zhang +10

We propose Lizard, a linearization framework that transforms pretrained Transformer-based Large Language Models (LLMs) into subquadratic architectures. Transformers faces severe co…