TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer

Zhen Qin, Dong Li, Weigao Sun +8

We present TransNormerLLM, the first linear attention-based Large Language Model (LLM) that outperforms conventional softmax attention-based models in terms of both accuracy and ef…