Distilling Sequential Computation in Transformer Language Models

Zixuan Lan, Jessica Yang, Yanhong Li +2

Transformer language models process sequences token by token in an autoregressive manner, making growing contexts increasingly expensive. Yet many adjacent token spans are highly p…