Pipelined Decoder for Efficient Context-Aware Text Generation

Zixian Huang, Chenxu Niu, Yu Gu +3

As the basis of generative AI, an autoregressive model requires the generation of a new token depending on all the previously generated tokens, which brings high quality but also r…