Hierarchical Skip Decoding for Efficient Autoregressive Text Generation

Yunqi Zhu, Xuebing Yang, Yuanyuan Wu +1

Autoregressive decoding strategy is a commonly used method for text generation tasks with pre-trained language models, while early-exiting is an effective approach to speedup the i…