Early Exiting with Ensemble Internal Classifiers

Tianxiang Sun, Yunhua Zhou, Xiangyang Liu +5

As a simple technique to accelerate inference of large-scale pre-trained models, early exiting has gained much attention in the NLP community. It allows samples to exit early at in…