Dynamic Vocabulary Pruning in Early-Exit LLMs

Jort Vincenti, Karim Abdel Sadek, Joan Velja +2

Increasing the size of large language models (LLMs) has been shown to lead to better performance. However, this comes at the cost of slower and more expensive inference. Early-exit…