Dynamic layer selection in decoder-only transformers

Theodore Glavas, Joud Chataoui, Florence Regol +4

The vast size of Large Language Models (LLMs) has prompted a search to optimize inference. One effective approach is dynamic inference, which adapts the architecture to the sample-…