Skip a Layer or Loop It? Learning Program-of-Layers in LLMs

Ziyue Li, Yang Li, Tianyi Zhou

Large language models (LLMs) perform inference by following a fixed depth and order, non-recurrent execution of all layers. We reveal the wide existence of training-free, flexible,…