DLO: Dynamic Layer Operation for Efficient Vertical Scaling of LLMs

Zhen Tan, Daize Dong, Xinyu Zhao +3

In this paper, we introduce Dynamic Layer Operations (DLO), a novel approach for vertically scaling transformer-based Large Language Models (LLMs) by dynamically expanding, activat…