Xinyu Zhao

most citedDLO: Dynamic Layer Operation for Efficient Vertical Scaling of LLMs

1 citations · 1 across the 2 of their papers we have counts for

collaborators

2 papers

cs.LG2024★ 1 cited

DLO: Dynamic Layer Operation for Efficient Vertical Scaling of LLMs

Zhen Tan, Daize Dong, Xinyu Zhao +3

In this paper, we introduce Dynamic Layer Operations (DLO), a novel approach for vertically scaling transformer-based Large Language Models (LLMs) by dynamically expanding, activat…

cs.LG2024

$MoE-RBench$ : Towards Building Reliable Language Models with Sparse Mixture-of-Experts

Guanjie Chen, Xinyu Zhao, Tianlong Chen +1

Mixture-of-Experts (MoE) has gained increasing popularity as a promising framework for scaling up large language models (LLMs). However, the reliability assessment of MoE lags behi…

DLO: Dynamic Layer Operation for Efficient Vertical Scaling of LLMs

MoE-RBench: Towards Building Reliable Language Models with Sparse Mixture-of-Experts

$MoE-RBench$ : Towards Building Reliable Language Models with Sparse Mixture-of-Experts