Zhen Jia

most citedLancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping

1 citations · 1 across the 2 of their papers we have counts for

collaborators

2 papers

math.CO2024

Non-uniform Cross-intersecting Families

Zhen Jia, Qing Xiang, Jimeng Xiao +1

Let $m \geq 2$ , $n$ be positive integers, and $R_{i} = {k_{i, 1} > k_{i, 2} > \dots > k_{i, t_{i}}}$ be subsets of $[n]$ for $i = 1, 2, \dots, m$ . The families $\mathcal{F}_1\subseteq \binom{[n]…

cs.DC2024★ 1 cited

Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping

Chenyu Jiang, Ye Tian, Zhen Jia +3

The Mixture-of-Expert (MoE) technique plays a crucial role in expanding the size of DNN model parameters. However, it faces the challenge of extended all-to-all communication laten…