3 citations · 4 across the 5 of their papers we have counts for
1 paper · 2 filters
Yanbing Chen, Ruilin Wang, Zihao Yang +2
Packing and shuffling tokens is a common practice in training auto-regressive language models (LMs) to prevent overfitting and improve efficiency. Typically documents are concatena…