Libra: Taming Attention Workload Skew in Long-Context LLM Training with Bounded Sequence Pool

Yan Wang, Xiulong Yuan, Kaiming Yang +16

Long-context LLM training suffers from a load-balancing problem that sequence packing does not solve. Packing samples into fixed-token sequences balances memory and linear-cost ope…