KnapFormer: An Online Load Balancer for Efficient Diffusion Transformers Training

Kai Zhang, Peng Wang, Sai Bi +2

We present KnapFormer, an efficient and versatile framework to combine workload balancing and sequence parallelism in distributed training of Diffusion Transformers (DiT). KnapForm…