2 citations · 3 across the 3 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2023★ 1 cited
ROAM: memory-efficient large DNN training via optimized operator ordering and memory layout
Huiyao Shu, Ang Wang, Ziji Shi +3
As deep learning models continue to increase in size, the memory requirements for training have surged. While high-level techniques like offloading, recomputation, and compression…
cs.LG2023
TAPAS: Fast and Automatic Derivation of Tensor Parallel Strategies for Large Neural Networks
Ziji Shi, Le Jiang, Ang Wang +6
Tensor parallelism is an essential technique for distributed training of large neural networks. However, automatically determining an optimal tensor parallel strategy is challengin…