48 citations · 81 across the 9 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023★ 1 cited
ROAM: memory-efficient large DNN training via optimized operator ordering and memory layout
Huiyao Shu, Ang Wang, Ziji Shi +3
As deep learning models continue to increase in size, the memory requirements for training have surged. While high-level techniques like offloading, recomputation, and compression…
cs.LG2023
TAPAS: Fast and Automatic Derivation of Tensor Parallel Strategies for Large Neural Networks
Ziji Shi, Le Jiang, Ang Wang +6
Tensor parallelism is an essential technique for distributed training of large neural networks. However, automatically determining an optimal tensor parallel strategy is challengin…