2 papers
cs.PF2026
Optimus: A Generic Operator-Level PyTorch Model Transformation Framework
Menglu Yu, Jiaqi Xu, Yuzhen Huang +19
In large-scale industrial applications, deep learning models that power recommendation and ranking have complex and diverse model architectures. These models are continuously devel…
cs.DC2024
SimpleFSDP: Simpler Fully Sharded Data Parallel with torch.compile
Ruisi Zhang, Tianyu Liu, Will Feng +4
Distributed training of large models consumes enormous computation resources and requires substantial engineering efforts to compose various training techniques. This paper present…