2 papers
cs.DC2025
FFTrainer: Fast Failover in Large-Language Model Training with Almost-Free State Management
Bohan Zhao, Yuanhong Wang, Chenglin Liu +6
Recent developments in large language models (LLMs) have introduced new requirements for efficient and robust training. As LLM clusters scale, node failures, lengthy recoveries, an…
cs.NI2025
ForestColl: Throughput-Optimal Collective Communications on Heterogeneous Network Fabrics
Liangyu Zhao, Saeed Maleki, Yuanhong Wang +4
As modern DNN models grow ever larger, collective communications between the accelerators (allreduce, etc.) emerge as a significant performance bottleneck. Designing efficient comm…