3 papers
cs.DC2026
Memory-Efficient Expert Routing for Distributed MoE Training
Arnab Kanti Tarafder, Jaume Guasch-Martí, Gokcen Kestor +1
As Mixture-of-Experts (MoE) models scale toward hundreds of experts and higher top- routing, memory efficiency in distributed training becomes a critical bottleneck. Peak memory…
cs.LG2024
Optimization of GNN Training Through Half-precision
Arnab Kanti Tarafder, Yidong Gong, Pradeep Kumar
Recent trends in lower precision, e.g. half-precision floating point, training have shown improved system performance and reduced memory usage for Deep Learning while maintaining a…
cs.LG2024
Single-GPU GNN Systems: Traps and Pitfalls
Yidong Gong, Arnab Tarafder, Saima Afrin +1
The current graph neural network (GNN) systems have established a clear trend of not showing training accuracy results, and directly or indirectly relying on smaller datasets for e…