Efficient and Adaptable Overlapping for Computation and Communication via Signaling and Reordering

Ke Hong, Xiuhong Li, Minxu Liu +9

Generative models have achieved remarkable success across various applications, driving the demand for multi-GPU computing. Inter-GPU communication becomes a bottleneck in multi-GP…