T-CCL: Resource Efficient and Performant Collective Communication using Tensor Memory Accelerator

Keyvan Dadashzadeh, Yuehong Zhou, Minyu Cui +1

Large transformer-based models increasingly depend on multi-GPU execution, which requires frequent collective communication among GPUs. Existing communication libraries often rely…