activity
20222024
most citedDEFT: Exploiting Gradient Norm Difference between Model Layers for Scalable Gradient Sparsification

6 citations · 6 across the 4 of their papers we have counts for

collaborators
Showing 2024Show all

1 paper · 1 filter

cs.LG2024

Preserving Near-Optimal Gradient Sparsification Cost for Scalable Distributed Deep Learning

Daegun Yoon, Sangyoon Oh

Communication overhead is a major obstacle to scaling distributed training systems. Gradient sparsification is a potential optimization approach to reduce the communication volume…