Towards Tight Communication Lower Bounds for Distributed Optimisation

Dan Alistarh, Janne H. Korhonen

We consider a standard distributed optimisation setting where $N$ machines, each holding a $d$-dimensional function $f_i$, aim to jointly minimise the sum of the functions $\sum_{i…