Proving the Limited Scalability of Centralized Distributed Optimization via a New Lower Bound Construction

Alexander Tyurin

We consider centralized distributed optimization in the classical federated learning setup, where $n$ workers jointly find an $\epsilon$-stationary point of an $L$-smooth, $d$-d…