Storage, Computation, and Communication: A Fundamental Tradeoff in Distributed Computing

Abstract

We consider a MapReduce-like distributed computing system. We derive a lower bound on the communication cost for any given storage and computation costs. This lower bound matches the achievable bound we proposed recently. As a result, we completely characterize the optimal tradeoff between the storage, the computation, and the communication. Our result generalizes the previous one by Li et al. to also account for the number of computed intermediate values.

1 figure, has been published in 2018 IEEE Information Theory Workshop (ITW)