Optimising cost vs accuracy of decentralised analytics in fog computing environments

Abstract

The exponential growth of devices and data at the edges of the Internet is rising scalability and privacy concerns on approaches based exclusively on remote cloud platforms. Data gravity, a fundamental concept in Fog Computing, points towards decentralisation of computation for data analysis, as a viable alternative to address those concerns. Decentralising AI tasks on several cooperative devices means identifying the optimal set of locations or Collection Points (CP for short) to use, in the continuum between full centralisation (i.e., all data on a single device) and full decentralisation (i.e., data on source locations). We propose an analytical framework able to find the optimal operating point in this continuum, linking the accuracy of the learning task with the corresponding network and computational cost for moving data and running the distributed training at the CPs. We show through simulations that the model accurately predicts the optimal trade-off, quite often an intermediate point between full centralisation and full decentralisation, showing also a significant cost saving w.r.t. both of them. Finally, the analytical model admits closed-form or numeric solutions, making it not only a performance evaluation instrument but also a design tool to configure a given distributed learning task optimally before its deployment.

arXiv commentAccepted for publication in IEEE TNSE

References in corpus (11)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
• Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
• Federated Optimization:Distributed Optimization Beyond the Datacenter
• Agnostic Federated Learning
• GossipGraD: Scalable Deep Learning using Gossip Communication based Asynchronous Gradient Descent
• Nonasymptotic Convergence Rates for Cooperative Learning Over Time-Varying Directed Graphs
• Distributed Stochastic Variance Reduced Gradient Methods and A Lower Bound for Communication Complexity
• Gossip training for deep learning
• A communication efficient distributed learning framework for smart environments
• Distributed Learning over Unreliable Networks