Channel-driven Decentralized Bayesian Federated Learning for Trustworthy Decision Making in D2D Networks

Abstract

Bayesian Federated Learning (FL) offers a principled framework to account for the uncertainty caused by limitations in the data available at the nodes implementing collaborative training. In Bayesian FL, nodes exchange information about local posterior distributions over the model parameters space. This paper focuses on Bayesian FL implemented in a device-to-device (D2D) network via Decentralized Stochastic Gradient Langevin Dynamics (DSGLD), a recently introduced gradient-based Markov Chain Monte Carlo (MCMC) method. Based on the observation that DSGLD applies random Gaussian perturbations of model parameters, we propose to leverage channel noise on the D2D links as a mechanism for MCMC sampling. The proposed approach is compared against a conventional implementation of frequentist FL based on compression and digital transmission, highlighting advantages and limitations.

arXiv commentsubmitted

References in corpus (9)

• Federated Learning: Challenges, Methods, and Future Directions
• Variational Inference: A Review for Statisticians
• On Calibration of Modern Neural Networks
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• Opportunities of Federated Learning in Connected, Cooperative and Automated Industrial Systems
• Understanding Top-k Sparsification in Distributed Deep Learning
• Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo
• An introduction to decentralized stochastic optimization with gradient tracking
• Partitioned Variational Inference: A Framework for Probabilistic Federated Learning