Adversary-resilient Distributed and Decentralized Statistical Inference and Machine Learning: An Overview of Recent Advances Under the Byzantine Threat Model

Abstract

While the last few decades have witnessed a huge body of work devoted to inference and learning in distributed and decentralized setups, much of this work assumes a non-adversarial setting in which individual nodes---apart from occasional statistical failures---operate as intended within the algorithmic framework. In recent years, however, cybersecurity threats from malicious non-state actors and rogue entities have forced practitioners and researchers to rethink the robustness of distributed and decentralized algorithms against adversarial attacks. As a result, we now have a plethora of algorithmic approaches that guarantee robustness of distributed and/or decentralized inference and learning under different adversarial threat models. Driven in part by the world's growing appetite for data-driven decision making, however, securing of distributed/decentralized frameworks for inference and learning against adversarial threats remains a rapidly evolving research area. In this article, we provide an overview of some of the most recent developments in this area under the threat model of Byzantine attacks.

arXiv comment24 pages, 6 figures, 2 tables; Published in IEEE Signal Processing Magazine, May 2020 (Special Issue on "Machine Learning From Distributed, Streaming Data")

References in corpus (6)

• The Hidden Vulnerability of Distributed Learning in Byzantium
• Distributed Bayesian Detection with Byzantine Data
• Resilient Distributed Estimation Through Adversary Detection
• ByRDiE: Byzantine-resilient distributed coordinate descent for decentralized learning
• Consensus based Detection in the Presence of Data Falsification Attacks
• The Internet of Things: Secure Distributed Inference

Cited by in corpus (10)

• Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks
• Combating Interference for Over-the-Air Federated Learning: A Statistical Approach via RIS
• BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackers
• A Linearly Convergent Algorithm for Distributed Principal Component Analysis
• Byzantine-Resilient Decentralized TD Learning with Linear Function Approximation
• FAST-PCA: A Fast and Exact Algorithm for Distributed Principal Component Analysis
• BROADCAST: Reducing Both Stochastic and Compression Noise to Robustify Communication-Efficient Federated Learning
• Scaling-up Distributed Processing of Data Streams for Machine Learning
• Robust Distributed Optimization With Randomly Corrupted Gradients
• Distributed Principal Subspace Analysis for Partitioned Big Data: Algorithms, Analysis, and Implementation