FedDebug: Systematic Debugging for Federated Learning Applications

Abstract

In Federated Learning (FL), clients independently train local models and share them with a central aggregator to build a global model. Impermissibility to access clients' data and collaborative training make FL appealing for applications with data-privacy concerns, such as medical imaging. However, these FL characteristics pose unprecedented challenges for debugging. When a global model's performance deteriorates, identifying the responsible rounds and clients is a major pain point. Developers resort to trial-and-error debugging with subsets of clients, hoping to increase the global model's accuracy or let future FL rounds retune the model, which are time-consuming and costly. We design a systematic fault localization framework, FedDebug, that advances the FL debugging on two novel fronts. First, FedDebug enables interactive debugging of realtime collaborative training in FL by leveraging record and replay techniques to construct a simulation that mirrors live FL. FedDebug's breakpoint can help inspect an FL state (round, client, and global model) and move between rounds and clients' models seamlessly, enabling a fine-grained step-by-step inspection. Second, FedDebug automatically identifies the client(s) responsible for lowering the global model's performance without any testing data and labels--both are essential for existing debugging techniques. FedDebug's strengths come from adapting differential testing in conjunction with neuron activations to determine the client(s) deviating from normal behavior. FedDebug achieves 100% accuracy in finding a single faulty client and 90.3% accuracy in finding multiple faulty clients. FedDebug's interactive debugging incurs 1.2% overhead during training, while it localizes a faulty client in only 2.1% of a round's training time.

Published at ICSE 2023. Link https://ieeexplore.ieee.org/document/10172839

References in corpus (17)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• The Future of Digital Health with Federated Learning
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• How To Backdoor Federated Learning
• Poisoning Attacks against Support Vector Machines
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• Analyzing Federated Learning through an Adversarial Lens
• FedML: A Research Library and Benchmark for Federated Machine Learning
• DLFuzz: Differential Fuzzing Testing of Deep Learning Systems
• Dimensionality-Driven Learning with Noisy Labels
• FMore: An Incentive Scheme of Multi-dimensional Auction for Federated Learning in MEC
• TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
• Battery-constrained Federated Edge Learning in UAV-enabled IoT for B5G/6G Networks
• Beyond Synthetic Noise: Deep Learning on Controlled Noisy Labels
• A Comprehensive Survey of Incentive Mechanism for Federated Learning
• FOCUS: Dealing with Label Quality Disparity in Federated Learning

Cited by in corpus (3)

• MeanCache: User-Centric Semantic Caching for LLM Web Services
• FedDefender: Backdoor Attack Defense in Federated Learning
• TraceFL: Interpretability-Driven Debugging in Federated Learning via Neuron Provenance