Model Pruning Enables Localized and Efficient Federated Learning for Yield Forecasting and Data Sharing

Abstract

Federated Learning (FL) presents a decentralized approach to model training in the agri-food sector and offers the potential for improved machine learning performance, while ensuring the safety and privacy of individual farms or data silos. However, the conventional FL approach has two major limitations. First, the heterogeneous data on individual silos can cause the global model to perform well for some clients but not all, as the update direction on some clients may hinder others after they are aggregated. Second, it is lacking with respect to the efficiency perspective concerning communication costs during FL and large model sizes. This paper proposes a new technical solution that utilizes network pruning on client models and aggregates the pruned models. This method enables local models to be tailored to their respective data distribution and mitigate the data heterogeneity present in agri-food data. Moreover, it allows for more compact models that consume less data during transmission. We experiment with a soybean yield forecasting dataset and find that this approach can improve inference performance by 15.5% to 20% compared to FedAvg, while reducing local model sizes by up to 84% and the data volume communicated between the clients and the server by 57.1% to 64.7%.

31 pages, 4 figures, 4 tables

References in corpus (19)

• Federated Learning: Challenges, Methods, and Future Directions
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• The Future of Digital Health with Federated Learning
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Pre-trained Models for Natural Language Processing: A Survey
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
• Predicting Parameters in Deep Learning
• FedKD: Communication Efficient Federated Learning via Knowledge Distillation
• A CNN-RNN Framework for Crop Yield Prediction
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
• Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
• SNIP: Single-shot Network Pruning based on Connection Sensitivity
• Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization
• Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
• Federated Learning of a Mixture of Global and Local Models
• Federated and Differentially Private Learning for Electronic Health Records
• Nemesyst: A Hybrid Parallelism Deep Learning-Based Framework Applied for Internet of Things Enabled Food Retailing Refrigeration Systems
• Spectral Pruning: Compressing Deep Neural Networks via Spectral Analysis and its Generalization Error

Cited by in corpus (3)

• Federated Learning Architectures: A Performance Evaluation with Crop Yield Prediction Application
• Communication-Efficient Training Workload Balancing for Decentralized Multi-Agent Learning
• Federated learning in food research