Moss: Proxy Model-based Full-Weight Aggregation in Federated Learning with Heterogeneous Models

Abstract

Modern Federated Learning (FL) has become increasingly essential for handling highly heterogeneous mobile devices. Current approaches adopt a partial model aggregation paradigm that leads to sub-optimal model accuracy and higher training overhead. In this paper, we challenge the prevailing notion of partial-model aggregation and propose a novel "full-weight aggregation" method named Moss, which aggregates all weights within heterogeneous models to preserve comprehensive knowledge. Evaluation across various applications demonstrates that Moss significantly accelerates training, reduces on-device training time and energy consumption, enhances accuracy, and minimizes network bandwidth utilization when compared to state-of-the-art baselines.

Accepted by ACM IMWUT/Ubicomp 2025

References in corpus (9)

• Federated Learning: Challenges, Methods, and Future Directions
• Knowledge Distillation: A Survey
• Robust Real-World Image Super-Resolution against Adversarial Attacks
• Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout
• LegoDNN: Block-grained Scaling of Deep Neural Networks for Mobile Vision
• FedSkel: Efficient Federated Learning on Heterogeneous Systems with Skeleton Gradients Update
• DistFL: Distribution-aware Federated Learning for Mobile Scenarios