Edge-cloud Collaborative Learning with Federated and Centralized Features

Abstract

Federated learning (FL) is a popular way of edge computing that doesn't compromise users' privacy. Current FL paradigms assume that data only resides on the edge, while cloud servers only perform model averaging. However, in real-life situations such as recommender systems, the cloud server has the ability to store historical and interactive features. In this paper, our proposed Edge-Cloud Collaborative Knowledge Transfer Framework (ECCT) bridges the gap between the edge and cloud, enabling bi-directional knowledge transfer between both, sharing feature embeddings and prediction logits. ECCT consolidates various benefits, including enhancing personalization, enabling model heterogeneity, tolerating training asynchronization, and relieving communication burdens. Extensive experiments on public and industrial datasets demonstrate ECCT's effectiveness and potential for use in academia and industry.

arXiv commentAccepted by Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '23)

References in corpus (14)

• Distilling the Knowledge in a Neural Network
• Deep Learning with Differential Privacy
• Federated Learning: Challenges, Methods, and Future Directions
• Towards Federated Learning at Scale: System Design
• AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge
• A Field Guide to Federated Optimization
• Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matching
• FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion
• CDKT-FL: Cross-Device Knowledge Transfer using Proxy Dataset in Federated Learning
• Federated Graph Machine Learning: A Survey of Concepts, Techniques, and Applications
• EdgeRec: Recommender System on Edge in Mobile Taobao
• FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server