Fast-adapting and Privacy-preserving Federated Recommender System
arXiv:2104.00919
Abstract
In the mobile Internet era, the recommender system has become an irreplaceable tool to help users discover useful items, and thus alleviating the information overload problem. Recent deep neural network (DNN)-based recommender system research have made significant progress in improving prediction accuracy, which is largely attributed to the access to a large amount of users' personal data collected from users' devices and then centrally stored in the cloud server. However, as there are rising concerns around the globe on user privacy leakage in the online platform, the public is becoming anxious by such abuse of user privacy. Therefore, it is urgent and beneficial to develop a recommender system that can achieve both high prediction accuracy and high degree of user privacy protection. To this end, we propose a DNN-based recommendation model called PrivRec running on the decentralized federated learning (FL) environment, which ensures that a user's data never leaves his/her during the course of model training. On the other hand, to better embrace the data heterogeneity commonly existing in FL, we innovatively introduce a first-order meta-learning method that enables fast in-device personalization with only few data points. Furthermore, to defense from potential malicious participant that poses serious security threat to other users, we develop a user-level differentially private DP-PrivRec model so that it is unable to determine whether a particular user is present or not solely based on the trained model. Finally, we conduct extensive experiments on two large-scale datasets in a simulated FL environment, and the results validate the superiority of our proposed PrivRec and DP-PrivRec.
arXiv commentAcceptd by VLDB J
References in corpus (27)
- Deep Learning with Differential Privacy
- Distributed Representations of Sentences and Documents
- Federated Learning with Non-IID Data
- Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
- Federated Learning for Mobile Keyboard Prediction
- Towards Federated Learning at Scale: System Design
- Learning Differentially Private Recurrent Language Models
- A Survey on Multi-Task Learning
- Differentially Private Federated Learning: A Client Level Perspective
- Federated Optimization in Heterogeneous Networks
- Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
- Improving Federated Learning Personalization via Model Agnostic Meta Learning
- Personalized Federated Learning: A Meta-Learning Approach
- Adaptive Personalized Federated Learning
- A generic framework for privacy preserving deep learning
- Three Approaches for Personalization with Applications to Federated Learning
- Federated Meta-Learning with Fast Convergence and Efficient Communication
- Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
- Meta-Learning: A Survey
- Subsampled Rényi Differential Privacy and Analytical Moments Accountant
- Differentially Private Learning with Adaptive Clipping
- Evaluating Differentially Private Machine Learning in Practice
- A Mutual Information Maximization Perspective of Language Representation Learning
- A Survey on Session-based Recommender Systems
- Entity Resolution and Federated Learning get a Federated Resolution
- Federated Multi-view Matrix Factorization for Personalized Recommendations
- AdaCliP: Adaptive Clipping for Private SGD