Federated Learning with Only Positive Labels

Abstract

We consider learning a multi-class classification model in the federated setting, where each user has access to the positive data associated with only a single class. As a result, during each federated learning round, the users need to locally update the classifier without having access to the features and the model parameters for the negative classes. Thus, naively employing conventional decentralized learning such as the distributed SGD or Federated Averaging may lead to trivial or extremely poor classifiers. In particular, for the embedding based classifiers, all the class embeddings might collapse to a single point. To address this problem, we propose a generic framework for training with only positive labels, namely Federated Averaging with Spreadout (FedAwS), where the server imposes a geometric regularizer after each round to encourage classes to be spreadout in the embedding space. We show, both theoretically and empirically, that FedAwS can almost match the performance of conventional learning where users have access to negative labels. We further extend the proposed method to the settings with large output spaces.

References in corpus (6)

• Deep Learning with Differential Privacy
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Practical Secure Aggregation for Federated Learning on User-Held Data
• PU Learning for Matrix Completion
• Generative Models for Effective ML on Private, Decentralized Datasets
• Stochastic Negative Mining for Learning with Large Output Spaces

Cited by in corpus (14)

• FedML: A Research Library and Benchmark for Federated Machine Learning
• Federated Learning on Non-IID Data Silos: An Experimental Study
• Oort: Efficient Federated Learning via Guided Participant Selection
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• FedEval: A Holistic Evaluation Framework for Federated Learning
• FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning
• TurboSVM-FL: Boosting Federated Learning through SVM Aggregation for Lazy Clients
• Unified Group Fairness on Federated Learning
• Federated Learning on Non-IID Data: A Survey
• A Robust Federated Learning Approach for Combating Attacks Against IoT Systems Under non-IID Challenges
• Federated Learning of User Authentication Models
• Federated Learning of User Verification Models Without Sharing Embeddings
• EasyFL: A Low-code Federated Learning Platform For Dummies
• Distantly Supervised Relation Extraction in Federated Settings