Distantly Supervised Relation Extraction in Federated Settings

Abstract

This paper investigates distantly supervised relation extraction in federated settings. Previous studies focus on distant supervision under the assumption of centralized training, which requires collecting texts from different platforms and storing them on one machine. However, centralized training is challenged by two issues, namely, data barriers and privacy protection, which make it almost impossible or cost-prohibitive to centralize data from multiple platforms. Therefore, it is worthy to investigate distant supervision in the federated learning paradigm, which decouples the model training from the need for direct access to the raw data. Overcoming label noise of distant supervision, however, becomes more difficult in federated settings, since the sentences containing the same entity pair may scatter around different platforms. In this paper, we propose a federated denoising framework to suppress label noise in federated settings. The core of this framework is a multiple instance learning based denoising method that is able to select reliable instances via cross-platform collaboration. Various experimental results on New York Times dataset and miRNA gene regulation relation dataset demonstrate the effectiveness of the proposed method.

References in corpus (16)

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Learning with Non-IID Data
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Federated Learning for Mobile Keyboard Prediction
• Federated Optimization in Heterogeneous Networks
• Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
• Relation Classification via Recurrent Neural Network
• Reinforcement Learning for Relation Classification from Noisy Data
• Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
• Privacy Preserving Distributed Machine Learning with Federated Learning
• Federated Learning Based on Dynamic Regularization
• Learning with Noise: Enhance Distantly Supervised Relation Extraction with Dynamic Transition Matrix
• Federated Learning with Only Positive Labels
• Federated pretraining and fine tuning of BERT using clinical notes from multiple silos