Preliminary Steps Towards Federated Sentiment Classification

Abstract

Automatically mining sentiment tendency contained in natural language is a fundamental research to some artificial intelligent applications, where solutions alternate with challenges. Transfer learning and multi-task learning techniques have been leveraged to mitigate the supervision sparsity and collaborate multiple heterogeneous domains correspondingly. Recent years, the sensitive nature of users' private data raises another challenge for sentiment classification, i.e., data privacy protection. In this paper, we resort to federated learning for multiple domain sentiment classification under the constraint that the corpora must be stored on decentralized devices. In view of the heterogeneous semantics across multiple parties and the peculiarities of word embedding, we pertinently provide corresponding solutions. First, we propose a Knowledge Transfer Enhanced Private-Shared (KTEPS) framework for better model aggregation and personalization in federated sentiment classification. Second, we propose KTEPS${}^{\star }$ with the consideration of the rich semantic and huge embedding size properties of word vectors, utilizing Projection-based Dimension Reduction (PDR) methods for privacy protection and efficient transmission simultaneously. We propose two federated sentiment classification scenes based on public benchmarks, and verify the superiorities of our proposed methods with abundant experimental investigations.

References in corpus (14)

• Efficient Estimation of Word Representations in Vector Space
• Distilling the Knowledge in a Neural Network
• Federated Learning with Non-IID Data
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• Recurrent Neural Network for Text Classification with Multi-Task Learning
• Domain Separation Networks
• Federated Learning with Personalization Layers
• Differentially Private Federated Learning: A Client Level Perspective
• Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
• Federated Adversarial Domain Adaptation
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• Federated Learning Of Out-Of-Vocabulary Words
• Private Federated Learning with Domain Adaptation
• Federated User Representation Learning