Privacy enabled Financial Text Classification using Differential Privacy and Federated Learning

Abstract

Privacy is important considering the financial Domain as such data is highly confidential and sensitive. Natural Language Processing (NLP) techniques can be applied for text classification and entity detection purposes in financial domains such as customer feedback sentiment analysis, invoice entity detection, categorisation of financial documents by type etc. Due to the sensitive nature of such data, privacy measures need to be taken for handling and training large models with such data. In this work, we propose a contextualized transformer (BERT and RoBERTa) based text classification model integrated with privacy features such as Differential Privacy (DP) and Federated Learning (FL). We present how to privately train NLP models and desirable privacy-utility tradeoffs and evaluate them on the Financial Phrase Bank dataset.

arXiv comment4 pages. Accepted at ECONLP-EMNLP'21

References in corpus (12)

• Deep Learning with Differential Privacy
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Learning Differentially Private Recurrent Language Models
• Federated Machine Learning: Concept and Applications
• Applied Federated Learning: Improving Google Keyboard Query Suggestions
• FinBERT: Financial Sentiment Analysis with Pre-trained Language Models
• A Survey on Transfer Learning in Natural Language Processing
• Federated Learning Meets Natural Language Processing: A Survey
• A survey on natural language processing (nlp) and applications in insurance
• Benchmarking Differential Privacy and Federated Learning for BERT Models
• FedPandemic: A Cross-Device Federated Learning Approach Towards Elementary Prognosis of Diseases During a Pandemic
• Differential Privacy and Natural Language Processing to Generate Contextually Similar Decoy Messages in Honey Encryption Scheme