BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis

Abstract

Question-answering plays an important role in e-commerce as it allows potential customers to actively seek crucial information about products or services to help their purchase decision making. Inspired by the recent success of machine reading comprehension (MRC) on formal documents, this paper explores the potential of turning customer reviews into a large source of knowledge that can be exploited to answer user questions.~We call this problem Review Reading Comprehension (RRC). To the best of our knowledge, no existing work has been done on RRC. In this work, we first build an RRC dataset called ReviewRC based on a popular benchmark for aspect-based sentiment analysis. Since ReviewRC has limited training examples for RRC (and also for aspect-based sentiment analysis), we then explore a novel post-training approach on the popular language model BERT to enhance the performance of fine-tuning of BERT for RRC. To show the generality of the approach, the proposed post-training is also applied to some other review-based tasks such as aspect extraction and aspect sentiment classification in aspect-based sentiment analysis. Experimental results demonstrate that the proposed post-training is highly effective. The datasets and code are available at https://www.cs.uic.edu/~hxu/.

accepted by NAACL 2019

References in corpus (16)

• Distributed Representations of Words and Phrases and their Compositionality
• Teaching Machines to Read and Comprehend
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• RACE: Large-scale ReAding Comprehension Dataset From Examinations
• Reading Wikipedia to Answer Open-Domain Questions
• DRCD: a Chinese Machine Reading Comprehension Dataset
• Aspect Level Sentiment Classification with Deep Memory Network
• NewsQA: A Machine Comprehension Dataset
• Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis
• Neural Generative Question Answering
• Exploiting Document Knowledge for Aspect-level Sentiment Classification
• Exploiting Coarse-to-Fine Task Transfer for Aspect-level Sentiment Classification
• WikiReading: A Novel Large-scale Language Understanding Task over Wikipedia

Cited by in corpus (19)

• Pre-trained Models for Natural Language Processing: A Survey
• Natural Language Processing Advancements By Deep Learning: A Survey
• A Survey on Aspect-Based Sentiment Classification
• Federated Continual Learning via Knowledge Fusion: A Survey
• Domain Adaptive Fake News Detection via Reinforcement Learning
• Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis
• Utilizing BERT Intermediate Layers for Aspect Based Sentiment Analysis and Natural Language Inference
• Arabic aspect sentiment polarity classification using BERT
• Exploiting BERT for End-to-End Aspect-based Sentiment Analysis
• EmotionX-KU: BERT-Max based Contextual Emotion Classifier
• Characterizing the 2022 Russo-Ukrainian Conflict Through the Lenses of Aspect-Based Sentiment Analysis: Dataset, Methodology, and Preliminary Findings
• Improving Sentiment Analysis with Multi-task Learning of Negation
• Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks
• ASAP: A Chinese Review Dataset Towards Aspect Category Sentiment Analysis and Rating Prediction
• Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classification
• Towards Computational Linguistics in Minangkabau Language: Studies on Sentiment Analysis and Machine Translation
• Re-Evaluating GermEval17 Using German Pre-Trained Language Models
• CHIME: Cross-passage Hierarchical Memory Network for Generative Review Question Answering
• Group, Extract and Aggregate: Summarizing a Large Amount of Finance News for Forex Movement Prediction