Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function

Abstract

In this paper, we study bidirectional LSTM network for the task of text classification using both supervised and semi-supervised approaches. Several prior works have suggested that either complex pretraining schemes using unsupervised methods such as language modeling (Dai and Le 2015; Miyato, Dai, and Goodfellow 2016) or complicated models (Johnson and Zhang 2017) are necessary to achieve a high classification accuracy. However, we develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results compared with more complex approaches. Furthermore, in addition to cross-entropy loss, by using a combination of entropy minimization, adversarial, and virtual adversarial losses for both labeled and unlabeled data, we report state-of-the-art results for text classification task on several benchmark datasets. In particular, on the ACL-IMDB sentiment analysis and AG-News topic classification datasets, our method outperforms current approaches by a substantial margin. We also show the generality of the mixed objective function by improving the performance on relation extraction task.

Published at AAAI 2019

References in corpus (8)

• Distributed Representations of Words and Phrases and their Compositionality
• Explaining and Harnessing Adversarial Examples
• Distributed Representations of Sentences and Documents
• Character-level Convolutional Networks for Text Classification
• Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling
• Adversarial Training Methods for Semi-Supervised Text Classification
• Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
• Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings

Cited by in corpus (4)

• Deep Learning-based Sentiment Classification: A Comparative Survey
• AllSpark: A Multimodal Spatio-Temporal General Intelligence Model with Ten Modalities via Language as a Reference Framework
• Explainable Natural Language Processing with Matrix Product States
• VirAAL: Virtual Adversarial Active Learning For NLU