Neural Text Classification by Jointly Learning to Cluster and Align

Abstract

Distributional text clustering delivers semantically informative representations and captures the relevance between each word and semantic clustering centroids. We extend the neural text clustering approach to text classification tasks by inducing cluster centers via a latent variable model and interacting with distributional word embeddings, to enrich the representation of tokens and measure the relatedness between tokens and each learnable cluster centroid. The proposed method jointly learns word clustering centroids and clustering-token alignments, achieving the state of the art results on multiple benchmark datasets and proving that the proposed cluster-token alignment mechanism is indeed favorable to text classification. Notably, our qualitative analysis has conspicuously illustrated that text representations learned by the proposed model are in accord well with our intuition.

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Distributed Representations of Words and Phrases and their Compositionality
• A Structured Self-attentive Sentence Embedding
• Convolutional Neural Networks for Sentence Classification
• Recurrent Neural Network for Text Classification with Multi-Task Learning
• Deep Learning of Nonnegativity-Constrained Autoencoders for Enhanced Understanding of Data
• Topic Memory Networks for Short Text Classification

Cited by in corpus (1)

• Entropy optimized semi-supervised decomposed vector-quantized variational autoencoder model based on transfer learning for multiclass text classification and generation