An Analysis of Hierarchical Text Classification Using Word Embeddings

Abstract

Efficient distributed numerical word representation models (word embeddings) combined with modern machine learning algorithms have recently yielded considerable improvement on automatic document classification tasks. However, the effectiveness of such techniques has not been assessed for the hierarchical text classification (HTC) yet. This study investigates the application of those models and algorithms on this specific problem by means of experimentation and analysis. We trained classification models with prominent machine learning algorithm implementations---fastText, XGBoost, SVM, and Keras' CNN---and noticeable word embeddings generation methods---GloVe, word2vec, and fastText---with publicly available data and evaluated them with measures specifically appropriate for the hierarchical context. FastText achieved an ${}_{LCA}{F}_1$ of 0.893 on a single-labeled version of the RCV1 dataset. An analysis indicates that using word embeddings and its flavors is a very promising approach for HTC.

Article accepted for publication in Information Sciences on Sep 1st, 2018

References in corpus (12)

• Efficient Estimation of Word Representations in Vector Space
• Deep Learning in Neural Networks: An Overview
• Machine Learning in Automated Text Categorization
• LSTM: A Search Space Odyssey
• Natural Language Processing (almost) from Scratch
• Distributed Representations of Sentences and Documents
• Character-level Convolutional Networks for Text Classification
• Convolutional Neural Networks for Sentence Classification
• Evaluation Measures for Hierarchical Classification: a unified view and novel approaches
• LSHTC: A Benchmark for Large-Scale Text Classification
• Kaggle LSHTC4 Winning Solution
• An empirical study on large scale text classification with skip-gram embeddings

Cited by in corpus (8)

• A new approach for query expansion using Wikipedia and WordNet
• Leveraging Hierarchical Structures for Few-Shot Musical Instrument Recognition
• Neural Language Models with Distant Supervision to Identify Major Depressive Disorder from Clinical Notes
• A Precisely Xtreme-Multi Channel Hybrid Approach For Roman Urdu Sentiment Analysis
• On the Information Content of Predictions in Word Analogy Tests
• Text Classification for Predicting Multi-level Product Categories
• Hybrid Improved Document-level Embedding (HIDE)
• Joint Embedding of Words and Category Labels for Hierarchical Multi-label Text Classification