Deep Learning for Hindi Text Classification: A Comparison

Abstract

Natural Language Processing (NLP) and especially natural language text analysis have seen great advances in recent times. Usage of deep learning in text processing has revolutionized the techniques for text processing and achieved remarkable results. Different deep learning architectures like CNN, LSTM, and very recent Transformer have been used to achieve state of the art results variety on NLP tasks. In this work, we survey a host of deep learning architectures for text classification tasks. The work is specifically concerned with the classification of Hindi text. The research in the classification of morphologically rich and low resource Hindi language written in Devanagari script has been limited due to the absence of large labeled corpus. In this work, we used translated versions of English data-sets to evaluate models based on CNN, LSTM and Attention. Multilingual pre-trained sentence embeddings based on BERT and LASER are also compared to evaluate their effectiveness for the Hindi language. The paper also serves as a tutorial for popular text classification techniques.

arXiv commentAccepted at International Conference on Intelligent Human Computer Interaction(IHCI) 2019

Cited by in corpus (8)

• Comparative Study of Long Document Classification
• Mono vs Multilingual BERT for Hate Speech Detection and Text Classification: A Case Study in Marathi
• Experimental Evaluation of Deep Learning models for Marathi Text Classification
• Evaluation of Deep Learning Models for Hostility Detection in Hindi Text
• Mono vs Multilingual BERT: A Case Study in Hindi and Marathi Named Entity Recognition
• On Sensitivity of Deep Learning Based Text Classification Algorithms to Practical Input Perturbations
• ShufText: A Simple Black Box Approach to Evaluate the Fragility of Text Classification Models
• Universal Cross-Lingual Text Classification