Strategies for Language Identification in Code-Mixed Low Resource Languages

Abstract

In recent years, substantial work has been done on language tagging of code-mixed data, but most of them use large amounts of data to build their models. In this article, we present three strategies to build a word level language tagger for code-mixed data using very low resources. Each of them secured an accuracy higher than our baseline model, and the best performing system got an accuracy around 91%. Combining all, the ensemble system achieved an accuracy of around 92.6%.

arXiv commentInternational Conference on Natural Language Processing (ICON 18) - Student Paper Competition, Patiala, India

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Matching Networks for One Shot Learning
• One-shot Learning with Memory-Augmented Neural Networks
• Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding
• L2 Regularization versus Batch and Weight Normalization
• Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
• Sentiment Analysis of Code-Mixed Indian Languages: An Overview of SAIL_Code-Mixed Shared Task @ICON-2017
• How Transferable are Neural Networks in NLP Applications?
• Preparing Bengali-English Code-Mixed Corpus for Sentiment Analysis of Indian Languages
• Language Identification of Bengali-English Code-Mixed data using Character & Phonetic based LSTM Models