Do Convolutional Networks need to be Deep for Text Classification ?

Abstract

We study in this work the importance of depth in convolutional models for text classification, either when character or word inputs are considered. We show on 5 standard text classification and sentiment analysis tasks that deep models indeed give better performances than shallow networks when the text input is represented as a sequence of characters. However, a simple shallow-and-wide network outperforms deep models such as DenseNet with word inputs. Our shallow word model further establishes new state-of-the-art performances on two datasets: Yelp Binary (95.9\%) and Yelp Full (64.9\%).

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Twitter mood predicts the stock market
• Natural Language Processing (almost) from Scratch
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Densely Connected Convolutional Networks
• Character-Aware Neural Language Models
• Temporal patterns of happiness and information in a global social network: Hedonometrics and Twitter
• A Convolutional Neural Network for Modelling Sentences
• Learning to Generate Reviews and Discovering Sentiment
• Gated Word-Character Recurrent Language Model

Cited by in corpus (5)

• Argument Identification in Public Comments from eRulemaking
• Did you take the pill? - Detecting Personal Intake of Medicine from Twitter
• #phramacovigilance - Exploring Deep Learning Techniques for Identifying Mentions of Medication Intake from Twitter
• An Explainable CNN Approach for Medical Codes Prediction from Clinical Text
• Search4Code: Code Search Intent Classification Using Weak Supervision