Text Understanding from Scratch

Abstract

This article demontrates that we can apply deep learning to text understanding from character-level inputs all the way up to abstract text concepts, using temporal convolutional networks (ConvNets). We apply ConvNets to various large-scale datasets, including ontology classification, sentiment analysis, and text categorization. We show that temporal ConvNets can achieve astonishing performance without the knowledge of words, phrases, sentences and any other syntactic or semantic structures with regards to a human language. Evidence shows that our models can work for both English and Chinese.

This technical report is superseded by a paper entitled "Character-level Convolutional Networks for Text Classification", arXiv:1509.01626. It has considerably more experimental results and a rewritten introduction

References in corpus (6)

• Improving neural networks by preventing co-adaptation of feature detectors
• Natural Language Processing (almost) from Scratch
• Distributed Representations of Sentences and Documents
• Deep Fragment Embeddings for Bidirectional Image Sentence Mapping
• Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
• Show and Tell: A Neural Image Caption Generator

Cited by in corpus (75)

• Opening the Black Box of Deep Neural Networks via Information
• A Survey on Data Augmentation for Text Classification
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• Contextual LSTM (CLSTM) models for Large scale NLP tasks
• Weakly-Supervised Neural Text Classification
• Long Text Generation via Adversarial Training with Leaked Information
• Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks
• Tweet2Vec: Learning Tweet Embeddings Using Character-level CNN-LSTM Encoder-Decoder
• UniParser: A Unified Log Parser for Heterogeneous Log Data
• A Bayesian Data Augmentation Approach for Learning Deep Models
• Learning Semantic Similarity for Very Short Texts
• StarSpace: Embed All The Things!
• Selection via Proxy: Efficient Data Selection for Deep Learning
• Character-Level Question Answering with Attention
• Efficient Vector Representation for Documents through Corruption
• An Online Multilingual Hate speech Recognition System
• Well-calibrated Confidence Measures for Multi-label Text Classification with a Large Number of Labels
• Omnivore: An Optimizer for Multi-device Deep Learning on CPUs and GPUs
• Word Embedding Perturbation for Sentence Classification
• Deep Weakly-supervised Anomaly Detection
• Generative Concatenative Nets Jointly Learn to Write and Classify Reviews
• Shallow reading with Deep Learning: Predicting popularity of online content using only its title
• Automatic Normalization of Word Variations in Code-Mixed Social Media Text
• Deepr: A Convolutional Net for Medical Records
• Unsupervised Cipher Cracking Using Discrete GANs
• Multilingual Language Processing From Bytes
• The SP theory of intelligence: distinctive features and advantages
• Convolutional Neural Network: Text Classification Model for Open Domain Question Answering System
• Analysis and Optimization of fastText Linear Text Classifier
• An Attentive Sequence Model for Adverse Drug Event Extraction from Biomedical Text
• Integrating Semantic Knowledge to Tackle Zero-shot Text Classification
• Combination of Domain Knowledge and Deep Learning for Sentiment Analysis
• ProtoAttend: Attention-Based Prototypical Learning
• Neural Personalized Response Generation as Domain Adaptation
• NewsEmbed: Modeling News through Pre-trained Document Representations
• Towards Sub-Word Level Compositions for Sentiment Analysis of Hindi-English Code Mixed Text
• EDUCE: Explaining model Decisions through Unsupervised Concepts Extraction
• Discovering New Intents via Constrained Deep Adaptive Clustering with Cluster Refinement
• Semi-supervised Clustering for Short Text via Deep Representation Learning
• Automatic Emotion Recognition (AER) System based on Two-Level Ensemble of Lightweight Deep CNN Models
• Filter Grafting for Deep Neural Networks
• Supporting Clustering with Contrastive Learning
• Combination of Domain Knowledge and Deep Learning for Sentiment Analysis of Short and Informal Messages on Social Media
• SocialInteractionGAN: Multi-person Interaction Sequence Generation
• SparCE: Sparsity aware General Purpose Core Extensions to Accelerate Deep Neural Networks
• Deep Semantic Architecture with discriminative feature visualization for neuroimage analysis
• CORE: Automating Review Recommendation for Code Changes
• Evolving Character-level Convolutional Neural Networks for Text Classification
• DeepLearningKit - an GPU Optimized Deep Learning Framework for Apple's iOS, OS X and tvOS developed in Metal and Swift
• Textual Membership Queries
• Predicting user intent from search queries using both CNNs and RNNs
• Pruning Filter in Filter
• Super Characters: A Conversion from Sentiment Classification to Image Classification
• Neural Random Projections for Language Modelling
• A Deep Neural Architecture for Sentence-level Sentiment Classification in Twitter Social Networking
• INFODENS: An Open-source Framework for Learning Text Representations
• AMSI-Based Detection of Malicious PowerShell Code Using Contextual Embeddings
• Learning to Fuse Music Genres with Generative Adversarial Dual Learning
• CEIL: A General Classification-Enhanced Iterative Learning Framework for Text Clustering
• Sequence to sequence learning for unconstrained scene text recognition
• Enhancing Semantic Word Representations by Embedding Deeper Word Relationships
• Infusing Collaborative Recommenders with Distributed Representations
• Data-Driven Regular Expressions Evolution for Medical Text Classification Using Genetic Programming
• TF-CR: Weighting Embeddings for Text Classification
• The Role of the Input in Natural Language Video Description
• Learning Interpretable and Discrete Representations with Adversarial Training for Unsupervised Text Classification
• Pairwise Supervised Contrastive Learning of Sentence Representations
• IoT Network Behavioral Fingerprint Inference with Limited Network Trace for Cyber Investigation: A Meta Learning Approach
• Filter Grafting for Deep Neural Networks: Reason, Method, and Cultivation
• Hamming Sentence Embeddings for Information Retrieval
• Making Sense of Hidden Layer Information in Deep Networks by Learning Hierarchical Targets
• Would you Like to Talk about Sports Now? Towards Contextual Topic Suggestion for Open-Domain Conversational Agents
• The RLLChatbot: a solution to the ConvAI challenge
• Melody-Conditioned Lyrics Generation with SeqGANs