How Transferable are Neural Networks in NLP Applications?

Abstract

Transfer learning is aimed to make use of valuable knowledge in a source domain to help model performance in a target domain. It is particularly important to neural networks, which are very likely to be overfitting. In some fields like image processing, many studies have shown the effectiveness of neural network-based transfer learning. For neural NLP, however, existing studies have only casually applied transfer learning, and conclusions are inconsistent. In this paper, we conduct systematic case studies and provide an illuminating picture on the transferability of neural networks in NLP.

Accepted by EMNLP-16

References in corpus (5)

• Frustratingly Easy Domain Adaptation
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• Discriminative Neural Sentence Modeling by Tree-Based Convolution
• Natural Language Inference by Tree-Based Convolution and Heuristic Matching
• Transfer Learning for Speech and Language Processing

Cited by in corpus (35)

• Deep learning for affective computing: text-based emotion recognition in decision support
• Clinical Concept Extraction: a Methodology Review
• Transfer Learning for Named-Entity Recognition with Neural Networks
• Exploiting Document Knowledge for Aspect-level Sentiment Classification
• Question Answering through Transfer Learning from Large Fine-grained Supervision Data
• On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation
• Identifying beneficial task relations for multi-task learning in deep neural networks
• Supervised and Unsupervised Transfer Learning for Question Answering
• Transfer Learning for Context-Aware Question Matching in Information-seeking Conversations in E-commerce
• Exploring and Predicting Transferability across NLP Tasks
• Multimodal deep learning for short-term stock volatility prediction
• Multi-task Learning for Universal Sentence Embeddings: A Thorough Evaluation using Transfer and Auxiliary Tasks
• Temporal Embeddings and Transformer Models for Narrative Text Understanding
• Review Helpfulness Prediction with Embedding-Gated CNN
• Joint Multi-Domain Learning for Automatic Short Answer Grading
• Improving Robustness and Generality of NLP Models Using Disentangled Representations
• Discovery of Natural Language Concepts in Individual Units of CNNs
• Neural Adaptation Layers for Cross-domain Named Entity Recognition
• Direct Network Transfer: Transfer Learning of Sentence Embeddings for Semantic Similarity
• Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains
• Transfer learning from language models to image caption generators: Better models may not transfer better
• Neural Task Representations as Weak Supervision for Model Agnostic Cross-Lingual Transfer
• Learning Robust, Transferable Sentence Representations for Text Classification
• Parameter Transfer Unit for Deep Neural Networks
• EasyTransfer -- A Simple and Scalable Deep Transfer Learning Platform for NLP Applications
• Domain Adaptation for Neural Networks by Parameter Augmentation
• A Hierarchical Deep Learning Natural Language Parser for Fashion
• Dropping Networks for Transfer Learning
• On Generality and Knowledge Transferability in Cross-Domain Duplicate Question Detection for Heterogeneous Community Question Answering
• Transfer and Multi-Task Learning for Noun-Noun Compound Interpretation
• Generating Contradictory, Neutral, and Entailing Sentences
• Strategies for Language Identification in Code-Mixed Low Resource Languages
• Transferable End-to-End Aspect-based Sentiment Analysis with Selective Adversarial Learning
• SNU_IDS at SemEval-2018 Task 12: Sentence Encoder with Contextualized Vectors for Argument Reasoning Comprehension
• SufiSent - Universal Sentence Representations Using Suffix Encodings