Deriving Machine Attention from Human Rationales
arXiv:1808.09367
Abstract
Attention-based models are successful when trained on large amounts of data. In this paper, we demonstrate that even in the low-resource scenario, attention can be learned effectively. To this end, we start with discrete human-annotated rationales and map them into continuous attention. Our central hypothesis is that this mapping is general across domains, and thus can be transferred from resource-rich domains to low-resource ones. Our model jointly learns a domain-invariant representation and induces the desired mapping between rationales and attention. Our empirical results validate this hypothesis and show that our approach delivers significant gains over state-of-the-art baselines, yielding over 15% average error reduction on benchmark datasets.
arXiv commentEMNLP 2018
References in corpus (6)
- Adam: A Method for Stochastic Optimization
- Categorical Reparameterization with Gumbel-Softmax
- Improved Training of Wasserstein GANs
- Convolutional Neural Networks for Sentence Classification
- Effective Approaches to Attention-based Neural Machine Translation
- Neural Machine Translation with Supervised Attention
Cited by in corpus (4)
- Evaluating Explanation Without Ground Truth in Interpretable Machine Learning
- Interpretations are useful: penalizing explanations to align neural networks with prior knowledge
- Using Machine Learning and Natural Language Processing to Review and Classify the Medical Literature on Cancer Susceptibility Genes
- Text Classification with Lexicon from PreAttention Mechanism