133 citations · 214 across the 14 of their papers we have counts for
4 papers · 1 filter
Neuron Interaction Based Representation Composition for Neural Machine Translation
Jian Li, Xing Wang, Baosong Yang +3
Recent NLP studies reveal that substantial linguistic information can be attributed to single neurons, i.e., individual dimensions of the representation vectors. We hypothesize tha…
Information Aggregation for Multi-Head Attention with Routing-by-Agreement
Jian Li, Baosong Yang, Zi-Yi Dou +3
Multi-head attention is appealing for its ability to jointly extract different types of information from multiple representation subspaces. Concerning the information aggregation,…
Context-Aware Self-Attention Networks
Baosong Yang, Jian Li, Derek Wong +3
Self-attention model have shown its flexibility in parallel computation and the effectiveness on modeling both long- and short-term dependencies. However, it calculates the depende…
Multi-Head Attention with Disagreement Regularization
Jian Li, Zhaopeng Tu, Baosong Yang +2
Multi-head attention is appealing for the ability to jointly attend to information from different representation subspaces at different positions. In this work, we introduce a disa…