297 citations · 716 across the 31 of their papers we have counts for
Showing 2019Show all
3 papers · 1 filter
cs.LG2019★ 106 cited
Root Mean Square Layer Normalization
Biao Zhang, Rico Sennrich
Layer normalization (LayerNorm) has been successfully applied to various deep neural networks to help stabilize training and boost model convergence because of its capability in ha…
cs.CL2019★ 5 cited
Improving Deep Transformer with Depth-Scaled Initialization and Merged Attention
Biao Zhang, Ivan Titov, Rico Sennrich
The general trend in NLP is towards increasing model capacity and performance via deeper neural networks. However, simply stacking more layers of the popular Transformer architectu…
cs.CL2019★ 1 cited
A Lightweight Recurrent Network for Sequence Modeling
Biao Zhang, Rico Sennrich
Recurrent networks have achieved great success on various sequential tasks with the assistance of complex recurrent units, but suffer from severe computational inefficiency due to…