189 citations · 300 across the 16 of their papers we have counts for
Showing 2019Show all
2 papers · 1 filter
cs.CL2019★ 42 cited
MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning
Guangxiang Zhao, Xu Sun, Jingjing Xu +2
In sequence to sequence learning, the self-attention mechanism proves to be highly effective, and achieves significant improvements in many tasks. However, the self-attention mecha…
cs.LG2019★ 189 cited
Adaptive Gradient Methods with Dynamic Bound of Learning Rate
Liangchen Luo, Yuanhao Xiong, Yan Liu +1
Adaptive optimization methods such as AdaGrad, RMSprop and Adam have been proposed to achieve a rapid training process with an element-wise scaling term on learning rates. Though p…