11 citations · 11 across the 2 of their papers we have counts for
2 papers
cs.CL2021
Revisiting Simple Neural Probabilistic Language Models
Simeng Sun, Mohit Iyyer
Recent progress in language modeling has been driven not only by advances in neural architectures, but also through hardware and optimization improvements. In this paper, we revisi…
cs.CL2020★ 11 cited
Hard-Coded Gaussian Attention for Neural Machine Translation
Weiqiu You, Simeng Sun, Mohit Iyyer
Recent work has questioned the importance of the Transformer's multi-headed attention for achieving high translation quality. We push further in this direction by developing a "har…