11 citations · 14 across the 5 of their papers we have counts for
6 papers · 1 filter
ChapterBreak: A Challenge Dataset for Long-Range Language Models
Simeng Sun, Katherine Thai, Mohit Iyyer
While numerous architectures for long-range language models (LRLMs) have recently been proposed, a meaningful evaluation of their discourse-level language understanding capabilitie…
Alternative Input Signals Ease Transfer in Multilingual Machine Translation
Simeng Sun, Angela Fan, James Cross +4
Recent work in multilingual machine translation (MMT) has focused on the potential of positive transfer between languages, particularly cases where higher-resourced languages can b…
Do Long-Range Language Models Actually Use Long-Range Context?
Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke +1
Language models are generally trained on short, truncated input sequences, which limits their ability to use discourse-level information present in long-range context to improve th…
Revisiting Simple Neural Probabilistic Language Models
Simeng Sun, Mohit Iyyer
Recent progress in language modeling has been driven not only by advances in neural architectures, but also through hardware and optimization improvements. In this paper, we revisi…
IGA : An Intent-Guided Authoring Assistant
Simeng Sun, Wenlong Zhao, Varun Manjunatha +5
While large-scale pretrained language models have significantly improved writing assistance functionalities such as autocomplete, more complex and controllable writing assistants h…
Hard-Coded Gaussian Attention for Neural Machine Translation
Weiqiu You, Simeng Sun, Mohit Iyyer
Recent work has questioned the importance of the Transformer's multi-headed attention for achieving high translation quality. We push further in this direction by developing a "har…