Improving Neural Language Models by Segmenting, Attending, and Predicting the Future

Hongyin Luo, Lan Jiang, Yonatan Belinkov +1

Common language models typically predict the next word given the context. In this work, we propose a method that improves language modeling by learning to align the given context a…