674 citations · 2.2k across the 8 of their papers we have counts for
8 papers
Single Headed Attention RNN: Stop Thinking With Your Head
Stephen Merity
The leading approaches in language modeling are all obsessed with TV shows of my youth - namely Transformers and Sesame Street. Transformers this, Transformers that, and over here…
An Analysis of Neural Language Modeling at Multiple Scales
Stephen Merity, Nitish Shirish Keskar, Richard Socher
Many of the leading approaches in language modeling introduce novel, complex and specialized architectures. We take existing state-of-the-art word level language models based on LS…
A Flexible Approach to Automated RNN Architecture Generation
Martin Schrimpf, Stephen Merity, James Bradbury +1
The process of designing neural architectures requires expert knowledge and extensive trial and error. While automated architecture search may simplify these requirements, the recu…
Regularizing and Optimizing LSTM Language Models
Stephen Merity, Nitish Shirish Keskar, Richard Socher
Recurrent neural networks (RNNs), such as long short-term memory networks (LSTMs), serve as a fundamental building block for many sequence learning tasks, including machine transla…
Revisiting Activation Regularization for Language RNNs
Stephen Merity, Bryan McCann, Richard Socher
Recurrent neural networks (RNNs) serve as a fundamental building block for many sequence tasks across natural language processing. Recent research has focused on recurrent dropout…
Quasi-Recurrent Neural Networks
James Bradbury, Stephen Merity, Caiming Xiong +1
Recurrent neural networks are a powerful tool for modeling sequential data, but the dependence of each timestep's computation on the previous timestep's output limits parallelism a…