231 citations · 759 across the 17 of their papers we have counts for
3 papers · 1 filter
Reverse engineering learned optimizers reveals known and novel mechanisms
Niru Maheswaranathan, David Sussillo, Luke Metz +2
Learned optimizers are algorithms that can themselves be trained to solve optimization problems. In contrast to baseline optimizers (such as momentum or Adam) that use simple updat…
The geometry of integration in text classification RNNs
Kyle Aitken, Vinay V. Ramasesh, Ankush Garg +3
Despite the widespread application of recurrent neural networks (RNNs) across a variety of tasks, a unified understanding of how RNNs solve these tasks remains elusive. In particul…
How recurrent networks implement contextual processing in sentiment analysis
Niru Maheswaranathan, David Sussillo
Neural networks have a remarkable capacity for contextual processing--using recent or nearby inputs to modify processing of current input. For example, in natural language, context…