146 citations · 895 across the 23 of their papers we have counts for
3 papers · 1 filter
Trainability, Expressivity and Interpretability in Gated Neural ODEs
Timothy Doyeon Kim, Tankut Can, Kamesh Krishnamurthy
Understanding how the dynamics in biological and artificial neural networks implement the computations required for a task is a salient open question in machine learning and neuros…
Flatter, faster: scaling momentum for optimal speedup of SGD
Aditya Cowsik, Tankut Can, Paolo Glorioso
Commonly used optimization algorithms often show a trade-off between good generalization and fast training times. For instance, stochastic gradient descent (SGD) tends to have good…
Gating creates slow modes and controls phase-space complexity in GRUs and LSTMs
Tankut Can, Kamesh Krishnamurthy, David J. Schwab
Recurrent neural networks (RNNs) are powerful dynamical models for data with complex temporal structure. However, training RNNs has traditionally proved challenging due to explodin…