19 citations · 25 across the 3 of their papers we have counts for
3 papers
Untangling tradeoffs between recurrence and self-attention in neural networks
Giancarlo Kerg, Bhargav Kanuparthi, Anirudh Goyal +3
Attention and self-attention mechanisms, are now central to state-of-the-art deep learning on sequential tasks. However, most recent progress hinges on heuristic approaches with li…
Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics
Giancarlo Kerg, Kyle Goyette, Maximilian Puelma Touzel +4
A recent strategy to circumvent the exploding and vanishing gradient problem in RNNs, and to allow the stable propagation of signals over long time scales, is to constrain recurren…
Deep Pepper: Expert Iteration based Chess agent in the Reinforcement Learning Setting
Sai Krishna G. V., Kyle Goyette, Ahmad Chamseddine +1
An almost-perfect chess playing agent has been a long standing challenge in the field of Artificial Intelligence. Some of the recent advances demonstrate we are approaching that go…