699 citations · 1.6k across the 10 of their papers we have counts for
10 papers
A Study of Gradient Variance in Deep Learning
Fartash Faghri, David Duvenaud, David J. Fleet +1
The impact of gradient noise on training deep models is widely acknowledged but not well understood. In this context, we study the distribution of gradients during training. We int…
The Scattering Compositional Learner: Discovering Objects, Attributes, Relationships in Analogical Reasoning
Yuhuai Wu, Honghua Dong, Roger Grosse +1
In this work, we focus on an analogical reasoning task that contains rich compositional structures, Raven's Progressive Matrices (RPM). To discover compositional structures of the…
Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learning
Silviu Pitis, Harris Chan, Stephen Zhao +2
What goals should a multi-goal reinforcement learning agent pursue during training in long-horizon tasks? When the desired (test time) goal distribution is too distant to offer a u…
BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning
Yeming Wen, Dustin Tran, Jimmy Ba
Ensembles, where multiple neural networks are trained individually and their predictions are averaged, have been shown to be widely successful for improving both the accuracy and p…
On Solving Minimax Optimization Locally: A Follow-the-Ridge Approach
Yuanhao Wang, Guodong Zhang, Jimmy Ba
Many tasks in modern machine learning can be formulated as finding equilibria in \emph{sequential} games. In particular, two-player zero-sum sequential games, also known as minimax…
DOM-Q-NET: Grounded RL on Structured Language
Sheng Jia, Jamie Kiros, Jimmy Ba
Building agents to interact with the web would allow for significant improvements in knowledge understanding and representation learning. However, web navigation tasks are difficul…