743 citations · 3k across the 44 of their papers we have counts for
6 papers · 1 filter
DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames
Erik Wijmans, Abhishek Kadian, Ari Morcos +5
We present Decentralized Distributed Proximal Policy Optimization (DD-PPO), a method for distributed reinforcement learning in resource-intensive simulated environments. DD-PPO is…
Luck Matters: Understanding Training Dynamics of Deep ReLU Networks
Yuandong Tian, Tina Jiang, Qucheng Gong +1
We analyze the dynamics of training deep ReLU networks and their implications on generalization capability. Using a teacher-student setting, we discovered a novel relationship betw…
The Generalization-Stability Tradeoff In Neural Network Pruning
Brian R. Bartoldson, Ari S. Morcos, Adrian Barbu +1
Pruning neural network parameters is often viewed as a means to compress models, but pruning has also been motivated by the desire to prevent overfitting. This motivation is partic…
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers
Ari S. Morcos, Haonan Yu, Michela Paganini +1
The success of lottery ticket initializations (Frankle and Carbin, 2019) suggests that small, sparsified networks can be trained so long as the network is initialized appropriately…
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP
Haonan Yu, Sergey Edunov, Yuandong Tian +1
The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) aids training by increasing the probability of a "lucky" sub-network initialization…
Learning to Make Analogies by Contrasting Abstract Relational Structure
Felix Hill, Adam Santoro, David G. T. Barrett +2
Analogical reasoning has been a principal focus of various waves of AI research. Analogy is particularly challenging for machines because it requires relational structures to be re…