10.8k citations · 17.5k across the 28 of their papers we have counts for
18 papers · 1 filter
Post-Workshop Report on Science meets Engineering in Deep Learning, NeurIPS 2019, Vancouver
Levent Sagun, Caglar Gulcehre, Adriana Romero +2
Science meets Engineering in Deep Learning took place in Vancouver as part of the Workshop section of NeurIPS 2019. As organizers of the workshop, we created the following report i…
Stabilizing Transformers for Reinforcement Learning
Emilio Parisotto, H. Francis Song, Jack W. Rae +10
Owing to their ability to both effectively integrate information over long time horizons and scale to massive amounts of data, self-attention architectures have recently shown brea…
Making Efficient Use of Demonstrations to Solve Hard Exploration Problems
Tom Le Paine, Caglar Gulcehre, Bobak Shahriari +11
This paper introduces R2D3, an agent that makes efficient use of demonstrations to solve hard exploration problems in partially observable environments with highly variable initial…
Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
Natasha Jaques, Angeliki Lazaridou, Edward Hughes +5
We propose a unified mechanism for achieving coordination and communication in Multi-Agent Reinforcement Learning (MARL), through rewarding agents for having causal influence over…
Sample Efficient Adaptive Text-to-Speech
Yutian Chen, Yannis Assael, Brendan Shillingford +11
We present a meta-learning approach for adaptive text-to-speech (TTS) with few data. During training, we learn a multi-speaker model using a shared conditional WaveNet core and ind…
Relational inductive biases, deep learning, and graph networks
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst +24
Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, i…
Plan, Attend, Generate: Planning for Sequence-to-Sequence Models
Francis Dutil, Caglar Gulcehre, Adam Trischler +1
We investigate the integration of a planning mechanism into sequence-to-sequence models using attention. We develop a model which can plan ahead in the future when it computes its…
Gated Orthogonal Recurrent Units: On Learning to Forget
Li Jing, Caglar Gulcehre, John Peurifoy +4
We present a novel recurrent neural network (RNN) based model that combines the remembering ability of unitary RNNs with the ability of gated RNNs to effectively forget redundant/i…
A Robust Adaptive Stochastic Gradient Method for Deep Learning
Caglar Gulcehre, Jose Sotelo, Marcin Moczulski +1
Stochastic gradient algorithms are the main focus of large-scale optimization problems and led to important successes in the recent advancement of the deep learning algorithms. The…
Memory Augmented Neural Networks with Wormhole Connections
Caglar Gulcehre, Sarath Chandar, Yoshua Bengio
Recent empirical results on long-term dependency tasks have shown that neural networks augmented with an external memory can learn the long-term dependency tasks more easily and ac…
Mollifying Networks
Caglar Gulcehre, Marcin Moczulski, Francesco Visin +1
The optimization of deep neural networks can be more challenging than traditional convex optimization problems due to the highly non-convex nature of the loss function, e.g. it can…
Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes
Caglar Gulcehre, Sarath Chandar, Kyunghyun Cho +1
We extend neural Turing machine (NTM) model into a dynamic neural Turing machine (D-NTM) by introducing a trainable memory addressing scheme. This addressing scheme maintains for e…
Noisy Activation Functions
Caglar Gulcehre, Marcin Moczulski, Misha Denil +1
Common nonlinear activation functions used in neural networks can cause training difficulties due to the saturation behavior of the activation function, which may hide dependencies…
Policy Distillation
Andrei A. Rusu, Sergio Gomez Colmenarejo, Caglar Gulcehre +6
Policies for complex visual tasks have been successfully learned with deep reinforcement learning, using an approach called deep Q-networks (DQN), but relatively large (task-specif…
EmoNets: Multimodal deep learning approaches for emotion recognition in video
Samira Ebrahimi Kahou, Xavier Bouthillier, Pascal Lamblin +15
The task of the emotion recognition in the wild (EmotiW) Challenge is to assign one of seven emotions to short video clips extracted from Hollywood style movies. The videos depict…
ADASECANT: Robust Adaptive Secant Method for Stochastic Gradient
Caglar Gulcehre, Marcin Moczulski, Yoshua Bengio
Stochastic gradient algorithms have been the main focus of large-scale learning problems and they led to important successes in machine learning. The convergence of SGD depends on…
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Yann Dauphin, Razvan Pascanu, Caglar Gulcehre +3
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-New…
Knowledge Matters: Importance of Prior Information for Optimization
Çağlar Gülçehre, Yoshua Bengio
We explore the effect of introducing prior information into the intermediate level of neural networks for a learning task on which all the state-of-the-art machine learning algorit…