10.8k citations · 17.5k across the 28 of their papers we have counts for
32 papers
Post-Workshop Report on Science meets Engineering in Deep Learning, NeurIPS 2019, Vancouver
Levent Sagun, Caglar Gulcehre, Adriana Romero +2
Science meets Engineering in Deep Learning took place in Vancouver as part of the Workshop section of NeurIPS 2019. As organizers of the workshop, we created the following report i…
Stabilizing Transformers for Reinforcement Learning
Emilio Parisotto, H. Francis Song, Jack W. Rae +10
Owing to their ability to both effectively integrate information over long time horizons and scale to massive amounts of data, self-attention architectures have recently shown brea…
Improving the Gating Mechanism of Recurrent Neural Networks
Albert Gu, Caglar Gulcehre, Tom Le Paine +2
Gating mechanisms are widely used in neural network models, where they allow gradients to backpropagate more easily through depth or time. However, their saturation property introd…
Making Efficient Use of Demonstrations to Solve Hard Exploration Problems
Tom Le Paine, Caglar Gulcehre, Bobak Shahriari +11
This paper introduces R2D3, an agent that makes efficient use of demonstrations to solve hard exploration problems in partially observable environments with highly variable initial…
Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
Natasha Jaques, Angeliki Lazaridou, Edward Hughes +5
We propose a unified mechanism for achieving coordination and communication in Multi-Agent Reinforcement Learning (MARL), through rewarding agents for having causal influence over…
Sample Efficient Adaptive Text-to-Speech
Yutian Chen, Yannis Assael, Brendan Shillingford +11
We present a meta-learning approach for adaptive text-to-speech (TTS) with few data. During training, we learn a multi-speaker model using a shared conditional WaveNet core and ind…
Relational inductive biases, deep learning, and graph networks
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst +24
Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, i…
Hyperbolic Attention Networks
Caglar Gulcehre, Misha Denil, Mateusz Malinowski +8
We introduce hyperbolic attention networks to endow neural networks with enough capacity to match the complexity of data with hierarchical and power-law structure. A few recent app…
Plan, Attend, Generate: Planning for Sequence-to-Sequence Models
Francis Dutil, Caglar Gulcehre, Adam Trischler +1
We investigate the integration of a planning mechanism into sequence-to-sequence models using attention. We develop a model which can plan ahead in the future when it computes its…
Plan, Attend, Generate: Character-level Neural Machine Translation with Planning in the Decoder
Caglar Gulcehre, Francis Dutil, Adam Trischler +1
We investigate the integration of a planning mechanism into an encoder-decoder architecture with an explicit alignment for character-level machine translation. We develop a model t…
Gated Orthogonal Recurrent Units: On Learning to Forget
Li Jing, Caglar Gulcehre, John Peurifoy +4
We present a novel recurrent neural network (RNN) based model that combines the remembering ability of unitary RNNs with the ability of gated RNNs to effectively forget redundant/i…
Machine Comprehension by Text-to-Text Neural Question Generation
Xingdi Yuan, Tong Wang, Caglar Gulcehre +5
We propose a recurrent neural model that generates natural-language questions from documents, conditioned on answers. We show how to train the model using a combination of supervis…
A Robust Adaptive Stochastic Gradient Method for Deep Learning
Caglar Gulcehre, Jose Sotelo, Marcin Moczulski +1
Stochastic gradient algorithms are the main focus of large-scale optimization problems and led to important successes in the recent advancement of the deep learning algorithms. The…
Memory Augmented Neural Networks with Wormhole Connections
Caglar Gulcehre, Sarath Chandar, Yoshua Bengio
Recent empirical results on long-term dependency tasks have shown that neural networks augmented with an external memory can learn the long-term dependency tasks more easily and ac…
Mollifying Networks
Caglar Gulcehre, Marcin Moczulski, Francesco Visin +1
The optimization of deep neural networks can be more challenging than traditional convex optimization problems due to the highly non-convex nature of the loss function, e.g. it can…
Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes
Caglar Gulcehre, Sarath Chandar, Kyunghyun Cho +1
We extend neural Turing machine (NTM) model into a dynamic neural Turing machine (D-NTM) by introducing a trainable memory addressing scheme. This addressing scheme maintains for e…
Theano: A Python framework for fast computation of mathematical expressions
The Theano Development Team, Rami Al-Rfou, Guillaume Alain +110
Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Since its introduction, it has bee…
Noisy Activation Functions
Caglar Gulcehre, Marcin Moczulski, Misha Denil +1
Common nonlinear activation functions used in neural networks can cause training difficulties due to the saturation behavior of the activation function, which may hide dependencies…
Pointing the Unknown Words
Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati +2
The problem of rare and unknown words is an important issue that can potentially influence the performance of many NLP systems, including both the traditional count-based and the d…
Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus
Iulian Vlad Serban, Alberto García-Durán, Caglar Gulcehre +4
Over the past decade, large-scale supervised learning corpora have enabled machine learning researchers to make substantial advances. However, to this date, there are no large-scal…
Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
Ramesh Nallapati, Bowen Zhou, Cicero Nogueira dos santos +2
In this work, we model abstractive text summarization using Attentional Encoder-Decoder Recurrent Neural Networks, and show that they achieve state-of-the-art performance on two di…
Policy Distillation
Andrei A. Rusu, Sergio Gomez Colmenarejo, Caglar Gulcehre +6
Policies for complex visual tasks have been successfully learned with deep reinforcement learning, using an approach called deep Q-networks (DQN), but relatively large (task-specif…
EmoNets: Multimodal deep learning approaches for emotion recognition in video
Samira Ebrahimi Kahou, Xavier Bouthillier, Pascal Lamblin +15
The task of the emotion recognition in the wild (EmotiW) Challenge is to assign one of seven emotions to short video clips extracted from Hollywood style movies. The videos depict…
On Using Monolingual Corpora in Neural Machine Translation
Caglar Gulcehre, Orhan Firat, Kelvin Xu +6
Recent work on end-to-end neural network-based architectures for machine translation has shown promising results for En-Fr and En-De translation. Arguably, one of the major factors…