1.5k citations
- Google (United States)US9 papers
- University of OxfordGB3 papers
- Google (United Kingdom)GB2 papers
- University of TorontoCA2 papers
- Australian National UniversityAU1 paper
- Canadian Institute for Advanced ResearchCA1 paper
- Gatsby Computational Neuroscience UnitGB1 paper
- Université de SherbrookeCA1 paper
- University College LondonGB1 paper
- University of AmsterdamNL1 paper
- University of British ColumbiaCA1 paper
- University of EdinburghGB1 paper
7 papers
Natural Neural Networks
Guillaume Desjardins, Karen Simonyan, Razvan Pascanu +1
We introduce Natural Neural Networks, a novel family of algorithms that speed up convergence by adapting their internal representation during training to improve conditioning of th…
Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages
Wittawat Jitkrittum, Arthur Gretton, Nicolas Heess +4
We propose an efficient nonparametric strategy for learning a message operator in expectation propagation (EP), which takes as input the set of incoming messages to a factor node,…
DRAW: A Recurrent Neural Network For Image Generation
Karol Gregor, Ivo Danihelka, Alex Graves +2
This paper introduces the Deep Recurrent Attentive Writer (DRAW) neural network architecture for image generation. DRAW networks combine a novel spatial attention mechanism that mi…
MADE: Masked Autoencoder for Distribution Estimation
Mathieu Germain, Karol Gregor, Iain Murray +1
There has been a lot of recent interest in designing neural network models to estimate a distribution from a set of examples. We introduce a simple modification for autoencoder neu…
Multiple Object Recognition with Visual Attention
Jimmy Ba, Volodymyr Mnih, Koray Kavukcuoglu
We present an attention-based model for recognizing multiple objects in images. The proposed model is a deep recurrent neural network trained with reinforcement learning to attend…
Move Evaluation in Go Using Deep Convolutional Neural Networks
Chris J. Maddison, Aja Huang, Ilya Sutskever +1
The game of Go is more challenging than other board games, due to the difficulty of constructing a position or move evaluation function. In this paper we investigate whether deep c…