Showing 2017 · cs.LGShow all

7 papers · 2 filters

cs.LG2017★ 57 cited

Twin Networks: Matching the Future for Sequence Generation

Dmitriy Serdyuk, Nan Rosemary Ke, Alessandro Sordoni +3

We propose a simple technique for encouraging generative RNNs to plan ahead. We train a "backward" recurrent network to generate a given sequence in reverse order, and we encourage…

cs.LG2017★ 13 cited

SEARNN: Training RNNs with Global-Local Losses

Rémi Leblond, Jean-Baptiste Alayrac, Anton Osokin +1

We propose SEARNN, a novel training algorithm for recurrent neural networks (RNNs) inspired by the "learning to search" (L2S) approach to structured prediction. RNNs have been wide…

cs.LG2017★ 24 cited

Convergent Tree Backup and Retrace with Function Approximation

Ahmed Touati, Pierre-Luc Bacon, Doina Precup +1

Off-policy learning is key to scaling up reinforcement learning as it allows to learn about a target policy from the experience generated by a different behavior policy. Unfortunat…

cs.LG2017★ 2 cited

A Robust Adaptive Stochastic Gradient Method for Deep Learning

Caglar Gulcehre, Jose Sotelo, Marcin Moczulski +1

Stochastic gradient algorithms are the main focus of large-scale optimization problems and led to important successes in the recent advancement of the deep learning algorithms. The…

cs.LG2017★ 134 cited

Sharp Minima Can Generalize For Deep Nets

Laurent Dinh, Razvan Pascanu, Samy Bengio +1

Despite their overwhelming capacity to overfit, deep learning architectures tend to generalize relatively well to unseen data, allowing them to be deployed in practice. However, ex…

cs.LG2017★ 9 cited

On Structured Prediction Theory with Calibrated Convex Surrogate Losses

Anton Osokin, Francis Bach, Simon Lacoste-Julien

We provide novel theoretical insights on structured prediction in the context of efficient convex surrogate loss minimization with consistency guarantees. For any task loss, we con…

cs.LG2017★ 23 cited

Calibrating Energy-based Generative Adversarial Networks

Zihang Dai, Amjad Almahairi, Philip Bachman +2

In this paper, we propose to equip Generative Adversarial Networks with the ability to produce direct energy estimates for samples.Specifically, we propose a flexible adversarial t…