Learning Stochastic Recurrent Networks
arXiv:1411.7610
Abstract
Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs). The model i) can be trained with stochastic gradient methods, ii) allows structured and multi-modal conditionals at each time step, iii) features a reliable estimator of the marginal likelihood and iv) is a generalisation of deterministic recurrent neural networks. We evaluate the method on four polyphonic musical data sets and motion capture data.
Submitted to conference track of ICLR 2015
References in corpus (4)
- ADADELTA: An Adaptive Learning Rate Method
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
- Speech Recognition with Deep Recurrent Neural Networks