582 citations · 1.2k across the 18 of their papers we have counts for
5 papers · 1 filter
Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters
Jelena Luketina, Mathias Berglund, Klaus Greff +1
Hyperparameter selection generally relies on running multiple full training trials, with selection based on validation set performance. We propose a gradient-based approach for loc…
DopeLearning: A Computational Approach to Rap Lyrics Generation
Eric Malmi, Pyry Takala, Hannu Toivonen +2
Writing rap lyrics requires both creativity to construct a meaningful, interesting story and lyrical skills to produce complex rhyme patterns, which form the cornerstone of good fl…
Lateral Connections in Denoising Autoencoders Support Supervised Learning
Antti Rasmus, Harri Valpola, Tapani Raiko
We show how a deep denoising autoencoder with lateral connections can be used as an auxiliary unsupervised learning task to support supervised learning. The proposed model is train…
Bidirectional Recurrent Neural Networks as Generative Models - Reconstructing Gaps in Time Series
Mathias Berglund, Tapani Raiko, Mikko Honkala +3
Bidirectional recurrent neural networks (RNN) are trained to predict both in the positive and negative time directions simultaneously. They have not been used commonly in unsupervi…
Pushing Stochastic Gradient towards Second-Order Methods -- Backpropagation Learning with Transformations in Nonlinearities
Tommi Vatanen, Tapani Raiko, Harri Valpola +1
Recently, we proposed to transform the outputs of each hidden neuron in a multi-layer perceptron network to have zero output and zero slope on average, and use separate shortcut co…