582 citations · 1.2k across the 18 of their papers we have counts for
Showing 2013Show all
2 papers · 1 filter
cs.NE2013★ 3 cited
Stochastic Gradient Estimate Variance in Contrastive Divergence and Persistent Contrastive Divergence
Mathias Berglund, Tapani Raiko
Contrastive Divergence (CD) and Persistent Contrastive Divergence (PCD) are popular methods for training the weights of Restricted Boltzmann Machines. However, both methods use an…
cs.LG2013
Pushing Stochastic Gradient towards Second-Order Methods -- Backpropagation Learning with Transformations in Nonlinearities
Tommi Vatanen, Tapani Raiko, Harri Valpola +1
Recently, we proposed to transform the outputs of each hidden neuron in a multi-layer perceptron network to have zero output and zero slope on average, and use separate shortcut co…