42 citations · 132 across the 20 of their papers we have counts for
4 papers · 1 filter
Random initialisations performing above chance and how to find them
Frederik Benzing, Simon Schug, Robert Meier +5
Neural networks trained with stochastic gradient descent (SGD) starting from different random initialisations typically find functionally very similar solutions, raising the questi…
Decoupling Hierarchical Recurrent Neural Networks With Locally Computable Losses
Asier Mujika, Felix Weissenberger, Angelika Steger
Learning long-term dependencies is a key long-standing challenge of recurrent neural networks (RNNs). Hierarchical recurrent neural networks (HRNNs) have been considered a promisin…
Optimal Kronecker-Sum Approximation of Real Time Recurrent Learning
Frederik Benzing, Marcelo Matheus Gauy, Asier Mujika +2
One of the central goals of Recurrent Neural Networks (RNNs) is to learn long-term dependencies in sequential data. Nevertheless, the most popular training method, Truncated Backpr…
Approximating Real-Time Recurrent Learning with Random Kronecker Factors
Asier Mujika, Florian Meier, Angelika Steger
Despite all the impressive advances of recurrent neural networks, sequential data is still in need of better modelling. Truncated backpropagation through time (TBPTT), the learning…