42 citations · 104 across the 10 of their papers we have counts for
10 papers
Hierarchical Attention Encoder Decoder
Asier Mujika
Recent advances in large language models have shown that autoregressive modeling can generate complex and novel sequences that have many real-world applications. However, these mod…
Online learning of long-range dependencies
Nicolas Zucchet, Robert Meier, Simon Schug +2
Online learning holds the promise of enabling efficient long-term credit assignment in recurrent neural networks. However, current algorithms fall short of offline backpropagation…
Open-Ended Reinforcement Learning with Neural Reward Functions
Robert Meier, Asier Mujika
Inspired by the great success of unsupervised learning in Computer Vision and Natural Language Processing, the Reinforcement Learning community has recently started to focus more o…
Improving Gradient Estimation in Evolutionary Strategies With Past Descent Directions
Florian Meier, Asier Mujika, Marcelo Matheus Gauy +1
Evolutionary Strategies (ES) are known to be an effective black-box optimization technique for deep neural networks when the true gradients cannot be computed, such as in Reinforce…
Decoupling Hierarchical Recurrent Neural Networks With Locally Computable Losses
Asier Mujika, Felix Weissenberger, Angelika Steger
Learning long-term dependencies is a key long-standing challenge of recurrent neural networks (RNNs). Hierarchical recurrent neural networks (HRNNs) have been considered a promisin…
Optimal Kronecker-Sum Approximation of Real Time Recurrent Learning
Frederik Benzing, Marcelo Matheus Gauy, Asier Mujika +2
One of the central goals of Recurrent Neural Networks (RNNs) is to learn long-term dependencies in sequential data. Nevertheless, the most popular training method, Truncated Backpr…