10 citations · 19 across the 11 of their papers we have counts for
3 papers · 1 filter
Frequency Domain Reservoir Computing
Klaus Schertler, Xiomara Runge, Andrea Ceni +2
While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balan…
Breaking Chains with Trees: Model-Parallel Deep Learning with Time Complexity
Neeraj Mohan Sushma, Aditya Nagarsekar, Cabrel Teguemne Fokam +4
Modern deep neural networks are trained using error backpropagation, which requires sequential forward and backward computations across network layers. As these networks become dee…
Dynamic sparsity in tree-structured feed-forward layers at scale
Reza Sedghi, Robin Schiewer, Anand Subramoney +1
At typical context lengths, the feed-forward MLP block accounts for a large share of a transformer's compute budget, motivating sparse alternatives to dense MLP blocks. We study sp…