2 citations · 2 across the 7 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Scaling Adaptive Depth with Norm-Agnostic Residual Networks
Tomás Figliolia, Beren Millidge
Residual architectures are ubiquitous in deep learning, but they suffer from a subtle structural limitation: the norm of the residual stream can grow rapidly with depth. As a resul…
cs.LG2026
Hybrid Associative Memories
Leon Lufkin, Tomás Figliolia, Beren Millidge +1
Recurrent neural networks (RNNs) and self-attention are both widely used sequence-mixing layers that maintain an internal memory. However, this memory is constructed using two orth…
cs.LG2026
Online Vector Quantized Attention
Nick Alonso, Tomas Figliolia, Beren Millidge
Standard sequence mixing layers used in language models struggle to balance efficiency and performance. Self-attention performs well on long context tasks but has expensive quadrat…