52 citations · 52 across the 3 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
The Orthogonalized Read Is a Removable Training Scaffold for Recurrent Memory
Keston Aquino-Michaels
Orthogonalizing the mLSTM memory matrix at read time with five differentiable Newton-Schulz iterations improves noisy associative recall. We replicate this effect and investigate i…
cs.LG2026
Routing Absorption in Sparse Attention: Why Random Gates Are Hard to Beat
Keston Aquino-Michaels
Learned gates can approximate sparse attention patterns on frozen transformers, yet provide limited benefit over random gates when trained jointly with the model. We investigate th…