15 citations · 16 across the 13 of their papers we have counts for
3 papers · 1 filter
Allocating Recurrent Compute in Looped Language Models
Ruhai Lin, Yiyang Guo, Rui-Jie Zhu +2
Looped language models improve reasoning and knowledge manipulation by applying shared computation repeatedly. Existing systems usually repeat an entire layer stack, although a mix…
SATFormer: Learning When to Use Early Value Residuals in Transformers
Skye Gunasekaran, Téa Wright, Rui-Jie Zhu +1
Recent Transformer architectures expose later layers to representations computed in earlier layers, motivated by the observation that low-level features can become harder to recove…
A Predictive Approach To Enhance Time-Series Forecasting
Skye Gunasekaran, Assel Kembay, Hugo Ladret +4
Accurate time-series forecasting is crucial in various scientific and industrial domains, yet deep learning models often struggle to capture long-term dependencies and adapt to dat…