1 citations · 1 across the 5 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024
Interpreting Affine Recurrence Learning in GPT-style Transformers
Samarth Bhargav, Alexander Gu
Understanding the internal mechanisms of GPT-style transformers, particularly their capacity to perform in-context learning (ICL), is critical for advancing AI alignment and interp…
cs.LG2024★ 1 cited
Mixture of Parrots: Experts improve memorization more than reasoning
Samy Jelassi, Clara Mohri, David Brandfonbrener +7
The Mixture-of-Experts (MoE) architecture enables a significant increase in the total number of model parameters with minimal computational overhead. However, it is not clear what…