10 citations · 20 across the 4 of their papers we have counts for
3 papers · 1 filter
Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
Nayoung Lee, Ziyang Cai, Avi Schwarzschild +2
Large language models often struggle with length generalization and solving complex problem instances beyond their training distribution. We present a self-improvement approach whe…
Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks
Jongho Park, Jaeseung Park, Zheyang Xiong +5
State-space models (SSMs), such as Mamba (Gu & Dao, 2023), have been proposed as alternatives to Transformer networks in language modeling, by incorporating gating, convolutions, a…
Teaching Arithmetic to Small Transformers
Nayoung Lee, Kartik Sreenivasan, Jason D. Lee +2
Large language models like GPT-4 exhibit emergent capabilities across general-purpose tasks, such as basic arithmetic, when trained on extensive text data, even though these tasks…