5 papers
Neural Neural Scaling Laws
Michael Y. Hu, Jane Pan, Ayush Rajesh Jhaveri +2
Neural scaling laws predict how language model performance improves with increased training inputs. While aggregate metrics like validation loss can follow smooth power-law curves,…
RELIC: Evaluating Complex Reasoning via the Recognition of Languages In-Context
Jackson Petty, Michael Y. Hu, Wentao Wang +3
Large language models (LLMs) are increasingly used to solve complex tasks where they must retrieve and compose many pieces of in-context information in long reasoning chains. For m…
Sample-Efficient Online Learning in LM Agents via Hindsight Trajectory Rewriting
Michael Y. Hu, Benjamin Van Durme, Jacob Andreas +1
Language model (LM) agents deployed in novel environments often exhibit poor sample efficiency when learning from sequential interactions. This significantly hinders the usefulness…
BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop
Lucas Charpentier, Leshem Choshen, Ryan Cotterell +11
BabyLM aims to dissolve the boundaries between cognitive modeling and language modeling. We call for both workshop papers and for researchers to join the 3rd BabyLM competition. As…
Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
Michael Y. Hu, Aaron Mueller, Candace Ross +7
The BabyLM Challenge is a community effort to close the data-efficiency gap between human and computational language learners. Participants compete to optimize language model train…