9 papers
QuasiMoTTo: Quasi-Monte Carlo Test-Time Scaling
Michael Y. Li, Anthony Zhan, Kanishk Gandhi +2
Scaling inference compute, by generating many parallel attempts per problem, is a costly but reliable lever for improving language model capabilities. By default these attempts are…
Simplified Sparse Attention via Gist Tokens
Yuzhen Mao, Michael Y. Li, Emily B. Fox
Sparse attention can reduce the cost of long-context inference, but most variants introduce new architectural components. We introduce Simplified Sparse Attention (SSA), a simpler…
auto-psych: Automating the science of mind using agent-driven theory discovery and experimentation
Ben Prystawski, Kushin Mukherjee, Daniel Wurgaft +4
AI-based scientific automation is increasingly possible by using agents to generate hypotheses, design experiments, and analyze data. Data collection is a major bottleneck in this…
SPIRAL: Learning to Search and Aggregate
Jubayer Ibn Hamid, Ifdita Hasan Orney, Michael Y. Li +5
Language model reasoning can be substantially improved at test time via scaffolds that scale inference compute across different primitives -- sequential reasoning within a trace, i…
Neural Garbage Collection: Learning to Forget while Learning to Reason
Michael Y. Li, Jubayer Ibn Hamid, Emily B. Fox +1
Chain-of-thought reasoning has driven striking advances in language model capability, yet every reasoning step grows the KV cache, creating a bottleneck to scaling this paradigm fu…
GIANTS: Generative Insight Anticipation from Scientific Literature
Joy He-Yueya, Anikait Singh, Ge Gao +5
Scientific breakthroughs often emerge from synthesizing prior ideas into novel contributions. While language models (LMs) show promise in scientific discovery, their ability to per…