125 citations · 346 across the 41 of their papers we have counts for
45 papers
A Four-by-Four Obstruction to Polynomial Removal for Ordered Binary Matrices
Zinan Wang
We exhibit a fixed binary matrix with no polynomial induced ordered removal bound. At depth , let . Our explicit host has order , exactly $2m^…
On the Importance of Gating: Memorization vs. In-Context Learning in State Space Models
William L. Tong, Aryo Lotfi, Emmanuel Abbe +6
State Space Models (SSMs) have emerged as a compelling alternative to Transformers, enabling sequence modeling with constant memory and linear compute. Although SSMs exhibit reason…
Annotations Mitigate Post-Training Mode Collapse
Jacob Mitchell Springer, Madhu Advani, Lukas Aichberger +7
Post-training (via supervised fine-tuning) improves instruction-following, but often induces semantic mode collapse by biasing models toward low-entropy fine-tuning data at the exp…
To Infinity and Beyond: Tool-Use Unlocks Length Generalization in State Space Models
Eran Malach, Omid Saremi, Sinead Williamson +5
State Space Models (SSMs) have become the leading alternative to Transformers for sequence modeling. Their primary advantage is efficiency in long-context and long-form generation,…
LLM Priors for ERM over Programs
Shivam Singhal, Priyadarsi Mishra, Eran Malach +1
We study program-learning methods that are efficient in both samples and computation. Classical learning theory suggests that when the target admits a short program description, fo…
How Reinforcement Learning After Next-Token Prediction Facilitates Learning
Nikolaos Tsilivis, Eran Malach, Karen Ullrich +1
Recent advances in reasoning domains with neural networks have primarily been enabled by a training recipe that optimizes Large Language Models, previously trained to predict the n…