3 papers
cs.LG2026
A Hierarchical Language Model with Predictable Scaling Laws and Provable Benefits of Reasoning
Jason Gaitonde, Frederic Koehler, Elchanan Mossel +2
We introduce a family of synthetic languages with hierarchical structure -- generated by a broadcast process on trees -- for which the role of context length and reasoning in autor…
cs.DS2025
Parallel Sampling via Autospeculation
Nima Anari, Carlo Baronio, CJ Chen +4
We present parallel algorithms to accelerate sampling via counting in two settings: any-order autoregressive models and denoising diffusion models. An any-order autoregressive mode…
cs.LG2024
Efficiently learning and sampling multimodal distributions with data-based initialization
Frederic Koehler, Holden Lee, Thuy-Duong Vuong
We consider the problem of sampling a multimodal distribution with a Markov chain given a small number of samples from the stationary measure. Although mixing can be arbitrarily sl…