71 citations · 186 across the 38 of their papers we have counts for
10 papers · 1 filter
LLM Evaluation is a Forecasting Problem
Rahul Thomas, Adam Block, Micah Goldblum
Standard procedure in machine learning is to randomly split train and test data, emulating IID draws from one underlying data distribution. In practice, models are trained on past…
Toward Simulations of Cold Baryogenesis via Supercooled Phase Transitions
Naoya Kitajima, Shota Nakagawa, Yuichiro Nakai +2
Supercooled first-order phase transitions provide an attractive framework for generating observable gravitational waves (GWs), but the accompanying entropy injection dilutes any pr…
Pinocchio: Fast Uncertainty Estimates for Black-Box Language Models
Kevin David Hayes, Arka Pal, Haosong Zhang +2
In high-stakes decision-making applications of large language models (LLMs), practitioners require not only accurate LLMs but also uncertainty estimates for their predictions. Exis…
Recovering Wasted Compute in Autoresearch Agents
Au Kwok Chun, Abhigyan Acherjee, Amrutha Rao +4
A slew of recent works develop agents for solving research problems end-to-end, a paradigm increasingly referred to as autoresearch. Such agents have inspired large industry invest…
Understanding Reasoning from Pretraining to Post-Training
Jingyan Shen, Ang Li, Salman Rahman +4
Reinforcement learning (RL) has become central to improving large language models (LLMs) on complex reasoning tasks, yet RL post-training is largely studied in isolation from the p…
End-to-End Context Compression at Scale
Ang Li, Sean McLeish, Haozhe Chen +12
Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length. Recent techniques to compress the KV cache fall short: they either degra…
On Training in Imagination
Nadav Timor, Ravid Shwartz-Ziv, Micah Goldblum +2
State-of-the-art model-based reinforcement learning methods train policies on imagined rollouts. These rollouts are trajectories generated by a learned dynamics model and are score…
LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches
Linyang He, Qiyao Yu, Hanze Dong +5
Mathematical reasoning is a hallmark of human intelligence, and whether large language models (LLMs) can meaningfully perform it remains a central question in artificial intelligen…
Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference of LLMs
Arka Pal, Louai Zahran, William Gvozdjak +2
As large language models (LLMs) continue to grow in size, fewer users are able to host and run models locally. This has led to increased use of third-party hosting services. Howeve…
Dynamic Delayed Tree Expansion For Improved Multi-Path Speculative Decoding
Rahul Thomas, Teo Kitanovski, Micah Goldblum +1
Multi-path speculative decoding accelerates lossless sampling from a target model by using a cheaper draft model to generate a draft tree of tokens, and then applies a verification…