121 citations · 260 across the 10 of their papers we have counts for
4 papers · 1 filter
CruxBench: A Benchmark of Information Discovery
Hui Dai, Lina Piao, Nick Merrill +3
Benchmarks for large language models (LLMs) typically evaluate the accuracy of answers against fixed reference labels. But a central step in many complex real-world tasks is identi…
ForecastBench-Sim: A Simulated-World Forecasting Benchmark
Jaeho Lee, Nick Merrill, Ezra Karger
Forecasting benchmarks for general-purpose AI systems usually inherit the constraints of the real world: outcomes resolve slowly, tail events are rare, and counterfactual questions…
Symmetry Defeats Auditing
Nick Merrill, Zeke Medley
We demonstrate an attack on Introspection Adapters (Shenoy et al., 2026).
Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Most
Nick Merrill, Jaeho Lee, Ezra Karger
We document inverse scaling in LLMs on forecasting problems whose underlying time series exhibit superlinear growth and tail risk of regime change, a structure common in finance an…