27 citations · 30 across the 3 of their papers we have counts for
4 papers
Towards a Realistic Long-Term Benchmark for Open-Web Research Agents
Peter Mühlbacher, Nikos I. Bosse, Lawrence Phillips
We present initial results of a forthcoming benchmark for evaluating LLM agents on white-collar tasks of economic value. We evaluate agents on real-world "messy" open-web research…
Assessing Human Judgment Forecasts in the Rapid Spread of the Mpox Outbreak: Insights and Challenges for Pandemic Preparedness
Thomas McAndrew, Maimuna S. Majumder, Andrew A. Lover +10
In May 2022, mpox (formerly monkeypox) spread to non-endemic countries rapidly. Human judgment is a forecasting approach that has been sparsely evaluated during the beginning of an…
Evaluating Forecasts with scoringutils in R
Nikos I. Bosse, Hugo Gruson, Anne Cori +3
Evaluating forecasts is essential to understand and improve forecasting and make forecasts useful to decision makers. A variety of R packages provide a broad variety of scoring rul…
Comparing trained and untrained probabilistic ensemble forecasts of COVID-19 cases and deaths in the United States
Evan L. Ray, Logan C. Brooks, Jacob Bien +12
The U.S. COVID-19 Forecast Hub aggregates forecasts of the short-term burden of COVID-19 in the United States from many contributing teams. We study methods for building an ensembl…