8 papers
Process Reward Agents for Steering Knowledge-Intensive Reasoning
Jiwoong Sohn, Tomasz Sternal, Kenneth Styppa +2
Reasoning in knowledge-intensive domains remains challenging as intermediate steps are often not locally verifiable: unlike math or code, evaluating step correctness may require sy…
Confounder Detection via Treatment Intent: A New Observational Study Design
Drago Plecko, Patrik Okanovic, Torsten Hoefler +1
Understanding the effects of interventions is central to scientific progress, with randomized controlled trials (RCTs) regarded as the gold standard for causal inference in many ap…
Large Language Model Selection with Limited Annotations
Yavuz Durmazkeser, Patrik Okanovic, Andreas Kirsch +2
Choosing a Large Language Model (LLM) for a given task requires comparing many strong candidates, yet standard evaluation relies on costly annotations over fixed evaluation sets. T…
Epidemiology of Large Language Models: A Benchmark for Observational Distribution Knowledge
Drago Plecko, Patrik Okanovic, Shreyas Havaldar +2
Artificial intelligence (AI) systems hold great promise for advancing various scientific disciplines, and are increasingly used in real-world applications. Despite their remarkable…
BLaST: High Performance Inference and Pretraining using BLock Sparse Transformers
Patrik Okanovic, Sameer Deshmukh, Grzegorz Kwasniewski +8
The energy consumption of large-scale ML models is dominated by data movement, shuffling billions of parameters across memory hierarchies and data centers. Sparsification offers a…
Active Model Selection for Large Language Models
Yavuz Durmazkeser, Patrik Okanovic, Andreas Kirsch +2
We introduce LLM SELECTOR, the first framework for active model selection of Large Language Models (LLMs). Unlike prior evaluation and benchmarking approaches that rely on fully an…