7 citations · 8 across the 13 of their papers we have counts for
15 papers
Causally Fair Generation with Large Language Models
Patrik Okanovic, Torsten Hoefler, Drago Plecko
Large language models (LLMs) are increasingly used to generate, complete, and transform information in settings where their outputs can shape consequential decisions, raising conce…
Which LLM to pick? Online Active Model Selection for Large Language Models
Alessandro Turrin, Patrik Okanovic, Torsten Hoefler +1
Large Language Models (LLMs) are increasingly applied to process streaming data, with practitioners relying on benchmarks to select the best model even though these signals only ap…
Performance Foundations of Parallel & Distributed Reasoning Language Models
Maciej Besta, Leonard Schmidt, Lara Nonino +7
Reinforcement Learning with Verifiable Rewards (RLVR) and other RL-style post-training paradigms have been used for aligning large language models (LLMs) with reasoning standards.…
When Data Is Scarce: Scaling Sparse Language Models with Repeated Training
Boqian Wu, Qiao Xiao, Patrik Okanovic +6
Scaling laws for dense LLMs under infinite data are well explored, but how sparsity interacts with limited data is not. In this work, we study sparse training in data-constrained r…
Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling
Qiao Xiao, Boqian Wu, Patrik Okanovic +6
Dynamic Sparse Training (DST) offers a promising paradigm for improving the training and inference efficiency of deep neural networks; however, we find that in large language model…
Confounder Detection via Treatment Intent: A New Observational Study Design
Drago Plecko, Patrik Okanovic, Torsten Hoefler +1
Understanding the effects of interventions is central to scientific progress, with randomized controlled trials (RCTs) regarded as the gold standard for causal inference in many ap…
Large Language Model Selection with Limited Annotations
Yavuz Durmazkeser, Patrik Okanovic, Andreas Kirsch +2
Choosing a Large Language Model (LLM) for a given task requires comparing many strong candidates, yet standard evaluation relies on costly annotations over fixed evaluation sets. T…
Epidemiology of Large Language Models: A Benchmark for Observational Distribution Knowledge
Drago Plecko, Patrik Okanovic, Shreyas Havaldar +2
Artificial intelligence (AI) systems hold great promise for advancing various scientific disciplines, and are increasingly used in real-world applications. Despite their remarkable…
Large Language Model Selection with Limited Annotations
Yavuz Durmazkeser, Patrik Okanovic, Andreas Kirsch +2
Choosing a Large Language Model (LLM) for a given task requires comparing many strong candidates, yet standard evaluation relies on costly annotations over fixed evaluation sets. T…
BLaST: High Performance Inference and Pretraining using BLock Sparse Transformers
Patrik Okanovic, Sameer Deshmukh, Grzegorz Kwasniewski +8
The energy consumption of large-scale ML models is dominated by data movement, shuffling billions of parameters across memory hierarchies and data centers. Sparsification offers a…
All models are wrong, some are useful: Model Selection with Limited Labels
Patrik Okanovic, Andreas Kirsch, Jannes Kasper +3
We introduce MODEL SELECTOR, a framework for label-efficient selection of pretrained classifiers. Given a pool of unlabeled target data, MODEL SELECTOR samples a small subset of hi…
EntryPrune: Neural Network Feature Selection using First Impressions
Felix Zimmer, Patrik Okanovic, Torsten Hoefler
There is an ongoing effort to develop feature selection algorithms to improve interpretability, reduce computational resources, and minimize overfitting in predictive models. Neura…
High Performance Unstructured SpMM Computation Using Tensor Cores
Patrik Okanovic, Grzegorz Kwasniewski, Paolo Sylos Labini +3
High-performance sparse matrix-matrix (SpMM) multiplication is paramount for science and industry, as the ever-increasing sizes of data prohibit using dense data structures. Yet, e…
Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learning
Patrik Okanovic, Roger Waleffe, Vasilis Mageirakos +5
Methods for carefully selecting or generating a small set of training data to learn from, i.e., data pruning, coreset selection, and data distillation, have been shown to be effect…
Gated Domain Units for Multi-source Domain Generalization
Simon Föll, Alina Dubatovka, Eugen Ernst +7
The phenomenon of distribution shift (DS) occurs when a dataset at test time differs from the dataset at training time, which can significantly impair the performance of a machine…