5 papers
Is Your LLM Overcharging You? Tokenization, Transparency, and Incentives
Ander Artola Velasco, Stratis Tsirtsis, Nastaran Okati +1
State-of-the-art large language models require specialized hardware and substantial energy to operate. As a consequence, cloud-based services that provide access to large language…
Measuring multi-calibration
Ido Guy, Daniel Haimovich, Fridolin Linder +4
A suitable scalar metric can help measure multi-calibration, defined as follows. When the expected values of observed responses are equal to corresponding predicted probabilities,…
Root Cause Analysis of Outliers with Missing Structural Knowledge
William Roy Orchard, Nastaran Okati, Sergio Hernan Garrido Mejia +2
The goal of Root Cause Analysis (RCA) is to explain why an anomaly occurred by identifying where the fault originated. Several recent works model the anomalous event as resulting f…
MCGrad: Multicalibration at Web Scale
Niek Tax, Lorenzo Perini, Fridolin Linder +5
We propose MCGrad, a novel and scalable multicalibration algorithm. Multicalibration - calibration in subgroups of the data - is an important property for the performance of machin…
Towards Human-AI Complementarity with Prediction Sets
Giovanni De Toni, Nastaran Okati, Suhas Thejaswi +2
Decision support systems based on prediction sets have proven to be effective at helping human experts solve classification tasks. Rather than providing single-label predictions, t…