4 papers · 1 filter
Aligning the Evaluation of Probabilistic Predictions with Downstream Value
Novin Shahroudi, Viacheslav Komisarenko, Meelis Kull
Every prediction is ultimately used in a downstream task. Consequently, evaluating prediction quality is more meaningful when considered in the context of its downstream use. Metri…
Probability Density from Latent Diffusion Models for Out-of-Distribution Detection
Joonas Järve, Karl Kaspar Haavel, Meelis Kull
Despite rapid advances in AI, safety remains the main bottleneck to deploying machine-learning systems. A critical safety component is out-of-distribution detection: given an input…
Improving Calibration by Relating Focal Loss, Temperature Scaling, and Properness
Viacheslav Komisarenko, Meelis Kull
Proper losses such as cross-entropy incentivize classifiers to produce class probabilities that are well-calibrated on the training data. Due to the generalization gap, these class…
Cautious Calibration in Binary Classification
Mari-Liis Allikivi, Joonas Järve, Meelis Kull
Being cautious is crucial for enhancing the trustworthiness of machine learning systems integrated into decision-making pipelines. Although calibrated probabilities help in optimal…