194 citations · 424 across the 14 of their papers we have counts for
8 papers · 1 filter
On the Usefulness of the Fit-on-the-Test View on Evaluating Calibration of Classifiers
Markus Kängsepp, Kaspar Valk, Meelis Kull
Every uncalibrated classifier has a corresponding true calibration map that calibrates its confidence. Deviations of this idealistic map from the identity map reveal miscalibration…
Cross-model Fairness: Empirical Study of Fairness and Ethics Under Model Multiplicity
Kacper Sokol, Meelis Kull, Jeffrey Chan +1
While data-driven predictive models are a strictly technological construct, they may operate within a social context in which benign engineering choices entail implicit, indirect a…
Classifier Calibration: A survey on how to assess and improve predicted class probabilities
Telmo Silva Filho, Hao Song, Miquel Perello-Nieto +3
This paper provides both an introduction to and a detailed overview of the principles and practice of classifier calibration. A well-calibrated classifier correctly quantifies the…
Shift Happens: Adjusting Classifiers
Theodore James Thibault Heiser, Mari-Liis Allikivi, Meelis Kull
Minimizing expected loss measured by a proper scoring rule, such as Brier score or log-loss (cross-entropy), is a common objective while training a probabilistic classifier. If the…
Instance-based Label Smoothing For Better Calibrated Classification Networks
Mohamed Maher, Meelis Kull
Label smoothing is widely used in deep neural networks for multi-class classification. While it enhances model generalization and reduces overconfidence by aiming to lower the prob…
Correlated daily time series and forecasting in the M4 competition
Anti Ingel, Novin Shahroudi, Markus Kängsepp +3
We participated in the M4 competition for time series forecasting and describe here our methods for forecasting daily time series. We used an ensemble of five statistical forecasti…
Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration
Meelis Kull, Miquel Perello-Nieto, Markus Kängsepp +3
Class probabilities predicted by most multiclass classifiers are uncalibrated, often tending towards over-confidence. With neural networks, calibration can be improved by temperatu…
HyperStream: a Workflow Engine for Streaming Data
Tom Diethe, Meelis Kull, Niall Twomey +5
This paper describes HyperStream, a large-scale, flexible and robust software package, written in the Python language, for processing streaming data with workflow creation capabili…