5 papers
The Manokhin Probability Matrix: A Diagnostic Framework for Classifier Probability Quality
Valery Manokhin
The Brier score conflates two distinct properties of probabilistic predictions: reliability (calibration error) and resolution (discriminatory power). We introduce the Manokhin Pro…
Training-Free Probabilistic Time-Series Forecasting with Conformal Seasonal Pools
Valery Manokhin
We propose Conformal Seasonal Pools (CSP), a training-free probabilistic time-series forecaster that mixes same-season empirical draws with signed residual draws around a seasonal…
Classifier Calibration at Scale: An Empirical Study of Model-Agnostic Post-Hoc Methods
Valery Manokhin, Daniel Grønhaug
We study model-agnostic post-hoc calibration methods intended to improve probabilistic predictions in supervised binary classification on real i.i.d. tabular data, with particular…
Computationally efficient versions of conformal predictive distributions
Vladimir Vovk, Ivan Petej, Ilia Nouretdinov +2
Conformal predictive systems are a recent modification of conformal predictors that output, in regression problems, probability distributions for labels of test observations rather…
Conformal predictive distributions with kernels
Vladimir Vovk, Ilia Nouretdinov, Valery Manokhin +1
This paper reviews the checkered history of predictive distributions in statistics and discusses two developments, one from recent literature and the other new. The first developme…