Countdown Regression: Sharp and Calibrated Survival Predictions
arXiv:1806.08324
Abstract
Probabilistic survival predictions from models trained with Maximum Likelihood Estimation (MLE) can have high, and sometimes unacceptably high variance. The field of meteorology, where the paradigm of maximizing sharpness subject to calibration is popular, has addressed this problem by using scoring rules beyond MLE, such as the Continuous Ranked Probability Score (CRPS). In this paper we present the \emph{Survival-CRPS}, a generalization of the CRPS to the survival prediction setting, with right-censored and interval-censored variants. We evaluate our ideas on the mortality prediction task using two different Electronic Health Record (EHR) data sets (STARR and MIMIC-III) covering millions of patients, with suitable deep neural network architectures: a Recurrent Neural Network (RNN) for STARR and a Fully Connected Network (FCN) for MIMIC-III. We compare results between the two scoring rules while keeping the network architecture and data fixed, and show that models trained with Survival-CRPS result in sharper predictive distributions compared to those trained by MLE, while still maintaining calibration.
arXiv commentUAI 2019
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- On Calibration of Modern Neural Networks
- Searching for Activation Functions
- Neural networks for post-processing ensemble weather forecasts
- Layer Normalization
- Accurate Uncertainties for Deep Learning Using Calibrated Regression
- Log-normal distribution based EMOS models for probabilistic wind speed forecasting
Cited by in corpus (7)
- Enabling Counterfactual Survival Analysis with Balanced Representations
- Survival Function Matching for Calibrated Time-to-Event Predictions
- SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data
- Short-Term Solar Irradiance Forecasting Using Calibrated Probabilistic Models
- WRSE -- a non-parametric weighted-resolution ensemble for predicting individual survival distributions in the ICU
- Inverse-Weighted Survival Games
- Harmonic Mean Point Processes: Proportional Rate Error Minimization for Obtundation Prediction