Ensemble Distillation for Structured Prediction: Calibrated, Accurate, Fast-Choose Three

Steven Reich, David Mueller, Nicholas Andrews

Modern neural networks do not always produce well-calibrated predictions, even when trained with a proper scoring function such as cross-entropy. In classification settings, simple…