Bias and Priors in Machine Learning Calibrations for High Energy Physics

Abstract

Machine learning offers an exciting opportunity to improve the calibration of nearly all reconstructed objects in high-energy physics detectors. However, machine learning approaches often depend on the spectra of examples used during training, an issue known as prior dependence. This is an undesirable property of a calibration, which needs to be applicable in a variety of environments. The purpose of this paper is to explicitly highlight the prior dependence of some machine learning-based calibration strategies. We demonstrate how some recent proposals for both simulation-based and data-based calibrations inherit properties of the sample used for training, which can result in biases for downstream analyses. In the case of simulation-based calibration, we argue that our recently proposed Gaussian Ansatz approach can avoid some of the pitfalls of prior dependence, whereas prior-independent data-based calibration remains an open problem.

arXiv comment17 pages, 7 figures, code available at https://github.com/hep-lbdl/calibrationpriors v2: Minor updates to match journal version

References in corpus (72)

• Array Programming with NumPy
• PYTHIA 6.4 Physics and Manual
• PYTHIA 6.2 Physics and Manual
• The anti-k_t jet clustering algorithm
• A Brief Introduction to PYTHIA 8.1
• FastJet user manual
• DELPHES 3, A modular framework for fast simulation of a generic collider experiment
• Dispelling the N^3 myth for the Kt jet-finder
• Normalizing Flows: An Introduction and Review of Current Methods
• Particle-flow reconstruction and global event description with the CMS detector
• Jet energy scale and resolution in the CMS experiment in pp collisions at 8 TeV
• Identification of heavy-flavour jets with the CMS detector in pp collisions at 13 TeV
• Muon reconstruction performance of the ATLAS detector in proton--proton collision data at $\sqrt{s}$=13 TeV
• Performance of electron reconstruction and selection with the CMS detector in proton-proton collisions at sqrt(s) = 8 TeV
• Topological cell clustering in the ATLAS calorimeters and its performance in LHC Run 1
• Electron and photon performance measurements with the ATLAS detector using the 2015-2017 LHC proton-proton collision data
• ATLAS $b$-jet identification performance and efficiency measurement with $t\bar{t}$ events in $pp$ collisions at $\sqrt{s}=13$ TeV
• Jet energy scale and resolution measured in proton-proton collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• Performance of photon reconstruction and identification with the CMS detector in proton-proton collisions at sqrt(s) = 8 TeV
• Performance of reconstruction and identification of $\tau$ leptons decaying to hadrons and ${\nu }_{\tau }$ in pp collisions at $\sqrt{s}=$ 13 TeV
• Identification of heavy, energetic, hadronically decaying particles using machine-learning techniques
• Adversarially-trained autoencoders for robust unsupervised new physics searches
• Measurement of the Underlying Event Activity at the LHC with sqrt(s)= 7 TeV and Comparison with sqrt(s) = 0.9 TeV
• Identification and energy calibration of hadronically decaying tau leptons with the ATLAS experiment in $pp$ collisions at $\sqrt{s}$=8 TeV
• Decorrelated Jet Substructure Tagging using Adversarial Neural Networks
• Nuisance hardened data compression for fast likelihood-free inference
• Pileup Mitigation with Machine Learning (PUMML)
• Automatic physical inference with information maximising neural networks
• Performance of top-quark and $W$-boson tagging with ATLAS in Run 2 of the LHC
• Calorimetry with Deep Learning: Particle Simulation and Reconstruction for Collider Physics
• Neural Networks for Full Phase-space Reweighting and Parameter Tuning
• Thinking outside the ROCs: Designing Decorrelated Taggers (DDT) for jet substructure
• DisCo Fever: Robust Networks Through Distance Correlation
• Machine Learning Uncertainties with Adversarial Neural Networks
• MLPF: Efficient machine-learned particle-flow reconstruction using graph neural networks
• Exploring the Space of Jets with CMS Open Data
• Performance of the reconstruction and identification of high-momentum muons in proton-proton collisions at $\sqrt{s}=$ 13 TeV
• ABCDisCo: Automating the ABCD Method with Machine Learning
• A Convolutional Neural Network based Cascade Reconstruction for the IceCube Neutrino Observatory
• Deep-Learning Jets with Uncertainties and More
• A deep neural network to search for new long-lived particles decaying to jets
• A guide for deploying Deep Learning in LHC searches: How to achieve optimality and account for uncertainty
• Uncertainty Aware Learning for High Energy Physics
• INFERNO: Inference-Aware Neural Optimisation
• uBoost: A boosting method for producing uniform selection efficiencies from multivariate classifiers
• DCTRGAN: Improving the Precision of Generative Models with Reweighting
• Per-Object Systematics using Deep-Learned Calibration
• E Pluribus Unum Ex Machina: Learning from Many Collider Events at Once
• Deep Neural Networks for Energy and Position Reconstruction in EXO-200
• Convolved Substructure: Analytically Decorrelating Jet Substructure Observables
• Cosmic Ray Spectrum from 250 TeV to 10 PeV using IceTop
• Mass Agnostic Jet Taggers
• A deep neural network for simultaneous estimation of b jet energy and resolution
• Improved Energy Reconstruction in NOvA with Regression Convolutional Neural Networks
• A Cautionary Tale of Decorrelating Theory Uncertainties
• Machine Learning based jet momentum reconstruction in heavy-ion collisions
• On the Use of Neural Networks for Energy Reconstruction in High-granularity Calorimeters
• QBDT, a new boosting decision tree method with systematic uncertainties into training for High Energy Physics
• Combine and Conquer: Event Reconstruction with Bayesian Ensemble Neural Networks
• Reducing the dependence of the neural network function to systematic uncertainties in the input space
• Pile-Up Mitigation using Attention
• Optimal statistical inference in the presence of systematic uncertainties using neural network optimization based on binned Poisson likelihoods with nuisance parameters
• Parametrizing the Detector Response with Neural Networks
• Learning Uncertainties the Frequentist Way: Calibration and Correlation in High Energy Physics
• Enhancing searches for resonances with machine learning and moment decomposition
• Transport away your problems: Calibrating stochastic simulations with optimal transport
• Mathematical Properties of Numerical Inversion for Jet Calibrations
• Calorimetric Measurement of Multi-TeV Muons via Deep Regression
• Convolutional Neural Networks for Shower Energy Prediction in Liquid Argon Time Projection Chambers
• Neural Conditional Reweighting
• Energy reconstruction in a liquid argon calorimeter cell using convolutional neural networks
• New approaches for boosting to uniformity

Cited by in corpus (1)

• Learning Uncertainties the Frequentist Way: Calibration and Correlation in High Energy Physics