OmniFold: A Method to Simultaneously Unfold All Observables

Abstract

Collider data must be corrected for detector effects ("unfolded") to be compared with many theoretical calculations and measurements from other experiments. Unfolding is traditionally done for individual, binned observables without including all information relevant for characterizing the detector response. We introduce OmniFold, an unfolding method that iteratively reweights a simulated dataset, using machine learning to capitalize on all available information. Our approach is unbinned, works for arbitrarily high-dimensional data, and naturally incorporates information from the full phase space. We illustrate this technique on a realistic jet substructure example from the Large Hadron Collider and compare it to standard binned unfolding methods. This new paradigm enables the simultaneous measurement of all observables, including those not yet invented at the time of the analysis.

arXiv comment8 pages, 3 figures, 1 table, 1 poem; v2: updated to approximate PRL version

References in corpus (41)

• Adam: A Method for Stochastic Optimization
• PYTHIA 6.4 Physics and Manual
• PYTHIA 6.2 Physics and Manual
• TensorFlow: A system for large-scale machine learning
• The anti-k_t jet clustering algorithm
• An Introduction to PYTHIA 8.2
• A Brief Introduction to PYTHIA 8.1
• FastJet user manual
• DELPHES 3, A modular framework for fast simulation of a generic collider experiment
• Herwig++ Physics and Manual
• Dispelling the N^3 myth for the Kt jet-finder
• Herwig 7.0 / Herwig++ 3.0 Release Note
• Identifying Boosted Objects with N-subjettiness
• Soft Drop
• Jet Trimming
• SVD Approach to Data Unfolding
• Towards an understanding of jet substructure
• Maximizing Boosted Top Identification by Minimizing N-subjettiness
• Recombination Algorithms and Jet Substructure: Pruning as a Tool for Heavy Particle Searches
• Techniques for improved heavy particle searches with jet substructure
• Jet Substructure at the Large Hadron Collider: Experimental Review
• Energy Flow Networks: Deep Sets for Particle Jets
• Systematics of quark/gluon tagging
• Mining gold from implicit models to improve likelihood-free inference
• Constraining Effective Field Theories with Machine Learning
• TUnfold: an algorithm for correcting migration effects in high energy physics
• A Guide to Constraining Effective Field Theories with Machine Learning
• Approximating Likelihood Ratios with Calibrated Discriminative Classifiers
• The Metric Space of Collider Events
• JUNIPR: a Framework for Unsupervised Machine Learning in Particle Physics
• Neural Networks for Full Phase-space Reweighting and Parameter Tuning
• Reweighting with Boosted Decision Trees
• Unfolding with Generative Adversarial Networks
• Exploring the Space of Jets with CMS Open Data
• A guide for deploying Deep Learning in LHC searches: How to achieve optimality and account for uncertainty
• Reweighting a parton shower using a neural network: the final-state case
• Observation of the decays ${\Lambda }_b^0\to {\chi }_{c1}p{K}^{-}$ and ${\Lambda }_b^0\to {\chi }_{c2}p{K}^{-}$
• Search for pair production of higgsinos in final states with at least three $b$-tagged jets in $\sqrt{s}=13$ TeV $pp$ collisions using the ATLAS detector
• Investigating the Topology Dependence of Quark and Gluon Jets
• Machine learning approach to inverse problem and unfolding procedure
• AI Safety for High Energy Physics

Cited by in corpus (82)

• Simulation Assisted Likelihood-free Anomaly Detection
• Machine Learning and LHC Event Generation
• Invertible Networks or Partons to Detector and Back Again
• Event Generators for High-Energy Physics Experiments
• Modern Machine Learning and Particle Physics
• GANplifying Event Samples
• A Neural Resampler for Monte Carlo Reweighting with Preserved Uncertainties
• The Present and Future of QCD
• A factorisation-aware Matrix element emulator
• A guide for deploying Deep Learning in LHC searches: How to achieve optimality and account for uncertainty
• Mapping Machine-Learned Physics into a Human-Readable Space
• Phenomenology of jet angularities at the LHC
• Presenting Unbinned Differential Cross Section Results
• DCTRGAN: Improving the Precision of Generative Models with Reweighting
• Learning to Simulate High Energy Particle Collisions from Unlabeled Data
• The LHC Olympics 2020: A Community Challenge for Anomaly Detection in High Energy Physics
• E Pluribus Unum Ex Machina: Learning from Many Collider Events at Once
• How to Understand Limitations of Generative Networks
• Two Invertible Networks for the Matrix Element Method
• Conformal Colliders Meet the LHC
• Neural Network-based Top Tagger with Two-Point Energy Correlations and Geometry of Soft Emissions
• SymmetryGAN: Symmetry Discovery with Deep Learning
• Towards a Deep Learning Model for Hadronization
• Reconstructing the Kinematics of Deep Inelastic Scattering with Deep Learning
• The MadNIS Reloaded
• Disentangling Quarks and Gluons with CMS Open Data
• Returning CP-Observables to The Frames They Belong
• Explainable machine learning of the underlying physics of high-energy particle collisions
• Machine learning-based jet and event classification at the Electron-Ion Collider with applications to hadron structure and spin physics
• Improving Generative Model-based Unfolding with Schrödinger Bridges
• Multidifferential study of identified charged hadron distributions in $Z$-tagged jets in proton-proton collisions at $\sqrt{s}=$13 TeV
• Non-Gaussianities in Collider Energy Flux
• CaloDREAM -- Detector Response Emulation via Attentive flow Matching
• The Landscape of Unfolding with Machine Learning
• Learning Likelihood Ratios with Neural Network Classifiers
• Unbinned Deep Learning Jet Substructure Measurement in High $Q^2$ ep collisions at HERA
• ${\nu }^2$-Flows: Fast and improved neutrino reconstruction in multi-neutrino final states with conditional normalizing flows
• Scaffolding Simulations with Deep Learning for High-dimensional Deconvolution
• Potential of the Julia programming language for high energy physics computing
• Precision-Machine Learning for the Matrix Element Method
• Solving Key Challenges in Collider Physics with Foundation Models
• ELSA -- Enhanced latent spaces for improved collider simulations
• A Method to Simultaneously Facilitate All Jet Physics Tasks
• A simultaneous unbinned differential cross section measurement of twenty-four $Z$+jets kinematic observables with the ATLAS detector
• Normalizing Flows for High-Dimensional Detector Simulations
• Parameter Estimation using Neural Networks in the Presence of Detector Effects
• Differentiable MadNIS-Lite
• Unbinned Profiled Unfolding
• Optimizing Observables with Machine Learning for Better Unfolding
• Full Event Particle-Level Unfolding with Variable-Length Latent Variational Diffusion
• Parameter Inference from Event Ensembles and the Top-Quark Mass
• Neural Empirical Bayes: Source Distribution Estimation and its Applications to Simulation-Based Inference
• Flavoured jet algorithms: a comparative study
• The Linear Template Fit
• Probing cold nuclear matter with energy correlators
• Electroweak Restoration at the LHC and Beyond: The $Vh$ Channel
• Hypergraphs in LHC Phenomenology -- The Next Frontier of IRC-Safe Feature Extraction
• Deep Learning Jet Substructure from Two-Particle Correlation
• Neural Conditional Reweighting
• Unsupervised and lightly supervised learning in particle physics
• Moment Unfolding
• Describing Hadronization via Histories and Observables for Monte-Carlo Event Reweighting
• Measurement of lepton-jet correlation in deep-inelastic scattering with the H1 detector using machine learning for unfolding
• Unifying Simulation and Inference with Normalizing Flows
• Generative Unfolding with Distribution Mapping
• Top-philic Machine Learning
• Machine Learning-Assisted Unfolding for Neutrino Cross-section Measurements with the OmniFold Technique
• Designing Observables for Measurements with Deep Learning
• Comparison of Machine Learning Approach to other Unfolding Methods
• Measurement of jet track functions in $pp$ collisions at $\sqrt{s}=13$ TeV with the ATLAS detector
• A Practical Guide to Unbinned Unfolding
• Latent Space Refinement for Deep Generative Models
• Preserving New Physics while Simultaneously Unfolding All Observables
• Tools for Unbinned Unfolding
• How to Unfold Top Decays
• ScatterPrism: convergence for generative simulation and inverse problems in particle and nuclear physics
• Machine learning in top quark physics at ATLAS and CMS
• Studying the QGP with Jets at the LHC and RHIC
• Measurement of event shapes in minimum-bias events from proton-proton collisions at $\sqrt{s}$ = 13 TeV
• Energy Flow in Particle Collisions
• Response Matrix Estimation in Unfolding Differential Cross Sections
• Data-Driven Predictions for Dark Photon and Millicharged Particle Production