Evaluating generative models in high energy physics

Abstract

There has been a recent explosion in research into machine-learning-based generative modeling to tackle computational challenges for simulations in high energy physics (HEP). In order to use such alternative simulators in practice, we need well-defined metrics to compare different generative models and evaluate their discrepancy from the true distributions. We present the first systematic review and investigation into evaluation metrics and their sensitivity to failure modes of generative models, using the framework of two-sample goodness-of-fit testing, and their relevance and viability for HEP. Inspired by previous work in both physics and computer vision, we propose two new metrics, the Fréchet and kernel physics distances (FPD and KPD, respectively), and perform a variety of experiments measuring their performance on simple Gaussian-distributed, and simulated high energy jet datasets. We find FPD, in particular, to be the most sensitive metric to all alternative jet distributions tested and recommend its adoption, along with the KPD and Wasserstein distances between individual feature distributions, for evaluating generative models in HEP. We finally demonstrate the efficacy of these proposed metrics in evaluating and comparing a novel attention-based generative adversarial particle transformer to the state-of-the-art message-passing generative adversarial network jet simulation model. The code for our proposed metrics is provided in the open source JetNet Python library.

arXiv comment11 pages, 5 figures, 3 tables, and a 5 page appendix

References in corpus (16)

• ParticleNet: Jet Tagging via Particle Clouds
• CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks
• Jet Substructure at the Large Hadron Collider: A Review of Recent Advances in Theory and Machine Learning
• Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis
• Accelerating Science with Generative Adversarial Networks: An Application to 3D Particle Showers in Multi-Layer Calorimeters
• Learning New Physics from a Machine
• Event Generation with Normalizing Flows
• How to GAN LHC Events
• Energy flow polynomials: A complete linear basis for jet substructure
• Calorimetry with Deep Learning: Particle Simulation and Reconstruction for Collider Physics
• Score-based Generative Models for Calorimeter Shower Simulation
• DijetGAN: A Generative-Adversarial Network Approach for the Simulation of QCD Dijet Events at the LHC
• Lund jet images from generative and cycle-consistent adversarial networks
• Calomplification -- The Power of Generative Calorimeter Models
• Particle-based Fast Jet Simulation at the LHC with Variational Autoencoders
• A survey of machine learning-based physics event generation

Cited by in corpus (26)

• CaloClouds: Fast Geometry-Independent Highly-Granular Calorimeter Simulation
• EPiC-GAN: Equivariant Point Cloud Generation for Particle Jets
• PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics
• L2LFlows: Generating High-Fidelity 3D Calorimeter Images
• How to Understand Limitations of Generative Networks
• Deep generative models for fast photon shower simulation in ATLAS
• Learning the language of QCD jets with transformers
• Deep Generative Models for Detector Signature Simulation: A Taxonomic Review
• Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models
• An implementation of neural simulation-based inference for parameter estimation in ATLAS
• Convolutional L2LFlows: Generating Accurate Showers in Highly Granular Calorimeters Using Convolutional Normalizing Flows
• Generalizing to new geometries with Geometry-Aware Autoregressive Models (GAAMs) for fast calorimeter simulation
• Goodness of fit by Neyman-Pearson testing
• Comparison of Affine and Rational Quadratic Spline Coupling and Autoregressive Flows through Robust Statistical Tests
• End-to-end simulation of particle physics events with Flow Matching and generator Oversampling
• CaloChallenge 2022: A Community Challenge for Fast Calorimeter Simulation
• Differentiable Earth Mover's Distance for Data Compression at the High-Luminosity LHC
• Aspen Open Jets: Unlocking LHC Data for Foundation Models in Particle Physics
• Galaxy Imaging with Generative Models: Insights from a Two-Models Framework
• Flow Matching Beyond Kinematics: Generating Jets with Particle-ID and Trajectory Displacement Information
• Refining fast simulation using machine learning
• LHC Hadronic Jet Generation Using Convolutional Variational Autoencoders with Normalizing Flows
• AdvNF: Reducing Mode Collapse in Conditional Normalising Flows using Adversarial Learning
• BitHEP -- The Limits of Low-Precision ML in HEP
• Event generation and consistency tests with sliced Wasserstein distance in high-energy physics
• Choose Your Diffusion: Efficient and flexible ways to accelerate the diffusion model in fast high energy physics simulation