CaloChallenge 2022: A Community Challenge for Fast Calorimeter Simulation

Abstract

We present the results of the "Fast Calorimeter Simulation Challenge 2022" - the CaloChallenge. We study state-of-the-art generative models on four calorimeter shower datasets of increasing dimensionality, ranging from a few hundred voxels to a few tens of thousand voxels. The 31 individual submissions span a wide range of current popular generative architectures, including Variational AutoEncoders (VAEs), Generative Adversarial Networks (GANs), Normalizing Flows, Diffusion models, and models based on Conditional Flow Matching. We compare all submissions in terms of quality of generated calorimeter showers, as well as shower generation time and model size. To assess the quality we use a broad range of different metrics including differences in 1-dimensional histograms of observables, KPD/FPD scores, AUCs of binary classifiers, and the log-posterior of a multiclass classifier. The results of the CaloChallenge provide the most complete and comprehensive survey of cutting-edge approaches to calorimeter fast simulation to date. In addition, our work provides a uniquely detailed perspective on the important problem of how to evaluate generative models. As such, the results presented here should be applicable for other domains that use generative AI and require fast and faithful generation of samples in a large phase space.

204 pages, 100+ figures, 30+ tables; v2: matches published version

References in corpus (41)

• XGBoost: A Scalable Tree Boosting System
• Normalizing Flows: An Introduction and Review of Current Methods
• Topological cell clustering in the ATLAS calorimeters and its performance in LHC Run 1
• CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks
• Accelerating Science with Generative Adversarial Networks: An Application to 3D Particle Showers in Multi-Layer Calorimeters
• The Machine Learning Landscape of Top Taggers
• AtlFast3: the next generation of fast simulation in ATLAS
• Classifying Anomalies THrough Outer Density Estimation (CATHODE)
• Score-based Generative Models for Calorimeter Shower Simulation
• Invertible Networks or Partons to Detector and Back Again
• CaloFlow: Fast and Accurate Generation of Calorimeter Showers with Normalizing Flows
• How to GAN away Detector Effects
• Scikit-dimension: a Python package for intrinsic dimension estimation
• CaloClouds: Fast Geometry-Independent Highly-Granular Calorimeter Simulation
• Evaluating generative models in high energy physics
• L2LFlows: Generating High-Fidelity 3D Calorimeter Images
• FETA: Flow-Enhanced Transportation for Anomaly Detection
• DCTRGAN: Improving the Precision of Generative Models with Reweighting
• Learning to Simulate High Energy Particle Collisions from Unlabeled Data
• How to Understand Limitations of Generative Networks
• ν-Flows: Conditional Neutrino Regression
• CaloScore v2: Single-shot Calorimeter Shower Simulation with Diffusion Models
• Deep generative models for fast photon shower simulation in ATLAS
• Inductive Simulation of Calorimeter Showers with Normalizing Flows
• Towards a Deep Learning Model for Hadronization
• The Interplay of Machine Learning--based Resonant Anomaly Detection Methods
• Deep Generative Models for Detector Signature Simulation: A Taxonomic Review
• Returning CP-Observables to The Frames They Belong
• Improving Generative Model-based Unfolding with Schrödinger Bridges
• CaloFlow for CaloChallenge Dataset 1
• The Landscape of Unfolding with Machine Learning
• CaloDREAM -- Detector Response Emulation via Attentive flow Matching
• Precision-Machine Learning for the Matrix Element Method
• Normalizing Flows for High-Dimensional Detector Simulations
• Calorimeter shower superresolution
• Convolutional L2LFlows: Generating Accurate Showers in Highly Granular Calorimeters Using Convolutional Normalizing Flows
• CaloShowerGAN, a Generative Adversarial Networks model for fast calorimeter shower simulation
• Towards a data-driven model of hadronization using normalizing flows
• Full Event Particle-Level Unfolding with Variable-Length Latent Variational Diffusion
• CaloGraph: Graph-based diffusion model for fast shower generation in calorimeters with irregular geometry
• GalaxyFlow: Upsampling Hydrodynamical Simulations for Realistic Mock Stellar Catalogs

Cited by in corpus (6)

• Amplitude Uncertainties Everywhere All at Once
• BitHEP -- The Limits of Low-Precision ML in HEP
• Extrapolating Jet Radiation with Autoregressive Transformers
• CaloHadronic: a diffusion model for the generation of hadronic showers
• Ultra Fast Calorimeter Simulation with Generative Machine Learning on FPGAs
• CaloClouds3: Ultra-Fast Geometry-Independent Highly-Granular Calorimeter Simulation