BitHEP -- The Limits of Low-Precision ML in HEP

Abstract

The increasing complexity of modern neural network architectures demands fast and memory-efficient implementations to mitigate computational bottlenecks. In this work, we evaluate the recently proposed BitNet architecture in HEP applications, assessing its performance in classification, regression, and generative modeling tasks. Specifically, we investigate its suitability for quark-gluon discrimination, SMEFT parameter estimation, and detector simulation, comparing its efficiency and accuracy to state-of-the-art methods. Our results show that while BitNet consistently performs competitively in classification tasks, its performance in regression and generation varies with the size and type of the network, highlighting key limitations and potential areas for improvement.

20 pages, 6 figures. v2: accepted for publication

References in corpus (164)

• FastJet user manual
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• NICE: Non-linear Independent Components Estimation
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• Ternary Weight Networks
• Fast inference of deep neural networks in FPGAs for particle physics
• ParticleNet: Jet Tagging via Particle Clouds
• CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks
• Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis
• Deep learning in color: towards automated quark/gluon jet discrimination
• Accelerating Science with Generative Adversarial Networks: An Application to 3D Particle Showers in Multi-Layer Calorimeters
• Energy Flow Networks: Deep Sets for Particle Jets
• Event Generation with Normalizing Flows
• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• Mining gold from implicit models to improve likelihood-free inference
• Constraining Effective Field Theories with Machine Learning
• Getting High: High Fidelity Simulation of High Granularity Calorimeters with High Speed
• A Guide to Constraining Effective Field Theories with Machine Learning
• Fast convolutional neural networks on FPGAs with hls4ml
• Approximating Likelihood Ratios with Calibrated Discriminative Classifiers
• JEDI-net: a jet identification algorithm based on interaction networks
• An Efficient Lorentz Equivariant Graph Neural Network for Jet Tagging
• Precise simulation of electromagnetic calorimeter showers using a Wasserstein Generative Adversarial Network
• Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML
• JUNIPR: a Framework for Unsupervised Machine Learning in Particle Physics
• AtlFast3: the next generation of fast simulation in ATLAS
• Machine Learning and LHC Event Generation
• Calorimetry with Deep Learning: Particle Simulation and Reconstruction for Collider Physics
• i-flow: High-dimensional Integration and Sampling with Normalizing Flows
• Score-based Generative Models for Calorimeter Shower Simulation
• Exploring phase space with Neural Importance Sampling
• CaloFlow: Fast and Accurate Generation of Calorimeter Showers with Normalizing Flows
• Neural Networks for Full Phase-space Reweighting and Parameter Tuning
• Diboson Interference Resurrection
• Recursive Neural Networks in Quark/Gluon Tagging
• Fast inference of Boosted Decision Trees in FPGAs for particle physics
• Autoencoders on FPGAs for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider
• Accelerating Monte Carlo event generation -- rejection sampling using neural network event-weight estimates
• Decoding Photons: Physics in the Latent Space of a BIB-AE Generative Network
• Neural Network-Based Approach to Phase Space Integration
• A factorisation-aware Matrix element emulator
• Controlling Physical Attributes in GAN-Accelerated Simulation of Electromagnetic Calorimeters
• Fast Point Cloud Generation with Diffusion Models in High Energy Physics
• Quark-Gluon Tagging: Machine Learning vs Detector
• CaloClouds: Fast Geometry-Independent Highly-Granular Calorimeter Simulation
• Optimising simulations for diphoton production at hadron colliders using amplitude neural networks
• Evaluating generative models in high energy physics
• Targeting Multi-Loop Integrals with Neural Networks
• Point Cloud Transformers applied to Collider Physics
• Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference
• EPiC-GAN: Equivariant Point Cloud Generation for Particle Jets
• Distance-Weighted Graph Neural Networks on FPGAs for Real-Time Particle Reconstruction in High Energy Physics
• PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics
• MadNIS -- Neural Multi-Channel Importance Sampling
• Particle Transformer for Jet Tagging
• L2LFlows: Generating High-Fidelity 3D Calorimeter Images
• Reweighting a parton shower using a neural network: the final-state case
• A neural simulation-based inference approach for characterizing the Galactic Center $\gamma$-ray excess
• Using neural networks for efficient evaluation of high multiplicity scattering amplitudes
• Improved Neural Network Monte Carlo Simulation
• Machine learning the Higgs boson-top quark CP phase
• How to Understand Limitations of Generative Networks
• E Pluribus Unum Ex Machina: Learning from Many Collider Events at Once
• Two Invertible Networks for the Matrix Element Method
• Measuring QCD Splittings with Invertible Networks
• Graph Neural Networks for Charged Particle Tracking on FPGAs
• Quarks and gluons in the Lund plane
• The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
• CaloMan: Fast generation of calorimeter showers with density estimation on learned manifolds
• Modern Machine Learning for LHC Physicists
• Towards Machine Learning Analytics for Jet Substructure
• Constraining CP-violation in the Higgs-top-quark interaction using machine-learning-based inference
• Deep generative models for fast photon shower simulation in ATLAS
• Accelerated Charged Particle Tracking with Graph Neural Networks on FPGAs
• Inductive Simulation of Calorimeter Showers with Normalizing Flows
• OmniJet-$\alpha$: The first cross-task foundation model for particle physics
• Nanosecond machine learning event classification with boosted decision trees in FPGA for high energy physics
• Efficient phase-space generation for hadron collider event simulation
• Deep Generative Models for Detector Signature Simulation: A Taxonomic Review
• Quark-Gluon Jet Discrimination Using Convolutional Neural Networks
• BitNet: Scaling 1-bit Transformers for Large Language Models
• Unweighting multijet event generation using factorisation-aware neural networks
• One-loop matrix element emulation with factorisation awareness
• CaloFlow for CaloChallenge Dataset 1
• Ultra-High-Resolution Detector Simulation with Intra-Event Aware GAN and Self-Supervised Relational Reasoning
• Bayesian Inference analysis of jet quenching using inclusive jet and hadron suppression measurements
• Black-Box Optimization with Local Generative Surrogates
• Learning Likelihood Ratios with Neural Network Classifiers
• CaloDREAM -- Detector Response Emulation via Attentive flow Matching
• Cubic-Spline Flows
• Precision-Machine Learning for the Matrix Element Method
• Tree boosting for learning EFT parameters
• Solving Key Challenges in Collider Physics with Foundation Models
• A Method to Simultaneously Facilitate All Jet Physics Tasks
• A Lorentz-Equivariant Transformer for All of the LHC
• Re-Simulation-based Self-Supervised Learning for Pre-Training Foundation Models
• Quark Gluon Jet Discrimination with Weakly Supervised Learning
• Tagging more quark jet flavours at FCC-ee at 91 GeV with a transformer-based neural network
• Normalizing Flows for High-Dimensional Detector Simulations
• Parameter Estimation using Neural Networks in the Presence of Detector Effects
• Calorimeter shower superresolution
• EPiC-ly Fast Particle Cloud Generation with Flow-Matching and Diffusion
• Refining Fast Calorimeter Simulations with a Schrödinger Bridge
• Convolutional L2LFlows: Generating Accurate Showers in Highly Granular Calorimeters Using Convolutional Normalizing Flows
• Differentiable MadNIS-Lite
• A method for approximating optimal statistical significances with machine-learned likelihoods
• Jet Tagging with More-Interaction Particle Transformer
• Loop Amplitudes from Precision Networks
• Fast Muon Tracking with Machine Learning Implemented in FPGA
• Muon trigger with fast Neural Networks on FPGA, a demonstrator
• Nanosecond machine learning regression with deep boosted decision trees in FPGA for high energy physics
• Targeted Likelihood-Free Inference of Dark Matter Substructure in Strongly-Lensed Galaxies
• Generative Machine Learning for Detector Response Modeling with a Conditional Normalizing Flow
• Parameter Inference from Event Ensembles and the Top-Quark Mass
• Real-time Graph Building on FPGAs for Machine Learning Trigger Applications in Particle Physics
• Emulator-based Bayesian Inference on Non-Proportional Scintillation Models by Compton-Edge Probing
• Simulation-based inference in the search for CP violation in leptonic WH production
• Reduced Precision Strategies for Deep Learning: A High Energy Physics Generative Adversarial Network Use Case
• Systematic Quark/Gluon Identification with Ratios of Likelihoods
• Is infrared-collinear safe information all you need for jet classification?
• Constraining the Higgs Potential with Neural Simulation-based Inference for Di-Higgs Production
• CaloChallenge 2022: A Community Challenge for Fast Calorimeter Simulation
• Advancing Tools for Simulation-Based Inference
• Accurate Surrogate Amplitudes with Calibrated Uncertainties
• Aspen Open Jets: Unlocking LHC Data for Foundation Models in Particle Physics
• Accelerating HEP simulations with Neural Importance Sampling
• PIPPIN: Generating variable length full events from partons
• Energy Efficiency trends in HPC: what high-energy and astrophysicists need to know
• HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAs
• Jet substructure observables for jet quenching in Quark Gluon Plasma: a Machine Learning driven analysis
• Fast Neural Network Inference on FPGAs for Triggering on Long-Lived Particles at Colliders
• A rotation-equivariant graph neural network for learning hadronic SMEFT effects
• Is Tokenization Needed for Masked Particle Modelling?
• Optimal Equivariant Architectures from the Symmetries of Matrix-Element Likelihoods
• Improvement and generalization of ABCD method with Bayesian inference
• Hierarchical Neural Simulation-Based Inference Over Event Ensembles
• Machine Learning and Kalman Filtering for Nanomechanical Mass Spectrometry
• Bayesian Active Search on Parameter Space: a 95 GeV Spin-0 Resonance in the ($B-L$)SSM
• Jet tagging in the Lund plane with graph networks
• Bumblebee: Foundation Model for Particle Physics Discovery
• Reconstructing axion-like particles from beam dumps with simulation-based inference
• Quark-versus-gluon tagging in CMS Open Data with CWoLa and TopicFlow
• Charged Particle Tracking with Machine Learning on FPGAs
• Bitnet.cpp: Efficient Edge Inference for Ternary LLMs
• Application of Particle Transformer to quark flavor tagging in the ILC project
• Hierarchical High-Point Energy Flow Network for Jet Tagging
• Scaling Up Diffusion and Flow-based XGBoost Models
• Quark/Gluon Discrimination and Top Tagging with Dual Attention Transformer
• Pretrained Event Classification Model for High Energy Physics Analysis
• Nanosecond hardware regression trees in FPGA at the LHC
• Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4ml
• New Machine Learning Techniques for Simulation-Based Inference: InferoStatic Nets, Kernel Score Estimation, and Kernel Likelihood Ratio Estimation
• Conditional Deep Generative Models for Simultaneous Simulation and Reconstruction of Entire Events
• Generating and refining particle detector simulations using the Wasserstein distance in adversarial networks
• Profile Likelihoods on ML-Steroids
• From Optimal Observables to Machine Learning: an Effective-Field-Theory Analysis of $e^{+}{e}^{-}\to W^{+}{W}^{-}$ at Future Lepton Colliders
• Deep Learning for the Matrix Element Method
• Implementation of a framework for deploying AI inference engines in FPGAs
• Machine-Learned Exclusion Limits without Binning
• A multicategory jet image classification framework using deep neural network
• Ultra-low latency quantum-inspired machine learning predictors implemented on FPGA
• Identifying the Quantum Properties of Hadronic Resonances using Machine Learning
• Two Watts is All You Need: Enabling In-Detector Real-Time Machine Learning for Neutrino Telescopes Via Edge Computing
• Reconstructing hadronically decaying tau leptons with a jet foundation model

Cited by in corpus (2)

• Ultra Fast Calorimeter Simulation with Generative Machine Learning on FPGAs
• Graph theory inspired anomaly detection at the LHC