On the impact of selected modern deep-learning techniques to the performance and celerity of classification models in an experimental high-energy physics use case

Abstract

Beginning from a basic neural-network architecture, we test the potential benefits offered by a range of advanced techniques for machine learning, in particular deep learning, in the context of a typical classification problem encountered in the domain of high-energy physics, using a well-studied dataset: the 2014 Higgs ML Kaggle dataset. The advantages are evaluated in terms of both performance metrics and the time required to train and apply the resulting models. Techniques examined include domain-specific data-augmentation, learning rate and momentum scheduling, (advanced) ensembling in both model-space and weight-space, and alternative architectures and connection methods. Following the investigation, we arrive at a model which achieves equal performance to the winning solution of the original Kaggle challenge, whilst being significantly quicker to train and apply, and being suitable for use with both GPU and CPU hardware setups. These reductions in timing and hardware requirements potentially allow the use of more powerful algorithms in HEP analyses, where models must be retrained frequently, sometimes at short notice, by small groups of researchers with limited hardware resources. Additionally, a new wrapper library for PyTorch called LUMIN is presented, which incorporates all of the techniques studied.

Preprint V4: Fixing typographical error and correcting two plots. Mach. Learn.: Sci. Technol (2020)

References in corpus (8)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• The NumPy array: a structure for efficient numerical computation
• Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
• Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
• Wide & Deep Learning for Recommender Systems
• Regularization for Deep Learning: A Taxonomy
• SuperTML: Two-Dimensional Word Embedding for the Precognition on Structured Tabular Data

Cited by in corpus (7)

• Boosted decision trees in the era of new physics: a smuon analysis case study
• Search for nonresonant Higgs boson pair production in final state with two bottom quarks and two tau leptons in proton-proton collisions at $\sqrt{s}$ = 13 TeV
• Self-Supervised Learning on MeerKAT Wide-Field Continuum Images
• Advances in Multi-Variate Analysis Methods for New Physics Searches at the Large Hadron Collider
• Improving smuon searches with Neural Networks
• How to use Machine Learning to improve the discrimination between signal and background at particle colliders
• Probing new physics in the Boosted $HH\to b\bar{b}\gamma \gamma$ channel at the LHC