Refine Neutrino Events Reconstruction with BEiT-3

Abstract

Neutrino Events Reconstruction has always been crucial for IceCube Neutrino Observatory. In the Kaggle competition "IceCube -- Neutrinos in Deep Ice", many solutions use Transformer. We present ISeeCube, a pure Transformer model based on TorchScale (the backbone of BEiT-3). When having relatively same amount of total trainable parameters, our model outperforms the 2nd place solution. By using TorchScale, the lines of code drop sharply by about 80% and a lot of new methods can be tested by simply adjusting configs. We compared two fundamental models for predictions on a continuous space, regression and classification, trained with MSE Loss and CE Loss respectively. We also propose a new metric, overlap ratio, to evaluate the performance of the model. Since the model is simple enough, it has the potential to be used for more purposes such as energy reconstruction, and many new methods such as combining it with GraphNeT can be tested more easily. The code and pretrained models are available at https://github.com/chenlinear/ISeeCube

17 pages, 7 figures

References in corpus (18)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Gaussian Error Linear Units (GELUs)
• The IceCube Neutrino Observatory: Instrumentation and Online Systems
• Observation of high-energy neutrinos from the Galactic plane
• Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks
• BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers
• A Convolutional Neural Network based Cascade Reconstruction for the IceCube Neutrino Observatory
• Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs
• Vision Transformers Need Registers
• Semantic Segmentation with a Sparse Convolutional Neural Network for Event Reconstruction in MicroBooNE
• Graph Neural Networks for Low-Energy Event Classification & Reconstruction in IceCube
• Event reconstruction for KM3NeT/ORCA using convolutional neural networks
• A flexible event reconstruction based on machine learning and likelihood principles
• Combining Maximum-Likelihood with Deep Learning for Event Reconstruction in IceCube
• Foundation Transformers
• TorchScale: Transformers at Scale
• 2D Convolutional Neural Network for Event Reconstruction in IceCube DeepCore
• IceCube -- Neutrinos in Deep Ice The Top 3 Solutions from the Public Kaggle Competition