9 papers
SPADE: Split-and-Delay Embeddings for Autoregressive High-Granularity Calorimeter Simulation
Joschka Birk, Frank Gaede, Anna Hallin +3
We introduce SPADE (SPlit And Delay Embeddings), an autoregressive transformer for sequences whose tokens carry multiple features. Rather than embedding these features jointly, SPA…
Neural Scaling Laws for Jet Generation
Oz Amram, Darius A. Faroughy, Tjarko Gerdes +5
Recently observed empirical scaling laws describe the performance of foundation-type models as three independent key quantities -- dataset size, compute, and model parameters -- ar…
Agents of Discovery
Sascha Diefenbacher, Anna Hallin, Gregor Kasieczka +3
The substantial data volumes encountered in modern particle physics and other domains of fundamental physics research allow (and require) the use of increasingly complex data analy…
Foundation models for high-energy physics
Anna Hallin
The rise of foundation models -- large, pretrained machine learning models that can be finetuned to a variety of tasks -- has revolutionized the fields of natural language processi…
Enhancing next token prediction based pre-training for jet foundation models
Joschka Birk, Anna Hallin, Gregor Kasieczka +3
Next token prediction is an attractive pre-training task for jet foundation models, in that it is simulation free and enables excellent generative capabilities that can transfer ac…
Aspen Open Jets: Unlocking LHC Data for Foundation Models in Particle Physics
Oz Amram, Luca Anzalone, Joschka Birk +7
Foundation models are deep learning models pre-trained on large amounts of data which are capable of generalizing to multiple datasets and/or downstream tasks. This work demonstrat…