8 papers
A Scientific Human-Agent Reproduction Pipeline
Joschka Birk, Gregor Kasieczka, Siddharth Mishra-Sharma +3
Reproducing scientific analyses is essential for preserving knowledge, building extensible codebases, and deepening researcher understanding - yet the effort often outweighs its ac…
Pre-Training for Simulation-Based Science: A Study on Jet Foundation Model Training Objectives
Ibrahim Elsharkawy, Joschka Birk, Vinicius Mikuni +3
Foundation models (FMs) trained on large datasets and fine-tuned on downstream tasks have emerged as a powerful paradigm in AI for science. Industrial FMs are typically trained usi…
SPADE: Split-and-Delay Embeddings for Autoregressive High-Granularity Calorimeter Simulation
Joschka Birk, Frank Gaede, Anna Hallin +3
We introduce SPADE (SPlit And Delay Embeddings), an autoregressive transformer for sequences whose tokens carry multiple features. Rather than embedding these features jointly, SPA…
Enhancing next token prediction based pre-training for jet foundation models
Joschka Birk, Anna Hallin, Gregor Kasieczka +3
Next token prediction is an attractive pre-training task for jet foundation models, in that it is simulation free and enables excellent generative capabilities that can transfer ac…
Aspen Open Jets: Unlocking LHC Data for Foundation Models in Particle Physics
Oz Amram, Luca Anzalone, Joschka Birk +7
Foundation models are deep learning models pre-trained on large amounts of data which are capable of generalizing to multiple datasets and/or downstream tasks. This work demonstrat…
OmniJet-: Learning point cloud calorimeter simulations using generative transformers
Joschka Birk, Frank Gaede, Anna Hallin +3
We show the first use of generative transformers for generating calorimeter showers as point clouds in a high-granularity calorimeter. Using the tokenizer and generative part of th…