5 citations · 11 across the 20 of their papers we have counts for
20 papers
Tracing Generated Samples to Training-Data Clusters in Flow-Matching Models
Rania Briq, Ohad Fried, Michael Kamp +1
Understanding which training samples influence a generated image is an important problem in generative modeling. In flow matching, training samples influence the generated image th…
Fourier Neural Operators for Rayleigh-Bénard Convection
Chelsea Maria John, Thibaut Lunet, Sebastian Götschel +3
We propose an improved Fourier Neural Operator (FNO) for modeling two-dimensional Rayleigh-Bénard convection by predicting time increments instead of full solutions, achieving high…
Exploring and Exploiting Stability in Latent Flow Matching
Rania Briq, Michael Kamp, Ohad Fried +2
In this work, we show that Latent Flow-Matching (LFM) models are robust to different types of perturbations, including data reduction and model capacity shrinkage. We characterize…
Machine Learning for neutron source distributions
Jose Ignacio Robledo, Norberto Schmidt, Klaus Lieutenant +3
In light of the recent advancements in machine learning, we propose a novel approach to neutron source distribution estimation through the utilisation of probabilistic generative m…
The Amazing Stability of Flow Matching
Rania Briq, Michael Kamp, Ohad Fried +2
The success of deep generative models in generating high-quality and diverse samples is often attributed to particular architectures and large training datasets. In this paper, we…
LEPA: Learning Geometric Equivariance in Satellite Remote Sensing Data with a Predictive Architecture
Erik Scheurer, Rocco Sedona, Stefan Kesselheim +1
Geospatial foundation models provide precomputed embeddings that serve as compact feature vectors for large-scale satellite remote sensing data. While these embeddings can reduce d…
Optimal Scaling Needs Optimal Norm
Oleg Filatov, Jiangtao Wang, Jan Ebert +1
Despite recent progress in optimal hyperparameter transfer under model and dataset scaling, no unifying explanatory principle has been established. For Adam and Scion optimizers, w…
Training LLMs on HPC Systems: Best Practices from the OpenGPT-X Project
Carolin Penke, Chelsea Maria John, Jan Ebert +2
The training of large language models (LLMs) requires substantial computational resources, complex software stacks, and carefully designed workflows to achieve scalability and effi…
Memory and Bandwidth are All You Need for Fully Sharded Data Parallel
Jiangtao Wang, Jan Ebert, Oleg Filatov +1
Transformer models have revolutionized a wide spectrum of disciplines, especially in language processing. The recent success has proven that model size scalability is crucial for a…
Lossy Neural Compression for Geospatial Analytics: A Review
Carlos Gomes, Isabelle Wittmann, Damien Robert +24
Over the past decades, there has been an explosion in the amount of available Earth Observation (EO) data. The unprecedented coverage of the Earth's surface and atmosphere by satel…
Polynomial, trigonometric, and tropical activations
Ismail Khalfaoui-Hassani, Stefan Kesselheim
Which functions can be used as activations in deep neural networks? This article explores families of functions based on orthonormal bases, including the Hermite polynomial basis a…
The Artificial Scientist -- in-transit Machine Learning of Plasma Simulations
Jeffrey Kelling, Vicente Bolea, Michael Bussmann +19
Increasing HPC cluster sizes and large-scale simulations that produce petabytes of data per run, create massive IO and storage challenges for analysis. Deep learning-based techniqu…
Scaling Image Tokenizers with Grouped Spherical Quantization
Jiangtao Wang, Zhen Qin, Yifan Zhang +4
Vision tokenizers have gained a lot of attraction due to their scalability and compactness; previous works depend on old-school GAN-based hyperparameters, biased comparisons, and a…
Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit
Oleg Filatov, Jan Ebert, Jiangtao Wang +1
One of the main challenges in optimal scaling of large language models (LLMs) is the prohibitive cost of hyperparameter tuning, particularly learning rate and batch size . W…
Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs
Mehdi Ali, Michael Fromm, Klaudia Thellmann +38
We present two multilingual LLMs, Teuken 7B-base and Teuken 7B-instruct, designed to embrace Europe's linguistic diversity by supporting all 24 official languages of the European U…
Tokenizer Choice For LLM Training: Negligible or Crucial?
Mehdi Ali, Michael Fromm, Klaudia Thellmann +18
The recent success of Large Language Models (LLMs) has been predominantly driven by curating the training dataset composition, scaling of model architectures and dataset sizes and…
Physics informed Neural Networks applied to the description of wave-particle resonance in kinetic simulations of fusion plasmas
Jai Kumar, David Zarzoso, Virginie Grandgirard +2
The Vlasov-Poisson system is employed in its reduced form version (1D1V) as a test bed for the applicability of Physics Informed Neural Network (PINN) to the wave-particle resonanc…
A Comparative Study on Generative Models for High Resolution Solar Observation Imaging
Mehdi Cherti, Alexander Czernik, Stefan Kesselheim +2
Solar activity is one of the main drivers of variability in our solar system and the key source of space weather phenomena that affect Earth and near Earth space. The extensive rec…
Hearts Gym: Learning Reinforcement Learning as a Team Event
Jan Ebert, Danimir T. Doncevic, Ramona Kloß +1
Amidst the COVID-19 pandemic, the authors of this paper organized a Reinforcement Learning (RL) course for a graduate school in the field of data science. We describe the strategy…
JUWELS Booster -- A Supercomputer for Large-Scale AI Research
Stefan Kesselheim, Andreas Herten, Kai Krajsek +15
In this article, we present JUWELS Booster, a recently commissioned high-performance computing system at the Jülich Supercomputing Center. With its system architecture, most import…