24 citations · 90 across the 25 of their papers we have counts for
5 papers · 1 filter
Removing Dust from CMB Observations with Diffusion Models
David Heurtel-Depeiges, Blakesley Burkhart, Ruben Ohana +1
In cosmology, the quest for primordial -modes in cosmic microwave background (CMB) observations has highlighted the critical need for a refined model of the Galactic dust foregr…
Multiple Physics Pretraining for Physical Surrogate Models
Michael McCabe, Bruno Régaldo-Saint Blancard, Liam Holden Parker +11
We introduce multiple physics pretraining (MPP), an autoregressive task-agnostic pretraining approach for physical surrogate modeling of spatiotemporal systems with transformers. I…
xVal: A Continuous Numerical Tokenization for Scientific Language Models
Siavash Golkar, Mariel Pettee, Michael Eickenberg +11
Due in part to their discontinuous and discrete default encodings for numbers, Large Language Models (LLMs) have not yet been commonly used to process numerically-dense scientific…
Linear Optical Random Projections Without Holography
Ruben Ohana, Daniel Hesslow, Daniel Brunner +2
We introduce a novel method to perform linear optical random projections without the need for holography. Our method consists of a computationally trivial combination of multiple i…
MoMo: Momentum Models for Adaptive Learning Rates
Fabian Schaipp, Ruben Ohana, Michael Eickenberg +2
Training a modern machine learning architecture on a new task requires extensive learning-rate tuning, which comes at a high computational cost. Here we develop new Polyak-type ada…