2 papers
stat.ML2024
xVal: A Continuous Numerical Tokenization for Scientific Language Models
Siavash Golkar, Mariel Pettee, Michael Eickenberg +11
Due in part to their discontinuous and discrete default encodings for numbers, Large Language Models (LLMs) have not yet been commonly used to process numerically-dense scientific…
cs.LG2024
Multiple Physics Pretraining for Physical Surrogate Models
Michael McCabe, Bruno Régaldo-Saint Blancard, Liam Holden Parker +11
We introduce multiple physics pretraining (MPP), an autoregressive task-agnostic pretraining approach for physical surrogate modeling of spatiotemporal systems with transformers. I…