319 citations · 1.6k across the 35 of their papers we have counts for
5 papers · 1 filter
OReole-FM: successes and challenges toward billion-parameter foundation models for high-resolution satellite imagery
Philipe Dias, Aristeidis Tsaris, Jordan Bowman +4
While the pretraining of Foundation Models (FMs) for remote sensing (RS) imagery is on the rise, models remain restricted to a few hundred million parameters. Scaling models to bil…
Scalable Artificial Intelligence for Science: Perspectives, Methods and Exemplars
Wesley Brewer, Aditya Kashi, Sajal Dash +4
In a post-ChatGPT world, this paper explores the potential of leveraging scalable artificial intelligence for scientific discovery. We propose that scaling up artificial intelligen…
Sequence Length Scaling in Vision Transformers for Scientific Images on Frontier
Aristeidis Tsaris, Chengming Zhang, Xiao Wang +9
Vision Transformers (ViTs) are pivotal for foundational models in scientific imagery, including Earth science applications, due to their capability to process large sequence length…
Pretraining Billion-scale Geospatial Foundational Models on Frontier
Aristeidis Tsaris, Philipe Ambrozio Dias, Abhishek Potnis +3
As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples incre…
ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability
Xiao Wang, Siyan Liu, Aristeidis Tsaris +8
Earth system predictability is challenged by the complexity of environmental dynamics and the multitude of variables involved. Current AI foundation models, although advanced by le…