8 citations · 10 across the 8 of their papers we have counts for
11 papers
From Pilots to Production: Lessons in Cross-Institutional Federated Training and Artificial Intelligence for Science
Olivera Kotevska, Max Carlson, Yan Gao +10
Many of the most valuable scientific datasets cannot be centralized: they are proprietary, export-controlled, classified, or bound by data-sovereignty restrictions. This inverts th…
FLYING SERVING: On-the-Fly Parallelism Switching for Large Language Model Serving
Shouwei Gao, Junqi Yin, Feiyi Wang +1
Production LLM serving must simultaneously deliver high throughput, low latency, and sufficient context capacity under non-stationary traffic and mixed request requirements. Data p…
OmniFed: A Modular Framework for Configurable Federated Learning from Edge to HPC
Sahil Tyagi, Andrei Cozma, Olivera Kotevska +1
Federated Learning (FL) is critical for edge and High Performance Computing (HPC) where data is not centralized and privacy is crucial. We present OmniFed, a modular framework desi…
X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms
Yueming Yuan, Ahan Gupta, Jianping Li +3
Emerging expert-specialized Mixture-of-Experts (MoE) architectures, such as DeepSeek-MoE, deliver strong model quality through fine-grained expert segmentation and large top-k rout…
Distributed Cross-Channel Hierarchical Aggregation for Foundation Models
Aristeidis Tsaris, Isaac Lyngaas, John Lagregren +6
Vision-based scientific foundation models hold significant promise for advancing scientific discovery and innovation. This potential stems from their ability to aggregate images fr…
Scalable Artificial Intelligence for Science: Perspectives, Methods and Exemplars
Wesley Brewer, Aditya Kashi, Sajal Dash +4
In a post-ChatGPT world, this paper explores the potential of leveraging scalable artificial intelligence for scientific discovery. We propose that scaling up artificial intelligen…
AI-coupled HPC Workflow Applications, Middleware and Performance
Wes Brewer, Ana Gainaru, Frédéric Suter +3
AI integration is revolutionizing the landscape of HPC simulations, enhancing the importance, use, and performance of AI-driven HPC workflows. This paper surveys the diverse and ra…
Sequence Length Scaling in Vision Transformers for Scientific Images on Frontier
Aristeidis Tsaris, Chengming Zhang, Xiao Wang +9
Vision Transformers (ViTs) are pivotal for foundational models in scientific imagery, including Earth science applications, due to their capability to process large sequence length…
Pretraining Billion-scale Geospatial Foundational Models on Frontier
Aristeidis Tsaris, Philipe Ambrozio Dias, Abhishek Potnis +3
As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples incre…
Optimizing Distributed Training on Frontier for Large Language Models
Sajal Dash, Isaac Lyngaas, Junqi Yin +5
Large language models (LLMs) have demonstrated remarkable success as foundational models, benefiting various downstream applications through fine-tuning. Recent studies on loss sca…
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang +89
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences. This could herald a new era of scient…