8 citations · 12 across the 9 of their papers we have counts for
10 papers
Scaling Up Data Parallelism in Decentralized Deep Learning
Bing Xie, Junqi Yin, Zhenyu Zhou +2
Although it has been extensively explored in theory, decentralized learning is not yet green-lighted for production use, largely due to a lack of stability, scalability, and genera…
Decoding Memories: An Efficient Pipeline for Self-Consistency Hallucination Detection
Weizhi Gao, Xiaorui Liu, Feiyi Wang +2
Large language models (LLMs) have demonstrated impressive performance in both research and real-world applications, but they still struggle with hallucination. Existing hallucinati…
Pixel-Resolved Long-Context Learning for Turbulence at Exascale: Resolving Small-scale Eddies Toward the Viscous Limit
Junqi Yin, Mijanur Palash, M. Paul Laiu +6
Turbulence plays a crucial role in multiphysics applications, including aerodynamics, fusion, and combustion. Accurately capturing turbulence's multiscale characteristics is essent…
Modulated Diffusion: Accelerating Generative Modeling with Modulated Quantization
Weizhi Gao, Zhichao Hou, Junqi Yin +3
Diffusion models have emerged as powerful generative models, but their high computation cost in iterative sampling remains a significant bottleneck. In this work, we present an in-…
Scalable Artificial Intelligence for Science: Perspectives, Methods and Exemplars
Wesley Brewer, Aditya Kashi, Sajal Dash +4
In a post-ChatGPT world, this paper explores the potential of leveraging scalable artificial intelligence for scientific discovery. We propose that scaling up artificial intelligen…
Sequence Length Scaling in Vision Transformers for Scientific Images on Frontier
Aristeidis Tsaris, Chengming Zhang, Xiao Wang +9
Vision Transformers (ViTs) are pivotal for foundational models in scientific imagery, including Earth science applications, due to their capability to process large sequence length…
Pretraining Billion-scale Geospatial Foundational Models on Frontier
Aristeidis Tsaris, Philipe Ambrozio Dias, Abhishek Potnis +3
As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples incre…
ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability
Xiao Wang, Siyan Liu, Aristeidis Tsaris +8
Earth system predictability is challenged by the complexity of environmental dynamics and the multitude of variables involved. Current AI foundation models, although advanced by le…
Optimizing Distributed Training on Frontier for Large Language Models
Sajal Dash, Isaac Lyngaas, Junqi Yin +5
Large language models (LLMs) have demonstrated remarkable success as foundational models, benefiting various downstream applications through fine-tuning. Recent studies on loss sca…
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang +89
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences. This could herald a new era of scient…