Publications (39)
FIRST: Federated Inference Resource Scheduling Toolkit for Scientific AI Model Access
Aditya Tanikanti, Benoit Côté, Yanfei Guo +9
We present the Federated Inference Resource Scheduling Toolkit (FIRST), a framework enabling Inference-as-a-Service across distributed High-Performance Computing (HPC) clusters. FI…
UniFaaS: Programming across Distributed Cyberinfrastructure with Federated Function Serving
Yifei Li, Ryan Chard, Yadu Babuji +3
Modern scientific applications are increasingly decomposable into individual functions that may be deployed across distributed and diverse cyberinfrastructure such as supercomputer…
funcX: Federated Function as a Service for Science
Zhuozhao Li, Ryan Chard, Yadu Babuji +9
funcX is a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. Unlike centralized FaaS systems, funcX…
Accelerated, Scalable and Reproducible AI-driven Gravitational Wave Detection
E. A. Huerta, Asad Khan, Xiaobo Huang +10
The development of reusable artificial intelligence (AI) models for wider use and rigorous validation by the community promises to unlock new opportunities in multi-messenger astro…
Parsl: Pervasive Parallel Programming in Python
Yadu Babuji, Anna Woodard, Zhuozhao Li +9
High-level programming languages such as Python are increasingly used to provide intuitive interfaces to libraries written in lower-level languages and for assembling applications…
nelli: a lightweight frontend for MLIR
Maksim Levental, Alok Kamatar, Ryan Chard +2
Multi-Level Intermediate Representation (MLIR) is a novel compiler infrastructure that aims to provide modular and extensible components to facilitate building domain specific comp…
Icicle: Scalable Metadata Indexing and Real-Time Monitoring for HPC File Systems
Haochen Pan, Ryan Chard, Song Young Oh +7
Modern HPC file systems can contain billions of files and hundreds of petabytes of data, making even simple questions increasingly intractable to answer. Traditional file system ut…
Targeting SARS-CoV-2 with AI- and HPC-enabled Lead Generation: A First Data Release
Yadu Babuji, Ben Blaiszik, Tom Brettin +15
Researchers across the globe are seeking to rapidly repurpose existing drugs or discover new drugs to counter the the novel coronavirus disease (COVID-19) caused by severe acute re…
DLHub: Model and Data Serving for Science
Ryan Chard, Zhuozhao Li, Kyle Chard +7
While the Machine Learning (ML) landscape is evolving rapidly, there has been a relative lag in the development of the "learning systems" needed to enable broad adoption. Furthermo…
Octopus: Experiences with a Hybrid Event-Driven Architecture for Distributed Scientific Computing
Haochen Pan, Ryan Chard, Sicheng Zhou +7
Scientific research increasingly relies on distributed computational resources, storage systems, networks, and instruments, ranging from HPC and cloud systems to edge devices. Even…
Steering a Fleet: Adaptation for Large-Scale, Workflow-Based Experiments
Jim Pruyne, Valerie Hayot-Sasson, Weijian Zheng +5
Experimental science is increasingly driven by instruments that produce vast volumes of data and thus a need to manage, compute, describe, and index this data. High performance and…
OpenHLS: High-Level Synthesis for Low-Latency Deep Neural Networks for Experimental Science
Maksim Levental, Arham Khan, Ryan Chard +3
In many experiment-driven scientific domains, such as high-energy physics, material science, and cosmology, high data rate experiments impose hard constraints on data acquisition s…
Colmena: Scalable Machine-Learning-Based Steering of Ensemble Simulations for High Performance Computing
Logan Ward, Ganesh Sivaraman, J. Gregory Pauloski +9
Scientific applications that involve simulation ensembles can be accelerated greatly by using experiment design methods to select the best simulations to perform. Methods that use…
Trillion Parameter AI Serving Infrastructure for Scientific Discovery: A Survey and Vision
Nathaniel Hudson, J. Gregory Pauloski, Matt Baughman +13
Deep learning methods are transforming research, enabling new techniques, and ultimately leading to new discoveries. As the demand for more capable AI models continues to grow, we…
Ultrafast Focus Detection for Automated Microscopy
Maksim Levental, Ryan Chard, Kyle Chard +2
Technological advancements in modern scientific instruments, such as scanning electron microscopes (SEMs), have significantly increased data acquisition rates and image resolutions…
Empowering Scientific Workflows with Federated Agents
Alok Kamatar, J. Gregory Pauloski, Yadu Babuji +5
Agentic systems, in which diverse agents cooperate to tackle challenging problems, are exploding in popularity in the AI community. However, existing agentic frameworks take a rela…
RADAR-Radio Afterglow Detection and AI-driven Response: A Federated Framework for Gravitational Wave Event Follow-Up
Parth Patel, Alessandra Corsi, E. A. Huerta +11
The landmark detection of both gravitational waves (GWs) and electromagnetic (EM) radiation from the binary neutron star merger GW170817 has spurred efforts to streamline the follo…
Flight: A FaaS-Based Framework for Complex and Hierarchical Federated Learning
Nathaniel Hudson, Valerie Hayot-Sasson, Yadu Babuji +5
Federated Learning (FL) is a decentralized machine learning paradigm where models are trained on distributed devices and are aggregated at a central server. Existing FL frameworks…
IMPECCABLE: Integrated Modeling PipelinE for COVID Cure by Assessing Better LEads
Aymen Al Saadi, Dario Alfe, Yadu Babuji +33
The drug discovery process currently employed in the pharmaceutical industry typically requires about 10 years and $2-3 billion to deliver one new drug. This is both too expensive…
Towards Online Steering of Flame Spray Pyrolysis Nanoparticle Synthesis
Maksim Levental, Ryan Chard, Joseph A. Libera +8
Flame Spray Pyrolysis (FSP) is a manufacturing technique to mass produce engineered nanoparticles for applications in catalysis, energy materials, composites, and more. FSP instrum…
funcX: A Federated Function Serving Fabric for Science
Ryan Chard, Yadu Babuji, Zhuozhao Li +5
Exploding data volumes and velocities, new computational methods and platforms, and ubiquitous connectivity demand new approaches to computation in the sciences. These new approach…
The Manufacturing Data and Machine Learning Platform: Enabling Real-time Monitoring and Control of Scientific Experiments via IoT
Jakob R. Elias, Ryan Chard, Joseph A. Libera +2
IoT devices and sensor networks present new opportunities for measuring, monitoring, and guiding scientific experiments. Sensors, cameras, and instruments can be combined to provid…
Enabling End-to-End Secure Federated Learning in Biomedical Research on Heterogeneous Computing Environments with APPFLx
Trung-Hieu Hoang, Jordan Fuhrman, Ravi Madduri +8
Facilitating large-scale, cross-institutional collaboration in biomedical machine learning projects requires a trustworthy and resilient federated learning (FL) environment to ensu…
A Data Ecosystem to Support Machine Learning in Materials Science
Ben Blaiszik, Logan Ward, Marcus Schwarting +5
Facilitating the application of machine learning to materials science problems will require enhancing the data ecosystem to enable discovery and collection of data from many source…
Cloud Services Enable Efficient AI-Guided Simulation Workflows across Heterogeneous Resources
Logan Ward, J. Gregory Pauloski, Valerie Hayot-Sasson +7
Applications that fuse machine learning and simulation can benefit from the use of multiple computing resources, with, for example, simulation codes running on highly parallel supe…
Employing Artificial Intelligence to Steer Exascale Workflows with Colmena
Logan Ward, J. Gregory Pauloski, Valerie Hayot-Sasson +6
Computational workflows are a common class of application on supercomputers, yet the loosely coupled and heterogeneous nature of workflows often fails to take full advantage of the…
Experiences with Model Context Protocol Servers for Science and High Performance Computing
Haochen Pan, Ryan Chard, Reid Mello +12
Large language model (LLM)-powered agents are increasingly used to plan and execute scientific workflows, yet most research cyberinfrastructure (CI) exposes heterogeneous APIs and…
FAIR principles for AI models with a practical application for accelerated high energy diffraction microscopy
Nikil Ravi, Pranshu Chaturvedi, E. A. Huerta +7
A concise and measurable set of FAIR (Findable, Accessible, Interoperable and Reusable) principles for scientific data is transforming the state-of-practice for data management and…
Serverless Supercomputing: High Performance Function as a Service for Science
Ryan Chard, Tyler J. Skluzacek, Zhuozhao Li +6
Growing data volumes and velocities are driving exciting new methods across the sciences in which data analytics and machine learning are increasingly intertwined with research. Th…
Linking Scientific Instruments and HPC: Patterns, Technologies, Experiences
Rafael Vescovi, Ryan Chard, Nickolaus Saint +10
Powerful detectors at modern experimental facilities routinely collect data at multiple GB/s. Online analysis methods are needed to enable the collection of only interesting subset…
Parsl+CWL: Towards Combining the Python and CWL Ecosystems
Nishchay Karle, Ben Clifford, Yadu Babuji +3
The Common Workflow Language (CWL) is a widely adopted language for defining and sharing computational workflows. It is designed to be independent of the execution engine on which…
High-Performance Ptychographic Reconstruction with Federated Facilities
Tekin Bicer, Xiaodong Yu, Daniel J. Ching +5
Beamlines at synchrotron light source facilities are powerful scientific instruments used to image samples and observe phenomena at high spatial and temporal resolutions. Typically…
Linking the Dynamic PicoProbe Analytical Electron-Optical Beam Line / Microscope to Supercomputers
Alexander Brace, Rafael Vescovi, Ryan Chard +4
The Dynamic PicoProbe at Argonne National Laboratory is undergoing upgrades that will enable it to produce up to 100s of GB of data per day. While this data is highly important for…
AI-assisted Automated Workflow for Real-time X-ray Ptychography Data Analysis via Federated Resources
Anakha V Babu, Tekin Bicer, Saugat Kandel +7
We present an end-to-end automated workflow that uses large-scale remote compute resources and an embedded GPU platform at the edge to enable AI/ML-accelerated real-time analysis o…
WRATH: Workload Resilience Across Task Hierarchies in Task-based Parallel Programming Frameworks
Sicheng Zhou, Zhuozhao Li, Valérie Hayot-Sasson +6
Failures in Task-based Parallel Programming (TBPP) can severely degrade performance and result in incomplete or incorrect outcomes. Existing failure-handling approaches, including…
Globus Automation Services: Research process automation across the space-time continuum
Ryan Chard, Jim Pruyne, Kurt McKee +5
Research process automation -- the reliable, efficient, and reproducible execution of linked sets of actions on scientific instruments, computers, data stores, and other resources…
APPFLx: Providing Privacy-Preserving Cross-Silo Federated Learning as a Service
Zilinghan Li, Shilan He, Pranshu Chaturvedi +9
Cross-silo privacy-preserving federated learning (PPFL) is a powerful tool to collaboratively train robust and generalized machine learning (ML) models without sharing sensitive (e…
Real-Time Streaming and Event-driven Control of Scientific Experiments
Jakob R. Elias, Ryan Chard, Maksim Levental +3
Advancements in scientific instrument sensors and connected devices provide unprecedented insight into ongoing experiments and present new opportunities for control, optimization,…
Deep learning at the edge enables real-time streaming ptychographic imaging
Anakha V Babu, Tao Zhou, Saugat Kandel +15
Coherent microscopy techniques provide an unparalleled multi-scale view of materials across scientific and technological fields, from structural materials to quantum devices, from…