Publications (171)
AEGIS: Assay-Aware Protocol Validation and Runtime Monitoring for Open-Source Liquid Handling Robots
Priyanka V. Setty, Arvind Ramanathan, Ian Foster +1
Self-driving laboratories increasingly rely on low-cost liquid handlers such as the Opentrons OT-2, which ship without the pressure-based aspiration monitoring of Hamilton or Tecan…
Comprehensive Exploration of Synthetic Data Generation: A Survey
André Bauer, Simon Trapp, Michael Stenger +5
Recent years have witnessed a surge in the popularity of Machine Learning (ML), applied across diverse domains. However, progress is impeded by the scarcity of training data due to…
HydroNet: Benchmark Tasks for Preserving Intermolecular Interactions and Structural Motifs in Predictive and Generative Models for Molecular Data
Sutanay Choudhury, Jenna A. Bilbrey, Logan Ward +5
Intermolecular and long-range interactions are central to phenomena as diverse as gene regulation, topological states of quantum materials, electrolyte transport in batteries, and…
Foundation Models for the Electric Power Grid
Hendrik F. Hamann, Thomas Brunschwiler, Blazhe Gjorgiev +24
Foundation models (FMs) currently dominate news headlines. They employ advanced deep learning architectures to extract structural information autonomously from vast datasets throug…
TaPS: A Performance Evaluation Suite for Task-based Execution Frameworks
J. Gregory Pauloski, Valerie Hayot-Sasson, Maxime Gonthier +5
Task-based execution frameworks, such as parallel programming libraries, computational workflow systems, and function-as-a-service platforms, enable the composition of distinct tas…
BioWorkbench: A High-Performance Framework for Managing and Analyzing Bioinformatics Experiments
Maria Luiza Mondelli, Thiago Magalhães, Guilherme Loss +8
Advances in sequencing techniques have led to exponential growth in biological data, demanding the development of large-scale bioinformatics experiments. Because these experiments…
Connecting Large Language Model Agent to High Performance Computing Resource
Heng Ma, Alexander Brace, Carlo Siebenschuh +3
The Large Language Model agent workflow enables the LLM to invoke tool functions to increase the performance on specific scientific domain questions. To tackle large scale of scien…
CUF-Links: Continuous and Ubiquitous FAIRness Linkages for reproducible research
Ian Foster, Carl Kesselman
Despite much creative work on methods and tools, reproducibility -- the ability to repeat the computational steps used to obtain a research result -- remains elusive. One reason fo…
The Anatomy of the Grid - Enabling Scalable Virtual Organizations
Ian Foster, Carl Kesselman, Steven Tuecke
"Grid" computing has emerged as an important new field, distinguished from conventional distributed computing by its focus on large-scale resource sharing, innovative applications,…
From Knowledge to Action: Outcomes of the 2025 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry
Aritra Roy, Kevin Shen, Andrew MacBride +350
Large language models (LLMs) are rapidly changing how researchers in materials science and chemistry discover, organize, and act on scientific knowledge. This paper analyzes a broa…
In-situ Workflow Auto-tuning via Combining Performance Models of Component Applications
Tong Shu, Yanfei Guo, Justin Wozniak +3
In-situ parallel workflows couple multiple component applications, such as simulation and analysis, via streaming data transfer. in order to avoid data exchange via shared file sys…
Toward Trustworthy Autonomous Science: A Two-Year Community Roadmap
Rafael Ferreira da Silva, Milad Abolhasani, Peter Beaucage +25
The paper updates a community roadmap for autonomous scientific laboratories, emphasizing trust, verification, reproducibility, safety, security, and governance as central challeng…
DLHub: Model and Data Serving for Science
Ryan Chard, Zhuozhao Li, Kyle Chard +7
While the Machine Learning (ML) landscape is evolving rapidly, there has been a relative lag in the development of the "learning systems" needed to enable broad adoption. Furthermo…
Security for Grid Services
Von Welch, Frank Siebenlist, Ian Foster +7
Grid computing is concerned with the sharing and coordinated use of diverse resources in distributed "virtual organizations." The dynamic and multi-institutional nature of these en…
DynoStore: A wide-area distribution system for the management of data over heterogeneous storage
Dante D. Sanchez-Gallegos, J. L. Gonzalez-Compean, Maxime Gonthier +6
Data distribution across different facilities offers benefits such as enhanced resource utilization, increased resilience through replication, and improved performance by processin…
Causal Discovery over High-Dimensional Structured Hypothesis Spaces with Causal Graph Partitioning
Ashka Shah, Adela DePavia, Nathaniel Hudson +2
The aim in many sciences is to understand the mechanisms that underlie the observed distribution of variables, starting from a set of initial hypotheses. Causal discovery allows us…
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang +89
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences. This could herald a new era of scient…
SciQAG: A Framework for Auto-Generated Science Question Answering Dataset with Fine-grained Evaluation
Yuwei Wan, Yixuan Liu, Aswathy Ajith +6
We introduce SciQAG, a novel framework for automatically generating high-quality science question-answer pairs from a large corpus of scientific literature based on large language…
KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks
J. Gregory Pauloski, Qi Huang, Lei Huang +4
Kronecker-factored Approximate Curvature (K-FAC) has recently been shown to converge faster in deep neural network (DNN) training than stochastic gradient descent (SGD); however, K…
Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing
Bill Allcock, Joe Bester, John Bresnahan +7
An emerging class of data-intensive applications involve the geographically dispersed extraction of complex scientific information from very large collections of measured or comput…
To Stream or Not to Stream: Towards A Quantitative Model for Remote HPC Processing Decisions
Flavio Castro, Weijian Zheng, Joaquin Chung +2
Modern scientific instruments generate data at rates that increasingly exceed local compute capabilities and, when paired with the staging and I/O overheads of file-based transfers…
IMPECCABLE: Integrated Modeling PipelinE for COVID Cure by Assessing Better LEads
Aymen Al Saadi, Dario Alfe, Yadu Babuji +33
The drug discovery process currently employed in the pharmaceutical industry typically requires about 10 years and $2-3 billion to deliver one new drug. This is both too expensive…
D-Rex: Heterogeneity-Aware Reliability Framework and Adaptive Algorithms for Distributed Storage
Maxime Gonthier, Dante D. Sanchez-Gallegos, Haochen Pan +8
The exponential growth of data necessitates distributed storage models, such as peer-to-peer systems and data federations. While distributed storage can reduce costs and increase r…
FAIR for AI: An interdisciplinary and international community building perspective
E. A. Huerta, Ben Blaiszik, L. Catherine Brinson +21
A foundational set of findable, accessible, interoperable, and reusable (FAIR) principles were proposed in 2016 as prerequisites for proper data management and stewardship, with th…
Computational Grids
Ian Foster, Carl Kesselman
In this introductory chapter, we lay the groundwork for the rest of the book by providing a more detailed picture of the expected purpose, shape, and architecture of future grid sy…
HACC Cosmological Simulations: First Data Release
Katrin Heitmann, Thomas D. Uram, Hal Finkel +10
We describe the first major public data release from cosmological simulations carried out with Argonne's HACC code. This initial release covers a range of datasets from large gravi…
BFTrainer: Low-Cost Training of Neural Networks on Unfillable Supercomputer Nodes
Zhengchun Liu, Rajkumar Kettimuthu, Michael E. Papka +1
Supercomputer FCFS-based scheduling policies result in many transient idle nodes, a phenomenon that is only partially alleviated by backfill scheduling methods that promote small j…
PSI/J: A Portable Interface for Submitting, Monitoring, and Managing Jobs
Mihael Hategan-Marandiuc, Andre Merzky, Nicholson Collier +10
It is generally desirable for high-performance computing (HPC) applications to be portable between HPC systems, for example to make use of more performant hardware, make effective…
Cloud Services Enable Efficient AI-Guided Simulation Workflows across Heterogeneous Resources
Logan Ward, J. Gregory Pauloski, Valerie Hayot-Sasson +7
Applications that fuse machine learning and simulation can benefit from the use of multiple computing resources, with, for example, simulation codes running on highly parallel supe…
Employing Artificial Intelligence to Steer Exascale Workflows with Colmena
Logan Ward, J. Gregory Pauloski, Valerie Hayot-Sasson +6
Computational workflows are a common class of application on supercomputers, yet the loosely coupled and heterogeneous nature of workflows often fails to take full advantage of the…
A generative artificial intelligence framework based on a molecular diffusion model for the design of metal-organic frameworks for carbon capture
Hyun Park, Xiaoli Yan, Ruijie Zhu +5
Metal-organic frameworks (MOFs) exhibit great promise for CO2 capture. However, finding the best performing materials poses computational and experimental grand challenges in view…
Towards a Modular Architecture for Science Factories
Rafael Vescovi, Tobias Ginsburg, Kyle Hippe +14
Advances in robotic automation, high-performance computing (HPC), and artificial intelligence (AI) encourage us to conceive of science factories: large, general-purpose computation…
14 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathon
Kevin Maik Jablonka, Qianxiang Ai, Alexander Al-Feghali +50
Large-language models (LLMs) such as GPT-4 caught the interest of many scientists. Recent studies suggested that these models could be useful in chemistry and materials science. To…
XaaS: Acceleration as a Service to Enable Productive High-Performance Cloud Computing
Torsten Hoefler, Marcin Copik, Pete Beckman +8
HPC and Cloud have evolved independently, specializing their innovations into performance or productivity. Acceleration as a Service (XaaS) is a recipe to empower both fields with…
Object Proxy Patterns for Accelerating Distributed Applications
J. Gregory Pauloski, Valerie Hayot-Sasson, Logan Ward +4
Workflow and serverless frameworks have empowered new approaches to distributed application design by abstracting compute resources. However, their typically limited or one-size-fi…
Optimizing Fine-Grained Parallelism Through Dynamic Load Balancing on Multi-Socket Many-Core Systems
Wenyi Wang, Maxime Gonthier, Poornima Nookala +4
Achieving efficient task parallelism on many-core architectures is an important challenge. The widely used GNU OpenMP implementation of the popular OpenMP parallel programming mode…
Sharing Begins at Home
William Dempsey, Ian Foster, Scott Fraser +1
The broad sharing of research data is widely viewed as of critical importance for the speed, quality, accessibility, and integrity of science. Despite increasing efforts to encoura…
Deep Learning Accelerated Light Source Experiments
Zhengchun Liu, Tekin Bicer, Rajkumar Kettimuthu +1
Experimental protocols at synchrotron light sources typically process and validate data only after an experiment has completed, which can lead to undetected errors and cannot enabl…
Globus Service Enhancements for Exascale Applications and Facilities
Weijian Zheng, Jack Kordas, Tyler J. Skluzacek +2
Many extreme-scale applications require the movement of large quantities of data to, from, and among leadership computing facilities, as well as other scientific facilities and the…
Fast and accurate learned multiresolution dynamical downscaling for precipitation
Jiali Wang, Zhengchun Liu, Ian Foster +3
This study develops a neural network-based approach for emulating high-resolution modeled precipitation data with comparable statistical properties but at greatly reduced computati…
Globus Automation Services: Research process automation across the space-time continuum
Ryan Chard, Jim Pruyne, Kurt McKee +5
Research process automation -- the reliable, efficient, and reproducible execution of linked sets of actions on scientific instruments, computers, data stores, and other resources…
Cloud Classification with Unsupervised Deep Learning
Takuya Kurihana, Ian Foster, Rebecca Willett +6
We present a framework for cloud characterization that leverages modern unsupervised deep learning technologies. While previous neural network-based cloud classification models hav…
Big Data Staging with MPI-IO for Interactive X-ray Science
Justin M. Wozniak, Hemant Sharma, Timothy G. Armstrong +3
New techniques in X-ray scattering science experiments produce large data sets that can require millions of high-performance processing hours per week of computation for analysis.…
fairDMS: Rapid Model Training by Data and Model Reuse
Ahsan Ali, Hemant Sharma, Rajkumar Kettimuthu +7
Extracting actionable information rapidly from data produced by instruments such as the Linac Coherent Light Source (LCLS-II) and Advanced Photon Source Upgrade (APS-U) is becoming…
UniFaaS: Programming across Distributed Cyberinfrastructure with Federated Function Serving
Yifei Li, Ryan Chard, Yadu Babuji +3
Modern scientific applications are increasingly decomposable into individual functions that may be deployed across distributed and diverse cyberinfrastructure such as supercomputer…
Bridging Data Center AI Systems with Edge Computing for Actionable Information Retrieval
Zhengchun Liu, Ahsan Ali, Peter Kenesei +11
Extremely high data rates at modern synchrotron and X-ray free-electron laser light source beamlines motivate the use of machine learning methods for data reduction, feature detect…
Enabling Loosely-Coupled Serial Job Execution on the IBM BlueGene/P Supercomputer and the SiCortex SC5832
Ioan Raicu, Zhao Zhang, Mike Wilde +1
Our work addresses the enabling of the execution of highly parallel computations composed of loosely coupled serial jobs with no modifications to the respective applications, on la…
Parsl: Pervasive Parallel Programming in Python
Yadu Babuji, Anna Woodard, Zhuozhao Li +9
High-level programming languages such as Python are increasingly used to provide intuitive interfaces to libraries written in lower-level languages and for assembling applications…
Icicle: Scalable Metadata Indexing and Real-Time Monitoring for HPC File Systems
Haochen Pan, Ryan Chard, Song Young Oh +7
Modern HPC file systems can contain billions of files and hundreds of petabytes of data, making even simple questions increasingly intractable to answer. Traditional file system ut…
Core Hours and Carbon Credits: Incentivizing Sustainability in HPC
Alok Kamatar, Maxime Gonthier, Valerie Hayot-Sasson +6
Realizing a shared responsibility between providers and consumers is critical to manage the sustainability of HPC. However, while cost may motivate efficiency improvements by infra…
HiPerRAG: High-Performance Retrieval Augmented Generation for Scientific Insights
Ozan Gokdemir, Carlo Siebenschuh, Alexander Brace +21
The volume of scientific literature is growing exponentially, leading to underutilized discoveries, duplicated efforts, and limited cross-disciplinary collaboration. Retrieval Augm…
The Data Station: Combining Data, Compute, and Market Forces
Raul Castro Fernandez, Kyle Chard, Ben Blaiszik +7
This paper introduces Data Stations, a new data architecture that we are designing to tackle some of the most challenging data problems that we face today: access to sensitive data…
Targeting SARS-CoV-2 with AI- and HPC-enabled Lead Generation: A First Data Release
Yadu Babuji, Ben Blaiszik, Tom Brettin +15
Researchers across the globe are seeking to rapidly repurpose existing drugs or discover new drugs to counter the the novel coronavirus disease (COVID-19) caused by severe acute re…
Scalable Cross-Facility Federated Learning for Scientific Foundation Models on Multiple Supercomputers
Yijiang Li, Zilinghan Li, Kyle Chard +4
Artificial Intelligence for scientific applications increasingly requires training large models on data that cannot be centralized due to privacy constraints, data sovereignty, or…
OpenHLS: High-Level Synthesis for Low-Latency Deep Neural Networks for Experimental Science
Maksim Levental, Arham Khan, Ryan Chard +3
In many experiment-driven scientific domains, such as high-energy physics, material science, and cosmology, high data rate experiments impose hard constraints on data acquisition s…
Oil & Water? Diffusion of AI Within and Across Scientific Fields
Eamon Duede, William Dolan, André Bauer +2
This study empirically investigates claims of the increasing ubiquity of artificial intelligence (AI) within roughly 80 million research publications across 20 diverse scientific f…
Telescope: An Automated Hybrid Forecasting Approach on a Level-Playing Field
André Bauer, Mark Leznik, Michael Stenger +4
In many areas of decision-making, forecasting is an essential pillar. Consequently, many different forecasting methods have been proposed. From our experience, recently presented f…
Machine Learning Prediction of Accurate Atomization Energies of Organic Molecules from Low-Fidelity Quantum Chemical Calculations
Logan Ward, Ben Blaiszik, Ian Foster +3
Recent studies illustrate how machine learning (ML) can be used to bypass a core challenge of molecular modeling: the tradeoff between accuracy and computational cost. Here, we ass…
StreamGuard: Low-Overhead Resilience for Real-time HPC Data Streams
Hai Duc Nguyen, Bogdan Nicolae, Tekin Bicer +4
Real-time scientific workflows operate on continuous data streams and must produce timely, high-quality results despite executing on complex, failure-prone infrastructure. Hardware…
A Problem-Specific Fault-Tolerance Mechanism for Asynchronous, Distributed Systems
Adriana Iamnitchi, Ian Foster
The idle computers on a local area, campus area, or even wide area network represent a significant computational resource---one that is, however, also unreliable, heterogeneous, an…
AdaParse: An Adaptive Parallel PDF Parsing and Resource Scaling Engine
Carlo Siebenschuh, Kyle Hippe, Ozan Gokdemir +10
Language models for scientific tasks are trained on text from scientific publications, most distributed as PDFs that require parsing. PDF parsing approaches range from inexpensive…
Flight: A FaaS-Based Framework for Complex and Hierarchical Federated Learning
Nathaniel Hudson, Valerie Hayot-Sasson, Yadu Babuji +5
Federated Learning (FL) is a decentralized machine learning paradigm where models are trained on distributed devices and are aggregated at a central server. Existing FL frameworks…
AI- and HPC-enabled Lead Generation for SARS-CoV-2: Models and Processes to Extract Druglike Molecules Contained in Natural Language Text
Zhi Hong, J. Gregory Pauloski, Logan Ward +3
Researchers worldwide are seeking to repurpose existing drugs or discover new drugs to counter the disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). A…
Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Models
Mansi Sakarvadia, Aswathy Ajith, Arham Khan +5
Answering multi-hop reasoning questions requires retrieving and synthesizing information from diverse sources. Large Language Models (LLMs) struggle to perform such reasoning consi…
Extended Abstract: Productive Parallel Programming with Parsl
Kyle Chard, Yadu Babuji, Anna Woodard +6
Parsl is a parallel programming library for Python that aims to make it easy to specify parallelism in programs and to realize that parallelism on arbitrary parallel and distribute…
Interpreting Language Model Hidden States at Scale
Jordan Pettyjohn, Mansi Sakarvadia, Nathaniel Hudson +3
Lens methods interpret large language models (LLMs) by mapping intermediate activations to the output vocabulary, revealing how next-token predictions develop through the network.…
TomoGAN: Low-Dose Synchrotron X-Ray Tomography with Generative Adversarial Networks
Zhengchun Liu, Tekin Bicer, Rajkumar Kettimuthu +3
Synchrotron-based x-ray tomography is a noninvasive imaging technique that allows for reconstructing the internal structure of materials at high spatial resolutions from tens of mi…
AICCA: AI-driven Cloud Classification Atlas
Takuya Kurihana, Elisabeth Moyer, Ian Foster
Clouds play an important role in the Earth's energy budget and their behavior is one of the largest uncertainties in future climate projections. Satellite observations should help…
The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators
Mansi Sakarvadia, Kareem Hegazy, Amin Totounferoush +4
A core challenge in scientific machine learning, and scientific computing more generally, is modeling continuous phenomena which (in practice) are represented discretely. Machine-l…
IRNet: A General Purpose Deep Residual Regression Framework for Materials Discovery
Dipendra Jha, Logan Ward, Zijiang Yang +5
Materials discovery is crucial for making scientific advances in many domains. Collections of data from experiments and first-principle computations have spurred interest in applyi…
Attention Lens: A Tool for Mechanistically Interpreting the Attention Head Information Retrieval Mechanism
Mansi Sakarvadia, Arham Khan, Aswathy Ajith +5
Transformer-based Large Language Models (LLMs) are the state-of-the-art for natural language tasks. Recent work has attempted to decode, by reverse engineering the role of linear l…
Coupling streaming AI and HPC ensembles to achieve 100-1000x faster biomolecular simulations
Alexander Brace, Igor Yakushin, Heng Ma +7
Machine learning (ML)-based steering can improve the performance of ensemble-based simulations by allowing for online selection of more scientifically meaningful computations. We p…
Cloud Computing and Grid Computing 360-Degree Compared
Ian Foster, Yong Zhao, Ioan Raicu +1
Cloud Computing has become another buzzword after Web 2.0. However, there are dozens of different definitions for Cloud Computing and there seems to be no consensus on what a Cloud…
Trustworthy Provenance for Big Data Science: a Modular Architecture Leveraging Blockchain in Federated Settings
Nicola Giuseppe Marchioro, Yannis Velegrakis, Valentine Anantharaj +2
Ensuring the trustworthiness and long-term verifiability of scientific data is a foundational challenge in the era of data-intensive, collaborative research. Provenance metadata pl…
MOFA: Discovering Materials for Carbon Capture with a GenAI- and Simulation-Based Workflow
Xiaoli Yan, Nathaniel Hudson, Hyun Park +15
We present MOFA, an open-source generative AI (GenAI) plus simulation workflow for high-throughput generation of metal-organic frameworks (MOFs) on large-scale high-performance com…
ScaffoldGPT: A Scaffold-based GPT Model for Drug Optimization
Xuefeng Liu, Songhao Jiang, Ian Foster +2
Drug optimization has become increasingly crucial in light of fast-mutating virus strains and drug-resistant cancer cells. Nevertheless, it remains challenging as it necessitates r…
Experiences with Model Context Protocol Servers for Science and High Performance Computing
Haochen Pan, Ryan Chard, Reid Mello +12
Large language model (LLM)-powered agents are increasingly used to plan and execute scientific workflows, yet most research cyberinfrastructure (CI) exposes heterogeneous APIs and…
DiPerF: an automated DIstributed PERformance testing Framework
Catalin Dumitrescu, Ioan Raicu, Matei Ripeanu +1
We present DiPerF, a distributed performance testing framework, aimed at simplifying and automating service performance evaluation. DiPerF coordinates a pool of machines that test…
Replica Selection in the Globus Data Grid
Sudharshan Vazhkudai, Steven Tuecke, Ian Foster
The Globus Data Grid architecture provides a scalable infrastructure for the management of storage resources and data that are distributed across Grid environments. These services…
Serverless Supercomputing: High Performance Function as a Service for Science
Ryan Chard, Tyler J. Skluzacek, Zhuozhao Li +6
Growing data volumes and velocities are driving exciting new methods across the sciences in which data analytics and machine learning are increasingly intertwined with research. Th…
Towards Accommodating Real-time Jobs on HPC Platforms
Sam Nickolay, Eun-Sung Jung, Rajkumar Kettimuthu +1
Increasing data volumes in scientific experiments necessitate the use of high-performance computing (HPC) resources for data analysis. In many scientific fields, the data generated…
Addressing Reproducibility Challenges in HPC with Continuous Integration
Valérie Hayot-Sasson, Nathaniel Hudson, André Bauer +3
The high-performance computing (HPC) community has adopted incentive structures to motivate reproducible research, with major conferences awarding badges to papers that meet reprod…
Steering an Active Learning Workflow Towards Novel Materials Discovery via Queue Prioritization
Marcus Schwarting, Logan Ward, Nathaniel Hudson +5
Generative AI poses both opportunities and risks for solving inverse design problems in the sciences. Generative tools provide the ability to expand and refine a search space auton…
Accelerating Electronic Stopping Power Predictions by 10 Million Times with a Combination of Time-Dependent Density Functional Theory and Machine Learning
Logan Ward, Ben Blaiszik, Cheng-Wei Lee +3
Knowing the rate at which particle radiation releases energy in a material, the stopping power, is key to designing nuclear reactors, medical treatments, semiconductor and quantum…
Optimizing Scientific Data Transfer on Globus with Error-bounded Lossy Compression
Yuanjian Liu, Sheng Di, Kyle Chard +2
The increasing volume and velocity of science data necessitate the frequent movement of enormous data volumes as part of routine research activities. As a result, limited wide-area…
Efficient Data-Parallel Continual Learning with Asynchronous Distributed Rehearsal Buffers
Thomas Bouvier, Bogdan Nicolae, Hugo Chaugier +3
Deep learning has emerged as a powerful method for extracting valuable information from large volumes of data. However, when new training data arrives continuously (i.e., is not fu…
Data Station: Delegated, Trustworthy, and Auditable Computation to Enable Data-Sharing Consortia with a Data Escrow
Siyuan Xia, Zhiru Zhu, Chris Zhu +7
Pooling and sharing data increases and distributes its value. But since data cannot be revoked once shared, scenarios that require controlled release of data for regulatory, privac…
Principles of the Battery Data Genome
Logan Ward, Susan Babinec, Eric J. Dufek +24
Electrochemical energy storage is central to modern society -- from consumer electronics to electrified transportation and the power grid. It is no longer just a convenience but a…
Real-Time Streaming and Event-driven Control of Scientific Experiments
Jakob R. Elias, Ryan Chard, Maksim Levental +3
Advancements in scientific instrument sensors and connected devices provide unprecedented insight into ongoing experiments and present new opportunities for control, optimization,…
Realizing Fast, Scalable and Reliable Scientific Computations in Grid Environments
Yong Zhao, Ioan Raicu, Ian Foster +3
The practical realization of managing and executing large scale scientific computations efficiently and reliably is quite challenging. Scientific computations often involve thousan…
Data-sharing relationships in the Web
Adriana Iamnitchi, Matei Ripeanu, Ian Foster
We propose a novel structure, the data-sharing graph, for characterizing sharing patterns in large-scale data distribution systems. We analyze this structure in two such systems an…
The History of the Grid
Ian Foster, Carl Kesselman
With the widespread availability of high-speed networks, it becomes feasible to outsource computing to remote providers and to federate resources from many locations. Such observat…
In Search of Simplicity: A Self-Organizing Multi-Source Multicast Overlay
Matei Ripeanu, Adriana Iamnitchi, Ian Foster +1
Multicast communication primitives have broad utility as building blocks for distributed applications. The challenge is to create and maintain the distributed structures that suppo…
Data-driven Cloud Clustering via a Rotationally Invariant Autoencoder
Takuya Kurihana, Elisabeth Moyer, Rebecca Willett +2
Advanced satellite-born remote sensing instruments produce high-resolution multi-spectral data for much of the globe at a daily cadence. These datasets open up the possibility of i…
The Earth System Grid: Supporting the Next Generation of Climate Modeling Research
David Bernholdt, Shishir Bharathi, David Brown +17
Understanding the earth's climate system and how it might be changing is a preeminent scientific challenge. Global climate models are used to simulate past, present, and future cli…
Human-Machine Symbiosis, 50 Years On
Ian Foster
Licklider advocated in 1960 the construction of computers capable of working symbiotically with humans to address problems not easily addressed by humans working alone. Since that…
funcX: Federated Function as a Service for Science
Zhuozhao Li, Ryan Chard, Yadu Babuji +9
funcX is a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. Unlike centralized FaaS systems, funcX…
Accelerated, Scalable and Reproducible AI-driven Gravitational Wave Detection
E. A. Huerta, Asad Khan, Xiaobo Huang +10
The development of reusable artificial intelligence (AI) models for wider use and rigorous validation by the community promises to unlock new opportunities in multi-messenger astro…
Data Diffusion: Dynamic Resource Provision and Data-Aware Scheduling for Data Intensive Applications
Ioan Raicu, Yong Zhao, Ian Foster +1
Data intensive applications often involve the analysis of large datasets that require large amounts of compute and storage resources. While dedicated compute and/or storage farms o…
PRISM: Protocol Refinement through Intelligent Simulation Modeling
Brian Hsu, Priyanka V Setty, Rory M Butler +7
Automating experimental protocol design and execution remains as a fundamental bottleneck in realizing self-driving laboratories. We introduce PRISM (Protocol Refinement through In…