48 citations · 236 across the 32 of their papers we have counts for
34 papers
About the Influence of Workflow Topology on Task Intensity Prediction through Graph Learning
Max Otto, Haci Ismail Aslan, Joel Witzke +2
Efficient resource provisioning for large-scale workflows on cloud infrastructures is a critical performance engineering challenge. These workflows are often structured as directed…
Optimizing Memory Allocation in Distributed Clusters with Predictive Modeling
Jonathan Bader, Edgar Blumenthal, Marten Eckardt +5
In modern distributed systems, efficient resource allocation is a vital aspect to maintain scalability, reduce operational costs, and ensure fast execution even across heterogeneou…
Intelligent resource prediction for SAP HANA continuous integration build workloads
Torsten Mandel, Jonathan Bader, Hanyoung Yoo +1
Large enterprises often operate extensive Continuous Integration (CI) pipelines on large, heterogeneous compute clusters, where conservative, statically defined resource requiremen…
Learning Process Energy Profiles from Node-Level Power Data
Jonathan Bader, Julius Irion, Jannis Kappel +4
The growing demand for data center capacity, driven by the growth of high-performance computing, cloud computing, and especially artificial intelligence, has led to a sharp increas…
Optimizing Microgrid Composition for Sustainable Data Centers
Julius Irion, Philipp Wiesner, Jonathan Bader +1
As computing energy demand continues to grow and electrical grid infrastructure struggles to keep pace, an increasing number of data centers are being planned with colocated microg…
Predicting the Performance of Scientific Workflow Tasks for Cluster Resource Management: An Overview of the State of the Art
Jonathan Bader, Kathleen West, Soeren Becker +5
Scientific workflow management systems support large-scale data analysis on cluster infrastructures. For this, they interact with resource managers which schedule workflow tasks on…
WOW: Workflow-Aware Data Movement and Task Scheduling for Dynamic Scientific Workflows
Fabian Lehmann, Jonathan Bader, Friedrich Tschirpke +6
Scientific workflows process extensive data sets over clusters of independent nodes, which requires a complex stack of infrastructure components, especially a resource manager (RM)…
Flora: Efficient Cloud Resource Selection for Big Data Processing via Job Classification
Jonathan Will, Lauritz Thamsen, Jonathan Bader +1
Distributed dataflow systems like Spark and Flink enable data-parallel processing of large datasets on clusters of cloud resources. Yet, selecting appropriate computational resourc…
Workflows Community Summit 2024: Future Trends and Challenges in Scientific Workflows
Rafael Ferreira da Silva, Deborah Bard, Kyle Chard +108
The Workflows Community Summit gathered 111 participants from 18 countries to discuss emerging trends and challenges in scientific workflows, focusing on six key areas: time-sensit…
KS+: Predicting Workflow Task Memory Usage Over Time
Jonathan Bader, Ansgar Lößer, Lauritz Thamsen +2
Scientific workflow management systems enable the reproducible execution of data analysis pipelines on cluster infrastructures managed by resource managers such as Kubernetes, Slur…
Ponder: Online Prediction of Task Memory Requirements for Scientific Workflows
Fabian Lehmann, Jonathan Bader, Ninon De Mecquenem +5
Scientific workflows are used to analyze large amounts of data. These workflows comprise numerous tasks, many of which are executed repeatedly, running the same custom program on d…
Sizey: Memory-Efficient Execution of Scientific Workflow Tasks
Jonathan Bader, Fabian Skalski, Fabian Lehmann +4
As the amount of available data continues to grow in fields as diverse as bioinformatics, physics, and remote sensing, the importance of scientific workflows in the design and impl…
Scaling on Frontier: Uncertainty Quantification Workflow Applications using ExaWorks to Enable Full System Utilization
Mikhail Titov, Robert Carson, Matthew Rolchigo +6
When running at scale, modern scientific workflows require middleware to handle allocated resources, distribute computing payloads and guarantee a resilient execution. While indivi…
The Common Workflow Scheduler Interface: Status Quo and Future Plans
Fabian Lehmann, Jonathan Bader, Lauritz Thamsen +1
Nowadays, many scientific workflows from different domains, such as Remote Sensing, Astronomy, and Bioinformatics, are executed on large computing infrastructures managed by resour…
Predicting Dynamic Memory Requirements for Scientific Workflow Tasks
Jonathan Bader, Nils Diedrich, Lauritz Thamsen +1
With the increasing amount of data available to scientists in disciplines as diverse as bioinformatics, physics, and remote sensing, scientific workflow systems are becoming increa…
Lotaru: Locally Predicting Workflow Task Runtimes for Resource Management on Heterogeneous Infrastructures
Jonathan Bader, Fabian Lehmann, Lauritz Thamsen +2
Many resource management techniques for task scheduling, energy and carbon efficiency, and cost optimization in workflows rely on a-priori task runtime knowledge. Building runtime…
Workflows Community Summit 2022: A Roadmap Revolution
Rafael Ferreira da Silva, Rosa M. Badia, Venkat Bala +102
Scientific workflows have become integral tools in broad scientific computing use cases. Science discovery is increasingly dependent on workflows to orchestrate large and complex s…
How Workflow Engines Should Talk to Resource Managers: A Proposal for a Common Workflow Scheduling Interface
Fabian Lehmann, Jonathan Bader, Friedrich Tschirpke +2
Scientific workflow management systems (SWMSs) and resource managers together ensure that tasks are scheduled on provisioned resources so that all dependencies are obeyed, and some…
Towards Advanced Monitoring for Scientific Workflows
Jonathan Bader, Joel Witzke, Soeren Becker +5
Scientific workflows consist of thousands of highly parallelized tasks executed in a distributed environment involving many components. Automatic tracing and investigation of the c…
Leveraging Reinforcement Learning for Task Resource Allocation in Scientific Workflows
Jonathan Bader, Nicolas Zunker, Soeren Becker +1
Scientific workflows are designed as directed acyclic graphs (DAGs) and consist of multiple dependent task definitions. They are executed over a large amount of data, often resulti…
Perona: Robust Infrastructure Fingerprinting for Resource-Efficient Big Data Analytics
Dominik Scheinert, Soeren Becker, Jonathan Bader +3
Choosing a good resource configuration for big data analytics applications can be challenging, especially in cloud environments. Automated approaches are desirable as poor decision…
Ruya: Memory-Aware Iterative Optimization of Cluster Configurations for Big Data Processing
Jonathan Will, Lauritz Thamsen, Jonathan Bader +2
Selecting appropriate computational resources for data processing jobs on large clusters is difficult, even for expert users like data engineers. Inadequate choices can result in v…
Macaw: The Machine Learning Magnetometer Calibration Workflow
Jonathan Bader, Kevin Styp-Rekowski, Leon Doehler +2
In Earth Systems Science, many complex data pipelines combine different data sources and apply data filtering and analysis steps. Typically, such data analysis processes are histor…
Reshi: Recommending Resources for Scientific Workflow Tasks on Heterogeneous Infrastructures
Jonathan Bader, Fabian Lehmann, Alexander Groth +5
Scientific workflows typically comprise a multitude of different processing steps which often are executed in parallel on different partitions of the input data. These executions,…