9 papers
Augur: Pre-Execution Energy Prediction for Workflow Tasks in Heterogeneous Clusters
Kathleen West, Vasilis Bountris, Philipp Thamm +3
Scientific workflows are widely used to process large quantities of data, leading to significant energy consumption and carbon emissions. To reduce this environmental impact, energ…
Nf-PEAK: Process-Based Energy Attribution for Nextflow Workflows on Kubernetes Clusters
Philipp Thamm, Somayeh Mohammadi, Kathleen West +3
Scientific workflows are pipelines of interdependent tasks. They are increasingly executed on shared Kubernetes clusters via workflow engines such as Nextflow. Their energy consump…
A Systematic Evaluation of the Potential of Carbon-Aware Execution for Scientific Workflows
Kathleen West, Youssef Moawad, Fabian Lehmann +4
Scientific workflows are critical to scientific data analysis and often involve computationally intensive processing of large datasets on compute clusters. As such, their execution…
HyProv: Hybrid Provenance Management for Scientific Workflows
Vasilis Bountris, Lauritz Thamsen, Ulf Leser
Provenance plays a crucial role in scientific workflow execution, for instance by providing data for failure analysis, real-time monitoring, or statistics on resource utilization f…
Strategies to Measure Energy Consumption Using RAPL During Workflow Execution on Commodity Clusters
Philipp Thamm, Ulf Leser
In science, problems in many fields can be solved by processing datasets using a series of computationally expensive algorithms, sometimes referred to as workflows. Traditionally,…
WOW: Workflow-Aware Data Movement and Task Scheduling for Dynamic Scientific Workflows
Fabian Lehmann, Jonathan Bader, Friedrich Tschirpke +6
Scientific workflows process extensive data sets over clusters of independent nodes, which requires a complex stack of infrastructure components, especially a resource manager (RM)…