48 citations · 236 across the 32 of their papers we have counts for
10 papers · 1 filter
Towards Advanced Monitoring for Scientific Workflows
Jonathan Bader, Joel Witzke, Soeren Becker +5
Scientific workflows consist of thousands of highly parallelized tasks executed in a distributed environment involving many components. Automatic tracing and investigation of the c…
Leveraging Reinforcement Learning for Task Resource Allocation in Scientific Workflows
Jonathan Bader, Nicolas Zunker, Soeren Becker +1
Scientific workflows are designed as directed acyclic graphs (DAGs) and consist of multiple dependent task definitions. They are executed over a large amount of data, often resulti…
Perona: Robust Infrastructure Fingerprinting for Resource-Efficient Big Data Analytics
Dominik Scheinert, Soeren Becker, Jonathan Bader +3
Choosing a good resource configuration for big data analytics applications can be challenging, especially in cloud environments. Automated approaches are desirable as poor decision…
Ruya: Memory-Aware Iterative Optimization of Cluster Configurations for Big Data Processing
Jonathan Will, Lauritz Thamsen, Jonathan Bader +2
Selecting appropriate computational resources for data processing jobs on large clusters is difficult, even for expert users like data engineers. Inadequate choices can result in v…
Macaw: The Machine Learning Magnetometer Calibration Workflow
Jonathan Bader, Kevin Styp-Rekowski, Leon Doehler +2
In Earth Systems Science, many complex data pipelines combine different data sources and apply data filtering and analysis steps. Typically, such data analysis processes are histor…
Reshi: Recommending Resources for Scientific Workflow Tasks on Heterogeneous Infrastructures
Jonathan Bader, Fabian Lehmann, Alexander Groth +5
Scientific workflows typically comprise a multitude of different processing steps which often are executed in parallel on different partitions of the input data. These executions,…
Get Your Memory Right: The Crispy Resource Allocation Assistant for Large-Scale Data Processing
Jonathan Will, Lauritz Thamsen, Jonathan Bader +2
Distributed dataflow systems like Apache Spark and Apache Hadoop enable data-parallel processing of large datasets on clusters. Yet, selecting appropriate computational resources f…
Collaborative Cluster Configuration for Distributed Data-Parallel Processing: A Research Overview
Lauritz Thamsen, Dominik Scheinert, Jonathan Will +2
Many organizations routinely analyze large datasets using systems for distributed data-parallel processing and clusters of commodity resources. Yet, users need to configure adequat…
Lotaru: Locally Estimating Runtimes of Scientific Workflow Tasks in Heterogeneous Clusters
Jonathan Bader, Fabian Lehmann, Lauritz Thamsen +3
Many scientific workflow scheduling algorithms need to be informed about task runtimes a-priori to conduct efficient scheduling. In heterogeneous cluster infrastructures, this prob…
SyncMesh: Improving Data Locality for Function-as-a-Service in Meshed Edge Networks
Daniel Habenicht, Kevin Kreutz, Soeren Becker +3
The increasing use of Internet of Things devices coincides with more communication and data movement in networks, which can exceed existing network capabilities. These devices ofte…