24 citations · 127 across the 21 of their papers we have counts for
21 papers
Flora: Efficient Cloud Resource Selection for Big Data Processing via Job Classification
Jonathan Will, Lauritz Thamsen, Jonathan Bader +1
Distributed dataflow systems like Spark and Flink enable data-parallel processing of large datasets on clusters of cloud resources. Yet, selecting appropriate computational resourc…
Experimentally Evaluating the Resource Efficiency of Big Data Autoscaling
Jonathan Will, Nico Treide, Lauritz Thamsen +1
Distributed dataflow systems like Spark and Flink enable data-parallel processing of large datasets on clusters. Yet, selecting appropriate computational resources for dataflow job…
Sizey: Memory-Efficient Execution of Scientific Workflow Tasks
Jonathan Bader, Fabian Skalski, Fabian Lehmann +4
As the amount of available data continues to grow in fields as diverse as bioinformatics, physics, and remote sensing, the importance of scientific workflows in the design and impl…
Privacy-Preserving Sharing of Data Analytics Runtime Metrics for Performance Modeling
Jonathan Will, Dominik Scheinert, Jan Bode +3
Performance modeling for large-scale data analytics workloads can improve the efficiency of cluster resource allocations and job scheduling. However, the performance of these workl…
Towards a Peer-to-Peer Data Distribution Layer for Efficient and Collaborative Resource Optimization of Distributed Dataflow Applications
Dominik Scheinert, Soeren Becker, Jonathan Will +2
Performance modeling can help to improve the resource efficiency of clusters and distributed dataflow applications, yet the available modeling data is often limited. Collaborative…
Karasu: A Collaborative Approach to Efficient Cluster Configuration for Big Data Analytics
Dominik Scheinert, Philipp Wiesner, Thorsten Wittkopp +3
Selecting the right resources for big data analytics jobs is hard because of the wide variety of configuration options like machine type and cluster size. As poor choices can have…
Selecting Efficient Cluster Resources for Data Analytics: When and How to Allocate for In-Memory Processing?
Jonathan Will, Lauritz Thamsen, Dominik Scheinert +1
Distributed dataflow systems such as Apache Spark or Apache Flink enable parallel, in-memory data processing on large clusters of commodity hardware. Consequently, the appropriate…
Perona: Robust Infrastructure Fingerprinting for Resource-Efficient Big Data Analytics
Dominik Scheinert, Soeren Becker, Jonathan Bader +3
Choosing a good resource configuration for big data analytics applications can be challenging, especially in cloud environments. Automated approaches are desirable as poor decision…
Ruya: Memory-Aware Iterative Optimization of Cluster Configurations for Big Data Processing
Jonathan Will, Lauritz Thamsen, Jonathan Bader +2
Selecting appropriate computational resources for data processing jobs on large clusters is difficult, even for expert users like data engineers. Inadequate choices can result in v…
Reshi: Recommending Resources for Scientific Workflow Tasks on Heterogeneous Infrastructures
Jonathan Bader, Fabian Lehmann, Alexander Groth +5
Scientific workflows typically comprise a multitude of different processing steps which often are executed in parallel on different partitions of the input data. These executions,…
Get Your Memory Right: The Crispy Resource Allocation Assistant for Large-Scale Data Processing
Jonathan Will, Lauritz Thamsen, Jonathan Bader +2
Distributed dataflow systems like Apache Spark and Apache Hadoop enable data-parallel processing of large datasets on clusters. Yet, selecting appropriate computational resources f…
Collaborative Cluster Configuration for Distributed Data-Parallel Processing: A Research Overview
Lauritz Thamsen, Dominik Scheinert, Jonathan Will +2
Many organizations routinely analyze large datasets using systems for distributed data-parallel processing and clusters of commodity resources. Yet, users need to configure adequat…
Lotaru: Locally Estimating Runtimes of Scientific Workflow Tasks in Heterogeneous Clusters
Jonathan Bader, Fabian Lehmann, Lauritz Thamsen +3
Many scientific workflow scheduling algorithms need to be informed about task runtimes a-priori to conduct efficient scheduling. In heterogeneous cluster infrastructures, this prob…
On the Potential of Execution Traces for Batch Processing Workload Optimization in Public Clouds
Dominik Scheinert, Alireza Alamgiralem, Jonathan Bader +3
With the growing amount of data, data processing workloads and the management of their resource usage becomes increasingly important. Since managing a dedicated infrastructure is i…
Training Data Reduction for Performance Models of Data Analytics Jobs in the Cloud
Jonathan Will, Onur Arslan, Jonathan Bader +2
Distributed dataflow systems like Apache Flink and Apache Spark simplify processing large amounts of data on clusters in a data-parallel manner. However, choosing suitable cluster…
Tarema: Adaptive Resource Allocation for Scalable Scientific Workflows in Heterogeneous Clusters
Jonathan Bader, Lauritz Thamsen, Svetlana Kulagina +3
Scientific workflow management systems like Nextflow support large-scale data analysis by abstracting away the details of scientific workflows. In these systems, workflows consist…
Dependable IoT Data Stream Processing for Monitoring and Control of Urban Infrastructures
Morgan K. Geldenhuys, Jonathan Will, Benjamin J. J. Pfister +3
The Internet of Things describes a network of physical devices interacting and producing vast streams of sensor data. At present there are a number of general challenges which exis…
Enel: Context-Aware Dynamic Scaling of Distributed Dataflow Jobs using Graph Propagation
Dominik Scheinert, Houkun Zhu, Lauritz Thamsen +4
Distributed dataflow systems like Spark and Flink enable the use of clusters for scalable data analytics. While runtime prediction models can be used to initially select appropriat…
Bellamy: Reusing Performance Models for Distributed Dataflow Jobs Across Contexts
Dominik Scheinert, Lauritz Thamsen, Houkun Zhu +4
Distributed dataflow systems enable the use of clusters for scalable data analytics. However, selecting appropriate cluster resources for a processing job is often not straightforw…
C3O: Collaborative Cluster Configuration Optimization for Distributed Data Processing in Public Clouds
Jonathan Will, Lauritz Thamsen, Dominik Scheinert +2
Distributed dataflow systems enable data-parallel processing of large datasets on clusters. Public cloud providers offer a large variety and quantity of resources that can be used…
Towards Collaborative Optimization of Cluster Configurations for Distributed Dataflow Jobs
Jonathan Will, Jonathan Bader, Lauritz Thamsen
Analyzing large datasets with distributed dataflow systems requires the use of clusters. Public cloud providers offer a large variety and quantity of resources that can be used for…