An introduction to Docker for reproducible research, with examples from the R environment
arXiv:1410.0846 · doi:10.1145/2723872.2723882
Abstract
As computational work becomes more and more integral to many aspects of scientific research, computational reproducibility has become an issue of increasing importance to computer systems researchers and domain scientists alike. Though computational reproducibility seems more straight forward than replicating physical experiments, the complex and rapidly changing nature of computer environments makes being able to reproduce and extend such work a serious challenge. In this paper, I explore common reasons that code developed for one research project cannot be successfully executed or extended by subsequent researchers. I review current approaches to these issues, including virtual machines and workflow systems, and their limitations. I then examine how the popular emerging technology Docker combines several areas from systems research - such as operating system virtualization, cross-platform portability, modular re-usable elements, versioning, and a `DevOps' philosophy, to address these challenges. I illustrate this with several examples of Docker use with a focus on the R statistical environment.
References in corpus (1)
Cited by in corpus (71)
- 5G network slicing using SDN and NFV- A survey of taxonomy, architectures and future challenges
- Opening the black box of energy modelling: Strategies and lessons learned
- Consistency of VDJ rearrangement and substitution parameters enables accurate B cell receptor sequence annotation
- A Privacy-Preserving Healthcare Framework Using Hyperledger Fabric
- NSML: Meet the MLaaS platform with a real-world case study
- INDIGO-DataCloud:A data and computing platform to facilitate seamless access to e-infrastructures
- A survey of biodiversity informatics: Concepts, practices, and challenges
- Containers for portable, productive and performant scientific computing
- Publishing computational research -- A review of infrastructures for reproducible and transparent scholarly communication
- Privacy and Trust Redefined in Federated Machine Learning
- Reproducibility, Replicability, and Repeatability: A survey of reproducible research with a focus on high performance computing
- ir_metadata: An Extensible Metadata Schema for IR Experiments
- Container-Based Cloud Virtual Machine Benchmarking
- Brain MRI Tumor Segmentation with Adversarial Networks
- What Makes a Popular Academic AI Repository?
- Privacy Preserving Passive DNS
- USID and Pycroscopy -- Open frameworks for storing and analyzing spectroscopic and imaging data
- KernelHaven -- An Experimentation Workbench for Analyzing Software Product Lines
- A framework for fostering transparency in shared artificial intelligence models by increasing visibility of contributions
- BioWorkbench: A High-Performance Framework for Managing and Analyzing Bioinformatics Experiments
- NEOExchange -- An online portal for NEO and Solar System science
- Reproducible and Portable Big Data Analytics in the Cloud
- DocLite: A Docker-Based Lightweight Cloud Benchmarking Tool
- Implementing a scalable and elastic computing environment based on Cloud Containers
- The Technologies Required for Fusing HPC and Real-Time Data to Support Urgent Computing
- The Rockerverse: Packages and Applications for Containerization with R
- Portable, high-performance containers for HPC
- A Comparative Study of Containers and Virtual Machines in Big Data Environment
- KernelHaven -- An Open Infrastructure for Product Line Analysis
- A Collaborative Approach to Computational Reproducibility
- Enabling End-To-End Machine Learning Replicability: A Case Study in Educational Data Mining
- Reproducible Workflow on a Public Cloud for Computational Fluid Dynamics
- Reproducibility of Build Environments through Space and Time
- Mitigating Docker Security Issues
- Proxying ROS communications -- enabling containerized ROS deployments in distributed multi-host environments
- d-SEAMS: Deferred Structural Elucidation Analysis for Molecular Simulations
- Source Code Archiving to the Rescue of Reproducible Deployment
- The imperative for reproducibility in building performance simulation research
- Using Unity to Help Solve Intelligence
- Formal Definition and Implementation of Reproducibility Tenets for Computational Workflows
- Confronting the Reproducibility Crisis: A Case Study of Challenges in Cybersecurity AI
- DiME and AGVis: A Distributed Messaging Environment and Geographical Visualizer for Large-scale Power System Simulation
- Monte Carlo modeling photon-tissue interaction using on-demand cloud infrastructure
- Improving Software Engineering Research through Experimentation Workbenches
- Kooplex: collaborative data analytics portal for advancing sciences
- SwarmRob: A Toolkit for Reproducibility and Sharing of Experimental Artifacts in Robotics Research
- Design and Execution of make-like, distributed Analyses based on Spotify's Pipelining Package Luigi
- Lightweight Container-based User Environment
- From Model Performance to Claim: How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility Gap
- End-to-End Analysis Automation over Distributed Resources with Luigi Analysis Workflows
- Facilitating Rapid Prototyping in the OODIDA Data Analytics Platform via Active-Code Replacement
- Production Ranking Systems: A Review
- On The Relation Between Outdated Docker Containers, Severity Vulnerabilities and Bugs
- Reasons, Challenges and Some Tools for Doing Reproducible Research in Transportation Research
- CUF-Links: Continuous and Ubiquitous FAIRness Linkages for reproducible research
- Integrated Benchmarking and Design for Reproducible and Accessible Evaluation of Robotic Agents
- Provenance tracking in the LHCb software
- Data Lakes, Clouds and Commons: A Review of Platforms for Analyzing and Sharing Genomic Data
- Reproducible and User-Controlled Software Environments in HPC with Guix
- Tactical Network Modeller Simulation Tool Combined discrete event and network back-ends
- A Visual Analytics Framework for Distributed Data Analysis Systems
- Reproducing Scientific Experiment with Cloud DevOps
- Build and Execution Environment (BEE): an Encapsulated Environment Enabling HPC Applications Running Everywhere
- WebMIaS on Docker: Deploying Math-Aware Search in a Single Line of Code
- Calibration of LOFAR data on the cloud
- Context-Aware Task Handling in Resource-Constrained Robots with Virtualization
- A Makefile for Developing Containerized LaTeX Technical Documents
- Idiomatic and Reproducible Software Builds using Containers for Reliable Computing
- Lack of evidence for a substantial rate of templated mutagenesis in B cell diversification
- Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems
- SecDocker: Hardening the Continuous Integration Workflow