Rucio - Scientific data management

Abstract

Rucio is an open-source software framework that provides scientific collaborations with the functionality to organize, manage, and access their data at scale. The data can be distributed across heterogeneous data centers at widely distributed locations. Rucio was originally developed to meet the requirements of the high-energy physics experiment ATLAS, and now is continuously extended to support the LHC experiments and other diverse scientific communities. In this article, we detail the fundamental concepts of Rucio, describe the architecture along with implementation details, and give operational experience from production usage.

arXiv comment21 pages, 11 figures

References in corpus (1)

• Search for TeV-scale gravity signatures in high-mass final states with leptons and jets with the ATLAS detector at $\sqrt{s}=13$ TeV

Cited by in corpus (16)

• The XENONnT Dark Matter Experiment
• Software and computing for Run 3 of the ATLAS experiment at the LHC
• Scientific Computing Plan for the ECCE Detector at the Electron Ion Collider
• The XLZD Design Book: Towards the Next-Generation Liquid Xenon Observatory for Dark Matter and Neutrino Physics
• Deployment of Elastic Virtual Hybrid Clusters Across Cloud Sites
• WLCG Authorisation from X.509 to Tokens
• Third-party transfers in WLCG using HTTP
• The Virtual Research Environment: towards a comprehensive analysis platform
• Accelerating Machine Learning Inference with GPUs in ProtoDUNE Data Processing
• Operational Experience and R&D results using the Google Cloud for High Energy Physics in the ATLAS experiment
• Managed Network Services for Exascale Data Movement Across Large Global Scientific Collaborations
• Total cost of ownership and evaluation of Google cloud resources for the ATLAS experiment at the LHC
• Dynamo -- Handling Scientific Data Across Sites and Storage Media
• Constraints on future analysis metadata systems in High Energy Physics
• Status and Perspectives of Neutrino Physics
• The Major Gamma-ray Imaging Cherenkov Telescopes (MAGIC)