Supporting High-Performance and High-Throughput Computing for Experimental Science
arXiv:1810.03056 · doi:10.1007/s41781-019-0022-7
Abstract
The advent of experimental science facilities-instruments and observatories, such as the Large Hadron Collider, the Laser Interferometer Gravitational Wave Observatory, and the upcoming Large Synoptic Survey Telescope-has brought about challenging, large-scale computational and data processing requirements. Traditionally, the computing infrastructure to support these facility's requirements were organized into separate infrastructure that supported their high-throughput needs and those that supported their high-performance computing needs. We argue that to enable and accelerate scientific discovery at the scale and sophistication that is now needed, this separation between high-performance computing and high-throughput computing must be bridged and an integrated, unified infrastructure provided. In this paper, we discuss several case studies where such infrastructure has been implemented. These case studies span different science domains, software systems, and application requirements as well as levels of sustainability. A further aim of this paper is to provide a basis to determine the common characteristics and requirements of such infrastructure, as well as to begin a discussion of how best to support the computing requirements of existing and future experimental science facilities.
arXiv comment13 pages, 7 figures. Accepted to Computing and Software for Big Science
References in corpus (57)
- Observation of Gravitational Waves from a Binary Black Hole Merger
- Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
- Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
- GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral
- Evidence for oscillation of atmospheric neutrinos
- Advanced Virgo: a 2nd generation interferometric gravitational wave detector
- Multi-messenger Observations of a Binary Neutron Star Merger
- Gravitational Waves and Gamma-rays from a Binary Neutron Star Merger: GW170817 and GRB 170817A
- GWTC-1: A Gravitational-Wave Transient Catalog of Compact Binary Mergers Observed by LIGO and Virgo during the First and Second Observing Runs
- Advanced LIGO
- GW151226: Observation of Gravitational Waves from a 22-Solar-Mass Binary Black Hole Coalescence
- GW170104: Observation of a 50-Solar-Mass Binary Black Hole Coalescence at Redshift 0.2
- GW170814: A Three-Detector Observation of Gravitational Waves from a Binary Black Hole Coalescence
- Measurement of the rate of nu_e + d --> p + p + e^- interactions produced by 8B solar neutrinos at the Sudbury Neutrino Observatory
- Evolution of Binary Black Hole Spacetimes
- Accurate Evolutions of Orbiting Black-Hole Binaries Without Excision
- Gravitational wave extraction from an inspiraling configuration of merging black holes
- GW170608: Observation of a 19-solar-mass Binary Black Hole Coalescence
- The DESI Experiment Part I: Science,Targeting, and Survey Design
- On the Numerical Integration of Einstein's Field Equations
- Frequency-domain gravitational waves from non-precessing black-hole binaries. II. A phenomenological model for the advanced detector era
- The Dark Energy Survey: more than dark energy - an overview
- Robust parameter estimation for compact binaries with ground-based gravitational-wave observations using the LALInference software library
- Frequency-domain gravitational waves from non-precessing black-hole binaries. I. New numerical waveforms and anatomy of the signal
- GW150914: The Advanced LIGO Detectors in the Era of First Discoveries
- Evolutions in 3D numerical relativity using fixed mesh refinement
- The PyCBC search for gravitational waves from compact binary coalescence
- An improved effective-one-body model of spinning, nonprecessing binary black holes for the era of gravitational-wave astrophysics with advanced detectors
- The Einstein Toolkit: A Community Computational Infrastructure for Relativistic Astrophysics
- Effective-one-body model for black-hole binaries with generic mass ratios and spins
- A catalog of 174 binary black-hole simulations for gravitational-wave astronomy
- Deep Learning and its Application to LHC Physics
- A single-domain spectral method for black hole puncture data
- Deep Learning for Real-time Gravitational Wave Detection and Parameter Estimation: Results with Advanced LIGO Data
- A Fast Apparent-Horizon Finder for 3-Dimensional Cartesian Grids in Numerical Relativity
- Deep Neural Networks to Enable Real-time Multimessenger Astrophysics
- Toward Early-Warning Detection of Gravitational Waves from Compact Binary Coalescence
- The Collision of Two Black Holes
- Estimating the Contribution of Dynamical Ejecta in the Kilonova Associated with GW170817
- Glitch Classification and Clustering for LIGO with Deep Transfer Learning
- Deep Transfer Learning: A new deep learning glitch classification method for advanced LIGO
- GRHydro: A new open source general-relativistic magnetohydrodynamics code for the Einstein Toolkit
- High accuracy binary black hole simulations with an extended wave zone
- IllinoisGRMHD: An Open-Source, User-Friendly GRMHD Code for Dynamical Spacetimes
- The Mira-Titan Universe II: Matter Power Spectrum Emulation
- SpECTRE: A Task-based Discontinuous Galerkin Code for Relativistic Astrophysics
- New, efficient, and accurate high order derivative and dissipation operators satisfying summation by parts, and applications in three-dimensional multi-block evolutions
- The Dark Energy Survey Data Processing and Calibration System
- Simulations of inspiraling and merging double neutron stars using the Spectral Einstein Code
- Fusing numerical relativity and deep learning to detect higher-order multipole waveforms from eccentric binary black hole mergers
- PICS: Simulations of Strong Gravitational Lensing in Galaxy Clusters
- Workload Analysis of Blue Waters
- BOSS-LDG: A Novel Computational Framework that Brings Together Blue Waters, Open Science Grid, Shifter and the LIGO Data Grid to Accelerate Gravitational Wave Discovery
- Container solutions for HPC Systems: A Case Study of Using Shifter on Blue Waters
- Efficient Construction of Mock Catalogs for Baryon Acoustic Oscillation Surveys
- Real-time regression analysis with deep convolutional neural networks
- High-Throughput Computing on High-Performance Platforms: A Case Study