Prospects for Wideband VLBI Correlation in the Cloud

Abstract

This paper proposes a cloud architecture for the correlation of wide bandwidth VLBI data. Cloud correlation facilitates processing of entire experiments in parallel using flexibly allocated and practically unlimited compute resources. This approach offers a potential improvement over dedicated correlation clusters, which are constrained by a fixed number of installed processor nodes and playback units. Additionally, cloud storage offers an alternative to maintaining a fleet of hard-disk drives that might be utilized intermittently. We describe benchmarks of VLBI correlation using the DiFX-2.5.2 software on the Google Cloud Platform to assess cloud-based correlation performance. The number of virtual CPUs per Virtual Machine was varied to determine the optimum configuration of cloud resources. The number of stations was varied to determine the scaling of correlation time with VLBI arrays of different sizes. Data transfer rates from Google Cloud Storage to the Virtual Machines performing the correlation were also measured. We also present an example cloud correlation configuration. Current cloud service and equipment pricing data is used to compile cost estimates allowing an approximate economic comparison between cloud and cluster processing. The economic comparisons are based on cost figures which are a moving target, and are highly dependent on factors such as the utilization of cluster and media, which are a challenge to estimate. Our model suggests that shifting to the cloud is an alternative path for high data rate, low duty cycle wideband VLBI correlation that should continue to be explored. In the production phase of VLBI correlation, the cloud has the potential to significantly reduce data processing times and allow the processing of more science experiments in a given year for the petabyte-scale data sets increasingly common in both astronomy and geodesy VLBI applications.

arXiv comment14 pages, 6 figures, accepted for publication in the Publications of the Astronomical Society of the Pacific journal

References in corpus (24)

• First M87 Event Horizon Telescope Results. I. The Shadow of the Supermassive Black Hole
• First M87 Event Horizon Telescope Results. IV. Imaging the Central Supermassive Black Hole
• A gravitational-wave standard siren measurement of the Hubble constant
• First M87 Event Horizon Telescope Results. II. Array and Instrumentation
• Trigonometric Parallaxes of Massive Star Forming Regions: VI. Galactic Structure, Fundamental Parameters and Non-Circular Motions
• Event-horizon-scale structure in the supermassive black hole candidate at the Galactic Centre
• DiFX2: A more flexible, efficient, robust and powerful software correlator
• Jet Launching Structure Resolved Near the Supermassive Black Hole in M87
• DiFX: A software correlator for very long baseline interferometry using multi-processor computing environments
• Resolved Magnetic-Field Structure and Variability Near the Event Horizon of Sagittarius A*
• The Trigonometric Parallax of Cygnus X-1
• 1.3 mm Wavelength VLBI of Sagittarius A*: Detection of Time-Variable Emission on Event Horizon Scales
• The first accurate parallax distance to a black hole
• Micro-Arcsecond Radio Astrometry
• 230 GHz VLBI observations of M87: event-horizon-scale structure at the enhanced very-high-energy $\gamma$-ray state in 2012
• The stellar velocity dispersion of the lens galaxy in MG2016+112 at z=1.004
• The central image of a gravitationally lensed quasar
• Detection of intrinsic source structure at ~3 Schwarzschild radii with Millimeter-VLBI observations of SAGITTARIUS A*
• The Discovery of the Radio Afterglow From the Optically Dim Gamma-Ray Burst of March 29, 1998
• Imaging an Event Horizon: submm-VLBI of a Super Massive Black Hole
• The Influence of Central Black Holes on Gravitational Lenses
• Gravitational lensing as a probe of cold dark matter subhalos
• Radio limits on off-axis GRB afterglows and VLBI observations of SN 2003gk
• High-Angular-Resolution and High-Sensitivity Science Enabled by Beamformed ALMA

Cited by in corpus (1)

• Reference Array and Design Consideration for the next-generation Event Horizon Telescope