HEALPix Alchemy: Fast All-Sky Geometry and Image Arithmetic in a Relational Database for Multimessenger Astronomy Brokers
arXiv:2112.06947 · doi:10.3847/1538-3881/ac5ab8
Abstract
Efficient searches for electromagnetic counterparts to gravitational wave, high-energy neutrino, and gamma-ray burst events demand rapid processing of image arithmetic and geometry set operations in a database to cross-match galaxy catalogs, observation footprints, and all-sky images. Here we introduce HEALPix Alchemy, an open-source, pure Python implementation of a set of methods that enables rapid all-sky geometry calculations. HEALPix Alchemy is built upon HEALPix, a spatial indexing strategy that is widely used in astronomical databases as well as the native format of LIGO-Virgo-KAGRA gravitational-wave sky localization maps. Our approach leverages new multirange types built into the PostgreSQL 14 database engine. This enables fast all-sky queries against probabilistic multimessenger event localizations and telescope survey footprints. Questions such as "What are the galaxies contained within the 90% credible region of an event?" and "What is the rank-ordered list of the fields within an observing footprint with the highest probability of containing the event?" can be performed in less than a few seconds on commodity hardware using off-the-shelf cloud-managed database implementations without server-side database extensions. Common queries scale roughly linearly with the number of telescope pointings. As the number of fields grows into the hundreds or thousands, HEALPix Alchemy is orders of magnitude faster than other implementations. HEALPix Alchemy is now used as the spatial geometry engine within SkyPortal, which forms the basis of the Zwicky Transient Facility transient marshal, called Fritz.
arXiv commentFor source code, see https://github.com/skyportal/healpix-alchemy
References in corpus (37)
- Array Programming with NumPy
- Astropy: A Community Python Package for Astronomy
- The Astropy Project: Building an inclusive, open-science project and status of the v2.0 core package
- HEALPix -- a Framework for High Resolution Discretization, and Fast Analysis of Data Distributed on the Sphere
- Advanced Virgo: a 2nd generation interferometric gravitational wave detector
- LSST: from Science Drivers to Reference Design and Anticipated Data Products
- Advanced LIGO
- The Zwicky Transient Facility: System Overview, Performance, and First Results
- The Fermi Gamma-Ray Burst Monitor
- The Zwicky Transient Facility: Data Processing, Products, and Archive
- Las Cumbres Observatory Global Telescope Network
- Astrometry.net: Blind astrometric calibration of arbitrary astronomical images
- The Zwicky Transient Facility: Science Objectives
- astroquery: An Astronomical Web-Querying Package in Python
- Observation of gravitational waves from two neutron star-black hole coalescences
- The 2MASS Redshift Survey - Description and Data Release
- Representations of celestial coordinates in FITS
- Rapid Bayesian position reconstruction for gravitational-wave transients
- The Zwicky Transient Facility: Observing System
- Going the Distance: Mapping Host Galaxies of LIGO and Virgo Sources in Three Dimensions Using Local Cosmography and Targeted Follow-up
- Representations of world coordinates in FITS
- The IceCube Realtime Alert System
- The Automatic Learning for the Rapid Classification of Events (ALeRCE) Alert Broker
- The GROWTH Marshal: A Dynamic Science Portal for Time-Domain Astronomy
- Localization of Gamma-Ray Bursts using the Fermi Gamma-Ray Burst Monitor
- Data-driven expectations for electromagnetic counterpart searches based on LIGO/Virgo public alerts
- Fink, a new generation of broker for the LSST community
- Transient processing and analysis using : Alert Management, Photometry and Evaluation of Lightcurves
- Kilonova Luminosity Function Constraints based on Zwicky Transient Facility Searches for 13 Neutron Star Mergers
- The ANTARES Astronomical Time-Domain Event Broker
- Hierarchical progressive surveys. Multi-resolution HEALPix data structures for astronomical images, catalogues, and 3-dimensional data cubes
- The Gravitational Wave Treasure Map: A Tool to Coordinate, Visualize, and Assess the Electromagnetic Follow-Up of Gravitational Wave Events
- Evaluation of Automated Fermi GBM Localizations of Gamma-ray Bursts
- IVOA Recommendation: HiPS - Hierarchical Progressive Survey
- MOC - HEALPix Multi-Order Coverage map Version 1.0
- Efficient data structures for masks on 2D grids
- The NASA Multi-Messenger Astrophysics Science Support Center (MOSSAIC)