catsHTM - A tool for fast accessing and cross-matching large astronomical catalogs

Abstract

Fast access to large catalogs is required for some astronomical applications. Here we introduce the catsHTM tool, consisting of several large catalogs reformatted into HDF5-based file format, which can be downloaded and used locally. To allow fast access, the catalogs are partitioned into hierarchical triangular meshes and stored in HDF5 files. Several tools are provided to perform efficient cone searches at resolutions spanning from a few arc seconds to degrees, within a few milliseconds time. The first released version includes the following catalogs (by alphabetical order): 2MASS, 2MASS extended sources, AKARI, APASS, Cosmos, DECaLS/DR5, FIRST, GAIA/DR1, GAIA/DR2, GALEX/DR6Plus7, HSC/v2, IPHAS/DR2, NED redshifts, NVSS, Pan-STARRS1/DR1, PTF photometric catalog, ROSAT faint source, SDSS sources, SDSS/DR14 spectroscopy, Spitzer/SAGE, Spitzer/IRAC galactic center, UCAC4, UKIDSS/DR10, VST/ATLAS/DR3, VST/KiDS/DR3, WISE and XMM. We provide Python code that allows to perform cone searches, as well as MATLAB code for performing cone searches, catalog cross-matching, general searches, as well as load and create these catalogs.

Submitted to PASP. Comments are welcome

References in corpus (12)

• The UKIRT Infrared Deep Sky Survey (UKIDSS)
• Gaia Data Release 1. Summary of the astrometric, photometric, and survey properties
• SkyMapper Southern Survey: First Data Release (DR1)
• The XMM-Newton serendipitous survey. VII. The third XMM-Newton serendipitous source catalogue
• The third data release of the Kilo-Degree Survey and associated data products
• Finding counterparts for All-sky X-ray surveys with Nway: a Bayesian algorithm for cross-matching multiple catalogues
• The Second Data Release of the INT Photometric H-Alpha Survey of the Northern Galactic Plane (IPHAS DR2)
• Probabilistic Cross-Identification of Astronomical Sources
• Version 1 of the Hubble Source Catalog
• Point Sources from a Spitzer IRAC Survey of the Galactic Center
• Probabilistic multi-catalogue positional cross-match
• C3, A Command-line Catalogue Cross-match tool for large astrophysical catalogues

Cited by in corpus (36)

• Seventeen Tidal Disruption Events from the First Half of ZTF Survey Observations: Entering a New Era of Population Studies
• A tidal disruption event coincident with a high-energy neutrino
• The Automatic Learning for the Rapid Classification of Events (ALeRCE) Alert Broker
• Discovery and confirmation of the shortest gamma ray burst from a collapsar
• Bright, months-long stellar outbursts announce the explosion of interaction-powered supernovae
• Optical follow-up of the neutron star-black hole mergers S200105ae and S200115j
• Kilonova Luminosity Function Constraints based on Zwicky Transient Facility Searches for 13 Neutron Star Mergers
• Transient processing and analysis using $\text{AMPEL}$: Alert Management, Photometry and Evaluation of Lightcurves
• A Search for Extragalactic Fast Blue Optical Transients in ZTF and the Rate of AT2018cow-like Transients
• The prevalence and influence of circumstellar material around hydrogen-rich supernova progenitors
• Fast-transient Searches in Real Time with ZTFReST: Identification of Three Optically-discovered Gamma-ray Burst Afterglows and New Constraints on the Kilonova Rate
• A Family Tree of Optical Transients from Narrow-Line Seyfert 1 Galaxies
• The ANTARES Astronomical Time-Domain Event Broker
• Triage of the Gaia DR3 astrometric orbits. I. A sample of binaries with probable compact companions
• Alert Classification for the ALeRCE Broker System: The Real-time Stamp Classifier
• Variability-selected intermediate mass black hole candidates in dwarf galaxies from ZTF and WISE
• Transient-optimised real-bogus classification with Bayesian Convolutional Neural Networks -- sifting the GOTO candidate stream
• The Large Array Survey Telescope -- System Overview and Performances
• A Systematic Study of Ia-CSM Supernovae from the ZTF Bright Transient Survey
• AGN on the move: A search for off-nuclear AGN from recoiling SMBHs and ongoing galaxy mergers with the Zwicky Transient Facility
• A high-rate foreground of sub-second flares from geosynchronous satellites
• A code for robust astrometric solution of astronomical images
• A catalog of over ten million variable source candidates in ZTF data release 1
• The Large Array Survey Telescope -- Science Goals
• AXS: A framework for fast astronomical data processing based on Apache Spark
• Towards the measurement of the mass of isolated neutron stars - Prediction of future astrometric microlensing events by pulsars
• AT2018lqh and the nature of the emerging population of day-scale duration optical transients
• Searching For Fermi GRB Optical Counterparts With The Prototype Gravitational-Wave Optical Transient Observer (GOTO)
• AstroCatR: a Mechanism and Tool for Efficient Time Series Reconstruction of Large-Scale Astronomical Catalogues
• The Weizmann Fast Astronomical Survey Telescope (W-FAST): System Overview
• HLC2: a highly efficient cross-matching framework for large astronomical catalogues on heterogeneous computing environments
• Accurate photometric calibration by fitting the system transmission
• The Gravitational-wave Optical Transient Observer (GOTO): prototype performance and prospects for transient science
• Candidate strongly lensed type Ia supernovae in the Zwicky Transient Facility archive
• Faint, high proper motion star candidates selected from the SDSS and PS1 catalogs
• The Large Array Survey Telescope-Pipeline. II. Image Subtraction and Transient Detection