paper

Marvin: A Toolkit for Streamlined Access and Visualization of the SDSS-IV MaNGA Data Set

arXiv:1812.03833 · doi:10.3847/1538-3881/ab2634

Abstract

The Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) survey, one of three core programs of the fourth-generation Sloan Digital Sky Survey (SDSS-IV), is producing a massive, high-dimensional integral field spectroscopic data set. However, leveraging the MaNGA data set to address key questions about galaxy formation presents serious data-related challenges due to the combination of its spatially inter-connected measurements and sheer volume. For each galaxy, the MaNGA pipelines produce relatively large data files to preserve the spatial correlations of the spectra and measurements, but this comes at the expense of storing the data set in a coarsely-chunked manner. The coarse chunking and total volume of the data make it time-consuming to download and curate locally-stored data. Thus, accessing, querying, visually exploring, and performing statistical analyses across the whole data set at a fine-grained scale is extremely challenging using just FITS files. To overcome these challenges, we have developed \marvin: a toolkit consisting of a Python package, Application Programming Interface (API), and web application utilizing a remote database. \marvin's robust and sustainable design minimizes maintenance, while facilitating user-contributed extensions such as high level analysis code. Finally, we are in the process of abstracting out \marvin's core functionality into a separate product so that it can serve as a foundation for others to develop \marvin-like systems for new science applications.

arXiv comment26 pages, 14 figures

References in corpus (20)

Cited by in corpus (83)