Trend Filtering -- I. A Modern Statistical Tool for Time-Domain Astronomy and Astronomical Spectroscopy
arXiv:1908.07151 · doi:10.1093/mnras/staa106
Abstract
The problem of denoising a one-dimensional signal possessing varying degrees of smoothness is ubiquitous in time-domain astronomy and astronomical spectroscopy. For example, in the time domain, an astronomical object may exhibit a smoothly varying intensity that is occasionally interrupted by abrupt dips or spikes. Likewise, in the spectroscopic setting, a noiseless spectrum typically contains intervals of relative smoothness mixed with localized higher frequency components such as emission peaks and absorption lines. In this work, we present trend filtering, a modern nonparametric statistical tool that yields significant improvements in this broad problem space of denoising signals. When the underlying signal is spatially heterogeneous, trend filtering is superior to any statistical estimator that is a linear combination of the observed data---including kernel smoothers, LOESS, smoothing splines, Gaussian process regression, and many other popular methods. Furthermore, the trend filtering estimate can be computed with practical and scalable efficiency via a specialized convex optimization algorithm, e.g. handling sample sizes of within a few minutes. In a companion paper, we explicitly demonstrate the broad utility of trend filtering to observational astronomy by carrying out a diverse set of spectroscopic and time-domain analyses.
arXiv commentPart 1 of 2; 15 pages, 3 figures
References in corpus (24)
- On the Complexity of Best Arm Identification in Multi-Armed Bandit Models
- Spectral Classification and Redshift Measurement for the SDSS-III Baryon Oscillation Spectroscopic Survey
- The solution path of the generalized lasso
- Towards a Precise Measurement of Matter Clustering: Lyman-alpha Forest Data at Redshifts 2-4
- Unusual Broad Absorption Line Quasars from the Sloan Digital Sky Survey
- The Carnegie Supernova Project: First Photometry Data Release of Low-Redshift Type Ia Supernovae
- A Gaussian process framework for modelling instrumental systematics: application to transmission spectroscopy
- A Trend Filtering Algorithm for wide field variability surveys
- from cosmic chronometers and Type Ia supernovae, with Gaussian Processes and the novel Weighted Polynomial Regression method
- Degrees of freedom in lasso problems
- Adaptive piecewise polynomial estimation via trend filtering
- Gemini Spectroscopy of Supernovae from SNLS: Improving High Redshift SN Selection and Classification
- K2SC: Flexible systematics correction and detrending of K2 light curves using Gaussian Process regression
- Type Ia Supernova Light Curves
- Fast and Flexible ADMM Algorithms for Trend Filtering
- Uncovering the Intrinsic Variability of Gamma-ray Bursts
- The Shape of the Primordial Power Spectrum: A Last Stand Before Planck
- The late-time light curve of the type Ia supernova SN 2011fe
- A wavelet analysis of QSO spectra
- Near-infrared light curves of Type Ia supernovae: Studying the properties of the second maximum
- Mock Quasar-Lyman-α Forest Data-sets for the SDSS-III Baryon Oscillation Spectroscopic Survey
- Gradient Particle Magnetohydrodynamics
- Light curve modeling of fast-evolving supernova KSN 2015K: explosion in circumstellar matter of Super-AGB progentor
- Trend Filtering -- II. Denoising Astronomical Signals with Varying Degrees of Smoothness