Enhancing the Interpretability of Radial Velocity Analysis Using Leave-one-out Cross-validation

Abstract

Radial velocity (RV) observations are reaching increasingly high precision, enabling the targeting of low-amplitude signals. Assessing results obtained at the limits of our capabilities requires understanding the relationship between data, models and inference. Leave-one-out cross-validation (LOO-CV) has become an increasingly popular tool for interpretable model criticism in astronomical data analysis. In this work, we demonstrate how LOO-CV can provide insight into the RV analysis of exoplanetary systems through three case studies. We first investigate the anomalously high mass of HD 119130 b inferred from CARMENES RVs, later revised significantly downward after follow-up observations with other instruments. We show that this high mass was strongly driven by two individual measurements, whose removal significantly reduces the tension between data sets. Next, we use LOO-CV as a model selection tool for the GJ 4276 system. Here, LOO-CV prefers an eccentric single-planet model over a two-planet model, and identifies the orbital phases at which follow-up observations would best discriminate between them. Finally, we extend LOO-CV to models with correlated noise, applying it to the GJ 357 system, whose RVs are modeled with a Gaussian process. We find no predictive support for a fourth planetary signal that is moderately favored by the Bayesian evidence, strengthening the interpretation of a three-planet architecture previously reported in the literature. We provide practical guidance for implementing LOO-CV in RV analysis, together with an online tutorial. Such methods will become ever more important as future RV surveys push the limits of planet detection and characterization.

arXiv commentAccepted for publication in The Astronomical Journal. We include an online tutorial to apply LOO-CV to RV data sets. The tutorial is hosted on GitHub at https://github.com/mcnixon/rv_loo_cv_tutorial and preserved on Zenodo at https://zenodo.org/records/22779652