Gaussian perturbation interpretability for regression techniques to understand retrieval black-boxes

Abstract

Characterizing the atmospheres of large numbers of exoplanets from their observed spectra can answer key questions about planetary populations. Traditional retrievals are resource-intensive; machine-learning methods offer a fast alternative, but their complexity renders them black-boxes. Understanding how they reach their predictions allows us to inspect them for biases and verify predictions are based on physically plausible spectral features. Commonly used perturbation-based methods for explaining machine learning struggle with the strong correlations between inputs in spectra. We introduce Gaussian perturbation methodology for explaining black-box regression models with one-dimensional structured inputs, which addresses these issues by leveraging physical knowledge of spectral correlation. We compare it with naive perturbation of one wavelength bin at a time. Although traditional retrievals are built on physical forward models, their mapping from spectrum to retrieved parameters is a black-box. We therefore first demonstrate this methodology using TauREx-III as a proxy black-box retrieval on HST observations of HD 209458 b and JWST observations of HAT-P-26 b, showing the efficacy of Gaussian perturbation-based interpretability against a well-established, physically motivated retrieval framework, before applying it to a convolutional neural network retrieval of HAT-P-26 b. For the gases the retrievals constrain, the importance from Gaussian perturbation agrees with the expected importance, including at the core of the $C{O}_2$ band of HAT-P-26 b, where a degeneracy inverts the response. The method shows which parts of a spectrum a retrieval relies on, which the physics of its forward model alone does not, and shows the neural network has learned the same degeneracy.

arXiv commentHeavily revised new version! 18 pages, 13 figures, in review