Comparing the frequentist and Bayesian periodic signal detection: rates of statistical mistakes and sensitivity to priors

Abstract

We perform extensive Monte Carlo simulations to systematically compare the frequentist and Bayesian treatments of the Lomb--Scargle periodogram. The goal is to investigate whether the Bayesian period search is advantageous over the frequentist one in terms of the detection efficiency, how much if yes, and how sensitive it is regarding the choice of the priors, in particular in case of a misspecified prior (whenever the adopted prior does not match the actual distribution of physical objects). We find that the Bayesian and frequentist analyses always offer nearly identical detection efficiency in terms of their tradeoff between type-I and type-II mistakes. Bayesian detection may reveal a formal advantage if the frequency prior is nonuniform, but this results in only $\sim 1$ per cent extra detected signals. In case if the prior was misspecified (adopting nonuniform one over the actual uniform) this may turn into an opposite advantage of the frequentist analysis. Finally, we revealed that Bayes factor of this task appears rather overconservative if used without a calibration against type-I mistakes (false positives), thereby necessitating such a calibration in practice.

arXiv comment16 pages, 9 figures; accepted by MNRAS

References in corpus (17)

• MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics
• The generalised Lomb-Scargle periodogram. A new formalism for the floating-mean and Keplerian periodograms
• Assessing statistical significance of periodogram peaks
• The Lick Planet Search: Detectability and Mass Thresholds
• Radial velocity planets de-aliased. A new, short period for Super-Earth 55 Cnc e
• Detectability of Extrasolar Planets in Radial Velocity Surveys
• BGLS: A Bayesian formalism for the generalised Lomb-Scargle periodogram
• Accounting for velocity jitter in planet search surveys
• The impact of red noise in radial velocity planet searches: Only three planets orbiting GJ581?
• Bayesian Approaches to Copula Modelling
• Radial Velocity Data Analysis with Compressed Sensing Techniques
• Quantifying the Bayesian Evidence for a Planet in Radial Velocity Data
• Detecting multiple periodicities in observational data with the multi-frequency periodogram. I. Analytic assessment of the statistical significance
• Enhanced models for stellar Doppler noise reveal hints of a 13-year activity cycle of 55 Cancri
• Distinguishing between a true period and its alias, and other tasks of model discrimination
• Detecting non-sinusoidal periodicities in observational data using multi-harmonic periodograms
• Keplerian periodogram for Doppler exoplanets detection: optimized computation and analytic significance thresholds

Cited by in corpus (2)

• Cleaning WASP-33 b transits from the host star photometric variability: analysis of TESS data from two sectors
• Information-based Preprocessing of PLC Data for Automatic Behavior Modeling