Does "model-free" forecasting really outperform the "true" model? A reply to Perretti et al
arXiv:1305.3544 · doi:10.1073/pnas.1308603110
Abstract
Estimating population models from uncertain observations is an important problem in ecology. Perretti et al. observed that standard Bayesian state-space solutions to this problem may provide biased parameter estimates when the underlying dynamics are chaotic. Consequently, forecasts based on these estimates showed poor predictive accuracy compared to simple "model-free" methods, which lead Perretti et al. to conclude that "Model-free forecasting outperforms the correct mechanistic model for simulated and experimental data". However, a simple modification of the statistical methods also suffices to remove the bias and reverse their results.
Letter submitted to PNAS, with additional supplementary information. R code included in the latex source