Dos and don'ts of reduced chi-squared

Abstract

Reduced chi-squared is a very popular method for model assessment, model comparison, convergence diagnostic, and error estimation in astronomy. In this manuscript, we discuss the pitfalls involved in using reduced chi-squared. There are two independent problems: (a) The number of degrees of freedom can only be estimated for linear models. Concerning nonlinear models, the number of degrees of freedom is unknown, i.e., it is not possible to compute the value of reduced chi-squared. (b) Due to random noise in the data, also the value of reduced chi-squared itself is subject to noise, i.e., the value is uncertain. This uncertainty impairs the usefulness of reduced chi-squared for differentiating between models or assessing convergence of a minimisation procedure. The impact of noise on the value of reduced chi-squared is surprisingly large, in particular for small data sets, which are very common in astrophysical problems. We conclude that reduced chi-squared can only be used with due caution for linear models, whereas it must not be used for nonlinear models at all. Finally, we recommend more sophisticated and reliable methods, which are also applicable to nonlinear models.

12 pages, 3 figures, 1 table

Cited by in corpus (91)

• Indirect to direct bandgap transition in methylammonium lead halide perovskite
• Neutron star mass and radius measurements from atmospheric model fits to X-ray burst cooling tail spectra
• Stellar mass functions: methods, systematics and results for the local Universe
• Comparative performance of selected variability detection techniques in photometric time series
• First test of Verlinde's theory of Emergent Gravity using Weak Gravitational Lensing measurements
• Lensing Constraints on the Mass Profile Shape and the Splashback Radius of Galaxy Clusters
• Broadband Spectral Modeling of the Extreme Gigahertz-Peaked Spectrum Radio Source PKS B0008-421
• A near-infrared census of the multi-component stellar structure of early-type dwarf galaxies in the Virgo cluster
• Dark Energy Survey Year 3 Results: Point-Spread Function Modeling
• Interpretation and diversity of exoplanetary material orbiting white dwarfs
• Cosmological Measurements from Angular Power Spectra Analysis of BOSS DR12 Tomography
• The physical properties of Fermi BL Lac objects jets
• The non-linear infrared-radio correlation of low-z galaxies: implications for redshift evolution, a new radio SFR recipe, and how to minimize selection bias
• The environment and distribution of emitting electrons as a function of source activity in Markarian 421
• The geometric distance and binary orbit of PSR B1259-63
• From ergodic to non-ergodic chaos in Rosenzweig-Porter model
• Dissection of the collisional and collisionless mass components in a mini sample of CLASH and HFF massive galaxy clusters at $z\approx 0.4$
• UV & U-band luminosity functions from CLAUDS and HSC-SSP -- I. Using four million galaxies to simultaneously constrain the very faint and bright regimes to $z\sim 3$
• Non-Maxwellian analysis of the transition-region line profiles observed by the Interface Region Imaging Spectrograph
• Observational constraints on Myrzakulov gravity
• Real-space density profile reconstruction of stacked voids
• Alkaline Exospheres of Exoplanet Systems: Evaporative Transmission Spectra
• The fraction of AGN in major merger galaxies and its luminosity dependence
• Precise Radial Velocities of Cool Low Mass Stars With iSHELL
• Collider limits on new physics within micrOMEGAs4.3
• Global Decay Chain Vertex Fitting at B-Factories
• The Extremely High Dark Matter Halo Concentration of the Relic Compact Elliptical Galaxy Mrk 1216
• The galaxy-subhalo connection in low-redshift galaxy clusters from weak gravitational lensing
• Revisiting CFHTLenS cosmic shear: Optimal E/B mode decomposition using COSEBIs and compressed COSEBIs
• Measuring protoplanetary disk gas surface density profiles with ALMA
• PUMA: The Positional Update and Matching Algorithm
• New Periodic 6.7 GHz Class II Methanol Maser Associated with G358.460-0.391
• SALT2 Versus SALT3: Updated Model Surfaces and Their Impacts on Type Ia Supernova Cosmology
• The Solar Neighborhood XXIX: The Habitable Real Estate of our Nearest Stellar Neighbors
• Distance to the IBEX Ribbon Source Inferred from Parallax
• New physics in $b\to s\ell \ell$ decays with complex Wilson coefficients
• A search for transits of GJ 581\lowercase{e} and characterization of the host star variability using MOST space telescope photometry
• Decoding signatures of extra dimensions and estimating spin of quasars from the continuum spectrum
• Loose Ends for the Exomoon Candidate Host Kepler-1625b
• Measuring the magnetic field of young stars using iSHELL observations: BP Tau and V347 Aur
• Plasmoid statistics in relativistic magnetic reconnection
• Detailed elemental abundances of binary stars: Searching for signatures of planet formation and atomic diffusion
• Warm and Cold Molecular Gas Conditions Modeled in 87 Galaxies Observed by the Herschel SPIRE Fourier Transform Spectrometer
• Cosmological constraints on early dark energy from the full shape analysis of eBOSS DR16
• Autonomous Gaussian decomposition of the Galactic Ring Survey. I. Global statistics and properties of the 13CO emission data
• Exploring the Correlations between Globular Cluster Populations and Supermassive Black Holes in Giant Galaxies
• Astrophysical S-factors, thermonuclear rates, and electron screening potential for the ${}^3$He(d,p)${}^4$He Big Bang reaction via a hierarchical Bayesian model
• Search for Lorentz Invariance Violation from stacked Gamma-Ray Burst spectral lag data
• The Hubble PanCET Program: A Metal-rich Atmosphere for the Inflated Hot Jupiter HAT-P-41b
• A Catalog of Calibrator Stars for Next-Generation Optical Interferometers
• A hot super-Earth planet in the WASP-84 planetary system
• Nuclear energy density functionals grounded in ab initio calculations
• The PAU Survey: Measurement of Narrow-band galaxy properties with Approximate Bayesian Computation
• Apsidal motion in the massive binary HD 152248 -- Constraining the internal structure of the stars
• RedMaPPer: Evolution and Mass Dependence of the Conditional Luminosity Functions of Red Galaxies in Galaxy Clusters
• Aerosols and Methane in the Ice Giant Atmospheres Inferred from Spatially Resolved, Near-Infrared Spectra: I. Uranus, 2001-2007
• Model-independent constraints on superfluidity from the cooling neutron star in Cassiopeia A
• The SNR of a Transit
• Emission from Volcanic SO Gas on Io at High Spectral Resolution
• Understanding the Effects of Systematics in Exoplanetary Atmospheric Retrievals
• A microlensing search of 700 million VVV light curves
• Does black hole continuum spectrum signal f(R) gravity in higher dimensions?
• The effects of ${}^{22}$Ne sedimentation and metallicity on the local 40 pc white dwarf luminosity function
• Identification of two new HMXBs in the LMC: a $\sim$2013 s pulsar and a probable SFXT
• Understanding matched filters for precision cosmology
• Galactic synchrotron emissivity measurements between 250° < l < 355° from the GLEAM survey with the MWA
• Bayesian analysis of Juno/JIRAM's NIR observations of Europa
• A resolved and asymmetric ring of PAHs within the young circumstellar disk of IRS 48
• Interpreting the variation phenomena of B2 1633+382 via the two-component model
• Radial profiles of the Phoebe ring, a vast debris disk around Saturn
• TOTEM data and the real part of the hadron elastic amplitude at 13 TeV
• Using Stable Distributions to Characterize Proton Pencil Beams
• On the Evidence for Cosmic Variation of the Fine Structure Constant (I): A Parametric Bayesian Model Selection Analysis of the Quasar Dataset
• Inferring AGN jet parameters using Bayesian analysis of VLBI data with non-uniform jet model
• Protein connectivity in chemotaxis receptor complexes
• A high-resolution view of the filament of gas between Abell 399 and Abell 401 from the Atacama Cosmology Telescope and MUSTANG-2
• Detectability of Polycyclic Aromatic Hydrocarbons in the Atmosphere of WASP-6 b with JWST NIRSpec PRISM
• Warm dark matter model with a few keV mass is bad for the too-big-to-fail problem
• A new tool to derive simultaneously exponent and extremes of power-law distributions
• On the integrated continuum radio spectrum of supernova remnant W44 (G34.7-0.4): new insights from Planck
• Ab initio models of atomic nuclei: challenges and new ideas
• tonalli: an asexual genetic code to characterise APOGEE-2 stellar spectra. I. Validation with synthetic and solar spectra
• Particle Production Scenario in an Algebraically Coupled Quintessence Field with a Dark Matter Fluid
• Implications of the Fermi-LAT Pass 8 Galactic Center Excess on Supersymmetric Dark Matter
• A Chi-Squared Analysis of the Measurements of Two Cosmological Parameters Over Time
• A possible solution of $B$ arrow $\pi \pi$ puzzle in Z' model
• Intermediate evolution using SNIa, and BAO
• Direct Comparison of using a Z-Transformation instead of the traditional $Q^2$ for for Extraction of the Proton Radius from $e-p$ Scattering Data
• How narrow is the M87* ring? I. The choice of closure likelihood function
• A High-Velocity Scatterer Revealed in the Thinning Ejecta of a Type II Supernova
• Development and Application of Tools to Characterize Transiting Astrophysical Systems