Balancing Type I Error and Power in Linear Mixed Models

Abstract

Linear mixed-effects models have increasingly replaced mixed-model analyses of variance for statistical inference in factorial psycholinguistic experiments. Although LMMs have many advantages over ANOVA, like ANOVAs, setting them up for data analysis also requires some care. One simple option, when numerically possible, is to fit the full variance-covariance structure of random effects (the maximal model; Barr et al. 2013), presumably to keep Type I error down to the nominal alpha in the presence of random effects. Although it is true that fitting a model with only random intercepts may lead to higher Type I error, fitting a maximal model also has a cost: it can lead to a significant loss of power. We demonstrate this with simulations and suggest that for typical psychological and psycholinguistic data, higher power is achieved without inflating Type I error rate if a model selection criterion is used to select a random effect structure that is supported by the data.

References in corpus (5)

• Parsimonious Mixed Models
• An investigation of the false discovery rate and the misinterpretation of P values
• Statistical methods for linguistic research: Foundational Ideas - Part II
• Bayesian linear mixed models using Stan: A tutorial for psychologists, linguists, and cognitive scientists
• On the Ambiguity of Interaction and Nonlinear Main Effects in a Regime of Dependent Covariates

Cited by in corpus (12)

• Statistical methods for linguistic research: Foundational Ideas - Part II
• Bayesian linear mixed models using Stan: A tutorial for psychologists, linguists, and cognitive scientists
• How much baseline correction do we need in ERP research? Extended GLM model can replace baseline correction while lifting its limits
• Statistical methods for linguistic research: Foundational Ideas - Part I
• Computation and application of generalized linear mixed model derivatives using lme4
• Confronting Quasi-Separation in Logistic Mixed Effects for Linguistic Data: A Bayesian Approach
• Application of the hierarchical bootstrap to multi-level data in neuroscience
• On the Ambiguity of Interaction and Nonlinear Main Effects in a Regime of Dependent Covariates
• When a Man Says He Is Pregnant: Event-related Potential Evidence for a Rational Account of Speaker-contextualized Language Comprehension
• Probabilistic adaptation of language comprehension for individual speakers: evidence from neural oscillations
• Cooperation and the social brain hypothesis in primate social networks
• A Framework for Learning Assessment through Multimodal Analysis of Reading Behaviour and Language Comprehension