Model Choice and Diagnostics for Linear Mixed-Effects Models Using Statistics on Street Corners

ABSTRACT

The complexity of linear mixed-effects (LME) models means that traditional diagnostics are rendered less effective. This is due to a breakdown of asymptotic results, boundary issues, and visible patterns in residual plots that are introduced by the model fitting process. Some of these issues are well known and adjustments have been proposed. Working with LME models typically requires that the analyst keeps track of all the special circumstances that may arise. In this article, we illustrate a simpler but generally applicable approach to diagnosing LME models. We explain how to use new visual inference methods for these purposes. The approach provides a unified framework for diagnosing LME fits and for model selection. We illustrate the use of this approach on several commonly available datasets. A large-scale Amazon Turk study was used to validate the methods. R code is provided for the analyses. Supplementary materials for this article are available online.

KEYWORDS:

• Lineup protocol
• Model diagnostics
• Model selection
• Statistical graphics
• Visual inference

Supplementary Material

Supplement.pdf The supplement materials include a description of the datasets used throughout this article, the experimental setup, and the calculation of p-values. Additional lineups that were omitted for brevity are also included. (PDF)

Study.csv Anonymized results from the Amazon Turk study. (CSV)

Analysis.r R script reproducing the results. (R)

Acknowledgments

We would like to thank Mahbubul Majumder and Eric Hare, who conducted the Amazon MTurk study (mahbub.stat.iastate.edu/feedback_turk11/homepage.html). R (R Core Team 2013) was used to implement all analyses and methods discussed in this article: lme4 (Bates et al. 2015) and HLMdiag (Loy and Hofmann 2014) were used for model fitting and calculation of diagnostics, respectively. ggplot2 (Wickham 2009), nullabor (Wickham 2012), and gridSVG (Murrell and Potter 2013) provided the basis for the visualizations. The authors gratefully acknowledge funding from the National Science Foundation Grant #DMS 1007697. All data collection has been conducted with approval from the Institutional Review Board IRB 10-347. This article was reviewed and accepted prior to Dianne Cook becoming Editor of JCGS.

Notes

1 We encode the panel number as a mathematical expression to pose a cognitive obstacle, allowing the reader to evaluate the lineup before being biased by knowing the answer.