A Plot is Worth a Thousand Tests: Assessing Residual Diagnostics with the Lineup Protocol

References

• Allaire, J. J., Xie, Y., Dervieux, C., R Foundation, Wickham, H., Journal of Statistical Software, Vaidyanathan, R., et al. (2022), rticles: Article formats for R Markdown. R package version 0.24. Available at https://CRAN.R-project.org/package=rticles.
   Google Scholar
• Box, G. E. P. (1976), “Science and Statistics,” Journal of the American Statistical Association, 71, 791–799.
   Web of Science ®Google Scholar
• Breusch, T. S., and Pagan, A. R. (1979), “A Simple Test for Heteroscedasticity and Random Coefficient Variation,” Econometrica: Journal of the Econometric Society, 47, 1287–1294.
   Web of Science ®Google Scholar
• Buja, A., Cook, D., Hofmann, H., Lawrence, M., Lee, E.-K., Swayne, D. F., and Wickham, H. (2009a), “Statistical Inference for Exploratory Data Analysis and Model Diagnostics,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 367, 4361–4383.
   PubMed Web of Science ®Google Scholar
• ——(2009b), “Statistical Inference for Exploratory Data Analysis and Model Diagnostics,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 367, 4361–4383.
   Google Scholar
• Cleveland, W. S., and McGill, R. (1984), “Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods,” Journal of the American Statistical Association, 79, 531–554.
   Web of Science ®Google Scholar
• Cook, R. D., and Weisberg, S. (1982), Residuals and Influence in Regression, New York: Chapman and Hall.
   Google Scholar
• ——(1999), Applied Rregression Including Computing and Graphics, New York: Wiley.
   Google Scholar
• De Leeuw, J. R. (2015), “jsPsych: A JavaScript Library for Creating Behavioral Experiments in a Web Browser,” Behavior Research Methods, 47, 1–12.
   PubMed Web of Science ®Google Scholar
• Draper, N. R., and Smith, H. (1998), Applied Regression Analysis (Vol. 326), Hoboken, NJ: Wiley.
   Google Scholar
• Farrar, T. J. (2020), skedastic: Heteroskedasticity Diagnostics for Linear Regression Models, Bellville, South Africa. R Package Version 1.0.0.
   Google Scholar
• Grinberg, M. (2018), Flask Web Development: Developing Web Applications with Python, Sebastopol, CA: O’Reilly Media, Inc.
   Google Scholar
• Hermite, M. (1864), Sur un nouveau développement en série des fonctions, Imprimerie de Gauthier-Villars.
   Google Scholar
• Jamshidian, M., Jennrich, R. I., and Liu, W. (2007), “A Study of Partial F tests for Multiple Linear Regression Models,” Computational Statistics & Data Analysis, 51, 6269–6284.
   Web of Science ®Google Scholar
• Jarque, C. M., and Bera, A. K. (1980), “Efficient Tests for Normality, Homoscedasticity and Serial Independence of Regression Residuals,” Economics Letters, 6, 255–259.
   Web of Science ®Google Scholar
• Jeppson, H., Hofmann, H., and Cook, D. (2021), ggmosaic: Mosaic Plots in the ’ggplot2’ Framework, R package version 0.3.3. Available at https://CRAN.R-project.org/package=ggmosaic.
   Google Scholar
• Kahle, D. (2013), “mpoly: Multivariate Polynomials in R,” The R Journal, 5, 162–170.
   Google Scholar
• Kahneman, D. (2011), Thinking, Fast and Slow, New York: Farrar, Straus and Giroux.
   Google Scholar
• Kimball, A. W. (1957), “Errors of the Third Kind in Statistical Consulting,” Journal of the American Statistical Association, 52, 133–142.
   Web of Science ®Google Scholar
• Kirk, R. E. (1996), “Practical Significance: A Concept Whose Time Has Come,” Educational and Psychological Measurement, 56, 746–759.
   Web of Science ®Google Scholar
• Kullback, S., and Leibler, R. A. (1951), “On Information and Sufficiency,” The Annals of Mathematical Statistics, 22, 79–86.
   Google Scholar
• Laplace, P.-S. (1820), Théorie analytique des probabilités (Vol. 7), Paris: Courcier.
   Google Scholar
• Loy, A. (2021), “Bringing Visual Inference to the Classroom,” Journal of Statistics and Data Science Education, 29, 171–182.
   Web of Science ®Google Scholar
• Loy, A., Follett, L., and Hofmann, H. (2016), “Variations of Q–Q Plots: The Power of Our Eyes!” The American Statistician, 70, 202–214.
   Web of Science ®Google Scholar
• Loy, A., and Hofmann, H. (2013), “Diagnostic Tools for Hierarchical Linear Models,” Wiley Interdisciplinary Reviews: Computational Statistics, 5, 48–61.
   Google Scholar
• ——(2014), “HLMdiag: A Suite of Diagnostics for Hierarchical Linear Models in R,” Journal of Statistical Software, 56, 1–28.
   Web of Science ®Google Scholar
• ——(2015), “Are You Normal? The Problem of Confounded Residual Structures in Hierarchical Linear Models,” Journal of Computational and Graphical Statistics, 24, 1191–1209.
   Web of Science ®Google Scholar
• Majumder, M., Hofmann, H., and Cook, D. (2013a), “Validation of Visual Statistical Inference, Applied to Linear Models,” Journal of the American Statistical Association, 108, 942–956.
   Web of Science ®Google Scholar
• ——(2013b), “Validation of Visual Statistical Inference, Applied to Linear Models,” Journal of the American Statistical Association, 108, 942–956.
   Google Scholar
• Montgomery, D. C., Peck, E. A., and Geoffrey Vining, G. (1982), Introduction to Linear Regression Analysis, Hoboken, NJ: Wiley.
   Google Scholar
• Nowosad, J. (2018), ’CARTOColors’ palettes. R package version 1.0, Available at https://nowosad.github.io/rcartocolor.
   Google Scholar
• Olvera Astivia, O. L., Gadermann, A., and Guhn, M. (2019), “The Relationship between Statistical Power and Predictor Distribution in Multilevel Logistic Regression: A Simulation-based Approach,” BMC Medical Research Methodology, 19, 1–20.
   PubMed Web of Science ®Google Scholar
• Palan, S., and Schitter, C. (2018), “Prolific. ac—A Subject Pool for Online Experiments,” Journal of Behavioral and Experimental Finance, 17, 22–27.
   Web of Science ®Google Scholar
• Pedersen, T. L. (2022), patchwork: The Composer of Plots, R package version 1.1.2, available at https://CRAN.R-project.org/package=patchwork.
   Google Scholar
• PythonAnywhere LLP. (2023), “PythonAnywhere,” available at https://www.pythonanywhere.com.
   Google Scholar
• Ramsey, J. B. (1969), “Tests for Specification Errors in Classical Linear Least-Squares Regression Analysis,” Journal of the Royal Statistical Society, Series B, 31, 350–371.
   Google Scholar
• Roy Chowdhury, N., Cook, D., Hofmann, H., Majumder, M., Lee, E.-K., and Toth, A. L. (2015), “Using Visual Statistical Inference to Better Understand Random Class Separations in High Dimension, Low Sample Size Data,” Computational Statistics, 30, 293–316.
   Web of Science ®Google Scholar
• Shapiro, S. S., and Wilk, M. B. (1965), “An Analysis of Variance Test for Normality (Complete Samples),” Biometrika, 52, 591–611.
   Web of Science ®Google Scholar
• Silvey, S. D. (1959), “The Lagrangian Multiplier Test,” The Annals of Mathematical Statistics, 30, 389–407.
   Google Scholar
• VanderPlas, S., and Hofmann, H. (2016), “Spatial Reasoning and Data Displays,” IEEE Transactions on Visualization and Computer Graphics, 22, 459–468. DOI: 10.1109/TVCG.2015.2469125.
   PubMed Web of Science ®Google Scholar
• VanderPlas, S., Röttger, C., Cook, D., and Hofmann, H. (2021), “Statistical Significance Calculations for Scenarios in Visual Inference,” Stat, 10, e337.
   Web of Science ®Google Scholar
• White, H. (1980), “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity,” Econometrica: Journal of the Econometric Society, 48, 817–838.
   Web of Science ®Google Scholar
• Wickham, H., Averick, M., Bryan, J., Chang, W., D’Agostino McGowan, L., François, R., Grolemund, G., et al. (2019), “Welcome to the tidyverse,” Journal of Open Source Software, 4, 1686.
   Google Scholar
• Wickham, H., Cook, D., Hofmann, H., and Buja, A. (2010), “Graphical Inference for Infovis,” IEEE Transactions on Visualization and Computer Graphics, 16, 973–979. DOI: 10.1109/TVCG.2010.161.
   PubMed Web of Science ®Google Scholar
• Xie, Y. (2014), “knitr: A Comprehensive Tool for Reproducible Research in R,” in Implementing Reproducible Computational Research, eds. V. Stodden, F. Leisch, and R. D. Peng. Boca Raton, FL: Chapman and Hall/CRC. Available at http://www.crcpress.com/product/isbn/9781466561595.
   Google Scholar
• Xie, Y., Dervieux, C., and Riederer, E. (2020), R Markdown Cookbook. Boca Raton, FL: Chapman and Hall/CRC. Available at https://bookdown.org/yihui/rmarkdown-cookbook.
   Google Scholar
• Yin, T., Majumder, M., Roy Chowdhury, N., Cook, D., Shoemaker, R., and Graham, M. (2013), “Visual Mining Methods for RNA-Seq Data: Data Structure, Dispersion Estimation and Significance Testing,” Journal of Data Mining in Genomics and Proteomics, 4, 137.
   Google Scholar
• Zeileis, A., and Hothorn, T. (2002), “Diagnostic Checking in Regression Relationships,” R News, 2, 7–10.
   Google Scholar
• Zhang, Z., and Yuan, K.-H. (2018). Practical Statistical Power Analysis Using Webpower and R, Granger, IN: Isdsa Press.
   Google Scholar
• Zhu, H. (2021), kableExtra: Construct Complex Table with kable and Pipe Syntax, R package version 1.3.4. Available at https://CRAN.R-project.org/package=kableExtra.
   Google Scholar