Deep Dive Into Visual Representation and Interrater Agreement Using Data From a High-School Diving Competition

References

• Agresti, A. (1992), “Modeling Patterns of Agreement and Disagreement,” Statistical Methods in Medical Research, 1, 201–218. DOI:10.1177/096228029200100205.
   PubMedGoogle Scholar
• Baker, B. O., Hardyck, C., and Petrinivic, L. F. (1966), “Weak Measurements vs Strong Statistics: An Empirical Critique of S.S. Stevens’ Proscriptions on Statistics,” Educational and Psychological Measurement, 26, 291–309. DOI:10.1177/001316446602600204.
   Web of Science ®Google Scholar
• Banerjee, M., Capozzoli, M., McSweeney, L., and Sinha, D. (1999), “Beyond Kappa: A Review of Interrater Agreement Measures,” The Canadian Journal of Statistics, 27, 3–23. DOI:10.2307/3315487.
   Web of Science ®Google Scholar
• Brennan, C. (2018), “U.S. Figure Skating Committee Picks Olympic Stars but Leaves Heartbreak in Its Wake,” USA Today, available at https://www.usatoday.com/story/sports/christinebrennan/2018/01/07/us-figure-skating-pyeongchang-controversy-ross-miner-adam-rippon/1011464001/.
   Google Scholar
• Cao, J. (2014), “Quantifying Randomness Versus Consensus in Wine Quality Ratings,” Journal of Wine Economics, 9, 202–213. DOI:10.1017/jwe.2014.8.
   Google Scholar
• Cohen, J. (1960), “A Coefficient of Agreement for Nominal Scales,” Educational and Psychological Measurement, 20, 37–46. DOI:10.1177/001316446002000104.
   Web of Science ®Google Scholar
• Emerson, J. W., Seltzer, M., and Lin, D. (2009), “Assessing Judging Bias: An Example From the 2000 Olympic Games,” The American Statistician, 63, 124–131. DOI:10.1198/tast.2009.0026.
   Web of Science ®Google Scholar
• Franklin, W. (2017), Calculating Degree of Difficulty for Dives: DD Formula in Springboard and Platform Diving, New York, NY: ThoughtCo, Inc.
   Google Scholar
• Franklin, W. (2018), High School Diving Competition Requirements, New York, NY: ThoughtCo, Inc.
   Google Scholar
• Gamer, M., Lemon, J., Fellows, I., and Sigh, P. (2012), “irr: Various Coefficients of Interrater Reliability and Agreement,” R Package Version 0.84 Edition.
   Google Scholar
• Graham, P. (1995), “Modeling Covariate Effects in Observer Agreement Studies: The Case of Nominal Scale Agreement,” Statistics in Medicine, 14, 299–310. DOI:10.1002/sim.4780140308.
   PubMed Web of Science ®Google Scholar
• Kendall, M. G. (1945), “The Treatment of Ties in Ranking Problems,” Biometrika, 33, 239–251. DOI:10.1093/biomet/33.3.239.
   PubMed Web of Science ®Google Scholar
• Kraemer, H. C. (1980), “Extension of the Kappa Coefficient,” Biometrics, 36, 207–216.
   PubMed Web of Science ®Google Scholar
• Kramer, R. S. S. (2017), “Sequential Effects in Olympic Synchronized Diving Scores,” Royal Society Open Science, 4, 160812, 2017. DOI:10.1098/rsos.160812,.
   PubMed Web of Science ®Google Scholar
• Lane, D. M. (2018), Online Statistics Education: A Multimedia Course of Study, Houston, TX: Rice University, available at http://onlinestatbook.com/.
   Google Scholar
• Looney, M. A. (2012), “Juding Anomalies at the 2010 Olympics in Men’s Figure Skating,” Measurement in Physical Education and Exercise Science, 16, 55–68. DOI:10.1080/1091367X.2012.639602.
   Google Scholar
• Morgan, H. N., and Rotthoff, K. W. (2014), “The Harder the Task, the Higher the Score: Findings of a Difficulty Bias,” Economic Inquiry, 52, 1014–1026, DOI:10.2139/ssrn.1555094.
   Web of Science ®Google Scholar
• Nelson, K., and Edwards, D. (2015), “Measures of Agreement Between Many Raters for Ordinal Classification,” Statistics in Medicine, 34, 3116–3132. DOI:10.1002/sim.6546.
   PubMed Web of Science ®Google Scholar
• Pilon, M., Lehren, A. W., Gosk, S., Sigel, E. R., and Abou-Sabe, K. (2018), “Think Olympic Figure Skating Judges Are Biased? The Data Says They Might Be,” Technical Report, NBC News.
   Google Scholar
• Prabhakaran, S. (2019), “Top 50 ggplot2 Visualizations—The Master List,” available at http://r-statistics.co/Top50-Ggplot2-Visualizations-MasterList-R-Code.html.
   Google Scholar
• R Studio (2019), “Tidyverse: R Packages for Data Science,” available at https://www.tidyverse.org/.
   Google Scholar
• Shrout, P. E., and Fleiss, J. L. (1979), “Intraclass Correlations: Uses in Assessing Rater Reliability,” Psychological Bulletin, 86, 420–428. DOI:10.1037/0033-2909.86.2.420.
   PubMed Web of Science ®Google Scholar
• Smith, J. (2018), “Olympic Judging: Fair or Biased?,” Technical Report, Minitab.
   Google Scholar
• Student (1921), “An Experimental Determination of the Probable Error of Dr Spearman’s Correlation Coefficient,” Biometrika, 29, 322.
   Google Scholar
• UCLA Statistical Consulting Group (2018), “How Can I Visualize Longitudinal Data in ggplot2?,” available at https://stats.idre.ucla.edu/r/faq/how-can-i-visualize-longitudinal-data-in-ggplot2/.
   Google Scholar
• Velleman, P., and Wilkinson, L. (1993), “Nominal, Ordinal, Interval, and Ratio Typologies Are Misleading,” The American Statistician, 47, 65–72. DOI:10.2307/2684788.
   Web of Science ®Google Scholar
• Warner, R. M. (2013), Applied Statistics: From Bivariate Through Multivariate Techniques (2nd ed.), Thousand Oaks, CA: Sage Publications, Inc.
   Google Scholar
• Wickham, H. (2014), “Tidy Data,” The Journal of Statistical Software, 59, 1–23. DOI:10.18637/jss.v059.i10.
   PubMed Web of Science ®Google Scholar
• Wickham, H. (2016), ggplot2: Elegant Graphics for Data Analysis, New York: Springer-Verlag.
   Google Scholar
• Wickham, H. (2017), tidyverse: Easily Install and load the tidyverse.
   Google Scholar
• Wilcoxon, F. (1945), “Individual Comparisons by Ranking Methods,” Biometrics, 1, 80–83. DOI:10.2307/3001968.
   Web of Science ®Google Scholar
• Woodbury, M. A. (1940), “Rank Correlation When There Are Equal Variates,” Annals of Mathematical Statistics, 11, 358. DOI:10.1214/aoms/1177731875.
   Google Scholar
• Zaccardi, N. (2012), “Scoring Controversy Takes Center Stage at Men’s Gymnastics Final,” Technical Report, Sports Illustrated.
   Google Scholar
• Zumbo, B. D., and Zimmerman, D. W. (1993), “Is the Selection of Statistical Methods Governed by Level of Measurement?,” Canadian Psychology, 34
   Web of Science ®Google Scholar