An Empirical Unraveling of Lord's Paradox

References

• Altman, D. G., Moher, D., & Schulz, K. F. (2012). Improving the reporting of randomised trials: The CONSORT Statement and beyond. Statistics in Medicine, 31 (25), 2985–2997. doi: 10.1002/sim.5402
   PubMed Web of Science ®Google Scholar
• Arah, O. A. (2008). The role of causal reasoning in understanding Simpson's paradox, Lord's paradox, and the suppression effect: Covariate selection in the analysis of observational studies. Emerging Themes in Epidemiology, 5 (5), 1–5. doi: 10.1186/1742-7622-5-5
   PubMedGoogle Scholar
• Breiman, L. (2001). Statistical modeling: The two cultures. Statistical Science, 16 (3), 199–231. doi: 10.2307/2676681
   Web of Science ®Google Scholar
• Buchanan, E., Morrison, J., Walker, M., Aston, H., & Cook, R. (2015). Tutor Trust Secondary (No. July). London, UK: Education Endowment Foundation. Retrieved from https://goo.gl/cEe4SG
   Google Scholar
• Dimitrov, D. M., & Rumrill, P. D. (2003). Pretest-posttest designs and measurement of change. Work, 20 (2), 159–165.
   PubMedGoogle Scholar
• Donoho, D. (2015). 50 years of data science. In Tukey centennial workshop (pp. 1–41). Princeton. Retrieved from http://goo.gl/9gWEQp
   Google Scholar
• Eriksson, K., & Haggstrom, O. (2014). Lord's paradox in a continuous setting and a regression artifact in numerical cognition research. PLoS ONE, 9 (4), 1–7. doi: 10.1371/journal.pone.0095949
   Web of Science ®Google Scholar
• Gelman, A., Hill, J., & Yajima, M. (2012). Why we (usually) don't have to worry about multiple comparisons. Journal of Research on Educational Effectiveness, 5 (2), 189–211. doi: 10.1080/19345747.2011.618213
   Web of Science ®Google Scholar
• Gorard, S., Siddiqui, N., & See, B. H. (2014). Effective Feedback (No. February). London, UK: Education Endowment Foundation. Retrieved from https://goo.gl/mTbNL3
   Google Scholar
• Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77 (1), 81–112. doi: 10.3102/003465430298487
   Web of Science ®Google Scholar
• Hayes, R., & Moulton, L. (2009). Cluster randomised trials. London, UK: Chapman and Hall/CRC.
   Google Scholar
• Higgins, S., Katsipataki, M., Coleman, R., Henderson, P., Major, L. E., Coe, R., & Mason, D. (2015). The Sutton Trust-Education Endowment Foundation Teaching and Learning Toolkit (Tech. Rep.). London, UK: Education Endowment Foundation. Retrieved from https://educationendowmentfoundation.org.uk/our-work/about-the-toolkits/
   Google Scholar
• Holland, P. W. (2005). Lord's Paradox. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 2, pp. 1106–1108). John Wiley & Sons.
   Google Scholar
• James, G., Witten, D., Hastie, T., & Tibshirani, R. (2015). An introduction to statistical learning: With applications in R (6th ed., Vol. 64) (No. 9–12). New York, NY: Springer.
   Google Scholar
• Knapp, T. R., & Schafer, W. (2009). From gain score t to ANCOVA F (and vice versa). Practical Assessment, Research & Evaluation, 14 (6), 1–7.
   Google Scholar
• Liu, S., Lebeau, J.-C., & Tenenbaum, G. (2016). Does exercise improve cognitive performance? A conservative message from Lord's Paradox. Frontiers in Psychology, 7 (1092), 1–10. doi: 10.3389/fpsyg.2016.01092
   PubMed Web of Science ®Google Scholar
• Lord, F. M. (1967). A paradox in the interpretation of group comparisons. Psychological Bulletin, 68 (5), 304–305.
   PubMed Web of Science ®Google Scholar
• Lord, F. M. (1969). Statistical adjustments when comparing preexisting groups. Psychological Bulletin, 72 (5), 336–337.
   Web of Science ®Google Scholar
• McElreath, R. (2016). Statistical rethinking: A Bayesian course with examples in R and Stan. New York, NY: CRC Press.
   Google Scholar
• Menzies, V., Collyer, C., Kokotsaki, D., Hewitt, C., Younger, K., Wiggins, A., & Torgerson, C. (2015). SHINE in Secondaries (No. June). London, UK: Education Endowment Foundation.
   Google Scholar
• O'Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. New York, NY: Crown Publishing.
   Google Scholar
• Pascarella, E. T., Wolniak, G. C., & Pierson, C. T. (2003). Explaining student growth in college when you don't think you are. Journal of College Student Development, 44 (1), 122–126. doi: 10.1353/csd.2003.0007
   Web of Science ®Google Scholar
• Petscher, Y., & Schatschneider, C. (2011). A simulation study on the performance of the simple difference and covariance-adjusted scores in randomized experimental designs. Journal of Educational Measurement, 48 (1), 31–43. doi: 10.1111/j.1745-3984.2010.00129.x
   PubMed Web of Science ®Google Scholar
• Pike, G. R. (2004). Lord's Paradox and the assessment of change during college. Journal of College Student Development, 45 (3), 348–353. doi: 10.1353/csd.2004.0040
   Web of Science ®Google Scholar
• Senn, S. (2006). Change from baseline and analysis of covariance revisited. Statistics in Medicine, 25, 4334–4344. doi: 10.1002/sim.2682
   PubMed Web of Science ®Google Scholar
• Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22 (11), 1359–1366. doi: 10.1177/0956797611417632
   PubMed Web of Science ®Google Scholar
• Torgerson, D. J., & Torgerson, C. J. (2008). Designing randomised trials in health, education and the social sciences: An introduction. London, UK: Palgrave Macmillan.
   Google Scholar
• Tu, Y.-K. (2010). Statistical paradoxes. International Encyclopedia of Education, 429–435.
   Google Scholar
• Tu, Y.-K., Baelum, V., & Gilthorpe, M. S. (2008). A structural equation modelling approach to the analysis of change. European Journal of Oral Sciences, 116 (4), 291–296. doi: 10.1111/j.1600-0722.2008.00549.x
   PubMed Web of Science ®Google Scholar
• Tu, Y.-K., Gunnell, D., & Gilthorpe, M. S. (2008). Simpson's Paradox, Lord's Paradox, and suppression effects are the same phenomenon-the reversal paradox. Emerging Themes in Epidemiology, 5, 1–9. doi: 10.1186/1742-7622-5-2
   PubMedGoogle Scholar
• van Breukelen, G. J. P. (2006). ANCOVA versus change from baseline had more power in randomized studies and more bias in nonrandomized studies. Journal of Clinical Epidemiology, 59 (9), 920–925. doi: 10.1016/j.jclinepi.2006.02.007
   PubMed Web of Science ®Google Scholar
• van Breukelen, G. J. P. (2013). ANCOVA versus CHANGE from baseline in nonrandomized studies: The difference. Multivariate Behavioral Research, 48 (6), 895–922. doi: 10.1080/00273171.2013.831743
   PubMed Web of Science ®Google Scholar
• Vickers, A. J., & Altman, D. G. (2001). Analysing controlled trials with baseline and follow up measurements. British Medical Journal, 323 (7321), 1123–4. doi: 10.1136/bmj.323.7321.1123
   PubMed Web of Science ®Google Scholar
• Wainer, H., & Brown, L. M. (2004). Two statistical paradoxes in the interpretation of group differences: Illustrated with medical school admission and licensing data. The American Statistician, 58 (2), 117–123. doi: 10.1198/0003130043268
   Web of Science ®Google Scholar
• Werts, C. E., & Linn, R. L. (1969). Lord's Paradox: A generic problem. Psychological Bulletin, 72 (6), 423–425.
   Web of Science ®Google Scholar
• Xiao, Z., & Higgins, S. (2017, August 12). The Power of Noise and the Art of Prediction. Retrieved from osf.io/preprints/socarxiv/zu64w
   Google Scholar
• Xiao, Z., Kasim, A., & Higgins, S. (2016). Same difference? Understanding variation in the estimation of Effect sizes from educational trials. International Journal of Educational Research, 77, 1–14. NJ, Sept 18 2015. doi: 10.1016/j.ijer.2016.02.00118
   Web of Science ®Google Scholar