From old school to open science: The implications of new research norms for educational psychology and beyond

References

• Association for Psychological Science (2017). Open practice badges. https://www.psychologicalscience.org/publications/badges
   Google Scholar
• Blattman, C. (2016, March 27). Preregistration of clinical trials causes medicines to stop working. Chris Blattman. https://chrisblattman.com/2016/03/01/13719/
   Google Scholar
• Center for Open Science. (2020). OSF Pre-registration template. https://osf.io/jea94/
   Google Scholar
• Cohen, G. L., Garcia, J., Apfel, N., & Master, A. (2006). Reducing the racial achievement gap: A social-psychological intervention. Science, 313(5791), 1307–1310. https://doi.org/10.1126/science.1128317
   PubMed Web of Science ®Google Scholar
• Cron, L. (2012). Wired for story: The writer's guide to using brain science to hook readers from the very first sentence. (1st ed.). Ten Speed Press.
   Google Scholar
• Dee, T. S. (2015). Social identity and achievement gaps: Evidence from an affirmation intervention. Journal of Research on Educational Effectiveness, 8(2), 149–168. https://doi.org/10.1080/19345747.2014.906009
   Web of Science ®Google Scholar
• Dominus, S. (2017). Oct. 18). When the revolution came for Amy Cuddy. The New York Times Magazine. https://www.nytimes.com/2017/10/18/magazine/when-the-revolution-came-for-amy-cuddy.html
   Google Scholar
• Flake, J. K. (2021/this issue). Strengthening the foundation of educational psychology by integrating construct validation into open science reform. Educational Psychologist, 56(2), 132–141. https://doi.org/10.1080/00461520.2021.1898962
   PubMed Web of Science ®Google Scholar
• Fleming, J. I., Wilson, S. E., Hart, S. A., Therrien, W. J., & Cook, B. G. (2021/this issue). Open accessibility in education research: Enhancing the credibility, equity, impact, and efficiency of research. Educational Psychologist, 56(2), 110–121. https://doi.org/10.1080/00461520.2021.1897593
   PubMed Web of Science ®Google Scholar
• Gehlbach, H. (2015). Seven survey sins. The Journal of Early Adolescence, 35(5–6), 883–897. https://doi.org/10.1177/0272431615578276
   Web of Science ®Google Scholar
• Gehlbach, H., Brinkworth, M. E., King, A. M., Hsu, L. M., McIntyre, J., & Rogers, T. (2016). Creating birds of similar feathers: Leveraging similarity to improve teacher–student relationships and academic achievement. Journal of Educational Psychology, 108(3), 342–352. https://doi.org/10.1037/edu0000042
   Web of Science ®Google Scholar
• Gehlbach, H., & Robinson, C. D. (2018). Mitigating illusory results through preregistration in education. Journal of Research on Educational Effectiveness, 11(2), 296–315. https://doi.org/10.1080/19345747.2017.1387950
   Web of Science ®Google Scholar
• Gelman, A., & Loken, E. (2014). The statistical crisis in science. American Scientist, 102(6), 460–465. https://doi.org/10.1511/2014.111.460
   Web of Science ®Google Scholar
• Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. (2016). Comment on “Estimating the reproducibility of psychological science". Science, 351(6277), 1037–1037. https://doi.org/10.1126/science.aad7243
   PubMed Web of Science ®Google Scholar
• Hanselman, P., Rozek, C. S., Grigg, J., & Borman, G. D. (2017). New evidence on self-affirmation effects and theorized sources of heterogeneity from large-scale replications. Journal of Educational Psychology, 109(3), 405–424. https://doi.org/10.1037/edu0000141
   PubMed Web of Science ®Google Scholar
• Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Giffen, C. J., Blair, S. S., Rouse, D. I., & Hyde, J. S. (2014). Closing the social class achievement gap for first-generation students in undergraduate biology. Journal of Educational Psychology, 106(2), 375–389. https://doi.org/10.1037/a0034679
   PubMed Web of Science ®Google Scholar
• Ioannidis, J. P. A. (2005). Why most published research findings are false. PLOS Medicine, 2(8), Article e124. https://doi.org/10.1371/journal.pmed.0020124
   PubMed Web of Science ®Google Scholar
• Jacowitz, K. E., & Kahneman, D. (1995). Measures of anchoring in estimation tasks. Personality and Social Psychology Bulletin, 21(11), 1161–1166. https://doi.org/10.1177/01461672952111004
   Web of Science ®Google Scholar
• Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., Nosek, B. A. (2014). Investigating variation in replicability. Social Psychology, 45(3), 142–152. https://doi.org/10.1027/1864-9335/a000178
   Web of Science ®Google Scholar
• Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., Batra, R., Berkics, M., Bernstein, M. J., Berry, D. R., Bialobrzeska, O., Binan, E. D., Bocian, K., Brandt, M. J., Busching, R., Nosek, B. A. Š. (2018). Many Labs 2: Investigating variation in replicability across samples and settings. Advances in Methods and Practices in Psychological Science, 1(4), 443–490. https://doi.org/10.1177/2515245918810225
   Google Scholar
• Kristal, A. S., Whillans, A. V., Bazerman, M. H., Gino, F., Shu, L. L., Mazar, N., & Ariely, D. (2020). Signing at the beginning versus at the end does not decrease dishonesty. Proceedings of the National Academy of Sciences of the United States of America, 117(13), 7103–7107. https://doi.org/10.1073/pnas.1911695117
   PubMed Web of Science ®Google Scholar
• Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty: Replications in the education sciences. Educational Researcher, 43(6), 304–316. https://doi.org/10.3102/0013189X14545513
   Web of Science ®Google Scholar
• Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? The American Psychologist, 70(6), 487–498. https://doi.org/10.1037/a0039400
   PubMed Web of Science ®Google Scholar
• Mellor, D. (2021/this issue). Improving norms in research culture to incentivize transparency and rigor. Educational Psychologist, 56(2), 122–131. https://doi.org/10.1080/00461520.2021.1902329
   PubMed Web of Science ®Google Scholar
• Miyake, A., Kost-Smith, L. E., Finkelstein, N. D., Pollock, S. J., Cohen, G. L., & Ito, T. A. (2010). Reducing the gender achievement gap in college science: A classroom study of values affirmation. Science, 330(6008), 1234–1237. https://doi.org/10.1126/science.1195996
   PubMed Web of Science ®Google Scholar
• Muenks, K., Wigfield, A., Yang, J. S., & O'Neal, C. R. (2017). How true is grit? Assessing its relations to high school and college students’ personality characteristics, self-regulation, engagement, and achievement. Journal of Educational Psychology, 109(5), 599–620. https://doi.org/10.1037/edu0000153
   Web of Science ®Google Scholar
• Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., … Yarkoni, T. (2015). SCIENTIFIC STANDARDS. Promoting an open research culture. Science (New York, N.Y.), 348(6242), 1422–1425. https://doi.org/10.1126/science.aab2374
   PubMed Web of Science ®Google Scholar
• Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002). Harvesting implicit group attitudes and beliefs from a demonstration web site. Group Dynamics: Theory, Research, and Practice, 6(1), 101–115. https://doi.org/10.1037/1089-2699.6.1.101
   Web of Science ®Google Scholar
• Nuzzo, R. (2014). Scientific method: statistical errors. Nature, 506(7487), 150–152. https://doi.org/10.1038/506150a
   PubMed Web of Science ®Google Scholar
• Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science, 349(6251), 716–711. https://doi.org/10.1126/science.aac4716
   PubMed Web of Science ®Google Scholar
• Patall, E. A. (2021/this issue). Implications of the open science era for educational psychology research syntheses. Educational Psychologist, 56(2), 142–160. https://doi.org/10.1080/00461520.2021.1897009
   PubMed Web of Science ®Google Scholar
• Phillips, D. C., Floden, R., Gehlbach, H., Lee, C., Warren Little, J., Maynard, R. A., Metz, M. H., Moje, E. B., Sandoval, W. A., Silverman, S., & Soudien, C. (2009). The preparation of aspiring educational researchers in the empirical qualitative and quantitative traditions of social science: Methodological rigor, social and theoretical relevance, and more. Report of Task Force of the Spencer Foundation Educational Research Training Grant Institutions.
   Google Scholar
• Plucker, J. A., & Makel, M. C. (2021/this issue). Replication is important for educational psychology: Recent developments and key issues. Educational Psychologist, 56(2), 90–100. https://doi.org/10.1080/00461520.2021.1895796
   PubMed Web of Science ®Google Scholar
• Reich, J. (2021/this issue). Preregistration and registered reports. Educational Psychologist, 56(2), 101–109. https://doi.org/10.1080/00461520.2021.1900851
   PubMed Web of Science ®Google Scholar
• Reich, J., Gehlbach, H., & Albers, C. (2020). Like upgrading from a typewriter to a computer: Registered reports in education research. AERA Open. https://doi.org/10.1177/2332858420917640
   Google Scholar
• Robinson, C. D., Scott, W., & Gottfried, M. A. (2019). Taking it to the next level: A field experiment to improve instructor-student relationships in college. AERA Open, 5(1). https://doi.org/10.1177/2332858419839707
   Web of Science ®Google Scholar
• Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638
   Web of Science ®Google Scholar
• Serra-Garcia, M., Hansen, K. T., & Gneezy, U. (2020). Can short psychological interventions affect educational performance? Revisiting the effect of self-affirmation interventions. Psychological Science, 31(7), 865–872. https://doi.org/10.1177/0956797620923587
   PubMed Web of Science ®Google Scholar
• Sherman, D. K., Hartson, K. A., Binning, K. R., Purdie-Vaughns, V., Garcia, J., Taborsky-Barba, S., Tomassetti, S., Nussbaum, A. D., & Cohen, G. L. (2013). Deflecting the trajectory and changing the narrative: How self-affirmation affects academic performance and motivation under identity threat. Journal of Personality and Social Psychology, 104(4), 591–618. https://doi.org/10.1037/a0031495
   PubMed Web of Science ®Google Scholar
• Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
   PubMed Web of Science ®Google Scholar
• Simons, D. J. (2018). Introducing advances in methods and practices in psychological science. Advances in Methods and Practices in Psychological Science, 1(1), 3–6. https://doi.org/10.1177/2515245918757424
   Google Scholar
• Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve: A key to the file-drawer. Journal of Experimental Psychology: General, 143(2), 534–547. https://doi.org/10.1037/a0033242
   Google Scholar
• Slavin, R. E., & Lake, C. (2008). Effective programs in elementary mathematics: A best-evidence synthesis. Review of Educational Research, 78(3), 427–515. https://doi.org/10.3102/0034654308317473
   Web of Science ®Google Scholar
• Slavin, R. E., Lake, C., Chambers, B., Cheung, A., & Davis, S. (2009). Effective reading programs for the elementary grades: A best-evidence synthesis. Review of Educational Research, 79(4), 1391–1466. https://doi.org/10.3102/0034654309341374
   Web of Science ®Google Scholar
• Slavin, R. E., & Smith, D. (2009). The relationship between sample sizes and effect sizes in systematic reviews in education. Educational Evaluation and Policy Analysis, 31(4), 500–506. https://doi.org/10.3102/0162373709352369
   Web of Science ®Google Scholar
• van der Zee, T., & Reich, J. (2018). Open education science. AERA Open, 4(3). https://doi.org/10.1177/2332858418787466
   Web of Science ®Google Scholar
• Wentzel, K. R. (2021/this issue). Open science reforms: Strengths, challenges, and future directions. Educational Psychologist, 56(2), 161–173. https://doi.org/10.1080/00461520.2021.1901709
   Web of Science ®Google Scholar