Open science reforms: Strengths, challenges, and future directions

References

• Aelterman, N., Vansteenkiste, M., Haerens, L., Soenens, B., Fontaine, J. R. J., & Reeve, J. (2019). Toward an integrative and fine-grained insight in motivating and demotivating teaching styles: The merits of a circumplex approach. Journal of Educational Psychology, 111(3), 497–521. https://doi.org/10.1037/edu0000293
   Web of Science ®Google Scholar
• Ahn, J., Pellicone, A., & Butler, B. S. (2014). Open badges for education: What are the implications at the intersection of open systems and badging? Research in Learning Technology, 22, 23563. https://doi.org/10.3402/rlt.v22.23563
   Google Scholar
• Al-Emadi, A. A. (2001). The relationships among achievement, goal orientation, and study strategies. Social Behavior and Personality: An International Journal, 29(8), 823–832. https://doi.org/10.2224/sbp.2001.29.8.823
   Web of Science ®Google Scholar
• Anderson, M. S., Martinson, B. C., & De Vries, R. (2007). Normative dissonance in science: Results from a national survey of U.S. scientists. Journal of Empirical Research on Human Research Ethics, 2(4), 3–14. https://doi.org/10.1525/jer.2007.2.4.3
   PubMed Web of Science ®Google Scholar
• Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. The American Psychologist, 73(1), 3–25. https://doi.org/10.1037/amp0000191
   PubMed Web of Science ®Google Scholar
• Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
   Google Scholar
• Baumeister, R., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117(3), 497–529. https://doi.org/10.1037/0033-2909.117.3.497
   PubMed Web of Science ®Google Scholar
• Bergin, C., & Bergin, D. (2009). Attachment in the classroom. Educational Psychology Review, 21(2), 141–170. https://doi.org/10.1007/s10648-009-9104-0
   Web of Science ®Google Scholar
• Bronfenbrenner, U. (1989). Ecological systems theory. In R. Vasta (Ed.), Annals of child development (Vol. 6, pp. 187–250). JAI.
   Google Scholar
• Chambers, B., Slavin, R. E., Madden, N. A., Abrami, P., Logan, M. K., & Gifford, R. (2011). Small-group, computer-assisted tutoring to improve reading outcomes for struggling first and second graders. The Elementary School Journal, 111(4), 625–640. https://doi.org/10.1086/659035
   Web of Science ®Google Scholar
• Chambers, C. (2014). Psychology’s ‘registration revolution, The Guardian. www.theguardian.com/science/head-quarters/2014/may/20/psychology-registration-revolution
   Google Scholar
• Coalition for Psychology in Schools and Education. (2015). Top 20 principles from psychology for preK-12 teaching and learning. American Psychological Association.
   Google Scholar
• Cronbach, L. J. (1975). Beyond the two disciplines of scientific psychology. American Psychologist, 30(2), 116–127. https://doi.org/10.1037/h0076829
   Web of Science ®Google Scholar
• Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957
   PubMed Web of Science ®Google Scholar
• Darling, N., & Steinberg, L. (1993). Parenting style as context—An integrative model. Psychological Bulletin, 113(3), 487–496. https://doi.org/10.1037/0033-2909.113.3.487
   Web of Science ®Google Scholar
• Deci, E. L., & Ryan, R. M. (2014). Autonomy and need satisfaction in close relationships: Relationships motivation theory. In N. Weinstein (Ed.), Human motivation and interpersonal relationships (pp. 53–73). Springer.
   Google Scholar
• De Smedt, F., Graham, S., & Van Keer, H. (2020). “It takes two”: The added value of structured peer-assisted writing in explicit writing instruction. Contemporary Educational Psychology, 60, 101835. https://doi.org/10.1016/j.cedpsych.2019.101835
   Web of Science ®Google Scholar
• Dweck, C. S. (2017). From needs to goals and representations: Foundations for a unified theory of motivation, personality, and development. Psychological Review, 124(6), 689–719. https://doi.org/10.1037/rev0000082
   PubMed Web of Science ®Google Scholar
• Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95(2), 256–272. https://doi.org/10.1037/0033-295X.95.2.256
   Web of Science ®Google Scholar
• Eccles, J. S., & Wigfield, A. (2020). From expectancy-value theory to situated expectancy-value theory: A developmental, social cognitive, and sociocultural perspective on motivation. Contemporary Educational Psychology, 61, 101859. https://doi.org/10.1016/j.cedpsych.2020.101859
   Web of Science ®Google Scholar
• Elliot, A. J., & Church, M. A. (1997). A hierarchical model of approach and avoidance achievement motivation. Journal of Personality and Social Psychology, 72(1), 218–232. https://doi.org/10.1037/0022-3514.72.1.218
   Web of Science ®Google Scholar
• Elliot, A. J., & Hulleman, C. S. (2017). Achievement goals. In A. Elliot, C. Dweck, & D. Yaeger (Eds.), Handbook of competence and motivation (2nd ed., pp. 43–60). Guilford.
   Google Scholar
• Farmer, T. W. (2020). Reforming research to support culturally and ecologically responsive and developmentally meaningful practice in schools. Educational Psychologist, 55(1), 32–39. https://doi.org/10.1080/00461520.2019.1698298
   Web of Science ®Google Scholar
• Farmer, T. W., Dawes, M., Alexander, Q., & Brooks, D. S. (2016). Challenges associated with applications and interventions; Correlated constraints, shadows of synchrony, and teacher/institutional factors that impact social change. In K. Wentzel & G. Ramani (Eds.), Handbook of social influences in school contexts: Social-emotional, motivation, and cognitive outcomes (pp. 423–438). Taylor Francis.
   Google Scholar
• Flake, J. K. (2021/this issue). Strengthening the foundation of educational psychology by integrating construct validation into open science reform. Educational Psychologist, 56(2), 132–141. https://doi.org/10.1080/00461520.2021.1898962
   PubMed Web of Science ®Google Scholar
• Fleming, J. I., Wilson, S. E., Hart, S. A., Therrien, W. J., & Cook, B. G. (2021/this issue). Open accessibility in education research: Enhancing the credibility, equity, impact, and efficiency of research. Educational Psychologist, 56(2), 110–121. https://doi.org/10.1080/00461520.2021.1897593
   Web of Science ®Google Scholar
• Ford, M. (1992). Motivating humans: Goals, emotions, and personal agency beliefs. SAGE.
   Google Scholar
• Ford, M. E., & Smith, P. R. (2007). Thriving with social purpose: An integrative approach to the development of optimal human functioning. Educational Psychologist, 42(3), 153–171. https://doi.org/10.1080/00461520701416280
   Web of Science ®Google Scholar
• Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502–1505. https://doi.org/10.1126/science.1255484
   PubMed Web of Science ®Google Scholar
• Furrer, C., & Skinner, E. (2003). Sense of relatedness as a factor in children’s academic engagement and performance. Journal of Educational Psychology, 95(1), 148–162. https://doi.org/10.1037/0022-0663.95.1.148
   Web of Science ®Google Scholar
• Gehlbach, H., Brinkworth, M. E., King, A. M., Hsu, L. M., McIntyre, J., & Rogers, T. (2016). Creating birds of similar feathers: Leveraging similarity to improve teacher–student relationships and academic achievement. Journal of Educational Psychology, 108(3), 342–352. https://doi.org/10.1037/edu0000042
   Web of Science ®Google Scholar
• Gehlbach, H., & Robinson, C. D. (2021/this issue). From old school to open science: The implications of new research norms for educational psychology and beyond. Educational Psychologist, 56(2), 79–89. https://doi.org/10.1080/00461520.2021.1898961
   Web of Science ®Google Scholar
• Gelman, A. (2016). The ASA statement on p-values: Context, process, and purpose: The problems with p-values are not just with p-values [Supplemental material]. The American Statistician, 70(2), 129–133. https://doi.org/10.6084/m9.figshare.3085162.v6
   Google Scholar
• Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337–350. https://doi.org/10.1007/s10654-016-0149-3
   PubMed Web of Science ®Google Scholar
• Hafen, C., Hamre, B., Allen, J., Bell, C., Gitomer, D., & Pianta, R. (2015). Teaching through interactions in secondary school classrooms: Revisiting the factor structure and practical application of the classroom assessment scoring system-secondary. The Journal of Early Adolescence, 35(5–6), 651–680. https://doi.org/10.1177/0272431614537117
   PubMed Web of Science ®Google Scholar
• Hamre, B. K., & Pianta, R. C. (2005). Can instructional and emotional support in the first-grade classroom make a difference for children at risk of school failure? Child Development, 76(5), 949–967. https://doi.org/10.1111/j.1467-8624.2005.00889.x
   PubMed Web of Science ®Google Scholar
• Hartwig, F., & Dearing, B. E. (1979). Exploratory data analysis. SAGE.
   Google Scholar
• Haven, T., & Van Grootel, D. L. (2019). Preregistering qualitative research. Accountability in Research, 26(3), 229–244. https://doi.org/10.1080/08989621.2019.1580147
   PubMed Web of Science ®Google Scholar
• Hulleman, C. S., Schrager, S. M., Bodmann, S. M., & Harackiewicz, J. M. (2010). A meta-analytic review of achievement goal measures: Different labels for the same constructs or different constructs with similar labels? Psychological Bulletin, 136(3), 422–449. https://doi.org/10.1037/a0018947
   PubMed Web of Science ®Google Scholar
• Ioannidis, J. P. (2018). Why replication has more scientific value than original discovery. Behavioral and Brain Sciences, 41, e137. https://doi.org/10.1017/S0140525X18000729
   Web of Science ®Google Scholar
• John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532. https://doi.org/10.1177/0956797611430953
   PubMed Web of Science ®Google Scholar
• Kaplan, A., Cromley, J., Perez, T., Dai, T., Mara, K., & Balsai, M. (2020). The role of context in educational RCT findings: A call to redefine ‘Evidence-based practice’. Educational Researcher, 49(4), 285–288. https://doi.org/10.3102/0013189X20921862
   Web of Science ®Google Scholar
• Lepper, M., & Henderlong, J. (2000). Turning “play” into “work” and “work” into “play”. In C. Sansone & J. Harackiewicz (Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 257–307). Academic Press.
   Google Scholar
• Lerner, R. M., Schwartz, S. J., & Phelps, E. (2009). Problematics of time and timing in the longitudinal study of human development: Theoretical and methodological issues. Human Development, 52(1), 44–68. https://doi.org/10.1159/000189215
   PubMed Web of Science ®Google Scholar
• Levitt, H. M. (2019). Reporting qualitative research in psychology: How to meet APA style journal article reporting standards. American Psychological Association.
   Google Scholar
• Madden, N., & Slavin, R. E. (2017). Evaluations of technology-assisted small-group tutoring for struggling readers. Reading & Writing Quarterly, 33(4), 327–334. https://doi.org/10.1080/10573569.2016.1255577
   Web of Science ®Google Scholar
• Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316. https://doi.org/10.3102/0013189X14545513
   Web of Science ®Google Scholar
• Makel, M. C., Plucker, J. A., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 7(6), 537–542. https://doi.org/10.1177/1745691612460688
   PubMedGoogle Scholar
• Marsh, H. W., Pekrun, R., Murayama, K., Arens, K. A., Parker, P. D., Guo, J., & Dicke, T. (2018). An integrated model of academic self-concept development: Academic self-concept, grades, test scores, and tracking over 6 years. Developmental Psychology, 54(2), 263–280. https://doi.org/10.1037/dev0000393
   PubMed Web of Science ®Google Scholar
• Marsh, H. W., Pekrun, R., Parker, P. D., Murayama, K., Guo, J., Dicke, T., & Arens, A. K. (2019). The murky distinction between self-concept and self-efficacy: Beware of lurking jingle-jangle fallacies. Journal of Educational Psychology, 111(2), 331–353. https://doi.org/10.1037/edu0000281
   Web of Science ®Google Scholar
• Maslow, A. (1962). Toward a psychology of being. VanNostrand.
   Google Scholar
• McClelland, D. C. (1987). Human motivation. Cambridge University Press.
   Google Scholar
• Mellor, D. (2021/this issue). Improving norms in research culture to incentivize transparency and rigor. Educational Psychologist, 56(2), 122–131. https://doi.org/10.1080/00461520.2021.1902329
   Web of Science ®Google Scholar
• Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). American Council on Education.
   Google Scholar
• Meyer, D. K., & Schutz, P. (2020). Qualitative and mixed methods research in educational psychology, Special Issue. Educational Psychologist, 55(4).
   Web of Science ®Google Scholar
• Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., Gheen, M., Kaplan, A., Kumar, R., Middleton, M. J., Nelson, J., Roeser, R., & Urdan, T. (2000). Manual for the Patterns of Adaptive Learning Sccdes (PALS). University of Michigan. http://www.umich.edu/∼pals/pals/manuals.html
   Google Scholar
• Miller-Cotto, D., & Byrnes, J. P. (2020). What’s the best way to characterize the relationship between working memory and achievement?: An initial examination of competing theories. Journal of Educational Psychology, 112(5), 1074–1084. https://doi.org/10.1037/edu0000395
   Web of Science ®Google Scholar
• Nolen, S. B. (1988). Reasons for studying: Motivation orientations and study strategies. Cognition and Instruction, 5(4), 269–287. https://doi.org/10.1207/s1532690xci0504_2
   Web of Science ®Google Scholar
• Nuzzo, R. (2014). Scientific method: Statistical errors. Nature, 506(7487), 150–152. https://doi.org/10.1038/506150a
   PubMed Web of Science ®Google Scholar
• Osterman, K. F. (2000). Students’ need for belonging in the school community. Review of Educational Research, 70(3), 323–367. https://doi.org/10.3102/00346543070003323
   Web of Science ®Google Scholar
• Patall, E. A. (2021/this issue). Implications of the open science era for educational psychology research syntheses. Educational Psychologist, 56(2), 142–160. https://doi.org/10.1080/00461520.2021.1897009
   Web of Science ®Google Scholar
• Pianta, R. C. (2001). Student–teacher relationship scale: Professional manual. Psychological Assessment Resources.
   Google Scholar
• Pianta, R. C., Hamre, B. K., & Allen, J. P. (2012). Teacher-student relationships and engagement: Conceptualizing, measuring, and improving the capacity of classroom interaction. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook on student engagement (pp. 365–386). Springer.
   Google Scholar
• Plucker, J. A., & Makel, M. C. (2021/this issue). Replication is important for educational psychology: Recent developments and key issues. Educational Psychologist, 56(2), 90–100. https://doi.org/10.1080/00461520.2021.1895796
   Web of Science ®Google Scholar
• Reich, J. (2021/this issue). Preregistration and registered reports. Educational Psychologist, 56(2), 101–109. https://doi.org/10.1080/00461520.2021.1900851
   Web of Science ®Google Scholar
• Ross, L. (1977). The intuitive psychologist and his shortcomings: Distortions in the attribution process. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 10, pp. 173–220). Academic.
   Google Scholar
• Ruzek, E. A., Hafen, C. A., Allen, J. P., Gregory, A., Mikami, A. Y., & Pianta, R. C. (2016). How teacher emotional support motivates students: The mediating roles of perceived peer relatedness, autonomy support, and competence. Learning and Instruction, 42, 95–103. https://doi.org/10.1016/j.learninstruc.2016.01.004
   PubMed Web of Science ®Google Scholar
• Ryan, R., & Deci, E. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68
   PubMed Web of Science ®Google Scholar
• Ryan, R., & Deci, E. (2020). Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions. Contemporary Educational Psychology, 61, 101860.
   Web of Science ®Google Scholar
• Schunk, D. H., & DiBenedetto, M. K. (2016). Self-efficacy theory in education. In K. Wentzel & D. Miele (Eds.), Handbook of motivation at school (2nd ed., pp. 34–54). LEA.
   Google Scholar
• Scott, S. (2013). Pre-registration would put science in chains. Time Higher Education. http://www.timeshighereducation.co.uk/comment/opinion/pre-registration-would-put-science-in-chains/2005954.article
   Google Scholar
• Senko, C. (2016). Achievement goal theory: A story of early promises, eventual discords, and future possibilities. In K. Wentzel & D. Miele (Eds.), Handbook of motivation at school (2nd ed., pp. 75–95). LEA.
   Google Scholar
• Senko, C., Hulleman, C. S., & Harackiewicz, J. M. (2011). Achievement goal theory at the crossroads: Old controversies, current challenges, and new directions. Educational Psychologist, 46(1), 26–47. https://doi.org/10.1080/00461520.2011.538646
   Web of Science ®Google Scholar
• Slavin, R. E. (1980). A review of peer tutoring and cooperative learning projects in twenty-eight schools. Review of Educational Research, 50(2), 315–342. https://doi.org/10.3102/00346543050002315
   Web of Science ®Google Scholar
• Solomon, D., Battistich, V., Watson, M., Schaps, E., & Lewis, C. (2000). A six-district study of educational change: Direct and mediated effects of the Child Development Project. Social Psychology of Education, 4(1), 3–51. https://doi.org/10.1023/A:1009609606692
   Google Scholar
• Sternberg, R. J., & Fiske, S. T. (2015). Ethical principles for the behavioral and brain sciences: case studies and commentaries. Cambridge University Press.
   Google Scholar
• van Rooij, I. (2019). Psychological science needs theory development before preregistration. https://featuredcontent.psychonomic.org/psychological-science-needs-theory-development-before-preregistration/
   Google Scholar
• Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.
   Google Scholar
• Walton, G. M., & Cohen, G. L. (2007). A question of belonging: Race, social fit, and achievement. Journal of Personality and Social Psychology, 92(1), 82–96. https://doi.org/10.1037/0022-3514.92.1.82
   PubMed Web of Science ®Google Scholar
• Walton, G. M., & Yeager, D. S. (2020). Seed and soil: Psychological affordances in contexts help to explain where wise interventions succeed or fail. Current Directions in Psychological Science, 29(3), 219–226. https://doi.org/10.1177/0963721420904453
   PubMed Web of Science ®Google Scholar
• Weiner, B. (1985). An attribution theory of achievement motivation and emotion. Psychological Review, 92(4), 548–573. https://doi.org/10.1037/0033-295X.92.4.548
   PubMed Web of Science ®Google Scholar
• Wentzel, K. R. (1997). Student motivation in middle school: The role of perceived pedagogical caring. Journal of Educational Psychology, 89(3), 411–419. https://doi.org/10.1037/0022-0663.89.3.411
   Web of Science ®Google Scholar
• Wentzel, K. R. (2012). Developing and nurturing interesting and researchable ideas. In C. Conrad & R. Serlin (Eds.), SAGE handbook for research in education: Pursuing ideas as the keystone of exemplary inquiry (2nd ed., pp. 111–126). SAGE.
   Google Scholar
• Wentzel, K. R. (2016). Students’ relationships with teachers as motivational contexts. In K. Wentzel & D. Miele (Eds.), Handbook of motivation at school (2nd ed., pp. 211–230). LEA.
   Google Scholar
• Wentzel, K. R. (2018). A competence-in-context approach to understanding motivation at school. In D. McInerney & G. Liem (Eds.), Big theories revisited (pp. 193–212). IAP.
   Google Scholar
• Wentzel, K. R., & Muenks, K. (2016). Peer influence on students’ motivation, academic achievement and social behavior. In K. Wentzel & G. Ramani (Eds.), Handbook of social influences in school contexts: Social-emotional, motivation, and cognitive outcomes (pp. 13–30). Taylor Francis.
   Google Scholar
• Wentzel, K. R., Russell, S., & Baker, S. A. (2016). Emotional support and expectations for behavior from peers, teachers, and parents: predictors of adolescent competence at school. Journal of Educational Psychology, 108(2), 242–255. https://doi.org/10.1037/edu0000049
   Web of Science ®Google Scholar
• Yeager, D. S., & Dweck, C. S. (2020). What can be learned from growth mindset controversies? American Psychologist, 75(9), 1269–1284. https://doi.org/10.1037/amp0000794
   PubMed Web of Science ®Google Scholar
• Yeager, D. S., Hanselman, P., Walton, G. M., Murray, J. S., Crosnoe, R., Muller, C., Tipton, E., Schneider, B., Hulleman, C. S., Hinojosa, C. P., Paunesku, D., Romero, C., Flint, K., Roberts, A., Trott, J., Iachan, R., Buontempo, J., Yang, S. M., Carvalho, C. M., … Dweck, C. S. (2019). A national experiment reveals where a growth mindset improves achievement. Nature, 573(7774), 364–369. https://doi.org/10.1038/s41586-019-1466-y
   PubMed Web of Science ®Google Scholar