Advanced search
Publication Cover
The American Statistician Volume 73, 2019 - Issue sup1: Statistical Inference in the 21st Century: A World Beyond p < 0.05
Submit an article Journal homepage
9,662
Views
55
CrossRef citations to date
0
Altmetric
Getting to a Post “p<0.05” Era

Large-Scale Replication Projects in Contemporary Psychological Research

Blakeley B. McShaneDepartment of Marketing, Kellogg School of Management, Northwestern University, Evanston, IL
,
Jennifer L. TackettDepartment of Psychology, Northwestern University, Evanston, IL
,
Ulf BöckenholtDepartment of Marketing, Kellogg School of Management, Northwestern University, Evanston, IL
&
Andrew GelmanDepartment of Statistics and Department of Political Science, Columbia University, New York, NY
Pages 99-105 | Received 01 Oct 2017, Published online: 20 Mar 2019

Citations (55)

Keep up to date with the latest research on this topic with citation updates for this article.

Subscribe to citation updates

Read on this site (8)

Paul De Boeck, Jolynn Pek, Katherine Walton, Duane T. Wegener, Brandon M. Turner, Barbara L. Andersen, Theodore P. Beauchaine, Luc Lecavalier, Jay I. Myung & Richard E. Petty. (2023) Questioning Psychological Constructs: Current Issues and Proposed Changes. Psychological Inquiry 34:4, pages 239-257.
Read now
Daniel F. Gucciardi, Robin L.J. Lines & Nikos Ntoumanis. (2022) Handling effect size dependency in meta-analysis. International Review of Sport and Exercise Psychology 15:1, pages 152-178.
Read now
Blakeley B. McShane, Ulf Böckenholt & Karsten T. Hansen. (2022) Variation and Covariation in Large-Scale Replication Projects: An Evaluation of Replicability. Journal of the American Statistical Association 117:540, pages 1605-1621.
Read now
Joshua Habiger & Ye Liang. (2022) Publication Policies for Replicable Research and the Community-Wide False Discovery Rate. The American Statistician 76:2, pages 131-141.
Read now
Brian D. Segal. (2021) Toward Replicability With Confidence Intervals for the Exceedance Probability. The American Statistician 75:2, pages 128-138.
Read now
Mark L. Howe & Henry Otgaar. (2020) Theoretically important failures to reject the null hypothesis: introduction to a special section of Memory. Memory 28:7, pages 837-838.
Read now
Costas Panagopoulos & Sander van der Linden. (2019) Political identity moderates the effect of watchful eyes on voter mobilization: A reply to Matland and Murray (2019). Social Influence 14:3-4, pages 152-158.
Read now
Ronald L. Wasserstein, Allen L. Schirm & Nicole A. Lazar. (2019) Moving to a World Beyond “p < 0.05”. The American Statistician 73:sup1, pages 1-19.
Read now

Articles from other publishers (47)

MATTHIAS BREUER & ED DEHAAN. (2024) Using and Interpreting Fixed Effects Models. Journal of Accounting Research.
Crossref
Eveline E. Schippers, Wineke Smid, Larissa Hoogsteder & Vivienne de Vogel. (2023) Factor Analysis With Unusual Sexual Interests: A Replication Study in a Representative Population Sample. Sexual Abuse 36:4, pages 464-485.
Crossref
Przemysław G. Hensel & Agnieszka Kacprzak. (2024) Streamlining the self-correction process: a review of the use of replication research by organizational scholars. Journal of Organizational Change Management 37:3, pages 465-489.
Crossref
Mikaella Polyviou, Johnny Rungtusanatham, Rebecca W. Reczek, Kevin Dooley & A. Michael Knemeyer. (2024) Advancing OSCM scientific knowledge by replicating empirical findings: Step‐by‐step procedure and illustration for transformative replication endeavors. Decision Sciences 55:2, pages 111-136.
Crossref
Antonia Krefeld-Schwalb, Eli Rosen Sugerman & Eric J. Johnson. (2024) Exposing omitted moderators: Explaining why effect sizes differ in the social sciences. Proceedings of the National Academy of Sciences 121:12.
Crossref
Wes Bonifay, Sonja D. Winter, Hanamori F. Skoblow & Ashley L. Watts. (2024) Good Fit Is Weak Evidence of Replication: Increasing Rigor Through Prior Predictive Similarity Checking. Assessment.
Crossref
Pedro Mateu, Brooks Applegate & Chris L. Coryn. (2023) Towards more credible conceptual replications under heteroscedasticity and unbalanced designs. Quality & Quantity 58:1, pages 723-751.
Crossref
David M. Sanbonmatsu, E. Hanna Cooley & Steven S. Posavac. (2023) The institutional impact of research challenges and constraints on psychology and other social and behavioral sciences. New Ideas in Psychology 70, pages 101014.
Crossref
John G. LynchJr.Jr., Philip M. Fernbach & Christina Kan. (2023) Auditing the value of empirical audits. Proceedings of the National Academy of Sciences 120:26.
Crossref
Gregory M. Walton, Mary C. Murphy, Christine Logel, David S. Yeager, J. Parker Goyer, Shannon T. Brady, Katherine T. U. Emerson, David Paunesku, Omid Fotuhi, Alison Blodorn, Kathryn L. Boucher, Evelyn R. Carter, Maithreyi Gopalan, Amy Henderson, Kathryn M. Kroeper, Lisel Alice Murdock-Perriera, Stephanie L. ReevesTsotso T. Ablorh, Shahana Ansari, Susie ChenPeter FisherManuel Galvan, Madison Kawakami Gilbertson, Chris S. HullemanJoel M. Le Forestier, Christopher LokKatie Mathias, Gregg A. Muragishi, Melanie NetterElise OzierEric N. Smith, Dustin B. Thoman, Heidi E. Williams, Matthew O. WilmotCassie HartzogX. Alice LiNatasha Krol. (2023) Where and with whom does a brief social-belonging intervention promote progress in college?. Science 380:6644, pages 499-505.
Crossref
Maya B Mathur & Matthew P Fox. (2023) Toward Open and Reproducible Epidemiology. American Journal of Epidemiology 192:4, pages 658-664.
Crossref
Michael Hallsworth. (2023) A manifesto for applying behavioural science. Nature Human Behaviour 7:3, pages 310-322.
Crossref
Matthias Breuer & Ed deHaan. (2023) Using and Interpreting Fixed Effects Models. SSRN Electronic Journal.
Crossref
Michael L. DeKay, Nataliya Rubinchik, Zhaojun Li & Paul De Boeck. (2022) Accelerating Psychological Science With Metastudies: A Demonstration Using the Risky-Choice Framing Effect. Perspectives on Psychological Science 17:6, pages 1704-1736.
Crossref
Jae H. Kim. (2021) Moving to a world beyond p-value < 0.05: a guide for business researchers. Review of Managerial Science 16:8, pages 2467-2493.
Crossref
Daniel T. L. Shek, Diya Dou, Xiaoqin Zhu & Xiang Li. (2022) Law abidance leadership education for university students in Hong Kong: Post-lecture evaluation. Frontiers in Psychology 13.
Crossref
Blakeley B. McShane & Andrew Gelman. (2022) Selecting on statistical significance and practical importance is wrong. Journal of Information Technology 37:3, pages 312-315.
Crossref
S. Gabe Hatch, Kayla Knopp, Yunying Le, Maggie O. T. Allen, Karen Rothman, Galena K. Rhoades & Brian D. Doss. (2021) Online relationship education for help‐seeking low‐income couples: A Bayesian replication and extension of the OurRelationship and ePREP programs. Family Process 61:3, pages 1045-1061.
Crossref
David S. Yeager, Christopher J. Bryan, James J. Gross, Jared S. Murray, Danielle Krettek Cobb, Pedro H. F. Santos, Hannah Gravelding, Meghann Johnson & Jeremy P. Jamieson. (2022) A synergistic mindsets intervention protects adolescents from stress. Nature 607:7919, pages 512-520.
Crossref
Ed DienerRobert Northcott, Michael J. Zyphur & Stephen G. West. (2022) Beyond Experiments. Perspectives on Psychological Science 17:4, pages 1101-1119.
Crossref
Mario Gollwitzer & Johannes Schwabe. (2021) Context Dependency as a Predictor of Replicability. Review of General Psychology 26:2, pages 241-249.
Crossref
Scott Highhouse, Lindsey M. Freier, Brent A. Stevenor, Michael A. Shea, Marie Childers & Sarah R. Melick. (2021) Failure to replicate the basic dilution effect in performance prediction. International Journal of Selection and Assessment 30:2, pages 195-201.
Crossref
Ruth N. Bolton, Anders Gustafsson, Crina O. Tarasi & Lars Witell. (2022) Managing a Global Retail Brand in Different Markets: Meta-Analyses of Customer Responses to Service Encounters. Journal of Retailing 98:2, pages 294-314.
Crossref
Florian Lange & Maarten Truyens. (2022) Mixed evidence for the effect of virtual nature exposure on effortful pro-environmental behavior. Journal of Environmental Psychology 81, pages 101803.
Crossref
Daniel T. L. Shek, Diya Dou, Xiaoqin Zhu, Xiang Li & Lindan Tan. (2022) Materialism, Egocentrism and Delinquent Behavior in Chinese Adolescents in Mainland China: A Short-Term Longitudinal Study. International Journal of Environmental Research and Public Health 19:8, pages 4912.
Crossref
David S. Yeager, Jamie M. Carroll, Jenny Buontempo, Andrei Cimpian, Spencer Woody, Robert Crosnoe, Chandra Muller, Jared Murray, Pratik Mhatre, Nicole Kersting, Christopher Hulleman, Molly Kudym, Mary Murphy, Angela Lee Duckworth, Gregory M. Walton & Carol S. Dweck. (2021) Teacher Mindsets Help Explain Where a Growth-Mindset Intervention Does and Doesn’t Work. Psychological Science 33:1, pages 18-32.
Crossref
Michele VisciolaMichele Visciola. 2022. Sustainable Innovation. Sustainable Innovation 155 178 .
Ali H. Al-Hoorie, Phil Hiver, Diane Larsen-Freeman & Wander Lowie. (2021) From replication to substantiation: A complexity theory perspective. Language Teaching, pages 1-16.
Crossref
Arjen van Witteloostuijn, Marcus Dejardin & Jeffrey M. Pollack. (2021) Replication in entrepreneurship. Journal of Business Venturing Insights 16, pages e00271.
Crossref
Allan Drazen, Anna Dreber, Erkut Y. Ozbay & Erik Snowberg. (2021) Journal-based replication of experiments: An application to “Being Chosen to Lead”. Journal of Public Economics 202, pages 104482.
Crossref
Christopher J. Bryan, Elizabeth Tipton & David S. Yeager. (2021) Behavioural science is unlikely to change the world without a heterogeneity revolution. Nature Human Behaviour 5:8, pages 980-989.
Crossref
Eirik Strømland. (2021) Making our “meta-hypotheses” clear: heterogeneity and the role of direct replications in science. European Journal for Philosophy of Science 11:2.
Crossref
David M. Sanbonmatsu, Emily H. Cooley & Jonathan E. Butner. (2021) The Impact of Complexity on Methods and Findings in Psychological Science. Frontiers in Psychology 11.
Crossref
Sander Greenland. (2020) Analysis goals, error‐cost sensitivity, and analysis hacking: Essential considerations in hypothesis testing and multiple comparisons. Paediatric and Perinatal Epidemiology 35:1, pages 8-23.
Crossref
Mark J. Brandt, Toon Kuppens, Russell Spears, Luca Andrighetto, Frederique Autin, Peter Babincak, Constantina Badea, Jaechang Bae, Anatolia Batruch, Julia C. Becker, Konrad Bocian, Bojana Bodroža, David Bourguignon, Marcin Bukowski, Fabrizio Butera, Sarah E. Butler, Xenia Chryssochoou, Paul Conway, Jarret T. Crawford, Jean‐Claude Croizet, Soledad de Lemus, Juliane Degner, Piotr Dragon, Federica Durante, Matthew J. Easterbrook, Iniobong Essien, Joseph P. Forgas, Roberto González, Sylvie Graf, Peter Halama, Gyuseog Han, Ryan Y Hong, Petr Houdek, Eric R. Igou, Yoel Inbar, Jolanda Jetten, William Jimenez Leal, Gloria Jiménez‐Moya, Jaya Kumar Karunagharan, Anna Kende, Maria Korzh, Simon M. Laham, Joris Lammers, Li Lim, Antony S. R. Manstead, Janko Međedović, Zachary J. Melton, Matt Motyl, Spyridoula Ntani, Chuma Kevin Owuamalam, Müjde Peker, Michael J. Platow, J. P. Prims, Christine Reyna, Mark Rubin, Rim Saab, Sindhuja Sankaran, Lee Shepherd, Chris G. Sibley, Agata Sobkow, Bram Spruyt, Pernille Stroebaek, Nebi Sümer, Joseph Sweetman, Catia P. Teixeira, Claudia Toma, Adrienn Ujhelyi, Jojanneke van der Toorn, Alain van Hiel, Alejandro Vásquez‐Echeverría, Alexandra Vazquez, Michelangelo Vianello, Marek Vranka, Vincent Yzerbyt & Jennifer L. Zimmerman. (2020) Subjective status and perceived legitimacy across countries. European Journal of Social Psychology 50:5, pages 921-942.
Crossref
Erik L. Knight, Blakeley B. McShane, Hana H. Kutlikova, Pablo J. Morales, Colton B. Christian, William T. Harbaugh, Ulrich Mayr, Triana L. Ortiz, Kimberly Gilbert, Christine Ma-Kellams, Igor Riečanský, Neil V. Watson, Christoph Eisenegger, Claus Lamm, Pranjal H. Mehta & Justin M. Carré. (2020) Weak and Variable Effects of Exogenous Testosterone on Cognitive Reflection Test Performance in Three Experiments: Commentary on Nave, Nadler, Zava, and Camerer (2017). Psychological Science 31:7, pages 890-897.
Crossref
Lincoln J. Colling, Dénes Szűcs, Damiano De Marco, Krzysztof Cipora, Rolf Ulrich, Hans-Christoph Nuerk, Mojtaba Soltanlou, Donna Bryce, Sau-Chin Chen, Philipp Alexander Schroeder, Dion T. Henare, Christine K. Chrystall, Paul M. Corballis, Daniel Ansari, Celia Goffin, H. Moriah Sokolowski, Peter J. B. Hancock, Ailsa E. Millen, Stephen R. H. Langton, Kevin J. Holmes, Mark S. Saviano, Tia A. Tummino, Oliver Lindemann, Rolf A. Zwaan, Jiří Lukavský, Adéla Becková, Marek A. Vranka, Simone Cutini, Irene Cristina Mammarella, Claudio Mulatti, Raoul Bell, Axel Buchner, Laura Mieth, Jan Philipp Röer, Elise Klein, Stefan Huber, Korbinian Moeller, Brenda Ocampo, Juan Lupiáñez, Javier Ortiz-Tudela, Juanma de la Fuente, Julio Santiago, Marc Ouellet, Edward M. Hubbard, Elizabeth Y. Toomarian, Remo Job, Barbara Treccani & Blakeley B. McShane. (2020) Registered Replication Report on Fischer, Castel, Dodd, and Pratt (2003). Advances in Methods and Practices in Psychological Science 3:2, pages 143-162.
Crossref
Blakeley B. McShane, Ulf Böckenholt & Karsten T. Hansen. (2020) Average Power: A Cautionary Note. Advances in Methods and Practices in Psychological Science 3:2, pages 185-199.
Crossref
Samuel Pawel & Leonhard Held. (2020) Probabilistic forecasting of replication studies. PLOS ONE 15:4, pages e0231416.
Crossref
Blakeley B. McShane & Ulf Böckenholt. (2019) Enriching Meta-Analytic Models of Summary Data: A Thought Experiment and Case Study. Advances in Methods and Practices in Psychological Science 3:1, pages 81-93.
Crossref
Edward E. Rigdon, Marko Sarstedt & Jan-Michael Becker. (2020) Quantify uncertainty in behavioral research. Nature Human Behaviour 4:4, pages 329-331.
Crossref
Florian Lange. (2020) Are Difficult-To-Study Populations too Difficult to Study in a Reliable Way?. European Psychologist 25:1, pages 41-50.
Crossref
Sumitava Mukherjee. (2019) Revise the Belief in Loss Aversion. Frontiers in Psychology 10.
Crossref
John Maindonald. (2019) Book Review. Australian & New Zealand Journal of Statistics 61:4, pages 474-476.
Crossref
Vincent S. Staggs. (2019) Why statisticians are abandoning statistical significance. Research in Nursing & Health 42:3, pages 159-160.
Crossref
Jennifer L. TackettCassandra M. BrandesKevin M. King & Kristian E. Markon. (2019) Psychology's Replication Crisis and Clinical Psychological Science. Annual Review of Clinical Psychology 15:1, pages 579-604.
Crossref
Allan Drazen, Anna Dreber, Erkut Ozbay & Erik Snowberg. (2019) A Journal-Based Replication of 'Being Chosen to Lead'. SSRN Electronic Journal.
Crossref

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.