Knowing how effective an intervention, treatment, or manipulation is and increasing replication rates: accuracy in parameter estimation as a partial solution to the replication crisis

References

• Abraham, C., & Michie, S. (2008). A taxonomy of behavior change techniques used in interventions. Health Psychology, 27(3), 379–387. https://doi.org/10.1037/0278-6133.27.3.379
   PubMed Web of Science ®Google Scholar
• Albers, C. J., Kiers, H. A. L., & Van Ravenzwaaij, D. (2018). Credible confidence: A pragmatic view on the frequentist vs Bayesian debate. Collabra: Psychology, 4, 1–8.
   PubMedGoogle Scholar
• American Psychological Association. (2008). Reporting standards for research in psychology. The American Psychologist, 63(9), 839–851. https://doi.org/10.1037/0003-066X.63.9.839
   PubMed Web of Science ®Google Scholar
• American Psychological Association. (2009). Publication manual of the American psychological association (6th ed.). APA.
   Google Scholar
• Amrhein, V., Greenland, S., & McShane, B. (2019). Scientists rise up against statistical significance. Nature, 567(7748), 305–307. https://doi.org/10.1038/d41586-019-00857-9
   PubMed Web of Science ®Google Scholar
• Amrhein, V., Trafimow, D., & Greenland, S. (2019). Inferential statistics as descriptive statistics: There is no replication crisis if we don’t expect replication. The American Statistician, 73(sup1), 262–270. https://doi.org/10.1080/00031305.2018.1543137
   Web of Science ®Google Scholar
• Bakker, M., Dijk, A. V., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7(6), 543–554. https://doi.org/10.1177/1745691612459060
   Web of Science ®Google Scholar
• Bonett, D. G., & Wright, T. a. (2000). Sample size requirements for estimating pearson, kendall and spearman correlations. Psychometrika, 65(1), 23–28. https://doi.org/10.1007/BF02294183
   Web of Science ®Google Scholar
• Brand, A., & Bradley, M. T. (2016). The precision of effect size estimation from published psychological research: Surveying confidence intervals. Psychological Reports, 118(1), 154–170. https://doi.org/10.1177/0033294115625265
   PubMed Web of Science ®Google Scholar
• Champely, S. (2016). pwr: Basic functions for power analysis.
   Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
   Google Scholar
• Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45(12), 1304–1312. https://doi.org/10.1037/0003-066X.45.12.1304
   Web of Science ®Google Scholar
• Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155
   PubMed Web of Science ®Google Scholar
• Crutzen, R., Peters, G.-J Y., & Abraham, C. (2012). What about trialists sharing other study materials? BMJ, 345(6), e8352–e8352. https://doi.org/10.1136/bmj.e8352
   PubMedGoogle Scholar
• Crutzen, R., & Peters, G.-J Y. (2017). Targeting next generations to change the common practice of underpowered research. Frontiers in Psychology, 8, 1184. https://doi.org/10.3389/fpsyg.2017.01184
   PubMed Web of Science ®Google Scholar
• Crutzen, R., & Peters, G.-J Y. (2018). Evolutionary learning processes as the foundation for behaviour change. Health Psychology Review, 12(1), 43–57. https://doi.org/10.1080/17437199.2017.1362569
   PubMed Web of Science ®Google Scholar
• Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7–29. https://doi.org/10.1177/0956797613504966
   PubMed Web of Science ®Google Scholar
• Cumming, G., & Calin-Jageman, R. (2016). Introduction to the new statistics: Estimation, open science, and beyond. Routledge.
   Google Scholar
• Cumming, G., & Finch, S. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61(4), 532–574. https://doi.org/10.1177/0013164401614002
   Web of Science ®Google Scholar
• Dongen, N. N. N., van Doorn, J. B., Gronau, Q. F., van Ravenzwaaij, D., Hoekstra, R., Haucke, M. N., Lakens, D., Hennig, C., Morey, R. D., Homer, S., Gelman, A., Sprenger, J., & Wagenmakers, E.-J. (2019). Multiple perspectives on inference for two simple statistical scenarios. The American Statistician, 73(sup1), 328–339. https://doi.org/10.1080/00031305.2019.1565553
   Google Scholar
• Earp, B. D., & Trafimow, D. (2015). Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology, 6(5), 621. https://doi.org/10.3389/fpsyg.2015.00621
   PubMedGoogle Scholar
• Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146
   PubMed Web of Science ®Google Scholar
• Gardner, M. J., & Altman, D. G. (1986). Statistics in medicine confidence intervals rather than P values: Estimation rather than hypothesis testing. BMJ, 292(6522), 746–750. https://doi.org/10.1136/bmj.292.6522.746
   PubMed Web of Science ®Google Scholar
• Goulet-Pelletier, J.-C., & Cousineau, D. (2018). A review of effect sizes and their confidence intervals, Part {I}: The Cohen’s d family. The Quantitative Methods for Psychology, 14(4), 242–265. https://doi.org/10.20982/tqmp.14.4.p242
   Web of Science ®Google Scholar
• Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337–350. https://doi.org/10.1007/s10654-016-0149-3
   PubMed Web of Science ®Google Scholar
• Gruijters, S. L. K., & Peters, G.-J Y. (2017). Introducing the Numbers Needed for Change (NNC): A practical measure of effect size for intervention research.
   Google Scholar
• Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H., Anggono, C. O., Birt, A., Brand, R., Brandt, M. J., Brewer, G., Bruyneel, S., Calvillo, D. P., Campbell, W. K., Cannon, P. R., Carlucci, M., Carruth, N.P., Cheung, T., Crowell, A., De Ridder, D. T. D., Dewitte, S., Elson, M., … Zwienenberg, M. (2016). A multi-lab pre-registered replication of the ego-depletion effect. Perspectives on Psychological Science, 25(4), 1227–1234. https://doi.org/10.1177/0956797614526415.Data
   Google Scholar
• Inauen, J., Shrout, P. E., Bolger, N., Stadler, G., & Scholz, U. (2016). Mind the gap? An intensive longitudinal study of between-person and within-person intention-behavior relations. Annals of Behavioral Medicine, 50(4), 516–522. https://doi.org/10.1007/s12160-016-9776-x
   PubMed Web of Science ®Google Scholar
• JASP Team. (2018). JASP (Version 0.9) [Computer software]. https://jasp-stats.org/.
   Google Scholar
• Kelley, K. (2018). MBESS: The MBESS R Package. https://CRAN.R-project.org/package=MBESS.
   Google Scholar
• Kelley, K., & Pornprasertmanit, S. (2016). Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software for composite measures. Psychological Methods, 21(1), 69–92. https://doi.org/10.1037/a0040086
   PubMed Web of Science ®Google Scholar
• Kok, G. (2014). A practical guide to effective behavior change: How to apply theory- and evidence-based behavior change methods in an intervention. European Health Psychologist, 16(5), 156–170.
   Google Scholar
• Kok, G., Gottlieb, N. H., Peters, G.-J Y., Mullen, P. D., Parcel, G. S., Ruiter, R. A. C., Fernández, M. E., Markham, C., & Bartholomew, L. K. (2016). A taxonomy of behaviour change methods: An intervention mapping approach. Health Psychology Review, 10(3), 297–312. https://doi.org/10.1080/17437199.2015.1077155
   PubMed Web of Science ®Google Scholar
• Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006). Caution regarding the use of pilot studies to guide power calculations for study proposals. Archives of General Psychiatry, 63(5), 484–489. https://doi.org/10.1001/archpsyc.63.5.484
   PubMed Web of Science ®Google Scholar
• Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4(11), 1–12. https://doi.org/10.3389/fpsyg.2013.00863
   PubMedGoogle Scholar
• Lakens, D. (2017). Equivalence tests: A practical primer for t-tests, correlations, and meta-analyses. Social Psychological and Personality Science, 8(4), 355–362. https://doi.org/10.1177/1948550617697177
   PubMed Web of Science ®Google Scholar
• Lakens, D., & Evers, E. R. K. (2014). Sailing from the seas of chaos into the corridor of stability: Practical recommendations to increase the informational value of studies. Perspectives on Psychological Science, 9(3), 278–292. https://doi.org/10.1177/1745691614528520
   Web of Science ®Google Scholar
• Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008). Sample size planning for statistical power and accuracy in parameter estimation. Annual Review of Psychology, 59(1), 537–563. https://doi.org/10.1146/annurev.psych.59.103006.093735
   PubMed Web of Science ®Google Scholar
• McGrath, R. E., & Meyer, G. J. (2006). When effect sizes disagree: The case of r and d. Psychological Methods, 11(4), 386–401. https://doi.org/10.1037/1082-989X.11.4.386
   PubMed Web of Science ®Google Scholar
• Moinester, M., & Gottfried, R. (2014). Sample size estimation for correlations with pre-specified confidence interval. The Quantitative Methods for Psychology, 10(2), 124–130. https://doi.org/10.20982/tqmp.10.2.p0124
   Google Scholar
• Morey, R. D., Hoekstra, R., Rouder, J. N., Lee, M. D., & Wagenmakers, E. J. (2016). The fallacy of placing confidence in confidence intervals. Psychonomic Bulletin & Review, 23(1), 103–123. https://doi.org/10.3758/s13423-015-0947-8
   PubMed Web of Science ®Google Scholar
• Morey, R. D., Rouder, J. N., Verhagen, J., & Wagenmakers, E.-J. (2014). Why hypothesis tests are essential for psychological science: A comment on cumming. Psychological Science, 25(6), 1289–1290. https://doi.org/10.1177/0956797614525969
   PubMed Web of Science ®Google Scholar
• Naughton, F., & Johnston, D. (2014). A starter kit for undertaking n-of-1 trials. The European Health Psychologist, 16(5), 196–205.
   Google Scholar
• Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, 6251. https://doi.org/10.1126/science.aac4716
   Web of Science ®Google Scholar
• Peters, G.-J Y. (2017). Diamond plots: A tutorial to introduce a visualisation tool that facilitates interpretation and comparison of multiple sample estimates while respecting their inaccuracy. Health Psychology Bulletin, doi:10.31234/osf.io/fzh6c
   Google Scholar
• Peters, G.-J Y. (2019). ufs: Quantitative analysis made accessible. https://ufs.opens.science.
   Google Scholar
• Peters, G.-J Y., Abraham, C. S., & Crutzen, R. (2012). Full disclosure: Doing behavioural science necessitates sharing. The European Health Psychologist, 14(4), 77–84.
   Google Scholar
• Peters, G.-J Y., & Crutzen, R. (2017). Pragmatic Nihilism: How a theory of nothing can help health psychology progress. Health Psychology Review, 11(2), 103–121. https://doi.org/10.1080/17437199.2017.1284015
   PubMed Web of Science ®Google Scholar
• Peters, G.-J Y., de Bruin, M., & Crutzen, R. (2015). Everything should be as simple as possible, but no simpler: Towards a protocol for accumulating evidence regarding the active content of health behaviour change interventions. Health Psychology Review, 9(1), 1–14. https://doi.org/10.1080/17437199.2013.848409
   PubMed Web of Science ®Google Scholar
• Peters, G.-J Y., & Gruijters, S. L. K. (2017). Why most experiments in psychology failed: Sample sizes required for randomization to generate equivalent groups as a partial solution to the replication crisis [Doctoral dissertation]. Maastricht University.
   Google Scholar
• Peters, G.-J Y., Kok, G., Crutzen, R., & Sanderman, R. (2017). Health Psychology Bulletin: Improving publication practices to accelerate scientific progress. Health Psychology Bulletin, 1(1), 1–6. https://doi.org/10.5334/hpb.2
   Google Scholar
• Peters, T.-J., & Tichem, M. (2016). Electrothermal actuators for SiO2 photonic MEMS. Micromachines, 7(11), 200. https://doi.org/10.3390/mi7110200
   Web of Science ®Google Scholar
• R Core Team. (2018). R: A language and environment for statistical computing. https://www.R-project.org/.
   Google Scholar
• RStudio Team. (2019). RStudio: Integrated development environment for R. http://www.rstudio.com/.
   Google Scholar
• Schimmack, U. (2016). The replicability-index: Quantifying statistical research integrity. https://wordpress.com/post/replication-index.wordpress.com/920.
   Google Scholar
• Schönbrodt, F. D., & Perugini, M. (2013). At what sample size do correlations stabilize? Journal of Research in Personality, 47(5), 609–612. https://doi.org/10.1016/j.jrp.2013.05.009
   Web of Science ®Google Scholar
• Szucs, D., & Ioannidis, J. P. (2017). Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLOS Biology, 15(3), e2000797. https://doi.org/10.1371/journal.pbio.2000797
   PubMed Web of Science ®Google Scholar
• Thompson, B. (2002). What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher, 31(3), 25–32. https://doi.org/10.3102/0013189X031003025
   Google Scholar
• Trafimow, D. (2018). An a priori solution to the replication crisis. Philosophical Psychology, 31(8), 1188–1214. https://doi.org/10.1080/09515089.2018.1490707
   Web of Science ®Google Scholar
• Trafimow, D., Amrhein, V., Areshenkoff, C. N., Barrera-Causil, C. J., Beh, E. J., Bilgiç, Y. K., Bono, R., Bradley, M. T., Briggs, W. M., Cepeda-Freyre, H. A., Chaigneau, S. E., Ciocca, D. R., Correa, J. C., Cousineau, D., de Boer, M. R., Dhar, S. S., Dolgov, I., Gómez-Benito, J., Grendar, M., … Marmolejo-Ramos, F. (2018). Manipulating the alpha level cannot cure significance testing. Frontiers in Psychology, 9, 699. https://doi.org/10.3389/fpsyg.2018.00699
   PubMed Web of Science ®Google Scholar
• Wasserstein, R. L., & Lazar, N. A. (2016). The ASA’s statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108
   Web of Science ®Google Scholar
• Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019). Moving to a world beyond “p < 0.05”. The American Statistician, 73(sup1), 1–19. https://doi.org/10.1080/00031305.2019.1583913
   Web of Science ®Google Scholar
• Wheeler, B. (2016). SuppDists: Supplementary distributions. R package version 1.1-9.4.
   Google Scholar