An a priori solution to the replication crisis

References

• Asendorpf, J. B., Conner, M., DeFruyt, F., DeHouwer, J., Denissen, J. J., Fiedler, K., & Wicherts, J. M. (2013). Replication is more than hitting the lottery twice. European Journal of Personality, 27(2), 108–119.
   Web of Science ®Google Scholar
• Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66(6), 423–437.
   PubMed Web of Science ®Google Scholar
• Bakker, M., Van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7(6), 543–554.
   Web of Science ®Google Scholar
• Bargh, J. A., Chen, M., & Burrows, L. (1996). Automaticity of social behavior: Direct effects of trait construct and stereotype activation on action. Journal of Personality and Social Psychology, 71(2), 230–244.
   PubMed Web of Science ®Google Scholar
• Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner-Sorolla, R., & Veer, A. (2013). The replication recipe: What makes for a convincing replication? Journal of Experimental Social Psychology, 50, 217–224.
   Web of Science ®Google Scholar
• Briggs, W. (2016). Uncertainty: The soul of modeling, probability and statistics. New York: Springer.
   Google Scholar
• Carver, R. P. (1978). The case against statistical significance testing. Harvard Educational Review, 48(3), 378–399.
   Web of Science ®Google Scholar
• Carver, R. P. (1993). The case against statistical significance testing, revisited. The Journal of Experimental Education, 61(4), 287–292.
   Web of Science ®Google Scholar
• Chatterjee, P. (2016). Response to Pashler et al. (2016). Basic and Applied Social Psychology, 38(1), 19–29.
   Web of Science ®Google Scholar
• Chatterjee, P., Rose, R. L., & Sinha, J. (2013). Why money meanings matter in decisions to donate time and money. Marketing Letters, 21(2), 1–10.
   Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd Edition ed.). Hillsdale, N. J.: Erlbaum.
   Google Scholar
• Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49(12), 997–1003.
   Web of Science ®Google Scholar
• Collaboration, O. S. (2015). Estimating the reproducibility of psychological science. Science, 349, 6251.
   Web of Science ®Google Scholar
• Doyen, S., Klein, O., Pichon, C. L., & Cleeremans, A. (2012). Behavioral priming: It’s all in the mind, but whose mind? PLoS ONE, 7(1), e29081.
   PubMed Web of Science ®Google Scholar
• Earp, B. D. (2016). What did the OSC replication initiative reveal about the crisis in psychology? An open review of the draft paper entitled “Replication initiatives will not salvage the trustworthiness of psychology” by James C. Coyne. BMC Psychology, 4(28), 1–19.
   PubMedGoogle Scholar
• Earp, B. D., & Everett, J. A. C. (2015, October 26) How to fix psychology’s replication crisis. The Chronicle of Higher Education. http://chronicle.com/article/How-to-Fix-Psychology-s/233857.
   Google Scholar
• Earp, B. D., Everett, J. A. C., Madva, E. N., & Hamlin, J. K. (2014). Out, damned spot: Can the “Macbeth Effect” be replicated? Basic and Applied Social Psychology, 36(1), 91–98.
   Web of Science ®Google Scholar
• Earp, B. D., & Trafimow, D. (2015). Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology, 6(621), 1–11.
   PubMedGoogle Scholar
• Greenwald, A. G., Gonzalez, R., Harris, R. J., & Guthrie, D. (1996). Effect sizes and p-values: What should be reported and what should be replicated? Psychophysiology, 33(2), 175–183.
   PubMed Web of Science ®Google Scholar
• Grice, J. W. (2017). Comment on Locascio’s results blind manuscript evaluation proposal. Basic and Applied Social Psychology, 39(5), 254–255.
   Web of Science ®Google Scholar
• Halsey, L. G., Curran-Everett, D., Vowler, S. L., & Drummond, G. B. (2015). The fickle P value generates irreproducible results. Nature Methods, 12, 179–185.
   PubMed Web of Science ®Google Scholar
• Hubbard, R. (2016). Corrupt research: The case for reconceptualizing empirical management and social science. Los Angeles: Sage Publications.
   Google Scholar
• Hyman, M. (2017). Can ‘results blind manuscript evaluation’ assuage ‘publication bias’? Basic and Applied Social Psychology, 39(5), 247–251.
   Web of Science ®Google Scholar
• Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, e124.
   PubMed Web of Science ®Google Scholar
• Iverson, G. J., Wagenmakers, E.-J., & Lee, M. D. (2010). A model averaging approach to replication: The case of prep. Psychological Methods, 15(2), 172–181.
   PubMed Web of Science ®Google Scholar
• John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532.
   PubMed Web of Science ®Google Scholar
• Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795.
   Web of Science ®Google Scholar
• Killeen, P. R. (2005). An alternative to null-hypothesis significance tests. Psychological Science, 16(5), 345–353.
   PubMed Web of Science ®Google Scholar
• Kline, R. (2017). Comment on Locascio, results blind science publishing. Basic and Applied Social Psychology, 39(5), 256–257.
   Web of Science ®Google Scholar
• Lakatos, I. (1978). The methodology of scientific research programmes. Cambridge, England: Cambridge University Press.
   Google Scholar
• Locascio, J. (2017a). Results blind science publishing. Basic and Applied Social Psychology., 39(5), 239–246.
   PubMed Web of Science ®Google Scholar
• Locascio, J. (2017b). Rejoinder to responses to “results blind publishing.”. Basic and Applied Social Psychology, 39(5), 258–261.
   PubMed Web of Science ®Google Scholar
• Makel, M., & Plucker, J. (2014). Facts are more important than novelty. Educational Researcher, 43(6), 304–316.
   Web of Science ®Google Scholar
• Maraun, M., & Gabriel, S. (2010). Killeen’s (2005) prep coefficient: Logical and mathematical problems. Psychological Methods, 15(2), 182–191.
   PubMed Web of Science ®Google Scholar
• Marks, M. J. (2017). Commentary on Locascio 2017. Basic and Applied Social Psychology., 39(5), 252–253.
   Web of Science ®Google Scholar
• McBee, M. T., & Matthews, M. S. (2014). Welcoming quality non-significance and replication work, but not the p-values: Announcing new policies for quantitative research. Journal of Advanced Academics, 25, 68–78.
   Google Scholar
• Meehl, P. E. (1967). Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science, 34(2), 103–115.
   Web of Science ®Google Scholar
• Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46(4), 806–834.
   Web of Science ®Google Scholar
• Meehl, P. E. (1990). Appraising and amending theories: The strategy of lakatosian defense and two principles that warrant it. Psychological Inquiry, 1(2), 108–141.
   Google Scholar
• Michelson, A. A., & Morley, E. W. (1887). On the relative motion of earth and luminiferous ether. American Journal of Science, Third Series, 34(203), 233–245.
   Google Scholar
• Nguyen, H. (2016). On evidential measures of support for reasoning with integrated uncertainty: A lesson from the ban of p-values in statistical inference. In V. N. Huynh, M. Inuiguchi, D. H. Tran, & T. Denoeux (Eds.), Integrated uncertainty in knowledge modeling and decision making (pp. 3–15)). New York: Springer.
   Google Scholar
• Pashler, H., Rohrer, D., Abramson, I., Wolfson, T., & Harris, C. R. (2016a). A social priming data set with troubling oddities. Basic and Applied Social Psychology, 38(1), 3–18.
   Web of Science ®Google Scholar
• Pashler, H., Rohrer, D., Abramson, I., Wolfson, T., & Harris, C. R. (2016b). Response to comments by Chatterjee, Rose, and Sinha. Basic and Applied Social Psychology, 38(1), 41–46.
   Web of Science ®Google Scholar
• Popper, K. R. (1959). The logic of scientific discovery. New York: Basic Books.
   Google Scholar
• Popper, K. R. (1963). Conjectures and refutations. London: Routledge.
   Google Scholar
• Popper, K. R. (1972). Objective knowledge. Oxford: Oxford University Press.
   Google Scholar
• Popper, K. R. (1983). Realism and the aim of science. London: Routledge.
   Google Scholar
• Prinz, F., Schlange, T., & Asadullah, K. (2011). Believe it or not: How much can we rely on published data on potential drug targets? Nature Reviews Drug Discovery, 10, 712.
   PubMed Web of Science ®Google Scholar
• Reichenbach, H. (1938). Experience and prediction. An analysis of the foundations and the structure of knowledge. Chicago: University of Chicago Press.
   Google Scholar
• Rose, R. L. (2016). Cautious thoughts on “A social priming data set with troubling oddities.”. Basic and Applied Social Psychology, 38(1), 30–32.
   Web of Science ®Google Scholar
• Rozeboom, W. W. (1960). The fallacy of the null-hypothesis significance test. Psychological Bulletin, 57(5), 416–428. doi:10.1037/h0042040
   PubMed Web of Science ®Google Scholar
• Rozeboom, W. W. (1997). Good science is abductive, not hypothetico-deductive. In L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 335–391). Mahwah, NJ: Erlbaum.
   Google Scholar
• Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods, 1(2), 115–129.
   Web of Science ®Google Scholar
• Schmidt, F. L., & Hunter, J. E. (1997). Eight objections to the discontinuation of significance testing in the analysis of research data. In L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 37–64). Mahwah, NJ: Erlbaum.
   Google Scholar
• Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366.
   PubMed Web of Science ®Google Scholar
• Simonsohn, U. (2015). Small telescopes: Detectability and the evaluation of replication results. Psychological Science, 26(5), 559–569.
   PubMed Web of Science ®Google Scholar
• Sinha, J. (2016). Selective literature review and selective data analyses: Implications for the (re)analysis of public access research data. Basic and Applied Social Psychology, 38(1), 33–40.
   Web of Science ®Google Scholar
• Trafimow, D. (2003). Hypothesis testing and theory evaluation at the boundaries: Surprising insights from Bayes’s theorem. Psychological Review, 110(3), 526–535.
   PubMed Web of Science ®Google Scholar
• Trafimow, D. (2006). Using epistemic ratios to evaluate hypotheses: An imprecision penalty for imprecise hypotheses. Genetic, Social, and General Psychology Monographs, 132(4), 431–462.
   PubMed Web of Science ®Google Scholar
• Trafimow, D. (2009). The theory of reasoned action: A case study of falsification in psychology. Theory & Psychology, 19(4), 501–518.
   Web of Science ®Google Scholar
• Trafimow, D. (2017a). Implications of an initial empirical victory for the truth of the theory and additional empirical victories. Philosophical Psychology, 30(4), 415–437.
   Web of Science ®Google Scholar
• Trafimow, D. (2017b). Using the coefficient of confidence to make the philosophical switch from a posteriori to a priori inferential statistics. Educational and Psychological Measurement, 77(5), 831–854.
   Google Scholar
• Trafimow, D., & Earp, B. D. (2017). Null hypothesis significance testing and Type I error: The domain problem. New Ideas in Psychology, 45(1), 19–27.
   Google Scholar
• Trafimow, D., & MacDonald, J. A. (2017). Performing inferential statistics prior to data collection. Educational and Psychological Measurement, 77(2), 204–219.
   PubMed Web of Science ®Google Scholar
• Trafimow, D., MacDonald, J., Rice, S., & Clason, D. L. (2010). How often is prep close to the true replication probability? Psychological Methods, 15, 300–307.
   PubMed Web of Science ®Google Scholar
• Trafimow, D., & Marks, M. (2015). Editorial. Basic and Applied Social Psychology, 37(1), 1–2.
   Web of Science ®Google Scholar
• Trafimow, D., & Marks, M. (2016). Editorial. Basic and Applied Social Psychology, 38(1), 1–2.
   Web of Science ®Google Scholar
• Trafimow, D., & Uhalt, J. (2015). The alleged tradeoff between explanatory breadth and predictive power. Theory & Psychology, 25(6), 833–840.
   Web of Science ®Google Scholar
• Trafimow, D., & Rice, S. (2009). What if social scientists had reviewed great scientific works of the past? Perspectives in Psychological Science, 4(1), 65–78.
   Web of Science ®Google Scholar
• Valentine, J. C., Aloe, A. M., & Lau, T. S. (2015). Life after NHST: How to describe your data without “p-ing” everywhere. Basic and Applied Social Psychology, 37(5), 260–273.
   Web of Science ®Google Scholar
• Vieland, V. J., & Hodge, S. E. (2011). Measurement of evidence and evidence of measurement. Statistical Applications in Genetics and Molecular Biology, 10(1). Article 35.
   Web of Science ®Google Scholar
• Wasserstein, R. L., & Lazar, N. A. (2016). The ASA’s statement on p-values: Context, process, and purpose. The American Statistician, 70, 129–133.
   Web of Science ®Google Scholar
• Whewell, W. (1996) [1840]). The philosophy of the inductive sciences (Vol. II). London: Routledge.
   Google Scholar
• Woodside, A. (2016). The good practices manifesto: Overcoming bad practices pervasive in current research in business. Journal of Business Research, 69(2), 365–381.
   Web of Science ®Google Scholar
• Zhong, C., & Liljenquist, K. (2006). Washing away your sins: Threatened morality and physical cleansing. Science, 313(5792), 1451–1452.
   PubMed Web of Science ®Google Scholar
• Ziliak, S. T., & McCloskey, D. N. (2016). The cult of statistical significance: How the standard error costs us jobs, justice, and lives. Ann Arbor, Michigan: University of Michigan Press.
   Google Scholar