Possible Solution to Publication Bias Through Bayesian Statistics, Including Proper Null Hypothesis Testing

References

• Anderson, C., & Dill, K. (2000). Video games and aggressive thoughts, feelings and behavior in the laboratory and in life. Journal of Personality and Social Psychology, 78, 772–790. doi:10.1037/0022-3514.78.4.772
   PubMed Web of Science ®Google Scholar
• Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K. … Wicherts, J. M. (2013). Recommendations for increasing replicability in psychology. European Journal of Personality, 27, 108–119. doi:10.1002/per.1919
   Web of Science ®Google Scholar
• Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7, 543–554. doi:10.1177/1745691612459060
   Web of Science ®Google Scholar
• Bakker, M., & Wicherts, J. M. (2011). The (mis)reporting of statistical results in psychology journals. Behavior Research Methods, 43(3), 666–678. doi:10.3758/s13428-011-0089-5
   PubMed Web of Science ®Google Scholar
• Brown, J. D., & Bobkowski, P. S. (2011). Older and newer media: Patterns of use and effects on adolescents’ health and Well‐Being. Journal of Research on Adolescence, 21(1), 95–113. doi:10.1111/j.1532-7795.2010.00717.x
   Web of Science ®Google Scholar
• Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997–1003.
   Web of Science ®Google Scholar
• Fanelli, D. (2010). ‘Positive’ results increase down the hierarchy of the sciences. PLoS ONE, 5(4), e10068. doi:10.1371/journal.pone.0010068
   PubMed Web of Science ®Google Scholar
• Fang, F. C., Steen, R. G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences, doi:10.1073/pnas.1212247109
   Web of Science ®Google Scholar
• Ferguson, C. J., & Brannick, M. T. (2012). Publication bias in psychological science: Prevalence, methods for identifying and controlling and implications for the use of meta-analyses. Psychological Methods, 17(1), 120–128. doi:10.1037/a0024445
   PubMed Web of Science ®Google Scholar
• Ferguson, C. J., Cruz, A. M., & Rueda, S. M. (2008). Gender, video game playing habits and visual memory tasks. Sex Roles, 58, 279–286. doi:10.1007/s11199-007-9332-z
   Web of Science ®Google Scholar
• Ferguson, C. J., & Heene, M. (2012). A vast graveyard of undead theories: Publication bias and psychological science’s aversion to the null. Perspectives on Psychological Science, 7(6), 555–561. doi:10.1177/1745691612459059
   Web of Science ®Google Scholar
• Fritz, A., Scherndl, T., & Kühberger, A. (2012, April). Correlation between effect size and sample size in psychological research: Sources, consequences, and remedies. Presented at the 10th Conference of the Austrian Psychological Society, Graz, Austria.
   Google Scholar
• Gallistel, C. R. (2009). The importance of proving the null. Psychological Review, 116(2), 439–453. doi:10.1037/a0015251
   PubMed Web of Science ®Google Scholar
• Gollwitzer, M., & Melzer, A. (2012). Macbeth and the joystick: Evidence for moral cleansing after playing a violent video game. Journal of Experimental Social Psychology, 48, 1356–1360. doi:10.1016/j.jesp.2012.07.001
   Web of Science ®Google Scholar
• Hare, E. (2014, May 23). D&AD Winners (Sweetie by Lemz and Terre des Hommes won in the White Pencil category of D&AD award competition for social adverts) [web log post]. Retrieved from http://www.contagious.com/blogs/news-and-views/14215281-d-ad-winnners
   Google Scholar
• Harrison, J. S., Banks, G., Pollack, J. M., O’Boyle, E. H., & Short, J. (2014). Publication bias in strategic management research. Journal of Management. doi:10.1177/0149206314535438
   Google Scholar
• Hoijtink, H. (2011). Informative hypotheses: Theory and practice for behavioral and social scientists. Boca Raton, FL: Chapman and Hall/CRC.
   Google Scholar
• Hoorn, J. F., Konijn, E. A., Germans, D. M., Burger, S., & Munneke, A. (2015). The in-between machine: The unique value proposition of a robot or why we are modelling the wrong things. In S. Loiseau, J. Filipe, B. Duval, & J. van den Herik (Eds.), Proceedings of the 7th International Conference on Agents and Artificial Intelligence (ICAART) Jan. 10-12, 2015. Lisbon, Portugal (pp. 464–469). Lisbon, PT: SCITEPRESS.
   Google Scholar
• Ioannidis, J. (2012). Scientific inbreeding and same-team replication: Type D personality as an example. Journal of Psychosomatic Research, 73, 408–410. doi:10.1016/j.jpsychores.2012.09.014
   PubMed Web of Science ®Google Scholar
• Jeffreys, H. (1961). Theory of probability (3rd ed.). Oxford, UK: Oxford University Press.
   Google Scholar
• John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532. doi:10.1177/0956797611430953
   PubMed Web of Science ®Google Scholar
• Johnson, V. E. (2013). Revised standards for statistical evidence. Proceedings of the National Academy of Sciences, 110(48), 19313–19317. doi:10.1073/pnas.1313476110
   PubMed Web of Science ®Google Scholar
• Johnstone, D. J. (1990). Interpreting statistical insignificance: A Bayesian perspective. Psychological Reports, 66(1), 115–121. doi:10.2466/pr0.1990.66.1.115
   Web of Science ®Google Scholar
• Jovanovic, D. (2013, November 5). ‘Sweetie’ sting lures thousands of alleged pedophiles. ABC News. Retrieved from http://abcnews.go.com/WNT/video/meet-sweetie-virtual-girl-identified-1000-pedophiles-world-20797023
   Google Scholar
• Kaplan, D., & Depaoli, S. (2013). Bayesian statistical methods. In T. D. Little (Ed.), Oxford handbook of quantitative methods (pp. 407–437). Oxford, UK: Oxford University Press.
   Google Scholar
• Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795. doi:10.1080/01621459.1995.10476572
   Web of Science ®Google Scholar
• Kepes, S., & McDaniel, M. A. (2013). How trustworthy is the scientific literature in industrial and organizational psychology? Industrial and Organizational Psychology: Perspectives on Science and Practice, 6(3), 252–268. doi:10.1111/iops.12045
   Web of Science ®Google Scholar
• Killeen, P. R. (2005). An alternative to null-hypothesis significance tests. Psychological Science, 16, 345–353. doi:10.1111/j.0956-7976.2005.01538.x
   PubMed Web of Science ®Google Scholar
• Klugkist, I., Laudy, O., & Hoijtink, H. (2005). Inequality constrained analysis of variance: A Bayesian approach. Psychological Methods, 10, 477–493. doi:10.1037/1082-989X.10.4.477
   PubMed Web of Science ®Google Scholar
• Konijn, E. A. (2015). Moved by media and emotion. Amsterdam, The Netherlands: Breckfield Midway.
   Google Scholar
• Krueger, J. I. (2001). Null hypothesis significance testing: On the survival of a flawed method. American Psychologist, 56, 16–26. doi:10.1037/0003-066X.56.1.16
   PubMed Web of Science ®Google Scholar
• Lee, S.-Y., & Song, X.-Y. (2004). Evaluation of the Bayesian and maximum likelihood approaches in analyzing structural equation models with small sample sizes. Multivariate Behavioral Research, 39(4), 653–686. doi:10.1207/s15327906mbr3904_4
   PubMed Web of Science ®Google Scholar
• Levine, T. R., Weber, R., Hullett, C. R., Park, H. S., & Massi Lindsey, L. L. M. (2008). A critical assessment of null hypothesis significance testing in quantitative communication research. Human Communication Research, 34, 171–187. doi:10.1111/j.1468-2958.2008.00317.x
   Web of Science ®Google Scholar
• Levine, T. R., Weber, R., Park, H. S., & Hullett, C. R. (2008). A communication researchers’ guide to null hypothesis significance testing and alternatives. Human Communication Research, 34, 188–209. doi:10.1111/j.1468-2958.2008.00318.x
   Web of Science ®Google Scholar
• Love, J., Selker, R., Marsman, M., Jamil, T., Verhagen, A. J., Ly, A. … Wagenmakers, E.-J. (2015). JASP (Version 0.6.6) [Computer software]. Retrieved from https://jasp-stats.org/
   Google Scholar
• Meyers, J., & Meyers, K. (1995). Rey complex figure test under four different administration procedures. The Clinical Neuropsychologist, 9, 63–67. doi:10.1080/13854049508402059
   Google Scholar
• Morey, R. D., & Rouder, J. N. (2012) Bayes Factor: An R package for computing Bayes factor for a variety of psychological research designs. Web-based: available on the Comprehensive R Archive Network.
   Google Scholar
• Mulder, J., Hoijtink, H., & De Leeuw, C. (2012). BIEMS: A Fortran 90 program for calculating Bayes factor for inequality and equality constrained models. Journal of Statistical Software, 46, 1–39.
   PubMed Web of Science ®Google Scholar
• Mulder, J., Hoijtink, H., & Klugkist, I. (2010). Equality and inequality constrained multivariate linear models: Objective model selection using constrained posterior priors. Journal of Statistical Planning and Inference, 140, 887–906. doi:10.1016/j.jspi.2009.09.022
   Web of Science ®Google Scholar
• Mulder, J., Klugkist, I., Van de Schoot, R., Meeus, W., Selfhout, M., & Hoijtink, H. (2009). Bayesian model selection of informative hypotheses for repeated measurements. Journal of Mathematical Psychology, 53, 530–546. doi:10.1016/j.jmp.2009.09.003
   Web of Science ®Google Scholar
• Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5(2), 241–301. doi:10.1037/1082-989X.5.2.241
   PubMed Web of Science ®Google Scholar
• Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7(6), 615–631. doi:10.1177/1745691612459058
   Web of Science ®Google Scholar
• Oppy, G., & Dowe, D. (2011). The Turing test. In E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy. Stanford, CA: The Metaphysics Research Lab, Stanford University. ISSN 1095-5054.
   Google Scholar
• Pericchi, L. (2015, March). The chronicle of a death foretold rejection rule. Invited Contribution, ISBA Bulletin, 22(1), 7.
   Google Scholar
• Pontier, M., Hoorn, J. F., Konijn, E. A., & vd Schoot, R. (under review). Dating a Robot is Like Dating a Man.
   Google Scholar
• Romeijn, J. W., & Van de Schoot, R. (2008). A philosopher’s view on Bayesian evaluation of informative hypotheses. In H. Hoijtink, I. Klugkist, & P. Boelen (Eds.), Bayesian evaluation of informative hypotheses (pp. 329–358). New York, NY: Springer.
   Google Scholar
• Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641.
   Web of Science ®Google Scholar
• Rosenthal, R., Rosnow, R. L., & Rubin, D. B. (2000). Contrasts and effect sizes in behavioral research: A correlational approach. Cambridge, UK: Cambridge University Press.
   Google Scholar
• Rosenthal-Von Der Pütten, A. M., Schulte, F. P., Eimler, S. C., Sobieraj, S., Hoffmann, L., Maderwald, S. … Krämer, N. C. (2014). Investigations on empathy towards humans and robots using fMRI. Computers in Human Behavior, 33, 201–212. doi:10.1016/j.chb.2014.01.004
   Web of Science ®Google Scholar
• Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276–1284. doi:10.1037/0003-066X.44.10.1276
   Web of Science ®Google Scholar
• Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225–237. doi:10.3758/PBR.16.2.225
   PubMed Web of Science ®Google Scholar
• Schooler, J. (2011). Unpublished results hide the decline effect. Nature, 470(7335), 437. doi:10.1038/470437a
   PubMed Web of Science ®Google Scholar
• Sellke, T., Bayarri, M. J., & Berger, J. O. (2001). Calibration of p values for testing precise null hypotheses. The American Statistician, 55, 62–71. doi:10.1198/000313001300339950
   Web of Science ®Google Scholar
• Silvapulle, M. J., & Sen, P. K. (2004). Constrained statistical inference: Order, inequality, and shape constraints. London, UK: Wiley.
   Google Scholar
• Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366. doi:10.1177/0956797611417632
   PubMed Web of Science ®Google Scholar
• Simonsohn, U. (2014, January 2), Posterior-Hacking: Selective reporting invalidates Bayesian results also. University of Pennsylvania, The Wharton School. Retrieved from http://papers.ssrn.com/sol3/Papers.cfm?abstract_id=2374040
   Google Scholar
• Sutton, A. J., & Abrams, K. R. (2001). Bayesian methods in meta-analysis and evidence synthesis. Statistical Methods in Medical Research, 10, 277–303. doi:10.1177/096228020101000404
   PubMed Web of Science ®Google Scholar
• Turing, A. M. (1950). Computing machinery and intelligence. Mind, LIX(236), 433–460. doi:10.1093/mind/LIX.236.433
   Google Scholar
• van de Schoot, R., & Depaoli, S. (2014). Bayesian analyses: Where to start and what to report. European Health Psychologist, 16(2), 75–84.
   Google Scholar
• van de Schoot, R., Hoijtink, H., & Deković, M. (2010). Testing inequality constrained hypotheses in SEM Models. Structural Equation Modeling, 17, 443–463. doi:10.1080/10705511.2010.489010
   Web of Science ®Google Scholar
• van de Schoot, R., Hoijtink, H., Mulder, J., Van Aken, M. A. G., Orobio de Castro, B., Meeus, W., & Romeijn, J.-W. (2011). Evaluating expectations about negative emotional states of aggressive boys using Bayesian model selection. Developmental Psychology, 47, 203–212. doi:10.1037/a0020957
   PubMed Web of Science ®Google Scholar
• van de Schoot, R., Kaplan, D., Denissen, J., Asendorpf, J. B., Neyer, F. J., & Van Aken, M. A. G. (2014). A gentle introduction to bayesian analysis: Applications to developmental research. Child Development, 85(3), 842–860. doi:10.1111/cdev.2014.85.issue-3
   PubMed Web of Science ®Google Scholar
• van Kemenade, M. A. M., Konijn, E. A., & Hoorn, J. F. (2015). Robots humanize care: Moral concerns versus witnessed benefits for the elderly. In C. Verdier, M. Bienkiewicz, A. Fred, H. Gamboa, & D. Elias (Eds.), Health Informatics (pp. 648–653). Lisbon, Portugal: SCITEPRESS.
   Google Scholar
• Wagenmakers, E. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14, 779–804. doi:10.3758/BF03194105
   PubMed Web of Science ®Google Scholar
• Wagenmakers, E. J., Verhagen, J., Ly, A., Bakker, M., Lee, M. D., Matzke, D., … & Morey, R. D. (2014). A power fallacy. Behavior Research Methods, 1–5.
   Google Scholar
• Wagenmakers, E.-J., Verhagen, A. J., Ly, A., Matzke, D., Steingroever, H., Rouder, J. N., & Morey, R. D. (in press). The need for Bayesian hypothesis testing in psychological science. In S. O. Lilienfeld & I. Waldman (Eds.), Psychological science under scrutiny: Recent challenges and proposed solutions. Hoboken, NJ: John Wiley and Sons.
   Google Scholar
• Wetzels, R., Raaijmakers, J. G., Jakab, E., & Wagenmakers, E. J. (2009). How to quantify support for and against the null hypothesis: A flexible WinBUGS implementation of a default Bayesian t test. Psychonomic Bulletin & Review, 16(4), 752–760. doi:10.3758/PBR.16.4.752
   PubMed Web of Science ®Google Scholar
• Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006). The poor availability of psychological research data for reanalysis. American Psychologist, 61, 726–728. doi:10.1037/0003-066X.61.7.726
   PubMed Web of Science ®Google Scholar