What Statistical Significance Testing Is, and What It Is Not

References

• Bakan, D. (1967). On method: Toward a reconstruction of psychological investigation. San Francisco: Jossey-Bass.
   Google Scholar
• Bangert-Drowns, R. L. (1986). Review of developments in meta-analytic methods. Psychological Bulletin, 99, 388–399.
   Web of Science ®Google Scholar
• Barber, B. (1961). Resistance by scientists to scientific discovery. Science, 134, 596–602.
   PubMed Web of Science ®Google Scholar
• Becker, G. (1991). Alternative methods of reporting research results. American Psychologist, 46(6), 654–655.
   Web of Science ®Google Scholar
• Berk, R. A., & Brewer, M. (1978). Feet of clay in hobnail boots: An assessment of statistical inference in applied research. Evaluation Studies Review Annual, 3, 190–214.
   Google Scholar
• Bracey, G. W. (1991). Sense, non-sense, and statistics. Phi Delta Kappan, 73(4), 335.
   Web of Science ®Google Scholar
• Brewer, J. K. (1985). Behavioral statistics textbooks: Source of myths and misconceptions? Journal of Educational Statistics, 10(3), 252–268.
   Google Scholar
• Campbell, K. E., & Jackson, T. T. (1979). The role of and need for replication research in social psychology. Replications in Social Psychology, 1(1), 3–14.
   Google Scholar
• Carlberg, C. G., & Miller, T. L. (1984). Introduction. The Journal of Special Education, 18, 9–10.
   Web of Science ®Google Scholar
• Carver, R. P. (1978). The case against statistical significance testing. Harvard Educational Review, 48(3), 378–399.
   Web of Science ®Google Scholar
• Cohen, J. (1969). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
   Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
   Google Scholar
• Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45(12), 1304–1312.
   Web of Science ®Google Scholar
• Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
   Google Scholar
• Cohen, S. A., & Hyman, J. S. (1981, April). Testing research hypotheses with critical ES instead of statistical significance in educational research. Paper presented at the annual conference of the American Educational Research Association, Los Angeles.
   Google Scholar
• Cronbach, L. J. (1975). Beyond the two disciplines of scientific psychology. American Psychologist, 30, 116–127.
   Web of Science ®Google Scholar
• Dar, R. (1987). Another look at Meehl, Lakatos, and the scientific practices of psychologists. American Psychologist, 42(2), 145–151.
   Web of Science ®Google Scholar
• Ferguson, G. A., & Takane, Y. (1989). Statistical analysis in psychology and education (6th ed.). New York: McGraw-Hill.
   Google Scholar
• Festinger, L. (1957). A theory of cognitive dissonance. Evanston, IL: Row, Peterson.
   Google Scholar
• Fiske, D. W. (1983). The meta-analytic revolution in outcome research. Journal of Consulting and Clinical Psychology, 51, 65–70.
   PubMed Web of Science ®Google Scholar
• Gigerenzer, G., Swijtink, Z., Porter, T., Daston, L., Beatty, J., & Krüger, L. (1989). The empire of chance: How probability changed science and everyday life. Cambridge: Cambridge University Press.
   Google Scholar
• Glass, G. V (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10), 3–8.
   Google Scholar
• Glass, G. V., & Hopkins, K. D. (1984). Statistical methods in education and psychology (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall.
   Google Scholar
• Glass, G. V. McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
   Google Scholar
• Harcum, E. R. (1989). The highly inappropriate calibrations of statistical significance. American Psychologist, 44(6), 964.
   Web of Science ®Google Scholar
• Hays, W. L. (1973). Statistics for the social sciences (2nd ed.). New York: Holt, Rinehart and Winston.
   Google Scholar
• Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. New York: Academic Press.
   Google Scholar
• Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
   Google Scholar
• Jackson, G. B. (1980). Methods for integrative reviews. Review of Educational Research, 50, 438–460.
   Web of Science ®Google Scholar
• Kupfersmid, J. (1988). Improving what is published: A model in search of an editor. American Psychologist, 43(8), 635–642.
   Web of Science ®Google Scholar
• Lightman, A., & Gingerich, O. (1992). When do anomalies begin? Science, 255, 690–695.
   PubMed Web of Science ®Google Scholar
• McHugh, R. B. (1964). Need the randomized block design be replicated? The Journal of Experimental Education, 33(2), 169–174.
   Web of Science ®Google Scholar
• Meehl, P. E. (1967). Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science, 34, 103–115.
   Web of Science ®Google Scholar
• Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 5–11.
   Google Scholar
• Morrison, D. E., & Henkel, R. E. (Eds.). (1970). The significance test controversy: A reader. Chicago: Aldine.
   Google Scholar
• Neuliep, J. W. (Ed.). (1991). Replication research in the social sciences. Newbury Park, CA: Sage.
   Google Scholar
• Pillemer, D. B. (1991). One-versus two-tailed hypothesis tests in contemporary educational research. Educational Researcher, 20(9), 13–17.
   Google Scholar
• Rosenthal, R. (1984). Meta-analytic procedures for social research. Beverly Hills, CA: Sage.
   Google Scholar
• Rosenthal, R. (1991). Replication in behavioral research. In J. W. Neuliep (Ed.), Replication research in the social sciences (pp. 1–30). Newbury Park, CA: Sage.
   Google Scholar
• Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological sciences. American Psychology, 44, 1276–1284.
   Web of Science ®Google Scholar
• Salsburg, D. S. (1985). The religion of statistics as practiced in medical journals. The American Statistician, 39(3), 220–223.
   Web of Science ®Google Scholar
• Schlaefli, A., Rest, J. R., & Thoma, S. J. (1985). Does moral education improve judgment? A meta-analysis of intervention studies using the Defining Issues Test. Review of Educational Research, 55, 319–352.
   Web of Science ®Google Scholar
• Seeman, J. (1973). On supervising student research. American Psychologist, 28, 900–906.
   Web of Science ®Google Scholar
• Shaver, J. P. (1979). The productivity of educational research and the applied-basic research distinction. Educational Researcher, 8(1), 3–9.
   Google Scholar
• Shaver, J. P. (1980, April). Readdressing the role of statistical tests of significance. Paper presented at the annual meeting of the American Educational Research Association, Boston.
   Google Scholar
• Shaver, J. P. (1985a). Chance and nonsense: A conversation about interpreting tests of statistical significance, Part 1. Phi Delta Kappan, 67, 57–60.
   Web of Science ®Google Scholar
• Shaver, J. P. (1985b). Chance and nonsense: A conversation about interpreting tests of statistical significance, Part 2. Phi Delta Kappan, 67, 138–141. Erratum, 1986, 67, 624.
   Web of Science ®Google Scholar
• Shaver, J. P. (1987, October). Mandates for curriculum and instruction from educational and psychological research. Keynote address presented at the annual meeting of the Northern Rocky Mountain Educational Research Association, Park City, UT.
   Google Scholar
• Shaver, J. P. (1991). Quantitative reviewing of research. In J. P. Shaver (Ed.), Handbook of research on social studies teaching and learning (pp. 83–95). New York: Macmillan.
   Google Scholar
• Shaver, J. P., & Norton, R. S. (1980a). Populations, samples, randomness, and replication in two social studies journals. Theory and Research in Social Education, 8(2), 1–20.
   Google Scholar
• Shaver, J. P., & Norton, R. S. (1980b). Randomness and replication in ten years of the American Educational Research Journal. Educational Researcher, 9(1), 9–15.
   Google Scholar
• Signorelli, A. (1974). Statistics: Tool or master of the psychologist? American Psychologist, 29, 774–777.
   Web of Science ®Google Scholar
• Skinner, B. F. (1956). A case history in scientific method. American Psychologist, 11, 221–223.
   Web of Science ®Google Scholar
• Thompson, B. (1987, April). The use (and misuse) of statistical significance testing: Some recommendations for improved editorial policy and practice. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC. (ERIC Document Reproduction Service No. ED 287 868)
   Google Scholar
• Thompson, B. (1989). The place of qualitative research in contemporary social science: The importance of post-paradigmatic thought. In B. Thompson (Ed.), Advances in social science methodology (Vol. 1, pp. 1–42). Greenwich, CT: JAI Press.
   Google Scholar
• Tukey, J. W. (1969). Analyzing data: Sanctification or detective work? American Psychologist, 24(2), 83–91.
   Web of Science ®Google Scholar
• Winch, R. F. & Campbell, D. T. (1969). Proof: No. Evidence? Yes. The significance of tests of significance. The American Sociologist, 4, 140–143.
   Google Scholar
• Winer, B. J., Brown, D. R., & Michels, K. M. (1991). Statistical principles in experimental design (3rd ed.). New York: McGraw-Hill.
   Google Scholar
• Wolf, F. M. (1986). Meta-analysis: Quantitative methods for research synthesis. Beverly Hills, CA: Sage.
   Google Scholar