Constructive Criticisms of Methodological and Editorial Practices

References

• Aiken , L. R. 1997 . Psychological testing and assessment, , 9th ed , Boston : Allyn & Bacon .
   Google Scholar
• Brewer , J. K. 1985 . Behavioral statistics textbooks: Source of myth and misconceptions? . Journal of Educational Statistics , 10 : 252 – 268 .
   Google Scholar
• Cangelosi , J. S. 1990 . Designing tests for evaluating student achievement , New York : Longman .
   Google Scholar
• Carver , R. P. 1993 . The case against statistical significance testing, revisited . The Journal of Experimental Education , 61 : 287 – 291 .
   Web of Science ®Google Scholar
• Cochran , W. G. 1977 . “ Early development of techniques in comparative experimentation ” . In Experimental design and interpretation , Edited by: Collier , R. O. Jr. and Hummel , T. J. 5 – 21 . Berkeley, CA : McCutchan, American Educational Research Association .
   Google Scholar
• Cohen , J. 1994 . The earth is round (p< .05) . American Psychologist , 49 : 997 – 1003 .
   Web of Science ®Google Scholar
• Duncan , D. B. 1955 . Multiple range and multiple F tests . Biometrics , 11 : 1 – 42 .
   Web of Science ®Google Scholar
• Dunn , O. J. 1961 . Multiple comparisons among means . Journal of the American Statistical Association , 56 : 52 – 64 .
   Web of Science ®Google Scholar
• Dunnet , C. W. 1955 . A multiple comparison procedure for comparing several treatments with a control . Journal of the American Statistical Association , 50 : 1096 – 1121 .
   Web of Science ®Google Scholar
• Frick , R. W. 1996 . The appropriate use of null hypothesis testing . Psychological Methods , 1 : 379 – 390 .
   Web of Science ®Google Scholar
• Grant , C. A. and Graue , E. 1999 . Re(Viewing) a review: A case history of the Review of Educational Research (1999) . Review of Educational Research , 69 : 384 – 396 .
   Google Scholar
• Hagen , R. L. 1997 . In praise of the null hypothesis statistical test . American Psychologist , 52 : 15 – 24 .
   Web of Science ®Google Scholar
• Harlow , L. L. , Mulaik , S. A. and Steiger , J. H. 1997 . What if there were no significance tests? , Mahwah, NJ : Erlbaum .
   Google Scholar
• Hays , W. L. 1981 . Statistics, , 3rd ed. , New York : Holt, Rinehart & Winston .
   Google Scholar
• Heldref Foundation . 1997 . Guidelines for contributors . The Journal of Experimental Education , 65 : 95 – 96 .
   Google Scholar
• Hochberg , Y. 1976 . A modification of the T-method of multiple comparisons for a one-way layout with unequal variances . Journal of the American Statistical Association , 71 : 200 – 203 .
   Web of Science ®Google Scholar
• Holm , S. 1979 . A simple sequentially rejective multiple test procedure . Scandinavian Journal of Statistics , 6 : 65 – 70 .
   Web of Science ®Google Scholar
• Hopkins , K. D. , Stanley , J. C. and Hopkins , B. R. 1990 . Educational and psychological measurement and evaluation, , 7th ed. , Englewood Cliffs, NJ : Prentice Hall .
   Google Scholar
• Hunter , M. A. and May , R. B. 1993 . Some myths concerning parametric and nonparametric tests . Canadian Psychology , 34 : 384 – 389 .
   Web of Science ®Google Scholar
• Hyman , R. 1995 . How to critique a published article . Psychological Bulletin , 118 : 178 – 182 .
   Web of Science ®Google Scholar
• Kaplan , R. M. and Saccuzzo , D. P. 1993 . Psychological testing: Principles, applications, and issues., , 3rd ed , Pacific Grove, CA : Brooks/Cole .
   Google Scholar
• Kirk , R. 1996 . Practical significance: A concept whose time has come . Educational and Psychological Measurement , 56 : 746 – 759 .
   Web of Science ®Google Scholar
• Knapp , T. R. 1999 . The use of tests of statistical significance . Mid-Western Educational Researcher , 12 ( 2 ) : 2 – 5 .
   Google Scholar
• Kromrey , J. D. and Dickinson , W. B. 1995 . The use of an overall F test to control Type I error rates in factorial analyses of variance: Limitations and better strategies . Journal of Applied Behavioral Science , 31 : 51 – 64 .
   Google Scholar
• Kubiszyn , T. and Borich , G. 1996 . Educational testing and measurement, , 5th ed. , New York : Harper Collins .
   Google Scholar
• Levin , J. R. 1993 . Statistical significance testing from three perspectives . The Journal of Experimental Education , 61 : 378 – 382 .
   Web of Science ®Google Scholar
• Levin , J. R. and Robinson , D. H. 1999 . Further reflections on hypothesis testing and editorial policy for primary research journals . Educational Psychology Review , 11 ( 2 ) : 143 – 155 .
   Web of Science ®Google Scholar
• Low , M. G. 1997 . On nonparametric confidence intervals . The Annals of Statistics , 25 : 2547 – 2554 .
   Google Scholar
• Marascuilo , L. A. and Serlin , R. C. 1988 . Statistical methods for the social and behavioral sciences , New York : Freeman .
   Google Scholar
• Maxwell , S. E. and Cole , D. A. 1995 . Tips for writing (and reading) methodological articles . Psychological Bulletin , 118 : 193 – 198 .
   Web of Science ®Google Scholar
• Meehl , P. E. 1978 . Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology . Journal of Consulting and Clinical Psychology , 46 : 806 – 834 .
   Web of Science ®Google Scholar
• Meehl , P. E. 1990 . Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it . Psychological Reports , 66 : 195 – 244 .
   Web of Science ®Google Scholar
• Mehrens , W. A. and Lehmann , I. J. 1987 . Using standardized tests in education, , 4th ed , New York : Longman .
   Google Scholar
• Melton , A. W. 1962 . Editorial . Journal of Experimental Psychology , 64 : 553 – 557 .
   Google Scholar
• Micceri , T. 1989 . The unicorn, the normal curve, and other improbable creatures . Psychological Bulletin , 105 : 156 – 166 .
   Web of Science ®Google Scholar
• Mittag , K. C. and Thompson , B. 2000 . A national survey of AERA members' perceptions of statistical significance tests and other statistical issues . Educational Researcher , 29 ( 4 ) : 14 – 20 .
   Google Scholar
• Nunnally , J. C. 1978 . Psychometric theory, , 2nd ed , New York : McGraw-Hill .
   Google Scholar
• Oosterhof , A. 1996 . Developing and using classroom assessments , Englewood Cliffs, NJ : Prentice Hall .
   Google Scholar
• Pedhazur , E. J. and Schmelkin , L. P. 1991 . Measurement, design, and analysis: An integrated approach , Hillsdale, NJ : Erlbaum .
   Google Scholar
• Popham , W. J. 1990 . Modern educational measurement: A practitioner's perspective, , 2nd ed. , Englewood Cliffs, NJ : Prentice Hall .
   Google Scholar
• Robinson , D. and Levin , J. 1997 . Reflections on statistical and substantive significance, with a slice of replication . Educational Researcher , 26 ( 5 ) : 21 – 26 .
   Google Scholar
• Sawilowsky , S. S. 2000a . Psychometrics vs. datametrics: Comment on Vacha-Haase's “reliability generalization” method and some EPM editorial policies . Educational and Psychological Measurement , 60 : 157 – 173 .
   Web of Science ®Google Scholar
• Sawilowsky , S. S. 2000b . Reliability: Rejoinder to Thompson and Vacha-Haase . Educational and Psychological Measurement , 60 : 196 – 200 .
   Web of Science ®Google Scholar
• Sawilowsky , S. S. “ Quasi-experimental design: The legacy of Campbell and Stanley ” . In Social indicators/quality of life research methodology: Methodological developments and issues, Yearbook 1999 , Edited by: Zumbo , B. D. Norwell, MA : Kluwer . (in press)
   Google Scholar
• Sawilowsky , S. S. and Blair , R. C. 1992 . A more realistic look at the robustness and Type II error properties of the t test to departures from population normality . Psychological Bulletin , 111 : 353 – 360 .
   Web of Science ®Google Scholar
• Scheffé , H. 1953 . A method for judging all contrasts in the analysis of variance . Biometrika , 40 : 87 – 104 .
   Web of Science ®Google Scholar
• Soric , B. 1989 . Statistical “discoveries” and effect-size estimation . Journal of the American Statistical Association , 84 : 608 – 610 .
   Web of Science ®Google Scholar
• Sterling , T. D. , Rosenbaum , W. L. and Weinkam , J. J. 1995 . Publication decisions revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa . The American Statistician , 49 : 108 – 112 .
   Web of Science ®Google Scholar
• Tamhane , A. 1977 . Multiple comparisons in Model I one-way ANOVA with unequal variances . Communications in Statistics , A6 : 15 – 32 .
   Google Scholar
• Thompson , B. 1990 . ALPHAMAX: A program that maximizes coefficient alpha by selective item deletion . Educational and Psychological Measurement , 50 : 585 – 589 .
   Web of Science ®Google Scholar
• Thompson , B. 1993 . The use of Statistical significance tests in research: Bootstrap and other alternatives . The Journal of Experimental Education , 61 : 361 – 377 .
   Web of Science ®Google Scholar
• Thompson , B. 1994 . Guidelines for authors . Educational and Psychological Measurement , 54 : 837 – 847 .
   Google Scholar
• Thompson , B. 1995 . Stepwise regression and stepwise discriminant analysis need not apply here: A guidelines editorial . Educational and Psychological Measurement , 55 : 525 – 534 .
   Web of Science ®Google Scholar
• Thompson , B. 1996 . AERA editorial policies regarding statistical significance testing: Three suggested reforms . Educational Researcher , 25 ( 2 ) : 26 – 30 .
   Google Scholar
• Thompson , B. 1998 . Editor's note . Educational and Psychological Measurement , 58 : 5
   Google Scholar
• Thompson , B. 1999a . “ Five methodology errors in educational research: A pantheon of statistical significance and other faux pas ” . In Advances in social science methodology , Edited by: Thompson , B. 23 – 86 . Greenwich, CT : JAI Press .
   Google Scholar
• Thompson , B. 1999b . Journal editorial policies regarding statistical significance tests: Heat is to fire as p is to importance . Educational Psychology Review , 11 ( 2 ) : 157 – 169 .
   Web of Science ®Google Scholar
• Thompson , B. and Borrello , G. M. 1985 . The importance of structure coefficients in regression research . Educational and Psychological Measurement , 45 : 203 – 209 .
   Web of Science ®Google Scholar
• Thompson , B. and Vacha-Haase , T. 2000 . Psychometrics is datametrics: The test is not reliable . Educational and Psychological Measurement , 60 : 174 – 195 .
   Web of Science ®Google Scholar
• Thorndike , R. M. , Cunningham , G. G. , Thorndike , R. L. and Hagen , E. P. 1991 . Measurement and evaluation in psychology and education, , 5th ed , New York : Macmillan .
   Google Scholar
• Tukey , J. W. 1953 . The problem of multiple comparisons Unpublished manuscript, Princeton University
   Google Scholar
• Tukey , J. W. 1991 . The philosophy of multiple comparisons . Statistical Science , 6 : 100 – 116 .
   Google Scholar
• Vacha-Haase , T. 1998 . Reliability generalization: Exploring variance in measurement error affecting score reliability across studies . Educational and Psychological Measurement , 58 : 6 – 20 .
   Web of Science ®Google Scholar
• Wilkinson L., & the APA Task Force on Statistical Inference . 1999 . Statistical methods in psychology journals: Guidelines and explanations . American Psychologist , 54 : 594 – 604 .
   Web of Science ®Google Scholar
• Worthen , B. R. , Borg , W. R. and White , K. R. 1993 . Measurement and evaluation in the schools , New York : Longman .
   Google Scholar
• Zumbo , B. D. and Hubley , A. M. 1998 . A note on misconceptions concerning prospective and retrospective power . The Statistician , 47 : 385 – 388 . Part 2
   Google Scholar