Statistical Techniques Employed in AERJ and JCP Articles from 1988 to 1997: A Methodological Review

References

• Abelson , R. P. 1997 . “ A retrospective on the significance test ban of 1999 (If there were no significance tests, they would be invented) ” . In What if there were no significance tests? , Edited by: Harlow , L. L. , Mulaik , S. A. and Steiger , J. H. 117 – 141 . Mahwah, NJ : Erlbaum .
   Google Scholar
• Aiken , L. S. , West , S. G. , Sechrest , L. , Reno , R. R. , Roediger , H. L. , Scarr , S. , Kazdin , A. E. and Sherman , S. J. 1990 . The training in statistics, methodology, and measurement in psychology . American Psychologist , 45 : 721 – 734 .
   Web of Science ®Google Scholar
• American Psychological Association. 1994 . Publication manual of the American Psychological Association, , 4th ed. , Washington, DC : Author .
   Google Scholar
• Azar , B. 1997 . APA task force urges a harder look at data . The APA Monitor , 28 ( 3 ) : 26
   Google Scholar
• Bagozzi , R. P. , Fornell , C. and Larcker , D. F. 1981 . Canonical correlation analysis as a special case of a structural relations model . Multivariate Behavioral Research , 16 : 437 – 454 .
   PubMed Web of Science ®Google Scholar
• Baumberger , J. P. and Bangert , A. W. 1996 . Research designs and statistical techniques used in the . Journal of Learning Disabilities , 29 : 313 – 316 .
   PubMed Web of Science ®Google Scholar
• Borgen , F. H. and Seling , M. J. 1978 . Uses of discriminant analysis following MANOVA: Multivariate statistics for multivariate purposes . Journal of Applied Psychology , 63 : 689 – 697 .
   Web of Science ®Google Scholar
• Burdenski , T. K. Jr. “ The importance of structure coefficients in multiple regression: A review with examples from published literature ” . In Advances in social science methodology (Vol. 6) , Edited by: Thompson , B. Stamford, CT : JAI Press . (in press)
   Google Scholar
• Cliff , N. 1987 . Analyzing multivariate data , San Diego, CA : Harcourt Brace Jovanovich .
   Google Scholar
• Cohen , J. 1962 . The statistical power of abnormal-social psychological research: A review . Journal of Abnormal and Social Psychology , 69 : 145 – 153 .
   Web of Science ®Google Scholar
• Cohen , J. 1968 . Multiple regression as a general data-analytic system . Psychological Bulletin , 70 : 426 – 433 .
   Web of Science ®Google Scholar
• Cohen , J. 1994 . The earth is round (p < .05) . American Psychologist , 49 : 997 – 1003 .
   Web of Science ®Google Scholar
• Cook , C. “ A review of intraclass correlation ” . In Advances in social science methodology (Vol. 6) , Edited by: Thompson , B. Stamford, CT : JAI Press . (in press)
   Google Scholar
• Cooley , W. W. and Lohnes , P. R. 1971 . Multivariate data analysis , New York : Wiley .
   Google Scholar
• Courville , T. and Thompson , B. 2001 . Use of structure coefficients in published multiple regression articles: b is not enough . Educational and Psychological Measurement , 61 : 229 – 248 .
   Web of Science ®Google Scholar
• Cowles , M. 1989 . Statistics in psychology: An historical perspective , Hillsdale, NJ : Erlbaum .
   Google Scholar
• Crocker , L. and Algina , J. 1986 . Introduction to classical and modern test theory , New York : Holt, Rinehart and Winston .
   Google Scholar
• Duncan , O. D. 1975 . Introduction to structural equation models , New York : Academic Press .
   Google Scholar
• Edgington , E. S. 1974 . A new tabulation of statistical procedures in APA journals . American Psychologist , 29 : 25 – 26 .
   Web of Science ®Google Scholar
• Elmore , P. B. and Woehlke , P. L. 1988 . Statistical methods employed in American Educational Research Journal, Educational Researcher Review of Educational Research from 1978 to 1987 . Educational Researcher , 17 ( 9 ) : 19 – 20 .
   Google Scholar
• Emmons , N. J. , Stallings , W. M. and Layne , B. H. Statistical methods used in American Educational Research Journal, Journal of Educational Psychology, and Sociology of Education from 1972 through 1987 . Paper presented at the annual meeting of the American Educational Research Association . Boston. April . (ERIC Document Reproduction Service No. ED 319 797)
   Google Scholar
• Fan , X. 1996 . “ Canonical correlation analysis as a general analytic model ” . In Advances in social science methodology , Edited by: Thompson , B. Vol. 4 , 71 – 94 . Greenwich, CT : JAI Press .
   Google Scholar
• Fish , L. 1988 . Why multivariate methods are usually vital . Measurement and Evaluation in Counseling and Development , 21 : 130 – 137 .
   Web of Science ®Google Scholar
• Glantz , S. A. 1980 . Biostatistics: How to detect, correct and prevent errors in the medical literature . Circulation , 61 : 1 – 7 .
   PubMed Web of Science ®Google Scholar
• Goodwin , L. D. and Goodwin , W. L. 1985 . Statistical techniques in AERJ articles, 1979–1983: The preparation of graduate students to read the educational literature . Educational Researcher , 14 ( 2 ) : 5 – 11 .
   Google Scholar
• Grimm , L. G. and Yarnold , P. R. , eds. 1995 . Reading and understanding multivariate statistics , Washington, DC : American Psychological Association .
   Google Scholar
• Harlow , L. L. , Mulaik , S. A. and Steiger , J. H. , eds. 1997 . What if there were no significance tests? , Mahwah, NJ : Erlbaum .
   Google Scholar
• Heldref Foundation. 1997 . Guidelines for contributors . The Journal of Experimental Education , 65 : 95 – 96 .
   Google Scholar
• Horst , P. 1966 . Psychological measurement and prediction , Belmont, CA : Wadsworth .
   Google Scholar
• Huberty , C. J. 1989 . “ Problems with stepwise methods–Better alternatives ” . In Advances in social science methodology , Edited by: Thompson , B. Vol. 1 , 43 – 70 . Greenwich, CT : JAI Press .
   Google Scholar
• Huberty , C. J. 1994 . Applied discriminant analysis , New York : Wiley .
   Google Scholar
• Huberty , C. J. and Barton , R. 1989 . An introduction to discriminant analysis . Measurement and Evaluation in Counseling and Development , 22 : 158 – 168 .
   Web of Science ®Google Scholar
• Huberty , C. J. and Morris , J. D. 1989 . Multivariate analysis versus multiple univariate analysis . Psychological Bulletin , 105 : 302 – 308 .
   Web of Science ®Google Scholar
• Huberty , C. J. and Pike , C. J. 1999 . “ On some history regarding statistical testing ” . In Advances in social science methodology , Edited by: Thompson , B. Vol. 5 , 1 – 22 . Stamford, CT : JAI Press .
   Google Scholar
• Huberty , C. J. and Wisenbaker , J. 1992 . “ Discriminant analysis: Potential improvements in typical practice ” . In Advances in social science methodology , Edited by: Thompson , B. Vol. 2 , 169 – 208 . Greenwich, CT : JAI Press .
   Google Scholar
• Hunter , J. E. 1997 . Needed: A ban on the significance test . Psychological Science , 8 ( 1 ) : 3 – 7 .
   Web of Science ®Google Scholar
• Joint Committee on Standards for Educational Evaluation. 1994 . The program evaluation standards: How to assess evaluations of educational programs, , 2nd ed. , Newbury Park, CA : Sage .
   Google Scholar
• Keselman , H. J. , Huberty , C. J. , Lix , L. M. , Olejnik , S. , Cribbie , R. , Donahue , B. , Kowalchuk , R. K. , Lowman , L. L. , Petoskey , M. D. , Keselman , J. C. and Levin , J. R. 1998 . Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses . Review of Educational Research , 68 : 350 – 386 .
   Web of Science ®Google Scholar
• Keselman , H. J. , Kowalchuk , R. K. and Lix , L. M. 1998 . Robust nonorthogonal analyses revisited: An update based on trimmed means . Psychometrika , 63 : 145 – 163 .
   Web of Science ®Google Scholar
• Keselman , H. J. , Lix , L. M. and Kowalchuk , R. K. 1998 . Multiple comparison procedures for trimmed means . Psychological Methods , 3 : 123 – 141 .
   Web of Science ®Google Scholar
• Kieffer , K. M. 1999a . An introductory primer on the appropriate use of exploratory and confirmatory factor analysis . Research in the Schools , 6 ( 2 ) : 75 – 92 .
   Google Scholar
• Kieffer , K. M. 1999b . “ Why generalizability theory is essential and classical test theory is often inadequate ” . In Advances in social science methodology , Edited by: Thompson , B. Vol. 5 , 149 – 170 . Stamford, CT : JAI Press .
   Google Scholar
• Kirk , R. 1996 . Practical significance: A concept whose time has come . Educational and Psychological Measurement , 56 : 746 – 759 .
   Web of Science ®Google Scholar
• Knapp , T. R. 1978 . Canonical correlation analysis: A general parametric significance testing system . Psychological Bulletin , 85 : 410 – 416 .
   Web of Science ®Google Scholar
• Lancaster , B. P. 1999 . “ Defining and interpreting suppressor effects: Advantages and limitations ” . In Advances in social science methodology , Edited by: Thompson , B. Vol. 5 , 139 – 148 . Stamford, CT : JAI Press .
   Google Scholar
• Levin , J. R. 1998 . To test or not to test Ho? . Educational and Psychological Measurement , 58 : 311 – 331 .
   Web of Science ®Google Scholar
• Maxwell , S. E. , Camp , J. C. and Arvey , R. D. 1981 . Measures of strength of association: A comparative examination . Journal of Applied Psychology , 66 : 525 – 534 .
   Web of Science ®Google Scholar
• Meier , S. T. and Davis , S. R. 1990 . Trends in reporting psychometric properties of scales used in counseling psychology research . Journal of Counseling Psychology , 37 : 113 – 115 .
   Web of Science ®Google Scholar
• Micceri , T. 1989 . The unicorn, the normal curve, and other improbable creatures . Psychological Bulletin , 105 : 156 – 166 .
   Web of Science ®Google Scholar
• Murphy , K. R. 1997 . Editorial . Journal of Applied Psychology , 82 : 3 – 5 .
   Web of Science ®Google Scholar
• Nickerson , R. S. 2000 . Null hypothesis significance testing: A review of an old and continuing controversy . Psychological Methods , 5 : 241 – 301 .
   PubMed Web of Science ®Google Scholar
• O'Grady , K. E. 1982 . Measures of explained variance: Cautions and limitations . Psychological Bulletin , 92 : 766 – 777 .
   Web of Science ®Google Scholar
• Owen , D. B. , ed. 1976 . On the history of statistics and probability , New York : Marcel Dekker .
   Google Scholar
• Pedhazur , E. J. 1982 . Multiple regression in behavioral research: Explanation and prediction, , 2nd ed. , New York : Holt, Rinehart and Winston .
   Google Scholar
• Pedhazur , E. J. 1997 . Multiple regression in behavioral research, , 3rd ed. , New York : Holt, Rinehart and Winston .
   Google Scholar
• Pedhazur , E. J. and Schmelkin , L. P. 1991 . Measurement, design, and analysis: An integrated approach , Hillsdale, NJ : Erlbaum .
   Google Scholar
• Rosenthal , R. 1979 . The “file drawer problem” and tolerance for null results . Psychological Bulletin , 86 : 638 – 641 .
   Web of Science ®Google Scholar
• Rosenthal , R. and Gaito , J. 1963 . The interpretation of level of significance by psychological researchers . Journal of Psychology , 55 : 33 – 38 .
   Web of Science ®Google Scholar
• Rosnow , R. L. and Rosenthal , R. 1989 . Statistical procedures and the justification of knowledge in psychological science . American Psychologist , 44 : 1276 – 1284 .
   Web of Science ®Google Scholar
• Schinka , J. A. , LaLone , L. and Broeckel , J. A. 1977 . Statistical methods in personality assessment . Journal of Personality Assessment , 68 : 487 – 496 .
   Web of Science ®Google Scholar
• Schmidt , F. L. 1996 . Statistical significance testing and cumulative knowledge in psychology: Implications for the training of researchers . Psychological Methods , 1 : 115 – 129 .
   Web of Science ®Google Scholar
• Schmidt , F. L. and Hunter , J. E. 1997 . “ Eight common but false objections to the discontinuation of significance testing in the analysis of research data ” . In What if there were no significance tests? , Edited by: Harlow , L. L. , Mulaik , S. A. and Steiger , J. H. 37 – 64 . Mahwah, NJ : Erlbaum .
   Google Scholar
• Sedlmeier , P. and Gigerenzer , G. 1989 . Do studies of statistical power have an effect on the power of studies? . Psychological Bulletin , 105 : 309 – 316 .
   Web of Science ®Google Scholar
• Shaver , J. 1985 . Chance and nonsense . Phi Delta Kappan , 67 ( 1 ) : 57 – 60 .
   Web of Science ®Google Scholar
• Shea , C. 1996 . Psychologists debate accuracy of “significance test.” . Chronicle of Higher Education , 42 ( 49 ) : A12 A16
   Google Scholar
• Snyder , P. and Lawson , S. 1993 . Evaluating results using corrected and uncorrected effect size estimates . The Journal of Experimental Education , 61 : 334 – 349 .
   Web of Science ®Google Scholar
• Thompson , B. 1984 . Canonical correlation analysis: Uses and interpretation , Thousand Oaks, CA : Sage .
   Google Scholar
• Thompson , B. 1992 . Misuse of ANCOVA and related “statistical control” procedures . Reading Psychology , 13 ( 1 ) : iii – xviii .
   Google Scholar
• Thompson , B. 1993 . The use of statistical significance tests in research: Bootstrap and other alternatives . The Journal of Experimental Education , 61 : 361 – 377 .
   Web of Science ®Google Scholar
• Thompson , B. 1994 . Guidelines for authors . Educational and Psychological Measurement , 54 : 837 – 847 .
   Google Scholar
• Thompson , B. 1995 . Stepwise regression and stepwise discriminant analysis need not apply here: A guidelines editorial . Educational and Psychological Measurement , 55 : 525 – 534 .
   Web of Science ®Google Scholar
• Thompson , B. 1996 . AERA editorial policies regarding statistical significance testing: Three suggested reforms . Educational Researcher , 25 ( 2 ) : 26 – 30 .
   Google Scholar
• Thompson , B. 1997 . The importance of structure coefficients in structural equation modeling confirmatory factor analysis . Educational and Psychological Measurement , 57 : 5 – 19 .
   Web of Science ®Google Scholar
• Thompson , B. Common methodology mistakes in educational research, revisited, along with a primer on both effect sizes and the bootstrap . Invited address presented at the annual meeting of the American Educational Research Association . Montreal. April . (ERIC Document Reproduction Service No. ED 429 110)
   Google Scholar
• Thompson , B. 1999b . “ Five methodology errors in educational research: A pantheon of statistical significance and other faux pas ” . In Advances in social science methodology , Edited by: Thompson , B. Vol. 5 , 23 – 86 . Stamford, CT : JAI Press .
   Google Scholar
• Thompson , B. 1999c . If statistical significance tests are broken/misused, what practices should supplement or replace them? . Theory & Psychology , 9 ( 2 ) : 167 – 183 .
   Web of Science ®Google Scholar
• Thompson , B. 1999d . Journal editorial policies regarding statistical significance tests: Heat is to fire as p is to importance . Educational Psychology Review , 11 : 157 – 169 .
   Web of Science ®Google Scholar
• Thompson , B. 2000 . “ Canonical correlation analysis ” . In Reading and understanding more multivariate statistics , Edited by: Grimm , L. and Yarnold , P. 285 – 316 . Washington, DC : American Psychological Association .
   Google Scholar
• Thompson , B. and Borrello , G. M. 1985 . The importance of structure coefficients in regression research . Educational and Psychological Measurement , 45 : 203 – 209 .
   Web of Science ®Google Scholar
• Thompson , B. and Snyder , P. A. 1998 . Statistical significance and reliability analyses in recent JCD research articles . Journal of Counseling and Development , 76 : 436 – 441 .
   Web of Science ®Google Scholar
• Thompson , B. and Vacha-Haase , T. 2000 . Psychometrics is datametrics: The test is not reliable . Educational and Psychological Measurement , 60 : 174 – 195 .
   Web of Science ®Google Scholar
• Vacha-Haase , T. 1998 . Reliability generalization: Exploring variance in measurement error affecting score reliability across studies . Educational and Psychological Measurement , 58 : 6 – 20 .
   Web of Science ®Google Scholar
• Vacha-Haase , T. , Kogan , L. R. and Thompson , B. 2000 . Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions . Educational and Psychological Measurement , 60 : 509 – 522 .
   Web of Science ®Google Scholar
• Vacha-Haase , T. , Ness , C. , Nilsson , J. and Reetz , D. 1999 . Practices regarding reporting of reliability coefficients: A review of three journals . The Journal of Experimental Education , 67 : 335 – 341 .
   Web of Science ®Google Scholar
• Vacha-Haase , T. , Nilsson , J. E. , Reetz , D. R. , Lance , T. S. and Thompson , B. 2000 . Reporting practices and APA editorial policies regarding statistical significance and effect size . Theory & Psychology , 10 : 413 – 425 .
   Web of Science ®Google Scholar
• Walker , H. M. 1975 . Studies in the history of statistical method , New York : Arno Press .
   Google Scholar
• Whittington , D. 1998 . How well do researchers report their measures? An evaluation of measurement in published educational research . Educational and Psychological Measurement , 58 : 21 – 37 .
   Web of Science ®Google Scholar
• Wilcox , R. R. 1997 . Introduction to robust estimation and hypothesis testing , San Diego, CA : Academic Press .
   Google Scholar
• Wilcox , R. R. 1998 . How many discoveries have been lost by ignoring modern statistical methods? . American Psychologist , 53 : 300 – 314 .
   Web of Science ®Google Scholar
• Wilkinson , L. and APA Task Force on Statistical Inference . 1999 . Statistical methods in psychology journals: Guidelines and explanations . American Psychologist , 54 : 594 – 604 . Reprint available through the APA Home Page: http://www.apa.org/journals/amp/amp548594.html
   Web of Science ®Google Scholar
• Willson , V. L. 1980 . Research techniques in AERJ articles: 1969 to 1978 . Educational Researcher , 9 ( 6 ) : 5 – 10 .
   Google Scholar