The (Non)Comparability of the Correlation Effect Size Across Different Measurement Procedures: A Challenge to Meta-Analysis as a Tool for Identifying “Evidence Based Practices”

REFERENCES

• Brennan , R. 2001 . Generalizability theory New York, NY : Springer-Verlag. .
   Google Scholar
• Campbell , D. 1960 . Recommendations for APA test standards regarding construct, trait, or discriminant validity . American Psychologist , 15 : 546 – 553 .
   Web of Science ®Google Scholar
• Cohen , J. , Cohen , P. , West , S. and Aiken , L. 2003 . Applied multiple regression/correlation analysis or the behavioral sciences, , (3rd ed.) Mahwah, NJ : Lawrence Erlbaum. .
   Google Scholar
• Cooper , H. and Hedges , L. 1994 . “ Research synthesis as a scientific enterprise. ” . In Handbook of Research Synthesis Edited by: Cooper , H. and Hedges , L. 3 – 14 . New York, NY : Russell Sage Foundation. .
   Google Scholar
• Cronbach , L. , Gleser , G. , Nanda , H. and Rajaratnam , N. 1972 . The dependability of behavioral measurement procedures: Theory of generalizability for scores and profiles New York, NY : John Wiley. .
   Google Scholar
• Cronbach , L. and Meehl , P. 1955 . Construct validity in psychological tests . Psychological Bulletin , 52 : 281 – 302 .
   PubMed Web of Science ®Google Scholar
• Cupillari , A. 2005 . The nuts and bolts of proofs, , (3rd ed.) Boston, MA : Elsevier. .
   Google Scholar
• Drasgow , F. 1984 . Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are the central issues . Psychological Bulletin , 95 : 134 – 135 .
   Web of Science ®Google Scholar
• Drasgow , F. and Kanfer , R. 1985 . Equivalence of psychological measurement in heterogeneous populations . Journal of Applied Psychology , 70 : 662 – 680 .
   PubMed Web of Science ®Google Scholar
• Embretson , S. 1983 . Construct validity: Construct representation versus nomothetic span . Psychological Bulletin , 93 : 179 – 197 .
   Web of Science ®Google Scholar
• Feldt , L. and Brennan , R. 1989 . “ Reliability. ” . In Educational measurement, , 2nd ed. Edited by: Linn , R. 105 – 146 . New York, NY : Macmillan. .
   Google Scholar
• Hand , D. 2004 . Measurement theory and practice: The world through quantification New York, NY : Oxford Press. .
   Google Scholar
• Hedges , L. and Olkin , I. 1985 . Statistical methods for meta-analysis New York, NY : Academic Press. .
   Google Scholar
• Hunt , M. 1997 . How science takes stock New York, NY : Russell Sage Foundation. .
   Google Scholar
• Kane , M. 1982 . A sampling model for validity . Applied Psychological Measurement , 6 ( 2 ) : 125 – 160 .
   Web of Science ®Google Scholar
• Lam , R. W. and Kennedy , S. H. 2005 . Using meta-analysis to evaluate evidence: Practical tips and traps . Canadian Journal of Psychiatry , 50 ( 3 ) : 167 – 174 .
   PubMed Web of Science ®Google Scholar
• Lipsey , M. and Wilson , D. 2001 . Practical Meta-Analysis Newbury Park, CA : Sage. .
   Google Scholar
• Littell , J. , Corcoran , J. and Pillai , V. 2008 . Systematic reviews and meta-analysis New York, NY : Oxford. .
   Google Scholar
• Lord , F. and Novick , M. 1968 . Statistical theories of mental test scores Reading, MA : Addison-Wesley. .
   Google Scholar
• Maier , W. and Moller , H. 2005 . Meta-analyses: Highest level of empirical evidence? . European Archives of Psychiatry & Clinical Neuroscience , 255 : 369 – 370 .
   PubMedGoogle Scholar
• Messick , S. 1989 . “ Validity. ” . In Educational measurement , (3rd ed.) Edited by: Linn , R. L. 13 – 104 . New York, NY : Macmillan. .
   Google Scholar
• Messick , S. 1981 . Constructs and their vicissitudes in educational and psychological measurement . Psychological Bulletin , 89 : 575 – 588 .
   Web of Science ®Google Scholar
• Nugent , W. 2006 . The comparability of the standardized mean difference effect size across different measures of the same construct: Measurement considerations . Educational and Psychological Measurement , 66 ( 8 ) : 612 – 623 .
   Google Scholar
• Nugent , W. 2009a . Meta-analysis as a research synthesis tool: Cause for concern . Journal of Social Service Research , 35 ( 2 ) : 181 – 192 .
   Web of Science ®Google Scholar
• Nugent , W. 2009b . Construct validity invariance and discrepancies in meta-analytic effect sizes based on different measures: A simulation study . Educational and Psychological Measurement , 69 ( 1 ) : 62 – 78 .
   Google Scholar
• Nunnally , J. and Bernstein , I. 1994 . Psychometric theory, , (3rd ed.) New York, NY : McGraw-Hill. .
   Google Scholar
• Oreg , S. , Bayazit , M. , Vakola , M. , Arciniega , L. , Armenakis , A. Barkauskiere , R. 2008 . Dispositional resistance to change: Measurement equivalence and the link to personal values across 17 nations . Journal of Applied Psychology , 93 ( 4 ) : 935 – 944 .
   PubMed Web of Science ®Google Scholar
• Petitti , D. 2000 . Meta-analysis, decision analysis, and cost-effectiveness analysis: Methods for quantitative synthesis in medicine, , (2nd ed.) New York, NY : Oxford Press. .
   Google Scholar
• Rosenthal , H. 1994 . “ Parametric measures of effect size. ” . In Handbook of Research Synthesis Edited by: Cooper , H. and Hedges , L. 231 – 244 . New York, NY : Russell Sage Foundation. .
   Google Scholar
• Shavelson , R. and Webb , N. 1991 . Generalizability theory: A primer Thousand Oaks, CA : Sage. .
   Google Scholar
• Solow , D. 2004 . How to read and do proofs, , (4th ed.) New York, NY : John Wiley. .
   Google Scholar
• Vandenberg , R. and Lance , C. 2000 . A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research . Organizational Research methods , 3 ( 1 ) : 4 – 70 .
   Web of Science ®Google Scholar
• Wilson , D. and Lipsey , M. 2001 . The role of method in treatment effectiveness research: Evidence from meta-analysis . Psychological Methods , 6 ( 4 ) : 413 – 429 .
   PubMed Web of Science ®Google Scholar