Using Reliability Generalization Methods to Explore Measurement Error: An Illustration Using the MMPI–2 PSY–5 Scales

REFERENCES

• References marked with an asterisk indicate studies used in the meta-analysis
   Google Scholar
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (AERA/APA/NCME) . 1985 . Standards for educational and psychological testing , Washington, DC : American Psychological Association .
   Google Scholar
• Almagor , M. and Nevo , B. 1996 . “ The MMPI–2: Translation and first steps in its adaptation into Hebrew ” . In International adaptations of the MMPI–2 , Edited by: Butcher , J. N. 487 – 505 . Minneapolis : University of Minnesota Press .
   Google Scholar
• Arbisi , P. A. , Ben-Porath , Y. S. and McNulty , J. L. 2003 . MMPI–2 empirical correlates in inpatient psychiatric settings Unpublished manuscript
   Google Scholar
• Arnau , R. C. , Handel , R. W. and Archer , R. P. 2005 . Principle components analyses of the MMPI–2 PSY–5 scales: Identification of facet subscales . Assessment , 12 : 186 – 198 .
   PubMed Web of Science ®Google Scholar
• Bagby , R. M. , Ryder , A. G. , Ben-Dat , D. , Bacchiochi , J. and Parker , J. D. A. 2002 . Validation of the dimensional factor structure of the Personality Psychopathology Five in clinical and nonclinical samples . Journal of Personality Disorders , 16 : 304 – 316 .
   PubMed Web of Science ®Google Scholar
• Barnes , L. L. B. , Harp , D. and Jung , W. S. 2002 . Reliability generalization of scores on the Spielberger State–Trait Anxiety Inventory . Educational and Psychological Measurement , 62 : 603 – 618 .
   Web of Science ®Google Scholar
• Beck , A. T. , Steer , R. A. and Brown , G. K. 1996 . Manual for the Beck Depression Inventory , San Antonio, TX : Psychological Corporation .
   Google Scholar
• Bem , S. L. 1981 . Bem Sex Role Inventory: Professional manual , Palo Alto, CA : Consulting Psychologists Press .
   Google Scholar
• Berning , L. C. and Weed , N. C. Relative external validity of the MMPI–2 substance abuse scales . The 34th annual symposium on Recent Developments in the Use of the MMPI, MMPI–2, and MMPI–A . Huntington Beach, CA. April .
   Google Scholar
• Butcher , J. N. , ed. 1996 . International adaptations of the MMPI–2 , Minneapolis : University of Minnesota Press .
   Google Scholar
• Butcher , J. N. 1997 . MMPI–2 user's guide. The Minnesota report: Reports for forensic settings , Minneapolis, MN : Pearson Assessments .
   Google Scholar
• Butcher , J. N. 1998 . The Butcher Treatment Planning Inventory: Manual , San Antonio, TX : The Psychological Corporation .
   Google Scholar
• Butcher , J. N. , Graham , J. R. , Ben-Porath , Y. S. , Tellegen , Y. S. , Dahlstrom , W. G. and Kaemmer , B. 2001 . Minnesota Multiphasic Personality Inventory–2: Manual for administration and scoring , Rev. ed. , Minneapolis : University of Minnesota Press .
   Google Scholar
• Butcher , J. N. , Graham , J. R. , Dahlstrom , W. G. and Bowman , E. 1990 . The MMPI–2 with college students . Journal of Personality Assessment , 54 : 1 – 15 .
   Web of Science ®Google Scholar
• Butcher , J. N. , Jeffrey , T. , Cayton , T. G. , Colligan , S. , DeVore , J. and Minnegawa , R. 1990 . A study of active duty military personnel with the MMPI–2 . Military Psychology , 2 : 47 – 61 .
   Google Scholar
• Byravan , A. 1997 . Structure of personality disorders from a five-factor model perspective, and the relative superiority of the MMPI–2 PSY–5, NEO–PI–R, and the 16 PF Fifth Edition scales for predicting personality disorders . Dissertation Abstracts International , 57 ( 10 ) : 6635 (UMI No. 9708742)
   Google Scholar
• Capraro , M. M. , Capraro , R. M. and Henson , R. K. 2001 . Measurement error of scores on the Mathematics Anxiety Rating Scale across studies . Educational and Psychological Measurement , 61 : 373 – 386 .
   Web of Science ®Google Scholar
• Capraro , R. M. and Capraro , M. M. 2002 . Myers–Briggs Type Indicator score reliability across studies: A meta-analytic reliability generalization study . Educational and Psychological Measurement , 62 : 590 – 602 .
   Web of Science ®Google Scholar
• Caruso , J. C. 2000 . Reliability generalization of the NEO Personality Scales . Educational and Psychological Measurement , 60 : 236 – 254 .
   Web of Science ®Google Scholar
• Caruso , J. C. and Edwards , S. 2001 . Reliability generalization of the Junior Eysenck Personality Questionnaire . Personality and Individual Differences , 31 : 173 – 184 .
   Google Scholar
• Cheung , F. M. , Weizheng , S. and Zhang , J. 1996 . “ The Chinese MMPI–2: Research and applications in Hong Kong and the People's Republic of China ” . In International adaptations of the MMPI–2 , Edited by: Butcher , J. N. 137 – 161 . Minneapolis : University of Minnesota Press .
   Google Scholar
• Costa , P. T. and McCrae , R. R. 1992 . Neo–PI, revised: Technical manual , Odessa, FL : Psychological Assessment Resources .
   Google Scholar
• Egger , J. I. M. , De Mey , H. R. A. , Derksen , J. J. L. and van der Staak , C. P. F. 2003 . Cross-cultural replication of the five-factor model and comparison of the NEO–PI–R and MMPI–2 PSY–5 scales in a Dutch psychiatric sample . Psychological Assessment , 15 : 81 – 88 .
   PubMed Web of Science ®Google Scholar
• Gillet , I. , Simon , M. , Guelfi , J. D. , Brun-Eberentz , A. , Monier , C. Seunevel , F. 1996 . “ The MMPI–2 in France ” . In International adaptations of the MMPI–2 , Edited by: Butcher , J. N. 395 – 416 . Minneapolis : University of Minnesota Press .
   Google Scholar
• Graham , J. R. and Butcher , J. N. Differentiating schizophrenic and major affective disorders with the revised form of the MMPI . The 23rd annual symposium on Recent Developments in the Use of the MMPI . St. Petersburg, FL. March .
   Google Scholar
• Greene , R. L. 2000 . MMPI–2: An interpretive manual , Needham Heights, MA : Allyn & Bacon .
   Google Scholar
• Han , K. 1996 . “ The Korean MMPI–2 ” . In International adaptations of the MMPI–2 , Edited by: Butcher , J. N. 88 – 136 . Minneapolis : University of Minnesota Press .
   Google Scholar
• Han , K. , Weed , N. C. , Calhoun , R. and Butcher , J. N. 1995 . Psychometric characteristics of the MMPI–2 Cook–Medley Hostility scale . Journal of Personality Assessment , 65 : 567 – 585 .
   PubMed Web of Science ®Google Scholar
• Han , K. , Weed , N. C. and McNeal , T. P. 1996 . Measuring conscientiousness with the MMPI–2 . Journal of Personality Assessment , 67 : 354 – 363 .
   PubMed Web of Science ®Google Scholar
• Harkness , A. R. and McNulty , J. L. 1994 . “ The Personality Psychopathology Five (PSY–5): Issue from the pages of a diagnostic manual instead of a dictionary ” . In Differentiating normal and abnormal personality , Edited by: Strack , S. and Lorr , M. 291 – 315 . New York : Springer .
   Google Scholar
• Harkness , A. R. and McNulty , J. L. 2006 . “ An overview of personality: The MMPI–2 Personality Psychopathology (PSY–5) scales ” . In MMPI–2: A practitioner's guide , Edited by: Butcher , J. J. 73 – 97 . Washington, DC : American Psychological Association .
   Google Scholar
• Harkness , A. R. , McNulty , J. L. and Ben-Porath , Y. S. 1995 . The Personality Psychopathology Five (PSY–5): Constructs and MMPI–2 scales . Psychological Assessment , 7 : 104 – 114 .
   Web of Science ®Google Scholar
• Hathaway , S. R. and McKinley , J. C. 1943 . The Minnesota Multiphasic Personality Inventory , Minneapolis : University of Minnesota Press .
   Google Scholar
• Henson , R. K. 2001 . Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha . Measurement and Evaluation in Counseling and Development , 34 : 177 – 189 .
   Web of Science ®Google Scholar
• Henson , R. K. and Hwang , D. 2002 . Variability and prediction of measurement error in Kolb's Learning Style Inventory scores: A reliability generalization study . Educational and Psychological Measurement , 62 : 712 – 727 .
   Web of Science ®Google Scholar
• Henson , R. K. , Kogan , L. R. and Vacha-Haase , T. 2001 . A reliability generalization study of the Teacher Efficacy Scale and related instruments . Educational and Psychological Measurement , 61 : 404 – 420 .
   Web of Science ®Google Scholar
• Henson , R. K. and Thompson , B. 2002 . Characterizing measurement error in scores across studies: Some recommendations for conducting “reliability generalization” studies . Measurement and Evaluation in Counseling and Development , 35 : 113 – 126 .
   Web of Science ®Google Scholar
• Hjemboe , S. and Butcher , J. N. 1991 . Couples in marital distress: A study of personality factors as measured by the MMPI–2 . Journal of Personality Assessment , 57 : 216 – 237 .
   PubMed Web of Science ®Google Scholar
• Hogan , T. P. , Benjamin , A. and Brezinski , K. L. 2000 . Reliability methods: A note on the frequency of use of various types . Educational and Psychological Measurement , 60 : 523 – 531 .
   Web of Science ®Google Scholar
• Keiffer , K. M. and Reese , R. J. 2002 . A reliability generalization study of the Geriatric Depression Scale . Educational and Psychological Measurement , 62 : 969 – 994 .
   Google Scholar
• Koscheyev , V. S. and Leon , G. R. 1996 . “ The Russian translation and preliminary adaptation of the MMPI–2 ” . In International adaptations of the MMPI–2 , Edited by: Butcher , J. N. 385 – 394 . Minneapolis : University of Minnesota Press .
   Google Scholar
• Kurokawa , N. K. S. 1998 . Development and validation of MMPI–2 scales to assess contemporary coping constructs , Oxford, MS : University of Mississippi . Unpublished doctoral dissertation
   Google Scholar
• Long , B. , Rouse , S. V. , Nelson , R. O. and Butcher , J. N. 2004 . The MMPI–2 in sexual harassment and discrimination litigation . Journal of Clinical Psychology , 60 : 643 – 657 .
   PubMedGoogle Scholar
• Lufi , D. , Okasha , S. and Cohen , A. 2004 . Test anxiety and its effect on the personality of students with learning disabilities . Learning Disability Quarterly , 27 : 176 – 184 .
   Web of Science ®Google Scholar
• Miach , P. P. , Berah , E. F. , Butcher , J. N. and Rouse , S. V. 2000 . Utility of the MMPI–2 in assessing gender dysphoric patients . Journal of Personality Assessment , 75 : 268 – 279 .
   PubMed Web of Science ®Google Scholar
• Miller , M. W. , Kaloupek , D. G. , Dillon , A. L. and Keane , T. M. 2004 . Externalizing and internalizing subtypes of combat-related PTSD: A replication and extension using the PSY–5 scales . Journal of Abnormal Psychology , 113 : 636 – 645 .
   PubMed Web of Science ®Google Scholar
• Myers , I. B. and McCauley , M. H. 1989 . Manual: A guide to the development and use of the Myers–Briggs Type Indicator , Palo Alto, CA : Consulting Psychologists Press .
   Google Scholar
• Neugarten , B. L. , Havighurst , R. J. and Tobin , S. S. 1961 . The measurement of life satisfaction . Journal of Gerontology , 16 : 134 – 143 .
   PubMedGoogle Scholar
• Pancheri , P. , Sirigatti , S. and Biondi , M. 1996 . “ Adaptation of the MMPI–2 in Italy ” . In International adaptations of the MMPI–2 , Edited by: Butcher , J. N. 416 – 441 . Minneapolis : University of Minnesota Press .
   Google Scholar
• Petroskey , L. J. , Ben-Porath , Y. S. and Stafford , K. P. 2003 . Correlates of the Minnesota Multiphasic Personality Inventory–2 (MMPI–2) Personality Psychopathology Five (PSY–5) scales in a forensic setting . Assessment , 10 : 393 – 399 .
   PubMed Web of Science ®Google Scholar
• Reese , R. J. , Keiffer , K. M. and Briggs , B. K. 2002 . A reliability generalization study of select measures of adult attachment style . Educational and Psychological Measurement , 62 : 619 – 646 .
   Web of Science ®Google Scholar
• Rissetti , F. J. , Himmel , E. and Gonzales-Moreno , J. A. 1996 . “ Use of the MMPI–2 in Chile: Translation and adaptation ” . In International adaptations of the MMPI–2 , Edited by: Butcher , J. N. 416 – 441 . Minneapolis : University of Minnesota Press .
   Google Scholar
• Rosenberg , M. 1965 . Society and the adolescent self-image , Princeton, NJ : Princeton University Press .
   Google Scholar
• Rosenthal , R. 1979 . The “file drawer” problem and tolerance for null results . Psychological Bulletin , 86 : 638 – 641 .
   Web of Science ®Google Scholar
• Rouse , S. V. 1997 . The construct validity of the MMPI–2 PSY–5 scales in a clinical sample . Dissertation Abstracts International , 58 ( 5-B ) : 2737 (UMI No. 9734734)
   Google Scholar
• Rouse , S. V. , Finger , M. S. and Butcher , J. N. 1999 . Advances in clinical personality measurement: An item response theory analysis of the MMPI–2 PSY–5 scales . Journal of Personality Assessment , 72 : 282 – 307 .
   Web of Science ®Google Scholar
• Schulze , R. 2004 . Meta-analyses: A comparison of approaches , Cambridge, MA : Hogrefe & Huber .
   Google Scholar
• Sharpe , J. P. and Desai , S. 2001 . The Revised NEO Personality Inventory and the MMPI–2 Psychopathology Five in the prediction of aggression . Personality and Individual Differences , 31 : 505 – 518 .
   Web of Science ®Google Scholar
• Sloore , H. , Derksen , J. , De May , H. and Hellenbosch , G. 1996 . “ The Flemish/Dutch version of the MMPI–2: Development and adaptation of the inventory for Belgium and the Netherlands ” . In International adaptations of the MMPI–2 , Edited by: Butcher , J. N. 329 – 349 . Minneapolis : University of Minnesota Press .
   Google Scholar
• Stransky , J. L. 2006 . The Usefulness of MMPI–2 Restructured Clinical (RC) Scale 4 in assessing substance abuse , Mt. Pleasant, MI : Central Michigan University . Unpublished doctoral dissertation
   Google Scholar
• Streiner , D. L. 2003 . Starting at the beginning: An introduction to coefficient alpha and internal consistency . Journal of Personality Assessment , 80 : 99 – 103 .
   PubMed Web of Science ®Google Scholar
• Thompson , B. 1994 . Guidelines for authors . Educational and Psychological Measurement , 54 : 837 – 847 .
   Google Scholar
• Thompson , B. 2003 . “ Understanding reliability and coefficient alpha, really ” . In Score reliability: Contemporary thinking on reliability issues , Edited by: Thompson , B. 3 – 23 . Thousand Oaks, CA : Sage .
   Google Scholar
• Thompson , B. and Vacha-Haase , T. 2000 . Psychometrics is datametrics: The test is not reliable . Educational and Psychological Measurement , 60 : 174 – 195 .
   Web of Science ®Google Scholar
• Vacha-Haase , T. 1998 . Reliability generalization: Exploring variance in measurement error affecting score reliability across studies . Educational and Psychological Measurement , 58 : 6 – 20 .
   Web of Science ®Google Scholar
• Vacha-Haase , T. , Henson , R. K. and Caruso , J. C. 2002a . Reliability generalization: Moving toward improved understanding and use of score reliability . Educational and Psychological Measurement , 62 : 562 – 569 .
   Web of Science ®Google Scholar
• 2002b . Reliability generalization (RG) as a measurement meta-analytic method [Special issue] . Educational and Psychological Measurement , 62 ( 4 ) Vacha-Haase, T., Henson, R. K., & Caruso J. C.
   Google Scholar
• Vacha-Haase , T. , Kogan , L. R. , Tani , C. R. and Woodall , R. A. 2001 . Reliability generalization: Exploring variation of reliability coefficients of MMPI clinical scale scores . Educational and Psychological Measurement , 61 : 45 – 59 .
   Web of Science ®Google Scholar
• Vacha-Haase , T. , Kogan , L. R. and Thompson , B. 2000 . Sample composition and variabilities in published studies versus those in test manuals: Validity of score reliability inductions . Educational and Psychological Measurement , 60 : 509 – 522 .
   Web of Science ®Google Scholar
• Vendrig , A. A. , Derksen , J. J. L. and De Mey , H. R. 2000 . MMPI–2 Personality Psychopathology Five (PSY–5) and the prediction of treatment outcome for patients with chronic back pain . Journal of Personality Assessment , 74 : 423 – 438 .
   PubMed Web of Science ®Google Scholar
• Viswesvaran , C. and Ones , D. S. 2000 . Measurement error in “Big Five factors” personality assessment: Reliability generalization across studies and measures . Educational and Psychological Measurement , 60 : 224 – 235 .
   Web of Science ®Google Scholar
• Wallace , K. A. and Wheeler , A. J. 2002 . Reliability generalization of the Life Satisfaction Index . Educational and Psychological Measurement , 62 : 674 – 684 .
   Web of Science ®Google Scholar
• Wilkinson , L. and the American Psychological Association Task Force on Statistical Inference . 1999 . Statistical methods in psychology journals: Guidelines and explanations . American Psychologist , 54 : 594 – 604 .
   Web of Science ®Google Scholar
• Wygant , D. B. , Sellbom , M. , Graham , J. R. and Schenk , P. W. 2006 . Incremental validity of the MMPI–2 PSY–5 scales in assessing self-reported personality disorder criteria . Assessment , 13 : 178 – 186 .
   PubMed Web of Science ®Google Scholar
• Yin , P. and Fan , X. 2000 . Assessing the reliability of Beck Depression Inventory scores: Reliability Generalization across studies . Educational and Psychological Measurement , 60 : 201 – 223 .
   Web of Science ®Google Scholar