Advanced search
Publication Cover
The Clinical Neuropsychologist Volume 27, 2013 - Issue 7
Submit an article Journal homepage
3,382
Views
111
CrossRef citations to date
0
Altmetric
Articles

The Robust Reliability of Neuropsychological Measures: Meta-Analyses of Test–Retest Correlations

Matthew Calamia Department of Psychology, University of Iowa, Iowa City, IA, USACorrespondence[email protected]
,
Kristian Markon Department of Psychology, University of Iowa, Iowa City, IA, USA
&
Daniel Tranel Department of Psychology, University of Iowa, Iowa City, IA, USA; Department of Neurology (Division of Behavioral Neurology and Cognitive Neuroscience), University of Iowa College of Medicine, Iowa City, IA, USA
Pages 1077-1105 | Received 12 Feb 2013, Accepted 24 May 2013, Published online: 25 Jun 2013

References

  • Baayen , R. H. , Davidson , D. J. and Bates , D. M. 2008 . Mixed-effects modeling with crossed random effects for subjects and items . Journal of Memory and Language , 59 : 390 – 412 . doi: 10.1016/j.jml.2007.12.005
  • Barrash , J. , Stillman , A. , Anderson , S. W. , Uc , E. Y. , Dawson , J. D. and Rizzo , M. 2010 . Prediction of driving ability with neuropsychological tests: Demographic adjustments diminish accuracy . Journal of the International Neuropsychological Society , 16 : 679 – 686 . doi: 10.1017/S1355617710000470
  • Bates, D. (2007). lme4: Linear mixed-effects models using S4 classes. http://CRAN.R-project.org. R package version 0.99875-9
  • Beretvas , S. N. and Pastor , D. A. 2003 . Using mixed-effects models in reliability generalization studies . Educational and Psychological Measurement , 63 : 75 – 95 . doi: 10.1177/0013164402239318
  • Calamia , M. , Markon , K. and Tranel , D. 2012 . Scoring higher the second time around: Meta-analyses of practice effects in neuropsychological assessment . The Clinical Neuropsychologist , 26 : 543 – 570 . doi: 10.1080/13854046.2012.680913
  • Cattell , R. B. 1986 . “ The psychometric properties of tests: Consistency, validity, and efficiency ” . In Functional psychological testing , Edited by: Cattell , R. B. and Johnson , R. C. 54 – 78 . New York , NY : Brunner/Mazel .
  • Charter , R. A. 2003 . A breakdown of reliability coefficients by test type and reliability method, and the clinical implications of low reliability . The Journal of General Psychology , 130 : 290 – 304 . doi: 10.1080/00221300309601160
  • Charter , R. A. and Feldt , L. S. 2001 . Meaning of reliability in terms of correct and incorrect clinical decisions: The art of decision making is still alive . Journal of Clinical and Experimental Neuropsychology , 23 : 530 – 537 . doi: 10.1076/jcen.23.4.530.1227
  • Chmielewski , M. and Watson , D. 2009 . What is being assessed and why it matters: The impact of transient error on trait research . Journal of Personality and Social Psychology , 97 : 186 – 202 . doi: 10.1037/a0015618
  • Cicchetti , D. V. 1999 . Sample size requirements for increasing the precision of reliability estimates: Problems and proposed solutions . Journal of Clinical and Experimental Neuropsychology , 21 : 567 – 570 . doi: 10.1076/jcen.21.4.567.886
  • Conley , J. J. 1984 . The hierarchy of consistency: A review and model of longitudinal findings on adult individual differences in intelligence, personality and self-opinion . Personality and Individual Differences , 5 : 11 – 25 .
  • Cronbach , L. J. 1951 . Coefficient alpha and the internal structure of tests . Psychometrika , 16 : 297 – 334 . doi: 10.1007/BF02310555
  • Delis , D. , Kramer , J. , Kaplan , E. and Holdnack , J. 2004 . Reliability and validity of the Delis-Kaplan Executive Function System: An update . Journal of the International Neuropsychological Society , 10 : 301 – 303 . doi: 10.10170S1355617704102191
  • Duff , K. 2012 . Evidence-based indicators of neuropsychological change in the individual patient: Relevant concepts and methods . Archives of Clinical Neuropsychology , 27 : 248 – 261 . doi: 10.1093/arclin/acr120
  • Fastenau , P. S. , Denburg , N. L. and Mauer , B. A. 1998 . Parallel short forms for the Boston Naming Test: Psychometric properties and norms for older adults . Journal of Clinical and Experimental Neuropsychology , 6 : 828 – 834 . doi: 10.1076/jcen.20.6.828.1105
  • Graves , R. E. , Bezeau , S. C. , Fogarty , J. and Blair , R. 2004 . Boston Naming Test short forms: A comparison of previous forms with new item response theory based forms . Journal of Clinical and Experimental Neuropsychology , 7 : 891 – 902 . doi: 10.1080/13803390490510716
  • Heilbronner , R. L. , Sweet , J. J. , Attix , D. K. , Krull , K. R. , Henry , G. K. and Hart , R. P. 2010 . Official position of the American Academy of Clinical Neuropsychology on serial neuropsychological assessments: The utility and challenges of repeat test administrations in clinical and forensic contexts . The Clinical Neuropsychologist , 24 : 1267 – 1278 . doi: 10.1080/13854046.2010.526785
  • Hinton-Bayre , A. D. 2010 . Deriving reliable change statistics from test-retest normative data: Comparison of models and mathematical expressions . Archives of Clinical Neuropsychology , 25 : 244 – 256 . doi: 10.1093/arclin/acq008
  • Hinton-Bayre , A. D. 2011 . Calculating the test-retest reliability co-efficient from normative retest data for determining reliable change . Archives of Clinical Neuropsychology , 26 : 76 – 77 . doi: 10.1093/arclin/acq084
  • Lezak , M. D. , Howieson , D. , Bigler , E. and Tranel , D. 2012 . Neuropsychological assessment , 5th ed. , New York , NY : Oxford University Press .
  • Maxwell , S. E. , Kelley , K. and Rausch , J. R. 2008 . Sample size planning for statistical power and accuracy in parameter estimation . Annual Review of Psychology , 59 : 537 – 563 . doi: 10.1146/annurev.psych.59.103006.093735
  • McCaffrey , R. J. , Duff , K. and Westervelt , H. J. 2000a . Practitioner’s guide to evaluating change with intellectual assessment instruments , New York , NY : Kluwer Academic, Plenum Publishers .
  • McCaffrey , R. J. , Duff , K. and Westervelt , H. J. 2000b . Practitioner’s guide to evaluating change with neuropsychological assessment instruments , New York , NY : Kluwer Academic, Plenum Publishers .
  • Mitrushina , M. , Boone , K. B. , Razani , J. and D’elia , L. F. 2005 . Handbook of normative data for neuropsychological assessment , 2nd ed. , New York , NY : Oxford University Press .
  • Morris , S. B. and DeShon , R. P. 2002 . Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs . Psychological Methods , 7 : 105 – 125 . doi: 10.1037//1082-989X.7.1.105
  • Nimon , K. , Zientek , L. R. and Henson , R. K. 2012 . The assumption of a reliable instrument and other pitfalls to avoid when considering the reliability of data . Frontiers in Psychology , 3 : 102 doi: 10.3389/fpsyg.2012.00102
  • Rabin , L. A. , Barr , W. B. and Burton , L. A. 2005 . Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA division 40 members . Archives of Clinical Neuropsychology , 20 : 33 – 65 . doi: 10.1016/j.acn.2004.02.005
  • Rupinski , M. T. and Dunlap , W. P. 1996 . Approximating Pearson product-moment correlations from Kendall’s tau and Spearman’s rho . Educational and Psychological Measurement , 56 : 419 – 429 . doi: 10.1177/0013164496056003004
  • Russell , E. 2001 . Toward an explanation of Dodrill’s observation: High neuropsychological test performance does not accompany high IQs . The Clinical Neuropsychologist , 15 : 423 – 428 . doi: 10.1076/clin.15.3.423.10267
  • Strauss , E. , Sherman , E. M. S. and Spreen , O. 2006 . A compendium of neuropsychological tests: Administration, norms, and commentary , New York , NY : Oxford University Press .
  • Tremblay, A. (2011). LMERConvenienceFunctions: A suite of functions to back-fit fixed effects and forward-fit random effects, as well as other miscellaneous functions. R package version 1.5.3. http://CRAN.R-roject.org/package/LMERConvenienceFunctions
  • Vacha-Haase , T. 1998 . Reliability generalization: Exploring variance in measurement error affecting score reliability across studies . Educational and Psychological Measurement , 58 : 6 – 20 . doi: 10.1177/0013164498058001002
  • Vacha-Haase , T. , Kogan , L. R. , Tani , C. R. and Woodall , R. A. 2001 . Reliability generalization: Exploring variation of reliability coefficients of MMPI clinical scales scores . Educational and Psychological Measurement , 61 : 45 – 59 . doi: 10.1177/00131640121971059
  • Viswesvaran , C. , Ones , D.S. and Schmidt , F. L. 1996 . Comparative analysis of the reliability of job performance ratings . Journal of Applied Psychology , 81 : 557 – 574 . doi: 10.1037/0021-9010.81.5.557
  • Watson , D. 2004 . Stability versus change, dependability versus error: Issues in the assessment of personality over time . Journal of Research in Personality , 38 : 319 – 350 . doi: 10.1016/j.jrp.2004.03.001
  • Wicherts , J. M. and Bakker , M. 2012 . Publish (your data) or (let the data) perish! Why not publish your data too? . Intelligence , 40 : 73 – 76 . doi: 10.1016/j.intell.2012.01.004
  • Yin , P. and Fan , X. 2000 . Assessing the reliability of Beck Depression Inventory Scores: Reliability generalization across studies . Educational and Psychological Measurement , 60 : 201 – 223 . doi: 10.1177/00131640021970466

REFERENCES INCLUDED IN THE META-ANALYSIS

  • Amir , T. 2001 . Benton Visual Retention Test: Reliability, gender, and the effect of extended practice on the performance of participants from UAE . Bulletin of the Faculty of the Arts: Cairo University , 61 : 7 – 16 .
  • Alder , A. G. , Adam , J. and Arenberg , D. 1990 . Individual-differences assessment of the relationship between change in and initial level of adult cognitive functioning . Psychology and Aging , 5 : 560 – 568 . doi: 10.1037//0882-7974.5.4.560
  • Attix , D. K. , Story , T. J. , Chelune , G. J. , Ball , J. D. , Stutts , M. L. , Hart , R. P. and Barth , J. 2009 . The prediction of change: Normative neuropsychological trajectories . The Clinical Neuropsychologist , 23 : 21 – 38 . doi: 10.1080/13854040801945078
  • Azuma , T. , Cruz , R. F. , Bayles , K. A. , Tomoeda , C. K. and Montgomery , E. B. Jr . 2003 . A longitudinal study of neuropsychological change in individuals with Parkinson’s disease . International Journal of Geriatric Psychiatry , 18 : 1115 – 1120 . doi: 10.1002/gps.1015
  • Basso , M. R. , Bornstein , R. A. and Lang , J. M. 1999 . Practice effects on commonly used measures of executive function across twelve months . The Clinical Neuropsychologist , 13 : 283 – 292 . doi: 10.1076/clin.13.3.283.1743
  • Beglinger , L. J. , Gaydos , B. , Tangphao-Daniels , O. , Duff , K. , Kareken , D. A. , Crawford , J. , … and Siemers , E. R. 2005 . Practice effects and the use of alternate forms in serial neuropsychological testing . Archives of Clinical Neuropsychology , 20 : 517 – 529 . doi: 10.1016/j.acn.2004.12.003
  • Beilin , B. , Hoofien , D. , Poran , R. , Gral , I. , Grinevich , G. , Butin , B. , … and Shavit , Y. 2008 . Comparison of two patient-controlled analgesia techniques on neuropsychological functioning in the immediate postoperative period . Journal of Clinical and Experimental Neuropsychology , 30 : 674 – 682 . doi: 10.1080/13803390701667310
  • Benedict , R. H. B. 2005 . Effects of using same-versus alternate-form memory tests during short-interval repeated assessments in multiple sclerosis . Journal of the International Neuropsychological Society , 11 : 727 – 736 . doi: 10.1017/S1355617705050782
  • Berry , D. T. R. , Allen , R. S. and Schmitt , F. A. 1991 . Rey-osterrieth complex figure: Psychometric characteristics in a geriatric sample . The Clinical Neuropsychologist , 5 : 143 – 153 . doi: 10.1080/13854049108403298
  • Boone , D. E. 1991 . Item-reduction vs. subtest-reduction short forms on the WAIS–R with psychiatric inpatients . Journal of Clinical Psychology , 47 : 271 – 276 .
  • Bornstein , R. A. , Baker , G. B. and Douglass , A. B. 1987 . Short-term retest reliability of the Halstead-Reitan Battery in a normal sample . The Journal of Nervous and Mental Disease , 175 : 229 – 232 . doi: 10.1097/00005053-198704000-00007
  • Bruggemans , E. F. , Vijver , F. J. R. V. and Huysmans , H. A. 1997 . Assessment of cognitive deterioration in individual patients following cardiac surgery: Correcting for measurement error and practice effects . Journal of Clinical and Experimental Neuropsychology , 19 : 543 – 559 . doi: 10.1080/01688639708403743
  • Burns , J. , Hitchcock , A. , Elias , M. and St. Germain , R. 1978 . Use of the Satz-Mogel abbreviated WAIS with hospitalized geriatric patients . Experimental Aging Research , 4 : 479 – 491 . doi: 10.1080/03610737808257170
  • Canter , A. H. 1951 . Direct and indirect measures of psychological deficit in multiple scleroris: Part II . The Journal of General Psychology , 44 : 27 – 50 . doi: 10.1080/00221309.1951.9711236
  • Carlozzi , N. E. , Stout , J. C. , Mills , J. A. , Duff , K. , Beglinger , L. J. , Aylward , E. H. and … . 2011 . The PREDICT-HD Investigators of the Huntington Study Group. Estimating premorbid IQ in the prodromal phase of a neurodegenerative disease . The Clinical Neuropsychologist , 25 : 757 – 777 . doi: 10.1080/13854046.2011.577811
  • Carvajal , H. , Schrader , M. S. and Holmes , C. B. 1996 . Retest reliability of the Wechsler Adult Intelligence Scale-Revised for 18- to 19-Year-Olds . Psychological Reports , 78 : 211 – 214 . doi: 10.2466/pr0.1996.78.1.211
  • Catron , D. W. 1978 . Immediate test-retest changes in WAIS scores among college males . Psychological Reports , 43 : 279 – 290 . doi: 10.2466/pr0.1978.43.1.279
  • Chaves , O. C. , Lombardo , L. E. , Bearden , C. E. , Woolsey , M. D. , Martinez , D. M. , Barrett , J. A. , … and Glahn , D. C. 2011 . Association of clinical symptoms and neurocognitive performance in bipolar disorder: A longitudinal study . Bipolar Disorders , 13 : 118 – 123 . doi: 10.1111/j.1399-5618.2011.00888.x
  • Cuesta , M. J. , Peralta , V. , Zarzuela , A. and Zandio , M. 2006 . Insight dimensions and cognitive function in psychosis: A longitudinal study . BMC Psychiatry , 6 : 26 doi: 10.1186/1471-244X-6-26
  • Dangour , A. D. , Allen , E. , Elbourne , D. , Fasey , N. , Fletcher , A. E. , Hardy , P. , … and Uauy , R. 2010 . Effect of 2-yn− 3 long-chain polyunsaturated fatty acid supplementation on cognitive function in older people: A randomized, double-blind, controlled trial . The American Journal of Clinical Nutrition , 91 : 1725 – 1732 . doi: 10.3945/ajcn.2009.29121
  • Delis , D. , Kaplan , E. , Kramer , J. and Ober , B. 2000 . California Verbal Learning Test–II , San Antonio , TX : The Psychological Corporation .
  • Delis , D. , Kramer , J. , Kaplan , E. and Ober , B. 1987 . California Verbal Learning Test , San Antonio , TX : The Psychological Corporation .
  • Delis , D. C. , McKee , R. , Massman , P. J. and Kramer , J.H. 1991 . Alternate form of the California Verbal Learning Test: Development and reliability . The Clinical Neuropsychologist , 5 : 154 – 162 . doi: 10.1080/13854049108403299
  • Demakis , G. J. 1999 . Serial malingering on verbal and nonverbal fluency and memory measures: An analog investigation . Archives of Clinical Neuropsychology , 14 : 401 – 410 . doi: 10.1016/S0887-6177(98)00039-0
  • Dikmen , S. S. , Heaton , R. K. , Grant , I. and Temkin , N. R. 1999 . Test–retest reliability and practice effects of Expanded Halstead–Reitan Neuropsychological Rest Battery . Journal of the International Neuropsychological Society , 5 : 346 – 356 . doi: 10.1111/j.1528-1157.1995.tb00976.x
  • Dodrill , C. B. and Troupin , A. S. 1975 . Effects of repeated administrations of a comprehensive neuropsychological battery among chronic epileptics . Journal of Nervous and Mental Disorders , 161 : 185 – 190 . doi: 10.1097/00005053-197509000-00006
  • Duff , K. , Beglinger , L. J. , Van Der Heiden , S. , Moser , D. J. , Arndt , S. , Schultz , S. K. and Paulsen , J. 2008 . Short-term practice effects in amnestic mild cognitive impairment: Implications for diagnosis and treatment . International Psychogeriatrics , 20 : 986 – 999 . doi: 10.1017/S1041610208007254
  • Duff , K. , Westervelt , H. J. , McCaffrey , R. J. and Haase , R. F. 2001 . Practice effects, test-retest stability, and dual baseline assessments with the California Verbal Learning Test in an HIV sample . Archives of Clinical Neuropsychology , 16 : 461 – 476 . doi: 10.1093/arclin/16.5.461
  • Duke , R. B. , Bloor , B. M. and Nugent , G. R. 1968 . Changes in performance on WAIS, trail making test and finger tapping test associated with carotid artery surgery . Perceptual and Motor Skills , 26 : 399 – 404 .
  • Eckardt , M. J. and Matarazzo , J. D. 1981 . Test-retest reliability of the Halstead Impairment Index in hospitalized alcoholic and nonalcoholic males with mild to moderate neuropsychological impairment . Journal of Clinical and Experimental Neuropsychology , 3 : 257 – 269 . doi: 10.1080/01688638108403129
  • Elwood , D. L. 1972 . Test-retest reliability and cost analyses of automated and face to face intelligence testing . International Journal of Man-Machine Studies , 4 : 1 – 22 . doi: 10.1016/S0020-7373(72)80017-8
  • Estevis , E. , Basso , M. and Combs , D. 2012 . Effects of practice on the Wechsler Adult Intelligence Scale–IV across 3- and 6-month intervals . The Clinical Neuropsychologist , 26 : 239 – 254 . doi: 10.1080/13854046.2012.659219
  • Ettenhofer , M. L. , Hambrick , D. Z. and Abeles , N. 2006 . Reliability and stability of executive functioning in older adults . Neuropsychology , 20 : 607 – 613 . doi: 10.1037/0894-4105.20.5.607
  • Flanagan , J. L. and Jackson , S. T. 1997 . Test-retest reliability of three aphasia tests: Performance of non-brain-damaged older adults . Journal of Communication Disorders , 30 : 33 – 43 . doi: 10.1016/S0021-9924(96)00039-1
  • Franzen , M. D. , Paul , D. and Iverson , G. L. 1996 . Reliability of alternate forms of the trail making test . The Clinical Neuropsychologist , 10 : 125 – 129 . doi: 10.1080/13854049608406672
  • Ganguli , M. , Seaberg , E. C. , Ratcliff , G. G. , Belle , S. H. and DeKosky , S. T. 1996 . Cognitive stability over 2 Years in a rural elderly population: The MoVIES project . Neuroepidemiology , 15 : 42 – 50 . doi: 10.1159/000109888
  • Geffen , G. M. , Butterworth , P. and Geffen , L. B. 1994 . Test-retest reliability of a new form of the auditory verbal learning test (AVLT) . Archives of Clinical Neuropsychology , 9 : 303 – 316 . doi: 10.1016/0887-6177(94)90018-3
  • Goldstein , G. and Watson , J. R. 1989 . Test-retest reliability of the Halstead-Reitan battery and the WAIS in a neuropsychiatric population . The Clinical Neuropsychologist , 3 : 265 – 272 . doi: 10.1080/13854048908404088
  • Greig , T. C. , Nicholls , S. S. , Wexler , B. E. and Bell , M. D. 2004 . Test-retest stability of neuropsychological testing and individual differences in variability in schizophrenia outpatients . Psychiatry Research , 129 : 241 – 247 . doi: 10.1016/j.psychres.2004.09.006
  • Gregory , T. , Nettelbeck , T. , Howard , S. and Wilson , C. 2008 . Inspection time: A biomarker for cognitive decline . Intelligence , 36 : 664 – 671 . doi: 10.1016/j.intell.2008.03.005
  • Greve , K. W. , Love , J. M. , Sherwin , E. , Mathias , C. W. , Houston , R. J. and Brennan , A. 2002 . Temporal stability of the Wisconsin Card Sorting Test in a chronic traumatic brain injury sample . Assessment , 9 : 271 doi: 10.1177/1073191102009003006
  • Harciarek , M. , Biedunkiewicz , B. , Lichodziejewska-Niemierko , M. , Debska-Slizen , A. and Rutkowski , B. 2009 . Cognitive performance before and after kidney transplantation: A prospective controlled study of adequately dialyzed patients with end-stage renal disease . Journal of the International Neuropsychological Society , 15 : 684 – 694 . doi: 10.1017/S1355617709990221
  • Harvey , P. D. , Palmer , B. W. , Heaton , R. K. , Mohamed , S. , Kennedy , J. and Brickman , A. 2005 . Stability of cognitive performance in older patients with schizophrenia: An 8-week test-retest study . American Journal of Psychiatry , 162 : 110 – 117 . doi: 10.1176/appi.ajp.162.1.110
  • Hatsukami , D. K. , Mitchell , J. E. , Morley , J. E. , Morgan , S. F. and Levine , A. S. 1986 . Effect of naltrexone on mood and cognitive functioning among overweight men . Biological Psychiatry , 21 : 293 – 300 . doi: 10.1016/0006-3223(86)90050-8
  • Heaton , R. K. , Temkin , N. , Dikmen , S. , Avitable , N. , Taylor , M. J. , Marcotte , T. D. and Grant , I. 2001 . Detecting change: A comparison of three neuropsychological methods, using normal and clinical samples . Archives of Clinical Neuropsychology , 16 : 75 – 91 . doi: 10.1093/arclin/16.1.75
  • Hermann , B. P. , Seidenberg , M. , Schoenfeld , J. , Peterson , J. , Leveroni , C. and Wyler , A. R. 1996 . Empirical techniques for determining the reliability, magnitude, and pattern of neuropsychological change after epilepsy surgery . Epilepsia , 37 : 942 – 950 . doi: 10.1111/j.1528-1157.1996.tb00531.x
  • Hopp , G. A. , Dixon , R. A. , Grut , M. and Bäckman , L. 1997 . Longitudinal and psychometric profiles of two cognitive status tests in very old adults . Journal of Clinical Psychology , 53 : 673 – 686 . doi: 10.1002/(SICI)1097-4679(199711)53:7<673::AID-JCLP5>3.0.CO;2-J
  • Hoth , K. F. , Haley , A. P. , Gunstad , J. , Paul , R. H. , Poppas , A. , Jefferson , A. L. , … and Cohen , R. 2008 . Elevated c-reactive protein is related to cognitive decline in older adults with cardiovascular disease . Journal of the American Geriatrics Society , 56 : 1898 – 1903 . doi: 10.1111/j.1532-5415.2008.01930.x
  • Hubley , A. M. and Jassal , S. 2006 . Comparability of the Rey-Osterrieth and Modified Taylor Complex Figures using total scores, completion times, and construct validation . Journal of Clinical and Experimental Neuropsychology , 28 : 1482 – 1497 .
  • Ingram , F. , Greve , K. W. , Ingram , P. T. F. and Soukup , V. M. 1999 . Temporal stability of the Wisconsin Card Sorting Test in an untreated patient sample . British Journal of Clinical Psychology , 38 : 209 – 211 . doi: 10.1348/014466599162764
  • Ishizaki , J. , Meguro , K. , Ohe , K. , Kimura , E. , Tsuchiya , E. , Ishii , H. , … and Yamadori , A. 2002 . Therapeutic psychosocial intervention for elderly subjects with very mild Alzheimer disease in a community: The tajiri project . Alzheimer Disease & Associated Disorders , 16 : 261 doi: 10.1097/00002093-200210000-00008
  • Kelland , D. Z. and Lewis , R. F. 1994 . Evaluation of the reliability and validity of the repeatable cognitive-perceptual-motor battery . The Clinical Neuropsychologist , 8 : 295 – 308 . doi: 10.1080/13854049408404136
  • Kent , P. S. and Luszcz , M. A. 2002 . A review of the Boston Naming Test and multiple-occasion normative data for older adults on 15-item versions . The Clinical Neuropsychologist , 16 : 555 – 574 . doi: 10.1076/clin.16.4.555.13916
  • Killian , G. A. , Holzman , P. S. , Davis , J. M. and Gibbons , R. D. 1984 . Effects of psychotropic medication on selected cognitive and perceptual measures . Journal of Abnormal Psychology , 93 : 58 – 70 . doi: 10.1037//0021-843X.93.1.58
  • Klonoff , H. , Fibiger , C. H. and Hutton , G. H. 1970 . Neuropsychological patterns in chronic schizophrenia . Journal of Nervous and Mental Disease , 150 : 291 – 300 . doi: 10.1097/00005053-197004000-00004
  • Krupp , L. , Christodoulou , C. , Melville , P. , Scherl , W. , MacAllister , W. and Elkins , L. 2004 . Donepezil improved memory in multiple sclerosis in a randomized clinical trial . Neurology , 63 : 1579 – 1585 .
  • Langenecker , S. A. , Zubieta , J. K. , Young , E. A. , Akil , H. and Nielson , K. A. 2007 . A task to manipulate attentional load, set-shifting, and inhibitory control: Convergent validity and test–retest reliability of the parametric Go/No-go test . Journal of Clinical and Experimental Neuropsychology , 29 : 842 – 853 . doi: 10.1080/13803390601147611
  • Lehrner , J. , Willfort , A. , Mlekusch , I. , Guttmann , G. , Minar , E. , Ahmadi , R. , … and Lang , W. 2005 . Neuropsychological outcome 6 months after unilateral carotid stenting . Journal of Clinical and Experimental Neuropsychology , 27 : 859 – 866 . doi: 10.1080/13803390490919083
  • Lemay , S. , Bedard , M. A. , Rouleau , I. and Tremblay , P. L. 2004 . Practice effect and test-retest reliability of attentional and executive tests in middle-aged to elderly subjects . The Clinical Neuropsychologist , 18 : 284 – 302 . doi: 10.1080/13854040490501718
  • Levine , A. , Miller , E. , Becker , J. , Selnes , O. and Cohen , B. 2004 . Normative data for determining significance of test-retest differences on eight common neuropsychological instruments . The Clinical Neuropsychologist , 18 : 373 – 384 . doi: 10.1080/1385404049052420
  • Lu , P. H. , Masterman , D. A. , Mulnard , R. , Cotman , C. , Miller , B. , Yaffe , K. , … and Cummings , J. 2006 . Effects of testosterone on cognition and mood in male patients with mild Alzheimer disease and healthy elderly men . Archives of Neurology , 63 : 177 doi: 10.1001/archneur.63.2.nct50002
  • Margolis , R. B. , Dunn , E. J. and Taylor , J. M. 1985 . Parallel-form reliability of the Wechsler Memory Scale in a geriatric population with suspected dementia . The Journal of Psychology , 119 : 81 – 85 . doi: 10.1080/00223980.1985.9712609
  • Martin , R. , Sawrie , S. , Gilliam , F. , Mackey , M. , Faught , E. , Knowlton , R. , … and Kuzniekcy , R. 2002 . Determining reliable cognitive change after epilepsy surgery: Development of reliable change indices and standardized Regression-based change norms for the WMS-III and WAIS-III . Epilepsia , 43 : 1551 – 1558 .
  • Matarazzo , J. D. and Herman , D. O. 1984 . Base rate data for the WAIS-R: Test–retest stability and VIQ–PIQ differences . Journal of Clinical Neuropsychology , 6 : 351 – 366 .
  • Matarazzo , J. D. , Wiens , A. N. , Matarazzo , R. G. and Goldstein , S. G. 1974 . Psychometric and clinical test-retest reliability of the Halstead Impairment Index in a sample of healthy, young, normal men . Journal of Nervous and Mental Disease , 158 : 37 – 49 . doi: 10.1097/00005053-197401000-00006
  • McCaffrey , R. J. , Cousins , J. P. , Westervelt , H. J. , Martynowicz , M. , Remick , S. C. , Szebenyi , S. , … and Haase , R. 1995 . Practice effects with the NIMH AIDS abbreviated neuropsychological battery . Archives of Clinical Neuropsychology , 10 : 241 – 250 . doi: 10.1093/arclin/10.3.241
  • McCaffrey , R. J. , Ortega , A. , Orsillo , S. M. and Nelles , W. B. 1992 . Practice effects in repeated neuropsychological assessments . The Clinical Neuropsychologist , 6 : 32 – 42 . doi: 10.1080/13854049208404115
  • McCracken , L. M. and Franzen , M. D. 1992 . Principal-components analysis of the equivalence of alternate forms of the Trail Making Test . Psychological Assessment , 4 : 235 – 238 . doi: 10.1037//1040-3590.4.2.235
  • McCaffrey , R. J. , Westervelt , H. J. and Haase , R. F. 2001 . Serial neuropsychological assessment with the National Institute of Mental Health (NIMH) AIDS Abbreviated Neuropsychological Battery . Archives of Clinical Neuropsychology , 16 : 9 – 18 . doi: 10.1093/arclin/16.1.9
  • Messinis , L. , Lyros , E. , Georgiou , V. and Papathanasopoulos , P. 2009 . Benton Visual Retention Test performance in normal adults and acute stroke patients: Demographic considerations, discriminant validity, and test-retest reliability . The Clinical Neuropsychologist , 23 : 962 – 977 . doi: 10.1080/13854040902740691
  • Mitrushina , M. and Satz , P. 1991 . Effect of repeated administration of a neuropsychological battery in the elderly . Journal of Clinical Psychology , 47 : 790 – 801 . doi: 10.1002/1097-4679(199111)47:6<790::AID-JCLP2270470610>3.0.CO;2-C
  • Moritz , S. , Iverson , G. L. and Woodward , T. S. 2003 . Reliable change indexes for memory performance in schizophrenia as a means to determine drug-induced cognitive decline . Applied Neuropsychology , 10 : 115 – 120 . doi: 10.1207/S15324826AN1002_07
  • Morris , J. C. , Heymann , A. , Mohs , R. C. , Hughes , J. R. , van Belle , G. , Fillenbaum , G. , … and Clark , C. 1989 . The Consortium to Establish a Registry for Alzheimer´s Disease (CERAD). Part I. Clinical and neuropsychological assessment of Alzheimer´s disease . Neurology , 39 : 1159 – 1165 .
  • Mortensen , E. L. and Kleven , M. 1993 . A WAIS longitudinal study of cognitive development during the life span from ages 50 to 70 . Developmental Neuropsychology , 9 : 115 – 130 . doi: 10.1080/87565649109540548
  • Muslimović , D. , Post , B. , Speelman , J. D. , de Haan , R. J. and Schmand , B. 2009 . Cognitive decline in Parkinson’s disease: A prospective longitudinal study . Journal of the International Neuropsychological Society , 15 : 426 – 437 . doi: 10.1017/S135561770909061
  • Nakhutina , L. , Pramataris , P. , Morrison , C. , Devinsky , O. and Barr , W. B. 2010 . Reliable change indices and regression-based measures for the Rey-Osterreith Complex Figure Test in partial epilepsy patients . The Clinical Neuropsychologist , 24 : 38 – 44 . doi: 10.1080/13854040902960091
  • Netherton , S. D. , Elias , J. W. , Albrecht , N. N. and Acosta , C. 1989 . Changes in the performance of parkinsonian patients and normal aged on the Benton Visual Retention Test . Experimental Aging Research , 15 : 13 – 18 . doi: 10.1080/03610738908259753
  • Newson , R. S. and Kemps , E. B. 2005 . General lifestyle activities as a predictor of current cognition and cognitive change in older adults: A cross-sectional and longitudinal examination . The Journals of Gerontology Series B: Psychological Sciences and Social Sciences , 60 : P113 – P120 . doi: 10.1093/geronb/60.3.P113
  • Paolo , A. M. , Axelrod , B. N. and Tröster , A. I. 1996 . Test-retest stability of the Wisconsin Card Sorting Test . Assessment , 3 : 137 – 143 . doi: 10.1177/107319119600300205
  • Paolo , A. M. , Tröster , A. I. and Ryan , J. J. 1997 . Test-retest stability of the California Verbal Learning Test in older persons . Neuropsychology , 11 : 613 – 616 . doi: 10.1037/0894-4105.11.4.613
  • Rabin , A. I. 1944 . Test constancy and variation in the mentally ill . Journal of General Psychology , 31 : 231 – 239 .
  • Rezvanfard , M. , Ekhtiari , H. , Maryam Noroozian , M. and Nilipour , R. 2011 . The Rey Auditory Verbal Learning Test: Alternate forms equivalency and reliability for the Iranian Adult Population (Persian Version) . Archives of Iranian Medicine , 14 : 104 – 109 .
  • Rönnlund , M. and Nilsson , L. G. 2006 . Adult life-span patterns in WAIS-R Block Design performance: Cross-sectional versus longitudinal age gradients and relations to demographic factors . Intelligence , 34 : 63 – 78 . doi: 10.1016/j.intell.2005.06.004
  • Rosen , M. , Stallings , L. , Floor , L. and Nowakiwska , M. 1968 . Reliability and stability of Wechsler IQ scores for institutionalized mental subnormals . American Journal of Mental Deficiency , 73 : 218 – 225 .
  • Ross , T. P. 2003 . The reliability of cluster and switch scores for the Controlled Oral Word Association Test . Archives of Clinical Neuropsychology , 18 : 153 – 164 . doi: 10.1016/S0887-6177(01)00192-5
  • Ruff , R. M. , Light , R. H. and Parker , H. S. 1996 . Benton Controlled Oral Word Association Test: Reliability and updated norms . Archives of Clinical Neuropsychology , 11 : 329 – 338 . doi: 10.1093/arclin/11.4.329
  • Ryan , J. , Geisser , M. E. , Randall , D. M. and Georgemiller , R. J. 1986 . Alternate form reliability and equivalency of the Rey Auditory Verbal Learning Test . Journal of Clinical and Experimental Neuropsychology , 8 : 611 – 616 . doi: 10.1080/01688638608405179
  • Ryan , J. , Georgemiller , R. J. , Geisser , M. E. and Randall , D. M. 1985 . Test-retest stability of the WAIS-R in a clinical sample . Journal of Clinical Psychology , 41 : 552 – 556 . doi: 10.1002/1097-4679(198507)41:4<552::AID-JCLP2270410418>3.0.CO;2-#
  • Ryan , J. J. , Paolo , A. M. and Brungardt , T. M. 1992 . WAIS-R test-retest stability in normal persons 75 years and older . The Clinical Neuropsychologist , 6 : 3 – 8 . doi: 10.1080/13854049208404111
  • Sachs , B. C. , Lucas , J. A. , Smith , G. E. , Ivnik , R. J. , Petersen , R. C. , Graff-Radford , N. R. and Pedraza , O. 2012 . Reliable change on the Boston Naming Test . Journal of the International Neuropsychological Society , 18 : 375 – 378 . doi: 10.1017/S1355617711001810
  • Sano , M. , Bell , K. , Cote , L. , Dooneief , G. , Lawton , A. , Legler , L. , … and Mayeux , R. 1992 . Double-blind parallel design pilot study of acetyl levocarnitine in patients with Alzheimer’s disease . Archives of Neurology , 49 : 1137 – 1141 .
  • Santos , R. F. , Galduroz , J. C. , Barbieri , A. , Castiglioni , M. L. , Ytaya , L. Y. and Bueno , O. F. 2003 . Cognitive performance, SPECT, and blood viscosity in elderly non-demented people using ginkgo biloba . Pharmacopsychiatry , 36 : 127 – 133 . doi: 10.1055/s-2003-41197
  • Sawrie , S. M. , Chelune , G. J. , Naugle , R. I. and Lüders , H. O. 1996 . Empirical methods for assessing meaningful neuropsychological change following epilepsy surgery . Journal of the International Neuropsychological Society , 2 : 556 – 564 . doi: 10.1017/S1355617700001739
  • Sawrie , S. M. , Marson , D. C. , Boothe , A. L. and Harrell , L. E. 1999 . A method for assessing clinically relevant individual cognitive change in older adult populations . The Journals of Gerontology Series B: Psychological Sciences and Social Sciences , 54 : P116 – P124 . doi: 10.1093/geronb/54B.2.P116
  • Shaw , P. J. , Bates , D. , Cartlidge , N. , French , J. M. , Heaviside , D. , Julian , D. G. and Shaw , D. A. 1987 . Neurologic and neuropsychological morbidity following major surgery: Comparison of coronary artery bypass and peripheral vascular surgery . Stroke , 18 : 700 – 707 . doi: 10.1161/01.STR.18.4.700
  • Shuttleworth-Edwards , A. B. , Smith , I. and Radloff , S. E. 2008 . Neurocognitive vulnerability amongst university rugby players versus noncontact sport controls . Journal of Clinical and Experimental Neuropsychology , 30 : 870 – 884 . doi: 10.1080/13803390701846914
  • Silberstein , C. H. , O’Dowd , M. A. , Chartock , P. , Schoenbaum , E. E. , Friedland , G. , Hartel , D. and McKegney , F. P. 1993 . A prospective four-year follow-up of neuropsychological function in HIV seropositive and seronegative methadone-maintained patients . General Hospital Psychiatry , 15 : 351 – 359 . doi: 10.1016/0163-8343(93)90001-5
  • Silverstein , S. M. , Jaeger , J. , Donovan-Lepore , A. M. , Wilkniss , S. M. , Savitz , A. , Malinovsky , I. , … and Dent , G. 2010 . A comparative study of the MATRICS and IntegNeuro cognitive assessment batteries . Journal of Clinical and Experimental Neuropsychology , 32 : 937 – 952 . doi: 10.1080/13803391003596496
  • Slate , J. R. , Frost , J. and Cross , B. 1991 . WAIS-R stability for college students with learning disabilities . Learning Disability Quarterly , 14 : 2 – 6 . doi: 10.2307/1510369
  • Snow , W. G. , Tierney , M. C. , Zorzitto , M. L. and Fisher , R. H. 1989 . WAIS–R test-retest reliability in a normal elderly sample . Journal of Clinical and Experimental Neuropsychology , 4 : 423 – 428 . doi: 10.1080/01688638908400903
  • Steinmetz , J. P. , Brunner , M. , Loarer , E. and Houssemand , C. 2010 . Incomplete psychometric equivalence of scores obtained on the manual and the computer version of the Wisconsin Card Sorting Test? . Psychological Assessment , 22 : 199 – 202 . doi: 10.1037/a0017661
  • Sweeney , J. A. , Haas , G. L. , Keilp , J. G. and Long , M. 1991 . Evaluation of the stability of neuropsychological functioning after acute episodes of schizophrenia: One-year followup study . Psychiatry Research , 38 : 63 – 76 . doi: 10.1016/0165-1781(91)90053-R
  • Tate , R. L. , Perdices , M. and Maggiotto , S. 1998 . Stability of the Wisconsin Card Sorting Test and the determination of reliability of change in scores . The Clinical Neuropsychologist , 12 : 348 – 357 . doi: 10.1076/clin.12.3.348.1988
  • Theisen, M. E. (1997). The effects of practice in repeated administration of the Wechsler Memory Scale-Revised and the California Verbal Learning Test in normal adults. (Doctoral Dissertation). Wayne State University, Detroit, Michigan.
  • Thompson , K. and Sergejew , A. A. 1998 . Two-week test-retest of the WAIS-R and WMS-R subtests, the Purdue Pegboard, Trailmaking, and the Mental Rotation Test in women . Australian Psychologist , 33 : 228 – 230 . doi: 10.1080/00050069808257410
  • Tröster , A. I. , Woods , S. P. and Morgan , E. E. 2007 . Assessing cognitive change in Parkinson’s disease: Development of practice effect-corrected reliable change indices . Archives of Clinical Neuropsychology , 22 : 711 – 718 . doi: 10.1016/j.acn.2007.05.004
  • Tuikka , R. A. , Laaksonen , R. K. and Somer , H. V. K. 1993 . Cognitive function in myotonic dystrophy: A follow-up study . European Neurology , 33 : 436 – 441 . doi: 10.1159/000116989
  • Uchiyama , C. L. , D’Elia , L. F. , Dellinger , A. M. , Becker , J. T. , Selnes , O. A. , Wesch , J. E. , … and Miller , E. 1995 . Alternate forms of the auditory-verbal learning test: Issues of test comparability, longitudinal reliability, and moderating demographic variables . Archives of Clinical Neuropsychology , 10 : 133 – 145 . doi: 10.1093/arclin/10.2.133
  • Wagner , E. E. and Caldwell , M. S. 1979 . WAIS test-retest reliability for a clinical out-patient sample . Perceptual and Motor Skills , 48 : 131 – 137 . doi: 10.2466/pms.1979.48.1.131
  • Watkins , C. Jr and Campbell , V. 1992 . The test-retest reliability and stability of the WAIS-R in a sample of mentally retarded adults . Journal of Intellectual Disability Research , 36 : 265 – 268 . doi: 10.1111/j.1365-2788.1992.tb00514.x
  • Wechsler , D. 1981 . The Wechsler Adult Intelligence Scale-Revised Manual , San Antonio , TX : Psychological Corporation .
  • Wechsler , D. 1997 . The Wechsler Adult Intelligence Scale- III Manual , San Antonio , TX : Psychological Corporation .
  • Wechsler , D. 1999 . The Wechsler Abbreviated Scale of Intelligence Manual , San Antonio , TX : Psychological Corporation .
  • Wechsler , D. 2008 . The Wechsler Adult Intelligence Scale-IV Manual , San Antonio , TX : Psychological Corporation .
  • Wechsler , D. 2011 . The Wechsler Abbreviated Scale of Intelligence-Second Edition Manual , San Antonio , TX : Psychological Corporation .
  • Woods , S. P. , Childers , M. , Ellis , R. J. , Guaman , S. , Grant , I. and Heaton , R. K. 2006a . A battery approach for measuring neuropsychological change . Archives of Clinical Neuropsychology , 21 : 83 – 89 . doi: 10.1016/j.acn.2005.07.008
  • Woods , S. P. , Delis , D. C. , Scott , J. C. , Kramer , J. H. and Holdnack , J. A. 2006b . The California Verbal Learning Test–second edition: Test-retest reliability, practice effects, and reliable change indices for the standard and alternate forms . Archives of Clinical Neuropsychology , 21 : 413 – 420 . doi: 10.1016/j.acn.2006.06.002
  • Youngjohn , J. R. , Larrabee , G. J. and Crook , T. H. 1992 . Test-retest reliability of computerized, everyday memory measures and traditional memory tests . The Clinical Neuropsychologist , 6 : 276 – 286 . doi: 10.1080/13854049208404129
  • Zec , R. F. , Markwell , S. J. , Burkett , N. R. and Larsen , D. L. 2005 . A longitudinal study of confrontation naming in the normal elderly . Journal of the International Neuropsychological Society , 11 : 716 – 726 . doi: 10.1017/S1355617705050897

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Order Reprints Request Corporate Permissions

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

Request Academic Permissions

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.