Assessing cognitive changes in multiple sclerosis: criteria for a reliable decision

References

• Amato, M. P., Ponziani, G., Pracucci, G., Bracco, L., Siracusa, G., & Amaducci, L. (1995). Cognitive impairment in early-onset multiple sclerosis. Pattern, predictors, and impact on everyday life in a 4-year follow-up. Archives of Neurology, 52(2), 168–172. https://doi.org/10.1001/archneur.1995.00540260072019
   PubMedGoogle Scholar
• Amato, M. P., Portaccio, E., Goretti, B., Zipoli, V., Iudice, A., Pina, D. D., Malentacchi, G., Sabatini, S., Annunziata, P., Falcini, M., Mazzoni, M., Mortilla, M., Fonda, C., & De Stefano, N. (2010). Relevance of cognitive deterioration in early relapsing-remitting MS: A 3-year follow-up study. Multiple Sclerosis Journal, 16(12), 1474–1482.https://doi.org/10.1177/1352458510380089
   PubMed Web of Science ®Google Scholar
• Aung, H. L., Siefried, K. J., Gates, T. M., Brew, B. J., Mao, L., Carr, A., & Cysique, L. A. (2023). Meaningful cognitive decline is uncommon in virally suppressed HIV, but sustained impairment, subtle decline and abnormal cognitive aging are not. eClinicalMedicine, 56. https://doi.org/10.1016/j.eclinm.2022.101792
   Google Scholar
• Benedict, R. H., Cookfair, D., Gavett, R., Gunther, M., Munschauer, F., Garg, N., & Weinstock-Guttman, B. (2006). Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). Journal of the International Neuropsychological Society: JINS, 12(4), 549–558. https://doi.org/10.1017/s1355617706060723
   PubMed Web of Science ®Google Scholar
• Benedict, R. H. B., DeLuca, J., Enzinger, C., Geurts, J. J. G., Krupp, L. B., & Rao, S. M. (2017). Neuropsychology of multiple sclerosis: Looking back and moving forward. Journal of the International Neuropsychological Society, 23(9–10), 832–842.
   PubMed Web of Science ®Google Scholar
• Borghi, M., Cavallo, M., Carletto, S., Ostacoli, L., Zuffranieri, M., Picci, R. L., Scavelli, F., Johnston, H., Furlan, P. M., Bertolotto, A., Malucchi, S., & Linker, R. A. (2013). Presence and significant determinants of cognitive impairment in a large sample of patients with multiple sclerosis. PLOS ONE, 8(7), e69820. https://doi.org/10.1017/S1355617717001060
   Web of Science ®Google Scholar
• Bruce, J. M., Hancock, L. M., Arnett, P., & Lynch, S. (2010). Treatment adherence in multiple sclerosis: Association with emotional status, personality, and cognition. Journal of Behavioral Medicine, 33(3), 219–227. https://doi.org/10.1007/s10865-010-9247-y
   PubMed Web of Science ®Google Scholar
• Bruggemans, E. F., Van de Vijver, F. J., & Huysmans, H. A. (1997). Assessment of cognitive deterioration in individual patients following cardiac surgery: Correcting for measurement error and practice effects. Journal of Clinical and Experimental Neuropsychology, 19(4), 543–559. https://doi.org/10.1080/01688639708403743
   PubMed Web of Science ®Google Scholar
• Buschke, H., & Fuld, P. A. (1974). Evaluating storage, retention, and retrieval in disordered memory and learning. Neurology, 24(11), 1019–1025. https://doi.org/10.1212/wnl.24.11.1019
   PubMed Web of Science ®Google Scholar
• Busch, R. M., Lineweaver, T. T., Ferguson, L., & Haut, J. S. (2015). Reliable change indices and standardized regression-based change score norms for evaluating neuropsychological change in children with epilepsy. Epilepsy and Behavior, 47, 45–54. https://doi.org/10.1016/j.yebeh.2015.04.052
   PubMed Web of Science ®Google Scholar
• Casaletto, K. B., & Heaton, R. K. (2017). Neuropsychological assessment: Past and future. Journal of the International Neuropsychological Society, 23(9–10), 778–790. https://doi.org/10.1017/S1355617717001060
   PubMed Web of Science ®Google Scholar
• Chelune, G. J. (2003). Assessing reliable neuropsychological change. In R. D. Franklin (Ed.), Prediction in forensic and neuropsychology: Sound statistical practices (pp. 123–147). Erlbaum.
   Google Scholar
• Chelune, G. J., Naugle, R. I., Lüders, H., Sedlak, J., & Awad, I. A. (1993). Individual change after epilepsy surgery: Practice effects and base-rate information. Neuropsychology, 7(1), 41–52. https://doi.org/10.1037/0894-4105.7.1.41
   Google Scholar
• Chiaravalloti, N. D., & DeLuca, J. (2008). Cognitive impairment in multiple sclerosis. The Lancet Neurology, 7(12), 1139–1151. https://doi.org/10.1016/S1474-4422(08)70259-X
   PubMed Web of Science ®Google Scholar
• Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104
   Web of Science ®Google Scholar
• Collie, A., Maruff, P., Makdissi, M., McStephen, M., Darby, D. G., & McCrory, P. (2004). Statistical procedures for determining the extent of cognitive change following concussion. British Journal of Sports Medicine, 38(3), 273–278. https://doi.org/10.1136/bjsm.2003.000293
   PubMed Web of Science ®Google Scholar
• Conradi, N., Behrens, M., Hermsen, A. M., Kannemann, T., Merkel, N., Schuster, A., Freiman, T. M., Strzelczyk, A., & Rosenow, F. (2020). Assessing cognitive change and quality of life 12 months after epilepsy surgery-development and application of reliable change indices and standardized regression-based change norms for a neuropsychological test battery in the German Language. Frontiers in Psychology, 11, 582836. https://doi.org/10.3389/fpsyg.2020.582836
   PubMed Web of Science ®Google Scholar
• Crawford, J. R., & Garthwaite, P. H. (2006). Methods of testing for a deficit in single-case studies: Evaluation of statistical power by Monte Carlo simulation. Cognitive Neuropsychology, 23(6), 877‑904. https://doi.org/10.1080/02643290500538372
   PubMed Web of Science ®Google Scholar
• Crawford, J. R., & Garthwaite, P. H. (2007). Using regression equations built from summary data in the neuropsychological assessment of the individual case. Neuropsychology, 21(5), 611–620. https://doi.org/10.1037/0894-4105.21.5.611
   PubMed Web of Science ®Google Scholar
• Crawford, J. R., Garthwaite, P. H., Denham, A. K., & Chelune, G. J. (2012). Using regression equations built from summary data in the psychological assessment of the individual case: Extension to multiple regression. Psychological Assessment, 24(4), 801‑814. https://doi.org/10.1037/a0027699
   Web of Science ®Google Scholar
• Crawford, J. R., & Howell, D. C. (1998). Regression equations in clinical neuropsychology: An evaluation of statistical methods for comparing predicted and obtained scores. Journal of Clinical and Experimental Neuropsychology, 20(5), 755–762. https://doi.org/10.1076/jcen.20.5.755.1132
   PubMed Web of Science ®Google Scholar
• DeLuca, G. C., Yates, R. L., Beale, H., & Morrow, S. A. (2015). Cognitive impairment in multiple sclerosis: Clinical, radiologic and pathologic insights. Brain Pathology (Zurich, Switzerland), 25(1), 79–98. https://doi.org/10.1111/bpa.12220
   PubMed Web of Science ®Google Scholar
• De Souza Costa, D., & de Paula, J. J. (2015). Usefulness of the reliable change index for psychology and psychiatry in clinical practice: A case report of cognitive-behavioral therapy. Clinical Neuropsychiatry, 12(5), 135–138.
   Web of Science ®Google Scholar
• Duff, K. (2012). Evidence-based indicators of neuropsychological change in the individual patient: Relevant concepts and methods. Archives of Clinical Neuropsychology, 27(3), 248–261. https://doi.org/10.1093/arclin/acr120
   PubMed Web of Science ®Google Scholar
• Duff, K., Atkinson, T. J., Suhrie, K. R., Dalley, B. C., Schaefer, S. Y., & Hammers, D. B. (2017). Short-term practice effects in mild cognitive impairment: Evaluating different methods of change. Journal of Clinical and Experimental Neuropsychology, 39(4), 396–407. https://doi.org/10.1080/13803395.2016.1230596
   PubMed Web of Science ®Google Scholar
• Eijlers, A. J. C., van Geest, Q., Dekker, I., Steenwijk, M. D., Meijer, K. A., Hulst, H. E., Barkhof, F., Uitdehaag, B. M. J., Schoonheim, M. M., & Geurts, J. J. G. (2018). Predicting cognitive decline in multiple sclerosis: A 5-year follow-up study. Brain, 141(9), 2605–2618. https://doi.org/10.1093/brain/awy202
   PubMed Web of Science ®Google Scholar
• Evans, M. (1992). Robustness of size of tests of autocorrelation and heteroscedasticity to nonnormality. Journal of Econometrics, 51(1), 7–24. https://doi.org/10.1016/0304-4076(92)90027-O
   Web of Science ®Google Scholar
• Feinstein, A., Freeman, J., & Lo, A. C. (2015). Treatment of progressive multiple sclerosis: What works, what does not, and what is needed. The Lancet Neurology, 14(2), 194–207. https://doi.org/10.1016/S1474-4422(14)70231-5
   PubMed Web of Science ®Google Scholar
• Format Dujardin, K., Sockeel, P., Cabaret, M., De Sèze, J., & Vermersch, P. (2004). BCcogSEP: A French test battery evaluating cognitive functions in multiple sclerosis. Revue Neurologique, 160(1), 51–62. https://doi.org/10.1016/s0035-3787(04)70847-4
   PubMed Web of Science ®Google Scholar
• Frerichs, R. J., & Tuokko, H. A. (2005). A comparison of methods for measuring cognitive change in older adults. Archives of Clinical Neuropsychology, 20(3), 321–333. https://doi.org/10.1016/j.acn.2004.08.002
   PubMed Web of Science ®Google Scholar
• Frerichs, R. J., & Tuokko, H. A. (2006). Reliable change scores and their relation to perceived change in memory: Implications for the diagnosis of mild cognitive impairment. Archives of Clinical Neuropsychology, 21(1), 109–115. https://doi.org/10.1016/j.acn.2005.08.003
   PubMed Web of Science ®Google Scholar
• Goverover, Y., Genova, H. M., Hillary, F. G., & DeLuca, J. (2007). The relationship between neuropsychological measures and the timed instrumental activities of daily living task in multiple sclerosis. Multiple Sclerosis (Houndmills, Basingstoke, England), 13(5), 636–644. https://doi.org/10.1177/1352458506072984
   PubMed Web of Science ®Google Scholar
• Hakim, E. A., Bakheit, A. M., Bryant, T. N., Roberts, M. W., McIntosh-Michaelis, S. A., Spackman, A. J., Martin, J. P., & McLellan, D. L. (2000). The social impact of multiple sclerosis—A study of 305 patients and their relatives. Disability and Rehabilitation, 22(6), 288–293. https://doi.org/10.1080/096382800296755
   PubMed Web of Science ®Google Scholar
• Haman, J., & Avery, M. (2020). ciTools: Confidence or prediction intervals, quantiles, and probabilities for statistical models. R package version 0.6.1. https://CRAN.Rproject.org/package=ciTools
   Google Scholar
• Hankomäki, E., Multanen, J., Kinnunen, E., & Hämäläinen, P. (2014). The progress of cognitive decline in newly diagnosed MS patients. Acta Neurologica Scandinavica, 129(3), 184–191. https://doi.org/10.1111/ane.12161
   PubMed Web of Science ®Google Scholar
• Hsu, L. M. (1989). Reliable changes in psychotherapy: Taking into account regression toward the mean. Behavioral Assessment, 11(4), 459–467.
   Google Scholar
• Huijbregts, S. C. J., Kalkers, N. F., de Sonneville, L. M. J., de Groot, V., & Polman, C. H. (2006). Cognitive impairment and decline in different MS subtypes. Journal of the Neurological Sciences, 245(1–2), 187–194. https://doi.org/10.1016/j.jns.2005.07.018
   PubMed Web of Science ®Google Scholar
• Huizenga, H. M., Agelink van Rentergem, J. A., Grasman, R. P., Muslimovic, D., & Schmand, B. (2016). Normative comparisons for large neuropsychological test batteries: User-friendly and sensitive solutions to minimize familywise false positives. Journal of Clinical and Experimental Neuropsychology, 38(6), 611–629. https://doi.org/10.1080/13803395.2015.1132299
   PubMed Web of Science ®Google Scholar
• Iverson, G. L. (2001). Interpreting change on the WAIS-III/WMS-III in clinical samples. Archives of Clinical Neuropsychology, 16(2), 183–191.
   PubMed Web of Science ®Google Scholar
• Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. https://doi.org/10.1037//0022-006x.59.1.12
   PubMed Web of Science ®Google Scholar
• Jennekens-Schinkel, A., van der Velde, E. A., Sanders, E. A., & Lanser, J. B. (1990). Memory and learning in outpatients with quiescent multiple sclerosis. Journal of the Neurological Sciences, 95(3), 311–325. https://doi.org/10.1016/0022-510x(90)90077-z
   PubMed Web of Science ®Google Scholar
• Johnen, A., Bürkner, P.-C., Landmeyer, N. C., Ambrosius, B., Calabrese, P., Motte, J., Hessler, N., Antony, G., König, I. R., Klotz, L., Hoshi, -M.-M., Aly, L., Groppa, S., Luessi, F., Paul, F., Tackenberg, B., Bergh, F. T., Kümpfel, T., Tumani, H., … Salmen, A. (2019). Can we predict cognitive decline after initial diagnosis of multiple sclerosis? Results from the German national early MS cohort (KKNMS). Journal of Neurology, 266(2), 386–397. https://doi.org/10.1007/s00415-018-9142-y
   PubMed Web of Science ®Google Scholar
• Johnen, A., Landmeyer, N. C., Bürkner, P.-C., Wiendl, H., Meuth, S. G., & Holling, H. (2017). Distinct cognitive impairments in different disease courses of multiple sclerosis – A systematic review and meta-analysis. Neuroscience and Biobehavioral Reviews, 83, 568–578. https://doi.org/10.1016/j.neubiorev.2017.09.005
   PubMed Web of Science ®Google Scholar
• Kalb, R., Beier, M., Benedict, R. H., Charvet, L., Costello, K., Feinstein, A., Gingold, J., Goverover, Y., Halper, J., Harris, C., Kostich, L., Krupp, L., Lathi, E., LaRocca, N., Thrower, B., & DeLuca, J. (2018). Recommendations for cognitive screening and management in multiple sclerosis care. Multiple Sclerosis, 24(13), 1665–1680. https://doi.org/10.1177/1352458518803785
   PubMed Web of Science ®Google Scholar
• Kneebone, A. C., Andrew, M. J., Baker, R. A., & Knight, J. L. (1998). Neuropsychologic changes after coronary artery bypass grafting: Use of reliable change indices. The Annals of Thoracic Surgery, 65(5), 1320–1325. https://doi.org/10.1016/s0003-4975(98)00158-1
   PubMed Web of Science ®Google Scholar
• Knief, U., & Forstmeier, W. (2021). Violating the normality assumption may be the lesser of two evils. Behavior Research Methods, 53(6), 2576–2590. https://doi.org/10.3758/s13428-021-01587-5
   PubMed Web of Science ®Google Scholar
• Langdon, D. W. (2011). Cognition in multiple sclerosis. Current Opinion in Neurology, 24(3), 244–249. https://doi.org/10.1097/WCO.0b013e328346a43b
   PubMed Web of Science ®Google Scholar
• Light, R. J. (1971). Measures of response agreement for qualitative data: Some generalizations and alternatives. Psychological Bulletin, 76(5), 365–377. https://doi.org/10.1037/h0031643
   Google Scholar
• McDonald, W. I., Compston, A., Edan, G., Goodkin, D., Hartung, H.-P., Lublin, F. D., McFarland, H. F., Paty, D. W., Polman, C. H., Reingold, S. C., Sandberg-Wollheim, M., Sibley, W., Thompson, A., Van Den Noort, S., Weinshenker, B. Y., & Wolinsky, J. S. (2001). Recommended diagnostic criteria for multiple sclerosis: Guidelines from the international panel on the diagnosis of multiple sclerosis. Annals of Neurology, 50(1), 121–127. https://doi.org/10.1002/ana.1032
   PubMed Web of Science ®Google Scholar
• McSweeny, A. J., Naugle, R. I., Chelune, G. J., & Lüders, H. (1993). “TScores for change”: An illustration of a regression approach to depicting change in clinical neuropsychology. Clinical Neuropsychologist, 7(3), 300–312. https://doi.org/10.1080/13854049308401901
   Google Scholar
• Morey, R. D., & Rouder, J. N. (2018). BayesFactor: Computation of Bayes factors for common designs. R Package Version, 0.9.12–4.2. https://CRAN.R-project.org/package=BayesFactor
   Google Scholar
• Nessler, S., & Brück, W. (2010). Advances in multiple sclerosis research in 2009. Journal of Neurology, 257(9), 1590–1593. https://doi.org/10.1007/s00415-010-5689-y
   PubMed Web of Science ®Google Scholar
• Popper, K. (1973). La logique de la découverte scientifique. (ed.) N. Thyssen-Rutten & P. Devaux Original work published 1959. Éditions Payot.
   Google Scholar
• Rao, S. M., Leo, G. J., Bernardin, L., & Unverzagt, F. (1991). Cognitive dysfunction in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology, 41(5), 685–691. https://doi.org/10.1212/wnl.41.5.685
   PubMed Web of Science ®Google Scholar
• R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. URL. https://www.R-project.org/
   Google Scholar
• Reuter, F., Zaaraoui, W., Crespy, L., Faivre, A., Rico, A., Malikova, I., Soulier, E., Viout, P., Ranjeva, J.-P., Pelletier, J., & Audoin, B. (2011). Frequency of cognitive impairment dramatically increases during the first 5 years of multiple sclerosis. Journal of Neurology, Neurosurgery, and Psychiatry, 82(10), 1157–1159. https://doi.org/10.1136/jnnp.2010.213744
   PubMed Web of Science ®Google Scholar
• Revelle, W. R. (2019). psych: Procedures for personality and psychological research. Northwestern University, Evanston, Illinois. http://personality-project.org/https://personality-project.org/r/psych-manual.pdf
   Google Scholar
• Rosti-Otajärvi, E., Ruutiainen, J., Huhtala, H., & Hämäläinen, P. (2014). Relationship between subjective and objective cognitive performance in multiple sclerosis. Acta neurologica Scandinavica, 130(5), 319–327. https://doi.org/10.1111/ane.12238
   PubMed Web of Science ®Google Scholar
• Roy, S., Benedict, R. H., Drake, A. S., & Weinstock-Guttman, B. (2016). Impact of pharmacotherapy on cognitive dysfunction in patients with multiple sclerosis. CNS Drugs, 30(3), 209–225. https://doi.org/10.1007/s40263-016-0319-6
   PubMed Web of Science ®Google Scholar
• Ruet, A., Deloire, M., Charré-Morin, J., Hamel, D., & Brochet, B. (2013). Cognitive impairment differs between primary progressive and relapsing-remitting MS. Neurology, 80(16), 1501–1508. https://doi.org/10.1212/WNL.0b013e31828cf82f
   PubMed Web of Science ®Google Scholar
• Sherman, E. M. S., Wiebe, S., Fay-McClymont, T. B., Tellez-Zenteno, J., Metcalfe, A., Hernandez-Ronquillo, L., Hader, W. J., & Jetté, N. (2011). Neuropsychological outcomes after epilepsy surgery: Systematic review and pooled estimates. Epilepsia, 52(5), 857‑869. https://doi.org/10.1111/j.1528-1167.2011.03022.x
   Web of Science ®Google Scholar
• Strober, L. B., Rao, S. M., Lee, J.-C., Fischer, E., & Rudick, R. (2014). Cognitive impairment in multiple sclerosis: An 18 year follow-up study. Multiple Sclerosis and Related Disorders, 3(4), 473–481. https://doi.org/10.1016/j.msard.2014.03.004
   PubMed Web of Science ®Google Scholar
• Walker, L. A. S., Mendella, P. D., Stewart, A., Freedman, M. S., & Smith, A. M. (2011). Meaningful change in cognition in multiple sclerosis: Method matters. The Canadian Journal of Neurological Sciences, 38(2), 282–288. https://doi.org/10.1017/s0317167100011471
   PubMed Web of Science ®Google Scholar
• Watson, J., Brown, C., Hirst, R. B., & Thompson, R. C. (2020). Commonality in abnormality: “Abnormal” neuropsychological scores are common in baseline testing of youth athletes. Archives of Clinical Neuropsychology, 35(5), 469–481. https://doi.org/10.1093/arclin/acaa015
   PubMed Web of Science ®Google Scholar
• Wechsler, D. (1981). WAIS-R: Manual: Wechsler adult intelligence scale–revised. Psychological Corporation.
   Google Scholar
• Weinstock, Z., Morrow, S., Conway, D., Fuchs, T., Wojcik, C., Unverdi, M., Zivadinov, R., Weinstock-Guttman, B., Iverson, G. L., Dwyer, M., & Benedict, R. H. (2022). Interpreting change on the symbol digit modalities test in people with relapsing multiple sclerosis using the reliable change methodology. Multiple Sclerosis, 28(7), 1101–1111. https://doi.org/10.1177/13524585211049397
   PubMed Web of Science ®Google Scholar
• Wetzels, R., Matzke, D., Lee, M. D., Rouder, J. N., Iverson, G. J., & Wagenmakers, E.-J. (2011). Statistical evidence in experimental psychology: An empirical comparison using 855 t-tests. Perspectives on Psychological Science, 6(3), 291–298. https://doi.org/10.1177/1745691611406923
   Web of Science ®Google Scholar
• Wickham, H. (2016). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag.
   Google Scholar
• Williams, M. N., Bååth, R., & Philipp, M. C. (2017). Using Bayes factors to test hypotheses in developmental research. Research in Human Development, 14(4), 321–337. https://doi.org/10.1080/15427609.2017.1370964
   Web of Science ®Google Scholar
• Wojcik, C., Jaworski, M., 3rd, Dwyer, M. G., Youngs, M., Unverdi, M., Weinstock-Guttman, B., & Benedict, R. H. (2022). Benchmarks of meaningful improvement on neurocognitive tests in multiple sclerosis. Multiple Sclerosis, 28(3), 487–491. https://doi.org/10.1177/13524585211044672
   PubMed Web of Science ®Google Scholar
• Yoshii, F., Takagi, S., Nagata, E., Hasegawa, Y., Kunika, N., Maki, F., Mochizuki, H., Ogino, M., & Kuroiwa, Y. (2015). Cognitive impairment of Japanese multiple sclerosis patients: Follow-up study using BRB-N assessment tool. Journal of the Neurological Sciences, 359(1–2), 323–327. https://doi.org/10.1016/j.jns.2015.11.002
   PubMed Web of Science ®Google Scholar
• Zheng, B., & Agresti, A. (2000). Summarizing the predictive power of a generalized linear model. Statistics in Medicine, 19(13), 1771–1781. https://doi.org/10.1002/1097-0258(20000715)19:13<1771::aid-sim485>3.0.co;2-p
   PubMed Web of Science ®Google Scholar
• Zipoli, V., Goretti, B., Hakiki, B., Siracusa, G., Sorbi, S., Portaccio, E., & Amato, M. P. (2010). Cognitive impairment predicts conversion to multiple sclerosis in clinically isolated syndromes. Multiple Sclerosis (Houndmills, Basingstoke, England), 16(1), 62–67. https://doi.org/10.1177/1352458509350311
   PubMed Web of Science ®Google Scholar