Construct validity, ecological validity and acceptance of self-administered online neuropsychological assessment in adults

Abstract

Objective: The goal of this project was to explore the initial psychometric properties (construct and ecological validity) of self-administered online (SAO) neuropsychological assessment (using the www.testmybrain.org platform), compared to traditional testing, in a clinical sample, as well as to evaluate participant acceptance. SAO assessment has the potential to expand the reach of in-person neuropsychological assessment approaches.

Method: Counterbalanced, within-subjects design comparing SAO performance to in-person performance in adults with diabetes with and without Chronic Kidney Disease (CKD). Forty-nine participants completed both assessment modalities (type 1 diabetes N = 14, type 2 diabetes N = 35; CKD N = 18).

Results: Associations between SAO and analogous in-person tests were adequate to good (r = 0.49-0.66). Association strength between divergent cognitive tests did not differ between SAO versus in-person tests. SAO testing was more strongly associated with age than in-person testing (age R²=0.54 versus 0.23), while prediction of education, HbA1c, and estimated glomerular filtration rate (eGFR) did not differ significantly between test modalities (education R²=0.37 versus 0.30; HbA1c R²=0.20 versus 0.12; eGFR R² = 0.41 versus 0.33). Associations with measures of everyday functioning were also similar (Functional Activities Questionnaire R²=0.08 versus 0.07; Neuro-QoL R²=0.14 versus 0.16; Diabetes Self-Management Questionnaire R²=0.19 versus 0.19).

Conclusions: The selected SAO neuropsychological tests had acceptable construct validity (including divergent, convergent, and criterion-related validity), and similar ecological validity to that of traditional testing. These SAO assessments were acceptable to participants and appear appropriate for use in research applications, although further research is needed to better understand the strengths and weaknesses in other clinical populations.

Keywords:

• Online
• neuropsychological
• validation
• diabetes

Acknowledgements

This study was supported by an Institute of Translational Health Sciences pilot award via the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1 TR002319. This work was also partially supported by NIH grant R01 DK 121240-01. The content is solely the responsibility of the authors.

Prior presentation

This work was presented at the 79^th Scientific Sessions of the American Diabetes Association, San Francisco, CA, 7-11 June 2019.

Author contributions

NSC researched data, contributed to the study design, participated in discussion and wrote the manuscript. CBL contributed to the study design, participated in discussion, researched data and reviewed/edited the manuscript. KRT, LTG and SMM contributed to the study design, participated in discussion and reviewed/edited the manuscript. LMF participated in discussion and reviewed/edited the manuscript. All authors reviewed and approved the final manuscript for submission. NSC is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and accuracy of the data analysis.

Duality of interest

NSC, LTG, LMF, CBL, SMM and KRT have no conflicts of interest related to the present work. NSC has received personal fees from Eli Lilly outside the submitted work. CBL has consulted for Providence Saint John's Health Center and has received research funding from the Bristol-Myers Squibb Foundation outside the submitted work. SMM has received research funding from the Bristol-Myers Squibb Foundation, Ringful Health, LLC, Managed Health Connections, LLC, the Orthopedic Specialty Institute, and consulted for Consistent Care, LLC and the Department of Justice outside the submitted work.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This project was supported by an Institute of Translational Health Sciences pilot award via the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1 TR002319.