Relevance of the assessment mode in the digital assessment of processing speed

ABSTRACT

Introduction: Many medical facilities and research institutes start using digital assessment methods to assess cognitive abilities, such as processing speed, instead of the traditional pen and paper versions. Even though many of the new digital assessment methods have shown to have a stable internal validity, the comparability of results across assessment modes is unclear. The study investigated whether results for assessing processing speed via (i) the traditional pen and paper version, (ii) a tablet and pen version, and (iii) a tablet and finger version are comparable.

Methods: In a within-subject design, each participant (N= 30) completed the 90-number version by Oswald and Roth (1987) of the trail-making test (TMT) in three different assessment modes in randomized order. Each participant completed four TMT versions in each assessment mode (3 × 4 within-subject design).

Results: Repeated measures ANOVA and mixed-effects analyses adjusted for age, gender, mode order, and trial number reveal significantly faster test TMT completion times (about 5 s) for the tablet and pen version compared to the pen and paper and the tablet and finger version.

Conclusions: Our findings indicate that assessing processing speed can lead to different results with different digital versions depending on their setup, especially different input devices. Medical professionals and researchers who use digital assessment methods to assess cognitive abilities need to be aware of mode effects, even within the digital assessment domain, because the results may not be comparable and the available norms may not be applicable.

KEYWORDS:

• Assessment mode
• processing speed
• trail-making test
• digitization

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The study was part of a project funded by the Federal Ministry of Education and Research (BMBF): grant# 01PZ16010B. F.S.R. reports a grant from the German Research Foundation (DFG): grant # TH2137/3-1.