The Oxford digital multiple errands test (OxMET): Validation of a simplified computer tablet based multiple errands test

References

• Alderman, N., Burgess, P. W., Knight, C., & Henman, C. (2003). Ecological validity of a simplified version of the multiple errands shopping test. Journal of the International Neuropsychological Society, 9(1), 31–44. https://doi.org/10.1017/S1355617703910046
   PubMed Web of Science ®Google Scholar
• Anderson, M., & Perrin, A. (2017). Tech adoption climbs among older adults. Pew Research Center, 1–22.
   Google Scholar
• Antoniak, K., Clores, J., Jensen, D., Nalder, E., Rotenberg, S., & Dawson, D. R. (2019). Developing and validating a big-store multiple errands test. Frontiers in Psychology, 10, 2575. https://doi.org/10.3389/fpsyg.2019.02575
   PubMedGoogle Scholar
• Bauer, R. M., Iverson, G. L., Cernich, A. N., Binder, L. M., Ruff, R. M., & Naugle, R. I. (2012). Computerized neuropsychological assessment devices: Joint position paper of the American academy of clinical neuropsychology and the national academy of neuropsychology. The Clinical Neuropsychologist, 26(2), 177–196. https://doi.org/10.1080/13854046.2012.663001
   PubMed Web of Science ®Google Scholar
• Bettcher, B. M., Mungas, D., Patel, N., Elofson, J., Dutt, S., Wynn, M., Watson, C. L., Stephens, M., Walsh, C. M., & Kramer, J. H. (2016). Neuroanatomical substrates of executive functions: Beyond prefrontal structures. Neuropsychologia, 85, 100–109. https://doi.org/10.1016/j.neuropsychologia.2016.03.001
   PubMed Web of Science ®Google Scholar
• Bielak, A. A. M., Mansueti, L., Strauss, E., & Dixon, R. A. (2006). Performance on the hayling and brixton tests in older adults: Norms and correlates. Archives of Clinical Neuropsychology, 21(2), 141–149. https://doi.org/10.1016/j.acn.2005.08.006
   PubMed Web of Science ®Google Scholar
• Burgess, P. W., Alderman, N., Evans, J., Emslie, H., & Wilson, B. A. (1998). The ecological validity of tests of executive function. Journal of the International Neuropsychological Society, 4(6), 547–558. https://doi.org/10.1017/S1355617798466037
   PubMed Web of Science ®Google Scholar
• Burgess, P. W., & Shallice, T. (1997). The Hayling and Brixton tests. Thames Valley Test Company.
   Google Scholar
• Burns, S. P., Pickens, N. D., Dawson, D. R., Perea, J. D., Vas, A. K., Marquez de la Plata, C., & Neville, M. (2018). In-home contextual reality: A qualitative analysis using the multiple errands test home version (MET-home). Neuropsychological Rehabilitation, 1–15. https://doi.org/10.1080/09602011.2018.1431134
   Google Scholar
• Cieslik, E. C., Mueller, V. I., Eickhoff, C. R., Langner, R., & Eickhoff, S. B. (2015). Three key regions for supervisory attentional control: Evidence from neuroimaging meta-analyses. Neuroscience & Biobehavioral Reviews, 48, 22–34. https://doi.org/10.1016/j.neubiorev.2014.11.003
   PubMed Web of Science ®Google Scholar
• Cipresso, P., Albani, G., Serino, S., Pedroli, E., Pallavicini, F., Mauro, A., & Riva, G. (2014). Virtual multiple errands test (VMET): A virtual reality-based tool to detect early executive functions deficit in Parkinson’s disease. Frontiers in Behavioral Neuroscience, 8, 405. https://doi.org/10.3389/fnbeh.2014.00405
   PubMed Web of Science ®Google Scholar
• Connor, L. T., & Maeir, A. (2011). Putting executive performance in a theoretical context. OTJR: Occupation, Participation and Health, 31(1_suppl), S3–S7. https://doi.org/10.3928/15394492-20101108-02
   PubMed Web of Science ®Google Scholar
• Dawson, D. R., Anderson, N. D., Burgess, P., Cooper, E., Krpan, K. M., & Stuss, D. T. (2009). Further development of the multiple errands test: Standardized scoring, reliability, and ecological validity for the baycrest version. Archives of Physical Medicine and Rehabilitation, 90(11), S41–S51. https://doi.org/10.1016/j.apmr.2009.07.012
   PubMed Web of Science ®Google Scholar
• Demeyere, N., Haupt, M., Webb, S. S., Strobel, L., Milosevich, E., Moore, M., Wright, H., Finke, K., & Duta, M. (2020, February 25). The Oxford Cognitive Screen – Plus (OCS-Plus): A tablet based short cognitive screening tool for milder cognitive impairment. https://doi.org/10.31234/osf.io/b2vgc
   Google Scholar
• Demeyere, N., Riddoch, M. J., Slavkova, E. D., Bickerton, W. L., & Humphreys, G. W. (2015). The Oxford cognitive screen (OCS): validation of a stroke-specific short cognitive screening tool. Psychological Assessment, 27(3), 883–894. https://doi.org/10.1037/pas0000082
   PubMed Web of Science ®Google Scholar
• Derringer, J. (2018, April 16). A simple correction for non-independent tests. https://doi.org/10.31234/osf.io/f2tyw
   Google Scholar
• Erez, N., Weiss, P. L., Kizony, R., & Rand, D. (2013). Comparing performance within a virtual supermarket of children with traumatic brain injury to typically developing children: A pilot study. OTJR: Occupation, Participation and Health, 33(4), 218–227. https://doi.org/10.3928/15394492-20130912-04
   PubMed Web of Science ®Google Scholar
• Gamer, M., Lemon, J., & Singh, P. (2019). irr: Various coefficients of interrater reliability and agreement. R package version 0.84.1. https://cran.r-project.org/web/packages/irr/index.html
   Google Scholar
• George, M., & Gilbert, S. (2018). Mental capacity act (2005) assessments: Why everyone needs to know about the frontal lobe paradox. In J. Fish (Ed.), The neuropsychologist (Vol. 5, pp. 59–66). British Psychological Society.
   Google Scholar
• Gilbert, S. J., & Burgess, P. W. (2008). Executive function. Current Biology, 18(3), R110–R114. https://doi.org/10.1016/j.cub.2007.12.014
   PubMed Web of Science ®Google Scholar
• Goldstein, S., Naglieri, J. A., Princiotta, D., & Otero, T. M. (2014). Introduction: A history of executive functioning as a theoretical and clinical construct. Handbook of executive functioning (pp. 3-12). Springer.
   Google Scholar
• Goverover, Y., & Josman, N. (2004). Everyday problem solving among four groups of individuals with cognitive impairments: Examination of the discriminant validity of the observed tasks of daily living—revised. OTJR: Occupation, Participation and Health, 24(3), 103–112. https://doi.org/10.1177/153944920402400304
   Web of Science ®Google Scholar
• Hanberg, V. L., MacKenzie, D. E., & Merritt, B. K. (2018). Scoping review of the Multiple Errands test: Is it relevant to youths with acquired brain injury? British Journal of Occupational Therapy, 81(12), 673–686. https://doi.org/10.1177/0308022618791714
   Web of Science ®Google Scholar
• Harrell Jr, F. E. (2019). Hmisc: Harrell Miscellaneous. R package version 4.3-0. https://CRAN.R-project.org/package=Hmisc
   Google Scholar
• Jokinen, H., Melkas, S., Ylikoski, R., Pohjasvaara, T., Kaste, M., Erkinjuntti, T., & Hietanen, M. (2015). Post-stroke cognitive impairment is common even after successful clinical recovery. European Journal of Neurology, 22(9), 1288–1294. https://doi.org/10.1111/ene.12743
   PubMed Web of Science ®Google Scholar
• Josman, N., Kizony, R., Hof, E., Goldenberg, K., Weiss, P. L., & Klinger, E. (2014). Using the virtual action planning-supermarket for evaluating executive functions in people with stroke. Journal of Stroke and Cerebrovascular Diseases, 23(5), 879–887. https://doi.org/10.1016/j.jstrokecerebrovasdis.2013.07.013
   PubMed Web of Science ®Google Scholar
• Jovanovski, D., Zakzanis, K., Campbell, Z., Erb, S., & Nussbaum, D. (2012). Development of a novel, ecologically oriented virtual reality measure of executive function: The multitasking in the city test. Applied Neuropsychology: Adult, 19(3), 171–182. https://doi.org/10.1080/09084282.2011.643955
   PubMed Web of Science ®Google Scholar
• Kassambara, A. (2020). Rstatix: Pipe-Friendly Framework for basic Statistical tests. R package version 0.4.0. https://CRAN.R-project.org/package=rstatix
   Google Scholar
• Knight, C., Alderman, N., & Burgess, P. W. (2002). Development of a simplified version of the multiple errands test for use in hospital settings. Neuropsychological Rehabilitation, 12(3), 231–255. https://doi.org/10.1080/09602010244000039
   Web of Science ®Google Scholar
• Koski, L., Brouillette, M., Lalonde, R., Hello, B., Wong, E., Tsuchida, A., & Fellows, L. K. (2011). Computerized testing augments pencil-and-paper tasks in measuring HIV-associated mild cognitive impairment. HIV Medicine, 12(8), 472–480. https://doi.org/10.1111/j.1468-1293.2010.00910.x
   PubMed Web of Science ®Google Scholar
• Kourtesis, P., Collina, S., Doumas, L. A. A., & MacPherson, S. E. (2020a). Validation of the virtual reality everyday assessment Lab (VR-EAL): An Immersive virtual reality neuropsychological battery with Enhanced ecological validity. Journal of the International Neuropsychological Society, 1–16. https://doi.org/10.1017/S1355617720000764
   Google Scholar
• Kourtesis, P., Korre, D., Collina, S., Doumas, L. A. A., & MacPherson, S. E. (2020b). Guidelines for the Development of Immersive virtual reality Software for cognitive Neuroscience and Neuropsychology: The Development of virtual reality everyday assessment Lab (VR-EAL), a neuropsychological test battery in Immersive virtual reality. Frontiers in Computer Science, 1. https://doi.org/10.3389/fcomp.2019.00012
   Web of Science ®Google Scholar
• La Paglia, F., La Cascia, C., Rizzo, R., Cangialosi, F., Sanna, M., Riva, G., & La Barbera, D. (2014). Cognitive assessment of OCD patients: NeuroVR vs neuropsychological test. Studies in Health Technology and Informatics, 199, 40–44.
   PubMedGoogle Scholar
• Law, A. S., Logie, R. H., & Pearson, D. G. (2006). The impact of secondary tasks on multitasking in a virtual environment. Acta Psychologica, 122(1), 27–44.
   PubMed Web of Science ®Google Scholar
• Lawton, M. P., & Brody, E. M. (1969). Assessment of older people: Self-maintaining and instrumental activities of daily living. The Gerontologist, 9(3), 179–186.
   PubMed Web of Science ®Google Scholar
• Lüdecke, D. (2018). Sjstats: Statistical functions for regression models. R package version 0.14. 0.
   Google Scholar
• Luria, A. R., Karpov, B. A., & Yarbuss, A. L. (1966). Disturbances of active visual perception with lesions of the frontal lobes. Cortex, 2(2), 202–212. https://doi.org/10.1016/S0010-9452(66)80003-5
   Google Scholar
• Mangiafico, S. (2019). Rcompanion: Functions to support Extension education Program Evaluation. R package version 2.3.7. https://CRAN.R-project.org/package=rcompanion
   Google Scholar
• Manly, T., Hawkins, K., Evans, J., Woldt, K., & Robertson, I. H. (2002). Rehabilitation of executive function: Facilitation of effective goal management on complex tasks using periodic auditory alerts. Neuropsychologia, 40(3), 271–281. https://doi.org/10.1016/S0028-3932(01)00094-X
   PubMed Web of Science ®Google Scholar
• Merriman, N. A., Sexton, E., McCabe, G., Walsh, M. E., Rohde, D., Gorman, A., Jeffares, I., Donnelly, N.-A., Pender, N., Williams, D. J., Horgan, F., Doyle, F., Wren, M.-A., Bennett, K. E., & Hickey, A. (2019). Addressing cognitive impairment following stroke: Systematic review and meta-analysis of non-randomised controlled studies of psychological interventions. BMJ Open, 9(2), e024429. https://doi.org/10.1136/bmjopen-2018-024429
   PubMed Web of Science ®Google Scholar
• Millis, S. R., Rosenthal, M., Novack, T. A., Sherer, M., Nick, T. G., Kreutzer, J. S., High, W. M., & Ricker, J. H. (2001). Long-term neuropsychological outcome after traumatic brain injury. Journal of Head Trauma Rehabilitation, 16(4), 343–355. https://doi.org/10.1097/00001199-200108000-00005
   PubMed Web of Science ®Google Scholar
• Mole, J. A., & Demeyere, N. (2020). The relationship between early post-stroke cognition and longer term activities and participation: A systematic review. Neuropsychological Rehabilitation, 30(2), 346–370. https://doi.org/10.1080/09602011.2018.1464934
   PubMed Web of Science ®Google Scholar
• Mollan, K., Trumble, I., Reifeis, S., Ferrer, O., Bay, C., Baldoni, P., & Hudgens, M. (2020). Precise and accurate power of the rank-sum test for a continuous outcome. Journal of Biopharmaceutical Statistics, 30(4), 639–648. https://doi.org/10.1080/10543406.2020.1730866
   PubMed Web of Science ®Google Scholar
• Norman, D. A., & Shallice, T. (1980). Attention to action: Willed and automatic control of behavior technical report no. 8006.
   Google Scholar
• Okahashi, S., Seki, K., Nagano, A., Luo, Z., Kojima, M., & Futaki, T. (2013). A virtual shopping test for realistic assessment of cognitive function. Journal of Neuroengineering and Rehabilitation, 10(1)), 59. https://doi.org/10.1186/1743-0003-10-59
   PubMedGoogle Scholar
• Pew Research Centre. (2019). Mobile fact sheet. http://www.pewinternet.org/fact-sheet/mobile/
   Google Scholar
• Pohjasvaara, T., Leskela, M., Vataja, R., Kalska, H., Ylikoski, R., Hietanen, M., Leppavuori, A., Kaste, M., & Erkinjuntti, T. (2002). Post-stroke depression, executive dysfunction and functional outcome. European Journal of Neurology, 9(3), 269–275. https://doi.org/10.1046/j.1468-1331.2002.00396.x
   PubMed Web of Science ®Google Scholar
• Quinn, T. (2014). Development of novel computerised tools to assess memory and planning problems in people with brain injury (Doctoral dissertation, University of Glasgow). http://theses.gla.ac.uk/5689/1/2014quinndclinpsy.pdf
   Google Scholar
• Rand, D., Katz, N., Shahar, M., Kizony, R., & Weiss, P. L. (2005). The virtual mall: A functional virtual environment for stroke rehabilitation. Annual Review of Cybertherapy and Telemedicine: A Decade of VR, 3, 193–198. https://doi.org/10.1037/e705572011-108
   Google Scholar
• Rand, D., Rukan, S. B. A., Weiss, P. L., & Katz, N. (2009). Validation of the Virtual MET as an assessment tool for executive functions. Neuropsychological Rehabilitation, 19(4), 583–602.
   PubMed Web of Science ®Google Scholar
• Raspelli, S., Pallavicini, F., Carelli, L., Morganti, F., Pedroli, E., Cipresso, P., Poletti, B., Corra, B., Sangalli, D., Silani, V., & Riva, G. (2012). Validating the neuro VR-based virtual version of the multiple errands test: Preliminary results. Presence: Teleoperators and Virtual Environments, 21(1), 31–42. https://doi.org/10.1162/PRES_a_00077
   Web of Science ®Google Scholar
• R Core Team. (2018). R: A language and environment for statistical computing.
   Google Scholar
• Revelle, W. (2018). Psych: Procedures for Personality and Psychological research. Northwestern University. https://CRAN.R-project.org/package=psych Version = 1.8.12
   Google Scholar
• Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J. C., & Müller, M. (2011). pROC: An open-source package for R and S to analyze and compare ROC curves. BMC Bioinformatics, 12(1), 1–8.
   PubMed Web of Science ®Google Scholar
• Robinson, G. A., Cipolotti, L., Walker, D. G., Biggs, V., Bozzali, M., & Shallice, T. (2015). Verbal suppression and strategy use: A role for the right lateral prefrontal cortex? Brain, 138(4), 1084–1096. https://doi.org/10.1093/brain/awv003
   PubMed Web of Science ®Google Scholar
• Robotham, R. J., Riis, J. O., & Demeyere, N. (2020). A Danish version of the Oxford cognitive screen: A stroke-specific screening test as an alternative to the MoCA. Aging, Neuropsychology, and Cognition, 27(1), 52–65. https://doi.org/10.1080/13825585.2019.1577352
   PubMed Web of Science ®Google Scholar
• Roebuck-Spencer, T. M., Glen, T., Puente, A. E., Denney, R. L., Ruff, R. M., Hostetter, G., & Bianchini, K. J. (2017). Cognitive screening tests versus comprehensive neuropsychological test batteries: A national academy of neuropsychology education paper. Archives of Clinical Neuropsychology, 32(4), 491–498. https://doi.org/10.1093/arclin/acx021
   PubMed Web of Science ®Google Scholar
• Rotenberg, S., Ruthralingam, M., Hnatiw, B., Neufeld, K., Yuzwa, K. E., Arbel, I., & Dawson, D. R. (2020). Measurement properties of the multiple errands test: A systematic review. Archives of Physical Medicine and Rehabilitation. https://www.sciencedirect.com/science/article/pii/S0003999320301106
   Web of Science ®Google Scholar
• Sasson, E., Doniger, G. M., Pasternak, O., Tarrasch, R., & Assaf, Y. (2013). White matter correlates of cognitive domains in normal aging with diffusion tensor imaging. Frontiers in Neuroscience, 7, 32. https://doi.org/10.3389/fnins.2013.00032
   PubMed Web of Science ®Google Scholar
• Shallice, I. (1982). Tower of London test. Sina Research Institute of Behavioral Cognitive Science (Ravantajhiz), 1387.
   Google Scholar
• Shallice, T., & Burgess, P. W. (1991). Deficits in strategy application following frontal lobe damage in man. Brain, 114(Pt 2), 727–741. https://doi.org/10.1093/brain/114.2.727
   PubMed Web of Science ®Google Scholar
• Shendyapina, M., Kuzmina, E., Kazymaev, S., Petrova, A., Demeyere, N., & Weekes, B. S. (2019). The Russian version of the Oxford cognitive screen: Validation study on stroke survivors. Neuropsychology, 33(1), 77–92. https://doi.org/10.1037/neu0000491
   PubMed Web of Science ®Google Scholar
• Shieh, G., Jan, S. L., & Randles, R. H. (2006). On power and sample size determinations for the Wilcoxon–Mann–Whitney test. Journal of Nonparametric Statistics, 18(1), 33–43. https://doi.org/10.1080/10485250500473099
   Web of Science ®Google Scholar
• Steverson, T., Adlam, A. R., & Langdon, P. E. (2017). Development and validation of a modified multiple errands test for adults with intellectual disabilities. Journal of Applied Research in Intellectual Disabilities, 30(2), 255–268. https://doi.org/10.1111/jar.12236
   PubMed Web of Science ®Google Scholar
• Su, C. Y., Wuang, Y. P., Lin, Y. H., & Su, J. H. (2015). The role of processing speed in post-stroke cognitive dysfunction. Archives of Clinical Neuropsychology, 30(2), 148–160. https://doi.org/10.1093/arclin/acu057
   PubMed Web of Science ®Google Scholar
• Titov, N., & Knight, R. G. (2005). A computer-based procedure for assessing functional cognitive skills in patients with neurological injuries: The virtual street. Brain Injury, 19(5), 315–322.
   PubMed Web of Science ®Google Scholar
• Van Den Berg, E., Nys, G. M., Brands, A. M., Ruis, C., Van Zandvoort, M. J., & Kessels, R. P. (2009). The brixton spatial anticipation test as a test for executive function: Validity in patient groups and norms for older adults. Journal of the International Neuropsychological Society, 15(5), 695–703. https://doi.org/10.1017/S1355617709990269
   PubMed Web of Science ®Google Scholar
• Van der Linden, M., & Andres, P. (2001). Supervisory attentional system in patients with focal frontal lesions. Journal of Clinical and Experimental Neuropsychology, 23(2), 225–239. https://doi.org/10.1076/jcen.23.2.225.1212
   PubMed Web of Science ®Google Scholar
• Vordenberg, J. A., Barrett, J. J., Doninger, N. A., Contardo, C. P., & Ozoude, K. A. (2014). Application of the Brixton spatial anticipation test in stroke: Ecological validity and performance characteristics. The Clinical Neuropsychologist, 28(2), 300–316. https://doi.org/10.1080/13854046.2014.881555
   PubMed Web of Science ®Google Scholar
• Waugh, N. (1999). Self-report of the the young, middle-aged, young-old and old-old individuals on prospective memory functioning (Doctoral dissertation).
   Google Scholar
• Wickham, H., & Bryan, J. (2019). Readxl: Read Excel files. R package version 1.3.1. https://CRAN.R-project.org/package=readxl
   Google Scholar
• Wickham, H., François, R., Henry, L., & Müller, K. (2019). Dplyr: A Grammar of data Manipulation. R package version 0.8.3. https://CRAN.R-project.org/package=dplyr
   Google Scholar
• Wilke, C. O. (2019). Cowplot: Streamlined Plot Theme and Plot Annotations for ‘ggplot2’. R package version 1.0.0. https://CRAN.R-project.org/package=cowplot
   Google Scholar
• Wilson, B. A., Clare, L., Cockburn, J., Baddeley, A. D., Tate, R., & Watson, P. (1999). The Rivermead behavioural memory test-extended version. Thames Valley Test Corporation.
   Google Scholar
• Wilson, B. A., Evans, J. J., Alderman, N., Burgess, P. W., & Emslie, H. (1996). Behavioural assessment of the dysexecutive syndrome. Thames Valley Test Company.
   Google Scholar
• Zartman, A. L., Hilsabeck, R. C., Guarnaccia, C. A., & Houtz, A. (2013). The pillbox test: An ecological measure of executive functioning and estimate of medication management abilities. Archives of Clinical Neuropsychology, 28(4), 307–319. https://doi.org/10.1093/arclin/act014
   PubMed Web of Science ®Google Scholar