Longitudinal analysis of sustained performance on gamified cognitive assessment tasks

References

• Arevalo-Rodriguez, I., Smailagic, N., Roqué I Figuls, M., Ciapponi, A., Sanchez-Perez, E., Giannakou, A., Pedraza, O. L., Bonfill Cosp, X., & Cullum, S. (2015). Mini-Mental State Examination (MMSE) for the detection of Alzheimer’s disease and other dementias in people with mild cognitive impairment (MCI). Cochrane Database of Systematic Reviews, 3, 68 pages. https://doi.org/10.1002/14651858.CD010783.pub2
   Web of Science ®Google Scholar
• Bartels, C., Wegrzyn, M., Wiedl, A., Ackermann, V., & Ehrenreich, H. (2010). Practice effects in healthy adults: A longitudinal study on frequent repetitive cognitive testing. BMC Neuroscience, 11, 118. https://doi.org/10.1186/1471-2202-11-118
   PubMed Web of Science ®Google Scholar
• Basso, M. R., Bornstein, R. A., & Lang, J. M. (1999). Practice effects on commonly used measures of executive function across twelve months. The Clinical Neuropsychologist, 13(3), 283–292. https://doi.org/10.1076/clin.13.3.283.1743
   PubMed Web of Science ®Google Scholar
• Beglinger, L. J., Gaydos, B., Tangphao-Daniels, O., Duff, K., Kareken, D. A., Crawford, J., Fastenau, P. S., & Siemers, E. R. (2005). Practice effects and the use of alternate forms in serial neuropsychological testing. Archives of Clinical Neuropsychology, 20(4), 517–529. https://doi.org/10.1016/j.acn.2004.12.003
   PubMed Web of Science ®Google Scholar
• Bird, S., Loper, E., & Klein, E. (2009). Natural language processing with python. O'Reilly Media, Inc.
   Google Scholar
• Boot, W. R., Souders, D., Charness, N., Blocker, K., Roque, N., & Vitale, T. (2016). The gamification of cognitive training: Older adults’ perceptions of and attitudes toward digital game-based interventions. Proceedings of the International conference on human aspects of it for the aged population (pp. 290-300). Springer. https://doi.org/10.1007/978-3-319-39943-0_28
   Google Scholar
• Busse, A., Angermeyer, M. C., & Riedel-Heller, S. G. (2006). Progression of mild cognitive impairment to dementia: A challenge to current thinking. The British Journal of Psychiatry: The Journal of Mental Science, 189(5), 399–404.
   PubMed Web of Science ®Google Scholar
• Calamia, M., Markon, K., & Tranel, D. (2012). Scoring higher the second time around: Meta-analyses of practice effects in neuropsychological assessment. Clinical Neuropsychologist, 26(4), 543-570. https://doi.org/10.1080/13854046.2012.680913
   PubMedGoogle Scholar
• Casey, J., Baghaei, N., & Nand, K. (2015). Improving learning outcomes by designing engaging educational tools [Paper presentation]. 2014 IEEE Frontiers in Education Conference (FIE) proceedings (pp. 1-4). IEEE. https://doi.org/10.1109/FIE.2014.7044042
   Google Scholar
• Cassady, J. C., & Johnson, R. E. (2002). Cognitive test anxiety and academic performance. Contemporary Educational Psychology, 27(2), 270–295. https://doi.org/10.1006/ceps.2001.1094
   Web of Science ®Google Scholar
• Dadon, G., & Henik, A. (2017). Adjustment of control in the numerical Stroop task. Memory and Cognition, 45(6), 891-902. https://doi.org/10.3758/s13421-017-0703-6
   Google Scholar
• Dao, P. (2021). Effects of task goal orientation on learner engagement in task performance. International Review of Applied Linguistics in Language Teaching, 59(3), 315–334. https://doi.org/10.1515/iral-2018-0188
   Web of Science ®Google Scholar
• Donovan, N. (2015). The effects of intrinsic and extrinsic motivation on cognitive performance in humans. The Huron University College Journal of Learning and Motivation, 53(1), 84-96.
   Google Scholar
• Feinstein, A., Brown, R., & Ron, M. (1994). Effects of practice of serial tests of attention in healthy subjects. Journal of Clinical and Experimental Neuropsychology, 16(3), 436–447. https://doi.org/10.1080/01688639408402654
   PubMed Web of Science ®Google Scholar
• Goldberg, T. E., Harvey, P. D., Wesnes, K. A., Snyder, P. J., & Schneider, L. S. (2015). Practice effects due to serial cognitive assessment: Implications for preclinical Alzheimer’s disease randomized controlled trials. Diagnosis, Assessment and Disease Monitoring. https://doi.org/10.1016/j.dadm.2014.11.003
   Google Scholar
• Gronwall, D. M. A. (1977). Paced auditory serial-addition task: A measure of recovery from concussion. Perceptual and Motor Skills, 44(2), 367–373. https://doi.org/10.2466/pms.1977.44.2.367
   PubMed Web of Science ®Google Scholar
• Hausknecht, J. P., Halpert, J. A., Di Paolo, N. T., & Moriarty Gerrard, M. O. (2007). Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Journal of Applied Psychology, 92(2), 373–385. https://doi.org/10.1037/0021-9010.92.2.373
   PubMed Web of Science ®Google Scholar
• Hawkins, G. E., Rae, B., Nesbitt, K. V., & Brown, S. D. (2013). Gamelike features might not improve data. Behavior Research Methods, 45(2), 301–318. https://doi.org/10.3758/s13428-012-0264-3
   PubMed Web of Science ®Google Scholar
• Ho, J., Tumkaya, T., Aryal, S., Choi, H., & Claridge-Chang, A. (2019). Moving beyond P values: Data analysis with estimation graphics. Nature Methods, 16(7), 565-566. https://doi.org/10.1038/s41592-019-0470-3
   Google Scholar
• Hu, Y. Z., Parimoo, S., & Chignell, M. (2021). Using a serious game to measure executive functioning: Working memory updating.
   Google Scholar
• Ikram, M. A., Vrooman, H. A., Vernooij, M. W., den Heijer, T., Hofman, A., Niessen, W. J., van der Lugt, A., Koudstaal, P. J., & Breteler, M. M. B. (2010). Brain tissue volumes in relation to cognitive function and risk of dementia. Neurobiology of Aging, 31(3), 378–386.
   PubMed Web of Science ®Google Scholar
• Kirchner, W. K. (1958). Age differences in short-term retention of rapidly changing information. Journal of Experimental Psychology, 55(4), 352–358. https://doi.org/10.1037/h0043688
   PubMedGoogle Scholar
• Leduc-McNiven, K., Dion, R. T., Mukhi, S. N., McLeod, R. D., & Friesen, M. R. (2018). Machine learning and serious games: opportunities and requirements for detection of mild cognitive impairment. Journal of Medical Artificial Intelligence, 1, 1–1. https://doi.org/10.21037/jmai.2018.07.02
   Google Scholar
• Lee, J. S., Tong, T., Tierney, M. C., Kiss, A., & Chignell, M. (2019). Predictive ability of a serious game to identify emergency patients with unrecognized delirium. Journal of the American Geriatrics Society, 67(11), 2370–2375.
   PubMed Web of Science ®Google Scholar
• Lumsden, J., Edwards, E. A., Lawrence, N. S., Coyle, D., & Munafò, M. R. (2016). Gamification of cognitive assessment and cognitive training: A systematic review of applications and efficacy. Jmir Serious Games, 4(2), e11. https://doi.org/10.2196/games.5888
   PubMed Web of Science ®Google Scholar
• Maimon, N. B., Molcho, L., Intrator, N., & Lamy, D. (2020). Single-channel EEG features during n-back task correlate with working memory load. arXiv Preprint arXiv:2008.04987.
   Google Scholar
• Marshall, G. A., Rentz, D. M., Frey, M. T., Locascio, J. J., Johnson, K. A., & Sperling, R. A. (2011). Executive function and instrumental activities of daily living in mild cognitive impairment and Alzheimer's disease. Alzheimer's & Dementia, 7(3), 300–308. https://doi.org/10.1016/j.jalz.2010.04.005
   PubMed Web of Science ®Google Scholar
• McCallum, S. (2012). Gamification and serious games for personalized health. In PHealth 2012: Proceedings of the 9th International conference on wearable micro and nano technologies for personalized health (pp. 85–96).
   Google Scholar
• McCallum, S., & Boletsis, C. (2013). A taxonomy of serious games for dementia. Games for Health: Proceedings of the 3rd European conference on gaming and playful interaction in health care 2013 (pp. 219-232). Springer Publishing.
   Google Scholar
• Miller, J. (1988). A warning about median reaction time. Journal of Experimental Psychology. Human Perception and Performance, 14(3), 539–543. https://doi.org/10.1037/0096-1523.14.3.539
   PubMed Web of Science ®Google Scholar
• Owen, A. M., McMillan, K. M., Laird, A. R., & Bullmore, E. (2005). N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Human Brain Mapping, 25(1), 46–59. https://doi.org/10.1002/hbm.20131
   PubMed Web of Science ®Google Scholar
• Petridis, P., Hadjicosta, K., Shi Guang, V., Dunwell, I., Baines, T., Bigdeli, A., Bustinza, O. F., & Uren, V. (2015). State-of-the-art in business games. International Journal of Serious Games, 2(1), 55-69. https://doi.org/10.17083/ijsg.v2i1.54
   Google Scholar
• Python Software Foundation. (2020). Python Language Reference, version 3.7. http://www.python.org
   Google Scholar
• R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/.
   Google Scholar
• Robert, P. H., König, A., Amieva, Hl., Andrieu, S., Bremond, Fço., Bullock, R., Ceccaldi, M., Dubois, B., Gauthier, S., Kenigsberg, P.-A., Nave, Sép., Orgogozo, J. M., Piano, J., Benoit, M., Touchon, J., Vellas, B., Yesavage, J., & Manera, V. (2014). Recommendations for the use of serious games in people with Alzheimer’s disease, related disorders and frailty. Frontiers in Aging Neuroscience, 6, 54. https://doi.org/10.3389/fnagi.2014.00054
   PubMed Web of Science ®Google Scholar
• Rousselet, G., & Wilcox, R. (2020). Reaction times and other skewed distributions: Problems with the mean and the median. Meta-Psychology, 4, MP.2019.1630. https://doi.org/10.15626/MP.2019.1630
   Google Scholar
• RStudio Team. (2020). RStudio: Integrated Development for R. RStudio, PBC, Boston, MA URL http://www.rstudio.com/.
   Google Scholar
• Savulich, G., Piercy, T., Fox, C., Suckling, J., Rowe, J. B., O’brien, J. T., & Sahakian, B. J. (2017). Cognitive training using a novel memory game on an iPad in patients with amnestic mild cognitive impairment (aMCI). International Journal of Neuropsychopharmacology, 20(8), 624–633. https://doi.org/10.1093/ijnp/pyx040
   PubMed Web of Science ®Google Scholar
• Shaban, A., & Pearson, E. (2020). Evaluation of user experience and cognitive load of a gamified cognitive training application for children with learning disabilities [Paper presentation]. In Proceedings of the 17th International web for All conference, W4A 2020 (pp. 1-10). https://doi.org/10.1145/3371300.3383341
   Google Scholar
• Smith, A. (1968). The symbol-digit modalities test: A neuropsychologic test for economic screening of learning and other cerebral disorders. Learning Disorders, 3, 83-91.
   Google Scholar
• Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662. https://doi.org/10.1037/h0054651
   Google Scholar
• Tong, T., & Chignell, M. (2013). Designing game-based cognitive assessments for elderly adults. In Proceedings of the First International conference on Gameful design, research, and applications (pp. 127-130).
   Google Scholar
• Tong, T., Chignell, M., & DeGuzman, C. A. (2021). Using a serious game to measure executive functioning: Response inhibition ability. Applied Neuropsychology: Adult, 28(6), 673-684.
   PubMed Web of Science ®Google Scholar
• Tong, T., Chignell, M., Lam, P., Tierney, M. C., & Lee, J. (2014). Designing serious games for cognitive assessment of the elderly. Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, 3(1), 28–35. https://doi.org/10.1177/2327857914031004
   Google Scholar
• Tong, T., Chignell, M., Tierney, M. C., & Lee, J. S. (2016a). A serious game for clinical assessment of cognitive status: Validation study. JMIR Serious Games, 4(1), e5006. https://doi.org/10.2196/games.5006
   Web of Science ®Google Scholar
• Tong, T., Chignell, M., Tierney, M. C., & Lee, J. S. (2016b). Test-retest reliability of a serious game for delirium screening in the emergency department. Frontiers in Aging Neuroscience, 8, 258. https://doi.org/10.3389/fnagi.2016.00258
   PubMed Web of Science ®Google Scholar
• Tong, T., Urakami, J., Chignell, M., Tierney, M. C., & Lee, J. S. (2020). Tracking cognitive decline with a serious game: benchmarking against the mini-mental state examination. Proceedings of the human factors and ergonomics society annual meeting, 64(1), 6–10. https://doi.org/10.1177/1071181320641002
   Google Scholar
• Tong, T., Yeung, J., Sandrakumar, J., Chignell, M., Tierney, M. C., & Lee, J. (2015). Improving the ergonomics of cognitive assessment with serious games. Proceedings of the International symposium on human factors and ergonomics in health care, 4(1), 1–5. https://doi.org/10.1177/2327857915041001
   Google Scholar
• Urakami, J., Hu, Y. Z., & Chignell, M. (2021). Conference on human factors in computing systems – proceedings, monitoring cognitive performance with a serious game: A longitudinal case study on online cognitive assessment using serious games. https://doi.org/10.1145/3411763.3443431
   Google Scholar
• Vermeir, J. F., White, M. J., Johnson, D., Crombez, G., & van Ryckeghem, D. M. L. (2020). The effects of gamification on computerized cognitive training: Systematic review and meta-analysis. JMIR Serious Games, 8(3), e18644. https://doi.org/10.2196/18644
   PubMed Web of Science ®Google Scholar
• Walsh, N. D., Williams, S. C. R., Brammer, M. J., Bullmore, E. T., Kim, J., Suckling, J., Mitterschiffthaler, M. T., Cleare, A. J., Pich, E. M., Mehta, M. A., & Fu, C. H. Y. (2007). A longitudinal functional magnetic resonance imaging study of verbal working memory in depression after antidepressant therapy. Biological Psychiatry, 62(11), 1236–1243. https://doi.org/10.1016/j.biopsych.2006.12.022
   PubMed Web of Science ®Google Scholar
• Zhang, B., & Chignell, M. (2020). A Framework for Using Cognitive Assessment Games for People Living with Dementia. In 2020 IEEE 8th International Conference on Serious Games and Applications for Health (SeGAH) (pp. 1–8). IEEE.
   Google Scholar