Special issue on ecological validity and cognitive assessment

Introduction

While each passing year brings new discoveries aimed at more efficiently, and accurately, diagnosing neurological conditions and a growing appreciation of the value of early initiation of treatments for such conditions, a fundamental question for patients, families, and caregivers, remains – is this person able independently to perform daily activities and, if not, can anything be done to improve his/her functioning?

The goal of this special issue is to highlight current research, from multi-disciplinary perspectives, regarding methods and approaches for assessing everyday functioning. The articles in this volume address ecological validity, various methodological approaches to ecologically valid assessment and the application of these to various populations. The issue of ecological validity is highly relevant to researchers, clinicians, funders and consumers, as virtually every form of brain damage or disease affects everyday life. A core goal of all neuropsychological rehabilitation is to positively benefit quality of life.

The term ecological validity is used here to refer to generalisability (veridicality, or the extent to which assessment results relate to and/or predict behaviours outside the test environment) and representativeness (verisimilitude, or the degree to which assessments resemble everyday life contexts in which the behaviours will be needed). While not the original meaning of the term (see Burgess et al., 2006 for discussion and Baum et al., 2017), it is the meaning that is now in general use.

Traditional cognitive assessments typically require discrete responses to single events, and are conducted in carefully controlled environments. Performance in the real world, however, often involves a serial, or occasionally parallel, stream of tasks, frequently in disordered environs. This juxtaposition of how/when we assess patients, versus the real-world context, may significantly limit the ecological validity of our measures. This problem gives rise to the question of whether “better” assessments are possible. We hope to address this through the series of papers in the current volume. These papers represent a variety of theoretical and disciplinary perspectives and in aggregate provide a comprehensive picture of the current state of research on ecologically valid assessment.

The papers in this issue extend our knowledge of assessments that use everyday life tasks and activities to understand the effect of cognitive impairments, particularly executive dysfunction, on real-world functioning. One group of papers focuses on performance-based measures.

The scoping review by Poncet, Swaine, Dutil, Chevignard, and Pradat-Diehl (2017), via their in-depth comparison of 12 performance-based measures, provides an excellent overview for readers. Additional content on two of the measures cited by Poncet et al. is provided in three papers in this volume. Baum et al. (2017) show data on the validity and clinical utility of Executive Function Performance Test in a large sample of adults with traumatic brain injury. Clark, Anderson, Nalder, Arshad and Dawson (2017) and Nalder, Clark, Anderson and Dawson (2017) add to a growing literature on the psychometric characteristics and clinical utility data of the Multiple Errands Test (MET). Lastly, the first psychometric data for the Complex Task Performance Assessment, a newer test modelled on the MET, but using library tasks, are provided by Wolf, Dahl, Auen, and Doherty (2017).

McFadyen, Gagné, Cossette, and Ouellet (2017) broaden our perspective regarding performance-based measures in their comprehensive review on the value of paradigms combining walking and cognitive tasks (dual-task walking, DTW) in the assessment of executive dysfunction. Finally, Cavuoto, Ong, Pike, Nicholas and Kinsella (2017) and Sanders and Schmitter-Edgecombe (2017) use performance-based tasks in real-world environments to further understand specific aspects of cognitive processing. Cavuoto et al. explore predictors of successful prospective memory for habitual tasks in older adults and Sanders and Schmitter-Edgecombe investigate how the use of a formal planning phase prior alters task execution, in this instance using the Amap task.

There are two primary approaches to developing everyday functioning assessments: function-led perspectives, which focus on the everyday task at hand (e.g., medication management), and construct-led, which focus on the underlying constructs needed for successful completion of task (e.g., habitual prospective memory). The distinction between these perspectives, in the context of virtual environments, is discussed in the review paper by Parsons, Carlew, Magtoto, and Stonecipher (2017), with the authors suggesting that virtual assessments designed from a function-led perspective align more closely with the aim of achieving a set of ecologically valid assessments. We encourage the reader to keep this idea in mind while reviewing the papers by Nir-Hadad, Weiss, Waizman, Schwartz, and Kizony (2017) and Canty, Neumann, Fleming, and Shum (2017). Nir-Hadad and colleagues provide data on the validity of a function-led virtual assessment, an adapted version of the Four-Item Shopping Task. In contrast, Canty et al. have developed a theory of mind assessment that occurs in a virtual environment, the Virtual Assessment of Mentalising Ability (VAMA). Their data suggest that the VAMA is more representative of social functioning in the real world than more traditional measures of theory of mind.

In the final paper in this special issue, Bottari, Gosselin, Chen, and Ptito (2017) illustrate that imaging paradigms may enhance our understanding of performance on ecologically valid tasks (in this instance, the Instrumental Activities of Daily Living Profile): adults with post-concussion syndrome performing poorly on this everyday life task also showed abnormalities on functional magnetic resonance imaging and evoked response potentials.

An underexplored issue, and an area of continuing importance, is the examination of the ecological validity of assessments between differing diagnostic and socio-demographic populations. Three recent papers addressing this issue include Chaytor et al. (2017), Goverover et al. (2017), and Thomas and Marsiske (2017).

Together, these studies provide a wealth of information not only with regards to specific ecologically valid assessments but also in relation to methodologies that will inform ongoing work in this area. It is our hope that these papers will serve as a catalyst for discussion of the best way to evaluate everyday functioning, in an ecologically valid approach, and inspire new and creative approaches to addressing the critical issue of how to assess, and help, patients in their everyday life.