Complexity in measuring outcomes after communication partner training: alignment between goals of intervention and methods of evaluation

ABSTRACT

Background: Researchers are currently discussing the need for consensus on a core set of outcome measures to assess interventions in aphasia. For indirect, environmental approaches to aphasia intervention, such as communication partner training (CPT), the roadmap to obtaining consensus on core outcome measures seems especially complex. While the purpose of CPT is to improve communication for people with communication disorders, the intervention is aimed at the communication partner. There is also a variety of goals, activities, and possible settings for CPT. This complexity increases the risk of a mismatch between the goals and content of the intervention and measures used to evaluate the outcome.

Aims: The purpose of this paper is to describe the complexity of measuring outcomes from CPT. The aim is to enable clinicians and researchers to reflect on the outcomes to be measured and also on how different types of measures may or may not be aligned with the goals and content of a specific CPT intervention.

Main contribution: The current proliferation of outcome measures used in CPT is considered in the light of a survey of general factors to be considered in evaluating intervention outcomes. The complexity of measuring outcomes in CPT is illustrated and the importance of alignment of main objectives, intervention tasks, and projected outcomes is exemplified by referencing two common types of CPT approaches. Objectives relating to knowledge of aphasia, interactional behaviour, and feelings and attitudes are considered in relation to specific outcome measurements. It is suggested that both study-specific and more general measures are needed for capturing and comparing outcomes. The measurement of relevant outcome in CPT is discussed along with implications for future research and clinical practice.

Conclusions: Different CPT approaches share the same purpose of facilitating communication in aphasia, but their application in research studies or in the clinic, is specific to the particular context. Special care must thus be taken in both clinical practice and research to safeguard the alignment between objectives, tasks, and projected intervention outcomes and the actual measures used. Further, it is concluded that there is a need for the development of new measures based on a consensus on key outcomes to be measured in CPT.

KEYWORDS:

• Communication partner training
• aphasia
• outcome measures
• patient-relevant outcome
• core outcome set

Introduction

Outcome measures have come into focus in stroke rehabilitation research in recent years, reflecting a general trend to call critical attention to the intended outcomes of interventions and the instruments used for measuring them (Ali, English, Bernhardt, Sunnerhagen, & Brady, 2013; Geroin et al., 2013; Wilde et al., 2010). In fact, appropriate outcome measures are vital to the correct interpretation of evidence regarding a particular clinical practice and essential to implementation of evidence-based practices. Poorly chosen outcome measures in intervention research may result in failure to support an effective treatment or in positive support for an ineffective or less effective treatment.

To ensure that results of research are valid and that research results across trials can be compared, initiatives have been undertaken to establish standardised sets of outcomes for clinical trials. The Core Outcome Measures in Effectiveness Trials (COMET; http://www.comet-initiative.org) is one example. In addition, tools for critical appraisal of the instruments used in outcome measurement have been suggested, for example, Consensus-based Standards for the selection of health Measurement Instruments (COSMIN; http://www.cosmin.nl).

In the field of aphasia, a recent Cochrane review shows that evidence about the efficacy of different intervention methods derive from studies using a large variety of outcome measures (Brady, Kelly, Godwin, Enderby, & Campbell, 2016). Besides preventing direct comparisons of intervention, this large variety of outcome measures makes it difficult to pool data across clinical trials, and it has motivated calls for greater consensus on the selection of outcome measurements in research (Brady et al., 2014; Wallace, Worrall, Rose, & Le Dorze, 2014). Given that aphasia affects the lives of individuals and their significant others in many different ways, and interventions target different consequences of aphasia, it is not surprising to find variety in the use of outcome measurements in aphasia research. However, without clear and comparable findings from aphasia treatment research, generalizations regarding effective treatments remain elusive. Wallace et al. (2014) have proposed an agenda for developing international consensus on a “core outcome set” and on instruments appropriate for measuring these core outcomes in the ROMA project: Improving Research Outcome Measurement in Aphasia (http://www.comet-initiative.org/studies/details/287). Thus, there is a well-reasoned strive for establishing a set of core outcome measures in aphasia intervention.

The heterogeneity of outcome measurement is clearly also evident in research on communication partner training (CPT) in aphasia, where a plethora of outcome measures used in CPT research makes interpretation of research findings across studies extremely difficult (Simmons-Mackie, Raymer, Armstrong, Holland, & Cherney, 2010; Simmons-Mackie, Raymer, & Cherney, 2016).

CPT is an umbrella term for several different approaches, which have the purpose of facilitating communication for both the person with aphasia (PWA) and the communication partner (CP). CPs, and often the dyad (i.e., the CP and the PWA), learn strategies to improve communicative interactions, including how to increase interactional well-being as well as how to increase quality and quantity of information exchange. CPT has been shown to be effective in improving partners’ skill in facilitating communication, and is now recommended in several national clinical guidelines for SLTs or stroke guidelines. Examples of such are the American Speech-Language-Hearing Association Practise Portal (n.d.); the UK National Institute for Health and Care Excellence (NICE, 2013) clinical guidelines for stroke; the Swedish national clinical guidelines for stroke care (Socialstyrelsen, 2018) and the Canadian Best Practice Recommendations (Hebert et al., 2016). Still, more research is needed to establish the effects from CPT for individuals with acute aphasia, and to assess the implementation of CPT in complex healthcare environments (Cherney, Simmons-Mackie, Raymer, Armstrong, & Holland, 2013; Simmons-Mackie et al., 2010, 2016; Simmons-Mackie, Savage, & Worrall, 2014; Turner & Whitworth, 2006; Wilkinson & Wielaert, 2012). Establishing a shared set of outcome measures for this type of intervention may enhance comparability of different studies and lead to improvements of the evidence base. However, many factors need to be considered when choosing an outcome measure in a particular study, and there may be a caveat against proposing any specific measure as a core outcome measure for evaluating CPT in general, as will be elaborated on in the following.

The purpose of this paper is to describe the complexity involved in measuring outcomes after CPT interventions. The specific aim is to provide clinicians and researchers with a conceptual framework enabling them to reflect in general on how different types of measures may or may not be aligned with the goals and training activities of a specific CPT intervention. Tools relevant to capturing outcomes and comparing different CPT approaches will also be discussed.

First, the complexity of CPT interventions will be described and how this complexity may affect measuring of outcome. Second, factors in relation to measuring outcome in general are considered in relation to CPT. In this second part, two different types of CPT approaches will be used to exemplify issues in measuring outcomes. Third, examples of existing measurements and the proposed ROMA Core Outcome Set (Wallace, Worrall, Rose, & Le Dorze, 2016b) are discussed in relation to the two CPT approaches as well as implications for future measurements.

Complexity of measuring outcome in communication partner training

One possible source of the diversity in outcome measures may derive from the fact that CPT is a cover term for several different approaches (for a descriptive taxonomy, see Simmons-Mackie et al., 2014). Differences between CPT approaches include, but are not limited to, who receives intervention, the specific goals of the intervention, and the focus of intervention activities. For example, some CPT approaches target only the CP, others target both the CP and the PWA, and the intervention may be generic or individualized, and delivered either in a group setting, in a dyad, or individually. CPs may also differ in the nature of their relationship to the PWA. CPs may be family members, volunteers, or different types of professionals, such as healthcare staff who engage in communication with PWAs as part of their job. The setting in which intervention takes place may also vary. Examples of different locations are in the participants’ home, clinical settings like hospital or rehabilitation clinics, and educational settings like universities/colleges. Common CPT intervention goals and focus of intervention activities are: (1) to establish explicit knowledge in order to promote behavioural change, (2) to affect behaviour through explicit or implicit methods of reflective learning, modelling, and coaching, and (3) to address feelings/attitudes about aphasia and communication (Cruice, Blom Johansson, Isaksen, & Horton, this issue). Some of the differences, which contribute to the complexity of obtaining consensus on outcome measurement of CPT, are illustrated in Table 1.

Table 1. Examples of differences between CPT approaches, and between methods of outcome evaluation.

Methods of Intervention		Methods of evaluation and measurement
What domain of change is the intervention focused on: knowledge, behaviour or feelings/attitudes? What are the included procedures, activities and tasks?	WHAT	What is measured to evaluate change: Constructs reflecting knowledge, behaviour or feelings/attitude? Which ICF domains are evaluated: body function, activity/participation, environmental or personal factors?
Who are the targeted participants: The dyad, only the CP, a family member, volunteer or healthcare staff?	WHO	Who is being assessed: CP, PWA or dyad? Who is assessing: PWA or CP giving self-reports, an “objective” other, for example, the SLT providing intervention, or a trained and blinded assessor?
How is intervention delivered: individually, in dyads, in groups? How long is the period of intervention, and how intensive?	HOW	Type of data explored: video-recorded interaction, interview, rating scale, or questionnaire data? Methods of analysing data: qualitative, quantitative, mixed? Psychometric properties of methods: Reliability, validity, sensitivity, feasibility?
What is the context or setting for the intervention: home, clinical setting, university/college, other?	WHERE	What is the social context for obtaining the outcome data: home, clinical setting, experimental, other?
What is the main objective of the intervention and its stated intermediate goals?	WHY	What is the purpose of evaluation: goal setting, baseline, outcome measurement?

Other sources of complexity may be that unlike traditional aphasia therapy, which directly targets the person with aphasia, CPT aims to facilitate communication in aphasia by modifying the communication environment, that is, through training of another person than the one in need of rehabilitation. Furthermore, the projected effects relate to conversation, which is itself a complex phenomenon. This is because conversational aspect may unfold in quite different ways depending not only on the participants’ behaviours, but also for example, on their relationship and roles, the specific context, and the topic of conversation (Ahlsén & Saldert, this issue).

In the next section, a survey of frequently used methods of evaluation will be provided and sources of complexity in outcome measurement will be further illustrated and exemplified.

Selecting appropriate methods for measurement and evaluation of intervention outcomes

Factors to consider in methods of evaluation

In both clinical practice and research, the assessment of outcomes from interventions is essential (Worral, 1999). Outcome may be evaluated with qualitative or quantitative methods, or a combination of both. Qualitative methods include interviews with stakeholders, ethnographic observation, or approaches such as Conversation Analysis (Silverman, 2001). Generally, qualitative methods can be adapted to the content which is studied, and to provide in-depth information. For example, they do not require a fully pre-specified target outcome, but permit evaluation of highly individualized outcomes. Qualitative methods thus have potential to provide detailed information about the actual impact of an intervention according to the lived experience of an individual and his or her family, but they tend to be time consuming in their application. Quantitative methods of assessment include traditional assessments of linguistic ability or functional communication, rating scales, or questionnaires with closed questions (Silverman, 2001; Wright, 1997). Quantitative methods are generally more economic in administration time than qualitative methods, and permit statistical tests of the significance and confidence intervals of obtained intervention effects.

Both qualitative and quantitative methods of evaluation and assessment need to be applied with methodological rigour by the researcher or the clinician. The quality of the information provided by self-ratings or assessments by SLT, for example, frequency counts of specific behaviours or rating scales, depends on the psychometric properties of the measures. Methods of evaluation need to be valid and sufficiently sensitive to capture the construct they are meant to evaluate with a specific population, and they need to be reliable in yielding consistent results regardless of operator (McDowell, 2006; Tinsley & Weiss, 1975). In applying qualitative approaches to outcome evaluation, the researcher may reinforce methodological validity, for example, by paying attention to negative data, which do not follow the main patterns, by triangulation of data, and participant validation (Mays & Pope, 2000).

When selecting outcome measures in a study or in the clinic, there is also an important question about the responsiveness of the selected outcome measures in relation to a specific intervention. The responsiveness of an instrument also has to do with its sensitivity, that is, the ability of the measure to actually detect change. Sensitivity has been described as an important aspect of validity (Hays & Hadorn, 1992). The instrument needs to include items dealing with different aspects of what is being measured and the scoring system needs to be sensitive to discrete changes. Again, to be responsive, the measure needs to be adapted to the specific population in question.

Another factor to consider when measuring outcome is the appropriateness of the measures selected, as pointed out by Coster (2013) in the field of occupational therapy. Thus, in the context of a particular intervention or study, the researcher or clinician must consider how selected outcome measures are appropriate for the intervention provided:

The ultimate value of a clinical trial or outcome study will be directly tied to how well the selected outcome measure matches the researcher’s understanding of what he or she expects to change, to what degree it is expected to change, over what period of time this change will happen, and how that change can best be identified. (Coster, 2013, p. 163)

In order to evaluate the appropriateness of an outcome measure in relation to a specific intervention, Coster (2013) suggests that the researcher develops a visual diagram of a causal model, which specifies his or her thinking about the hypothesized mechanisms of change in the intervention. The model should make explicit the goals of the intervention, the tasks, and the activities selected to match these goals, as well as to clarify in what aspects of the person’s life changes are most likely to be evident (Coster, p. 163). An important issue for the researcher to consider is whether there appears to be an appropriate alignment between the overall purpose of the intervention, the specific objectives and goals, and the selected activities or tasks of the intervention. If there is no proper alignment between objectives, goals, and intervention activities, the likelihood of obtaining an intervention effect is reduced (Coster, 2013). Thus, the researcher or clinician must consider the focus of the chosen activities and tasks and ask, if there is a proximal or more distal relationship between the activities and tasks, and the changes projected to follow from the intervention. For example, an intervention for anomia may include confrontation naming tasks, but no tasks for practicing word finding in conversation, as illustrated in Figure 1. In such a model, a projected change in confrontation naming or single word production will be proximal to the intervention activities and tasks, whereas a change in conversational-level word finding is more distal in relation to the intervention. This is not to say that generalisation of naming to conversation-level word finding is not important. Indeed, it may be the ultimate purpose of the intervention and therefore must be evaluated. However, it is not the most proximal intervention outcome.

Figure 1. Visual model inspired by Coster (2013) of a hypothetical type of anomia intervention, where therapy focuses on confrontation naming tasks. The projected proximal outcomes are likely to be effects relating to word retrieval in confrontation naming (full arrow). Possible effects relating to word finding in conversation would be dependent on generalization and seem to be more distal to the tasks and activities of the intervention (dotted arrow).

A causal model helps specify the projected outcomes, which are to be assessed or evaluated after the intervention. Once the projected intervention outcomes are clarified, it is possible for the researcher or clinician to make judgments about the appropriateness of specific tests or assessment methods for evaluating the intervention. When a specific assessment tool is taken into consideration, the researcher/clinician needs to question, if there is a reasonable relationship between the projected intervention outcomes and the underlying constructs or phenomena, which the assessment tool, for example, a questionnaire, is in fact designed to measure. Change is easier to demonstrate, if the chosen assessment tool measures outcomes, which are more proximal to the actual goals, activities and tasks of the intervention than outcomes, which are more distal.

The distinction (or continuum) proposed by Coster (2013) between proximal and distal outcomes is to be understood in relation to the tasks and activities of an intervention. It is therefore different from the distinction made in research studies between primary and secondary outcomes, where the primary outcome is the most important outcome motivating the intervention, for which a trial is powered.

At a very general level, CPT approaches differ from other types of aphasia therapy, for example, anomia therapy, by using tasks and activities, which includes another person, the CP. In CPT, the CP is to some extent a mediator of intervention effects pertaining to the PWA as suggested in the visual model of CPT in Figure 2. Furthermore, the effect from intervention on the CP is at the same time dependent on how the PWA responds to any change in the CP’s behaviour.

Figure 2. Visual model of a hypothetical type of CPT, which exclusively targets the CP. The projected proximal outcomes are likely to be effects relating to the communication partner (full arrow) or his or her interaction with the PWA (dashed arrow), whereas effects relating to the PWA seem to be more distal to the tasks and activities of the intervention (dotted arrow).

Two communication partner training approaches and projected outcomes

To be able to discuss the appropriateness of specific measurements in relation to the projected outcomes of a particular study, it is necessary to carefully consider the specific goals of a given CPT intervention, the tasks and activities associated with the therapy, and the phenomena most likely to change as a consequence of the therapy. For the purpose of illustration, two types of CPT approaches are presented and discussed in relation to the possible alignment between set goals, the intervention activities and tasks, and the projected outcome, see Table 2. For reasons of consistency and clarity, the term purpose is used for the overall goal shared by CPT approaches of facilitating communication for the PWA through intervention including the PWA’s communication partners. The term objective is used for the main goal(s) of a specific type of CPT. The term goal is reserved for particular (and individualized) goals or subgoals which may be set, when a specific type of CPT is applied in a particular setting, that is, in a study or in the clinic. These subgoals are often intermediate goals between the tasks and activities in the intervention and the main objective.

Table 2. Two different types of CPT approaches describing well-aligned main objectives, intermediate goals, activities and tasks in intervention and possible proximal and distal outcomes to measure.

CPT approach	Main objective	Intermediate goals	Activities and tasks in intervention	Outcomes suggested as proximal or distal
Dyad-focused CPT: Training dyads in the development of individually adapted communicative strategies	To facilitate communication in specific dyads	Increased knowledge about communication in aphasia. Increased use in the dyad of communicative behaviours which facilitates the interactional flow and reduction of barriers in the dyad’s everyday communication	Teaching about aphasia and conversational interaction. Use of own video-recorded interactions and coaching to gain insights and adaptation of specific communicative behaviours	Proximal: Knowledge about communication in aphasia including own barrier and facilitative behaviours; change in targeted behaviours in CP and/or PWA Distal: Change in the dyad’s feelings/attitudes about the quality of their communication
Generic CPT: Teaching healthcare staff to use supportive techniques and materials in healthcare contexts	To enable PWA to participate in planning and assessment of healthcare	Increased knowledge about communication and aphasia and behavioural increase in the appropriate use of generic supportive techniques in conversations with different PWAs	Teaching about aphasia and training through role play in use of generic strategies to support conversations with different PWAs in healthcare contexts	Proximal: CP’s knowledge about communication in aphasia; CP’s behaviour and skill in use of different supportive techniques Distal: Increased communicative participation (behaviour) of PWA, or PWAs’ feelings of inclusion and participation in the healthcare setting; change in CP’s feelings/attitudes about communicating with PWAs

Dyad-focused approach

In a dyad-focused approach, the CPT is aimed at dyads consisting of a PWA and an important and frequent CP, for example, a spouse or a key nurse in long-term residential care. The overarching main objective of using this approach may be to facilitate communication in a specific dyad. The dyad-focused CPT approach is in many ways similar to the CPT approach described in Booth and Perkins (1999), as well as similar to the established CPT programme Supporting Partners of People with Aphasia in Relationships and Conversation (SPPARC; Lock, Wilkinson, & Bryan, 2001), and the web-based e-resource Better Conversations with Aphasia (Beeke et al., 2013; see Cruice et al., this issue, for more examples). Within this CPT approach, the methodology of Conversation Analysis (Hutchby & Wooffitt, 1998) is often (but not in all instances) used to identify interactional behaviours, which are subsequently negotiated with the dyad in order to establish whether the dyad perceives these behaviours as facilitating or as barrier behaviours. The participants are also involved in the choice of which behaviours they want to be targeting in the intervention and the behavioural changes are considered as intermediate goals of the intervention. The intervention tasks typically include the clinician providing information about communication in general and in aphasia. Video recordings are used of the dyad’s everyday conversational interaction at home as examples of barriers or facilitating behaviours. The clinician provides online feedback to guide the participants (either the CP alone, or the dyad) in the use of individually adapted conversational behaviours, which facilitate the interaction and reduce barriers. In accordance with a causal model of the dyad-focused CPT approach, the projected proximal outcome, based on the stated goals and the included activities and tasks, include CP’s (and perhaps PWA’s) knowledge of aphasia, and of behaviours that facilitate or impede the dyad’s interaction, as well as behaviour changes in the targeted behaviours (see section on Measuring outcome in communication partner training, below).

We here give a first example of a study using the dyad-focused CPT approach. In this example, the outcomes and the activities and tasks in the intervention are aligned with the main objective: To facilitate communication in specific dyads. For example, in accordance with the projected proximal outcome, the CP may learn that asking the PWA to pronounce words correctly in their everyday conversation in fact prevents the PWA from elaborating on a topic and may be experienced by the PWA as embarrassing or humiliating (knowledge). Based on this insight, asking for the correct pronunciation may be selected as target behaviour for change by the CP in collaboration with the PWA and the speech language therapist (SLT). Through awareness raising and through learning alternate modes of responding to a word finding problem, the CP may decrease his or her initiation of repair, that is, asking for correct pronunciation (behaviour), which may result in the dyad experiencing increased conversational flow and more equal roles in their communicative interaction. Hopefully, this would also lead to the PWA experiencing less embarrassment and feeling better about the dyad’s communication, but this is a more distal outcome in relation to the training, which has not involved specific work on the PWA’s feelings/attitudes, and furthermore, the training has only been provided to the CP. In other words, the activities and tasks in the intervention are aimed at increased knowledge and behaviour changes. Still, it is possible, yet less directly so, that increased knowledge and change in behaviour will affect a positive change in the dyad’s feelings/attitudes, for example, increased enjoyment or satisfaction with conversations.

Unlike the first example, which illustrated alignment between the goal and activities and tasks, we shall now consider a second example of applying the dyad-focused approach, which illustrates a mismatch. The main objective of the intervention is the same as in the first example, that is, to facilitate communication in a specific dyad. In this hypothetical example, the participants in the dyad both wish for the PWA to be able to elaborate more on a topic. However, instead of working on the dyad’s understanding of their own communication patterns, the SLT informs them about the effects of aphasia on communication in general and provides CP with a list of generic strategies facilitating communication. The SLT further advises the CP generally to encourage the PWA to try to express herself. The proximal outcome that may be expected from this intervention is perhaps increased knowledge of aphasia and of existing generic facilitating communicative strategies, but there is a gap between the outcome projected by the tasks and activities and the objective of facilitating communication in a specific dyad. Although it is possible that the couple may try out different strategies from the list on their own, and thus find out which are facilitating for their communication, it is an outcome that is distal in relation to the activities performed in the intervention.

As a final illustration of the dyad-focused approach, we shall consider another example of a mismatch, where the main intervention objective is stated as better communication in general for the PWA. Some types of CPT interventions (e.g., Beeke et al., 2013) do include activities and tasks which train the PWA in using communicative strategies, which may be useful with many different CPs. However, the dyad-focused CPT approach would not be appropriate for this main objective, because the PWA probably experiences varying facilitators and barriers depending on the context and who the CP is. Thus, the projected outcome based on the intervention tasks and activities could not be expected to be better communication across multiple partners.

Generic communication partner training approach

In this CPT approach (Table 2), the provided intervention could be based on a program such as Supported Conversation for Adults with Aphasia (SCA^TM; Kagan, 1998). Examples of such studies may be found in, for example, Jensen et al. (2015) and Legg, Young, and Bryer (2005). The generic CPT approach is aimed at CPs only and the main objective is to enable participation in planning and assessment of healthcare for people with aphasia. That is, the intervention is aimed at healthcare staff, communicating with different people with different types of aphasia in a specific context such as a stroke unit. The intermediate goals in this approach are: increased knowledge about aphasia, and behavioural increase in appropriate use of generic supportive techniques and resources by the CP in their conversations with people with aphasia. Intervention tasks and activities consist in providing information about aphasia and supportive communicative techniques, recognition of such techniques through video analysis, and practice of their use in role-play or with volunteers with aphasia. In accordance with this, the projected proximal outcomes include CP’s knowledge concerning communication in aphasia and changes of CP’s behaviour due to an increased skill in the use of communicative techniques and resources. A possible distal outcome could be increased communicative participation for PWA as a patient in healthcare, and change in both the CPs’ and PWAs’ feelings about communication. The activities and tasks in this example of the generic CPT approach are also aligned with the main objective of the approach.

In the generic CPT approach, a mismatch between objective and activities and tasks would occur, if training was restricted to a non-interactive lecture. The assumption would then have to be that knowledge about aphasia and different techniques automatically makes CPs more skilled in using these techniques. However, having knowledge about different techniques does not necessarily mean that you are able to use them (Hausberg et al., 2012; Saldert, Forsgren, & Hartelius, 2016). Accordingly, an intervention where practice or role play is not included as part of the intervention activities would illustrate a gap between the goal of increasing skill in the use of communicative techniques and the activities and tasks of the intervention. With such a gap, the intervention might fail to lead to the projected outcome.

To sum up, while the two CPT approaches (dyad-focused and generic), described above and in Table 2, illustrate examples that are well aligned, we have here also given examples of applications of CPT where there is a mismatch between goals, activities and tasks in intervention, and projected outcomes. We have also tried to show how some outcomes are more proximal and others more distal in relation to the activities and goals. In both CPT approaches described here, the projected outcomes related to the intermediate goals of the specific approach are more proximal (e.g., improved knowledge of communication in aphasia) while the projected outcome related to the main objectives in both approaches is more distal (facilitated communication in a dyad and increased communicative participation of the PWA as a patient). It is also apparent that outcomes, which are well aligned with the goals, tend to be highly context specific. This is not surprising, since it is characteristic for CPT as a type of intervention that it is most often aimed at specific CPs in more or less specific contexts.

In the next section, some of the existing outcome measures used for CPT are discussed in more detail. The measures will be used to exemplify how different types of measures may or may not be aligned with the goals and training activities of a specific CPT intervention. For a more exhaustive presentation of the outcome measures used in CPT studies, see Simmons-Mackie and colleagues (2010, 2016).

Measuring outcome in dyad-focused and generic communication partner training

When selecting outcome measures, the researcher/clinician needs to ensure that there is an alignment between the underlying constructs measured by the assessment tools and the projected outcomes of the intervention. In the previous section, proximal and distal outcomes of different CPT approaches were presented more generally. To illustrate their application in practice, the following sections describe how outcomes might be measured with specific tools or methods of evaluation by referring to the same two types of CPT approaches as in the previous section.

Outcome measures in the dyad-focused communication partner training approach

In the dyad-focused CPT approach, the projected proximal outcomes concern change in knowledge and interactional behaviour and the more distal outcomes concern the participants’ feelings towards the dyad’s communication. Changes in frequency of specific behaviours are outcomes that may be observed by an independent rater, for example, an SLT, in a video recording. Increase in feelings of enjoyment or satisfaction with conversations are typically reported by the CP and/or PWA.

Assessment of knowledge of aphasia

Knowledge of communication and aphasia is often measured in a questionnaire, for example, by number of correct responses to factual questions or by qualitative analysis and evaluation of free text responses. For healthcare staff, The Knowledge of Aphasia Questionnaire (KAQ©; Simmons-Mackie et al., 2007) is such an instrument, see below. However, for family members and significant others of people with aphasia, there is not to our knowledge any internationally established instruments for evaluating knowledge and understanding of aphasia in relation to constructs targeted in CPT. There have been a few attempts to assess knowledge of aphasia with study-specific questionnaires; however, these instruments have usually not been evaluated in terms of validity and reliability (see, for example, Blom Johansson, Carlsson, Östberg, & Sonnander, 2013; Williams & Freer, 1986).

Quantification or rating of interactional behaviour

To assess change in targeted behaviours in the dyad’s interaction, frequency counts of discrete behaviours may be used. Despite a wariness of quantification in the Conversation Analysis tradition, frequency of treated behaviours, for example, topic initiations, repair, or pedagogic sequences, has been used as outcome measures within interaction-focused interventions (Best et al., 2016; Wilkinson, 2015; Wilkinson, Lock, Bryan, & Sage, 2011). The wariness of quantification arises from acknowledging that communicative interaction is a highly complex and context-dependent activity, where two instances of the same behaviour may have totally different implications when occurring in slightly different contexts within the same interaction (Hutchby & Wooffitt, 1998; pp 108–112; Schegloff, 1993). For example, the same behaviour may be functional in some contexts but dysfunctional in others. This may of course hamper the validity of the measure. The problem is that when a qualitative phenomenon is transformed into a construct, which allows quantification, the requirements for reliability of the measures raise problems in the assessment of conversational interaction (Eriksson et al., 2014). The validity of frequency counts is dependent on the reliability of the measure, which in turns depends on how well the targeted behaviours are defined.

Goal Attainment Scaling (GAS; Kiresuk, Smith, & Cardillo, 1994) is a type of rating scale which has been used to measure behavioural change as a proximal outcome from CPT (Eriksson, Forsgren, Hartelius, & Saldert, 2016). However, while the advantage of GAS is that it allows a highly individualized and contextualized approach to measurement, the method needs to be developed further in order to permit comparison of outcome between studies and patients (Tennant, 2007). Still, there is continued support and use of GAS within rehabilitation research (Krasny-Pacini, Hiebel, Pauly, Godon, & Chevignard, 2013). A well-established set of rating scales in CPT is the Measure of Skill in Supported Communication (MSC) and the Measure of Participation in Communication (MPC) (Kagan, Black, Felson Duchan, Simmons-Mackie, & Square, 2001; Kagan et al., 2004). The construct the MSC is designed to measure is the CP’s ability to use techniques or strategies, which are believed to facilitate the PWA’s ability to take an active part in conversation, as well as to understand and express himself. These rating scales will be further discussed below in relation to generic CPT.

In the first example of the dyad-focused CPT approach given above, one of the projected proximal outcomes was change in targeted behaviours. When choosing an outcome measure to evaluate change in targeted behaviours, the choice needs to be guided by whether only the CP’s behaviour has been targeted or if the goals are related to both participants. As mentioned above (Figure 2), when a CPT approach only targets the behaviour of the CP, change in the PWA’s behaviour is a distal rather than a proximal outcome, since it must be mediated by a change in the CP’s interactional behaviour. Further, when the performance of a targeted behaviour of the CP is dependent on the behaviour of the PWA, a measure focussing on the behaviour of the CP alone may provide confounded results as both the participants’ actions affect the interaction. For example, the targeted behaviour of the CP may be to suggest possible targets when a PWA experiences word finding difficulties. However, if the PWA does not produce any elaborated turns or experience any words retrieval problems, a measure focusing only on the CP’s behaviour is not sensitive or responsive enough to reflect a change in behaviour of the CP, although the CP might indeed have changed interactional behaviours.

Self-reported outcome in relation to behaviour and feelings/attitudes

If a particular behaviour is well chosen, well defined, and targeted in the training, this behaviour is probably the most well-aligned and appropriate outcome to measure in a CPT approach focusing on behavioural change. However, the main objective of the dyad-focused CPT approach described was to facilitate communication in specific dyads, and using frequency of occurrences of a specific CP behaviour as an outcome measure does not necessarily say anything about the impact of the change on the dyad’s interaction in general. How can we be sure that this change actually turned out to be a meaningful outcome for the dyad if we do not ask them? In Simmons-Mackie and colleagues’ (2010, 2016) reviews of CPT, only a few instruments measuring effects on participants’ feelings/attitudes towards their communication are reported. Most of these instruments only include a few items concerning feelings/attitudes, for example, KAQ© (Simmons-Mackie et al., 2007) and the Communication Outcome after Stroke scale (COAST; Long, Hesketh, Paszek, Booth, & Bowen, 2008). The lack of established instruments may perhaps explain why this is often not measured in CPT studies.

Some studies have used study-specific questionnaires. For example, based on information obtained during the initial interview and baseline phase of their intervention, Fox, Armstrong, and Boles (2009) formulated nine questions to probe a couple’s satisfaction with their conversations. The form included questions like “How enjoyable do you find your conversations?” and “How often do you avoid conversations with one another?” and the couple completed a rating scale for each question at several occasions during the intervention. Other study-specific questionnaires have been used by Lyon et al. (1997); Purdy and Hindenlang (2005), and Blom Johansson et al. (2013). For example, Lyon et al. (1997) used ratings by both PWA and CP regarding comfort and confidence along with the degree of functional communication. Purdy and Hindenlang (2005) gave a self-administered questionnaire to the caregivers (predominantly significant others) participating in their intervention, and a caregiver-administered questionnaire to their participants with aphasia. However, they distributed these questionnaires only post intervention, so they were less informative about the change induced by the intervention. And again, the measures used in these studies to capture participants’ evaluation or perception of change are study-specific measures and their validity and reliability have not been established.

Outcome in the form of a change in the participants’ behaviours may also be evaluated with self-reports in questionnaires or interviews. Instruments exist, which are intended to measure PWA’s or CP’s perception of change in communicative activity and/or participation in general, but not in particular dyads. See, for example, The Communicative Effectiveness Index (CETI; Lomas et al., 1989), The Aphasia Impact Questionnaire (AIQ; Swinburn, McVicker, & Cruice, 2012), and The Conversation and Communication Questionnaire for People with Aphasia (CCQA; Horton & Humby, 2014). However, regarding the first example of a study using the dyad-focused CPT approach above, the constructs measured in CETI, AIQ, and CCQA are quite distal or even unrelated to what has been targeted in the intervention described. For example, COAST (Long et al., 2008) has been shown to be a reliable and valid instrument and also has a version aimed for the CP (Carer-COAST; Long, Hesketh, & Bowen, 2009). Still, only nine of the twenty questions in the COAST may be related to the ability to interact in conversation and all of these items focus on the ability of the PWA in isolation. None of the questions concern the interaction in a specific dyad or the effect of the CP’s behaviour. The same goes for the Assessment of living with Aphasia (ALA; Kagan et al., 2011) and the disorder-generic short form of the Communicative Participation Item Bank (CPIB; Baylor et al., 2013). Again, these measures of participation in general may be too broad from a study-specific point of view to be sensitive to the outcome of our study example, because the measures aim to cover communicative life activities in general.

As pointed out by Coster (2013), it is important that intervention targets are represented in the constructs measured by the outcome measurement. If these targets are only represented in few items and embedded in a large number of less relevant items, the overall score of the instrument will lack in responsiveness to the intervention. It is also an important point for the researcher or clinician to consider that outcomes, like, quality of life, participation, or well-being may be subject to influence from a variety of factors in a person’s life, which are unrelated to the intervention. This creates a dilemma, since measurements of this type of multi-determined outcome may not be sensitive to the effect of an intervention, although they are indeed desired outcomes. This is not necessarily because the intervention has no effect on quality of life, but due to the fact that many other factors in the person’s life exert a substantial influence on this outcome.

It seems, then, that there are established instruments available to measure PWA’s or CP’s perception of change in communicative activity and/or participation in general. However, these instruments do not have a main focus on conversational interaction. To the extent that they contain items addressing conversation, these items are usually too generic to be sensitive to the individualized change targeted within a particular dyad.

A problem in using self-reports as outcome measures is that when CPs learn more about communication and aphasia, it is quite common that they report lower scores after intervention due to more knowledge and increased awareness (Rautakoski, Korpijaakko-Huuhka, & Klippi, 2008; Saldert, Backman, & Hartelius, 2013; Shewan & Cameron, 1984; Sorin-Peters & Behrmann, 1995). This is another issue that adds to the complexity in measuring outcome in CPT. Using self-report scores as outcome measure thus require the researcher or clinician to be prepared to treat both an increase and a decrease in scores as evidence of effect from the intervention.

Outcome measures in the generic communication partner training approach

The generic CPT approach described above aimed at teaching healthcare staff to adapt their use of supportive techniques in healthcare contexts. In measuring the proximal outcome from the examples using a generic CPT approach, again a measure of knowledge of communication and aphasia and also of feelings/attitudes towards communication with PWA would seem appropriate.

Knowledge of aphasia

As mentioned earlier, the KAQ© is such a questionnaire developed for healthcare staff (Simmons-Mackie et al., 2007; Sorin-Peters, McGilton, & Rochon, 2010). Some questions of the KAQ© are closed questions and use an analogue scale or 5-point Likert scale to establish self-assessed understanding of aphasia or the use of supportive strategies. Other questions are open-ended requiring respondents to describe their understanding “of aphasia” or “of the competence of people with aphasia”. Communicative Access Measures for Stroke (CAMS), developed by the Aphasia Institute in Toronto (Kagan et al., 2012; https://cams.aphasia.ca) is another set of measures that may be considered as potential outcome measures for CPT approaches targeting healthcare staff. CAMS include three surveys aiming to evaluate communicative accessibility at an institutional level, at front line staff level, and from a patient satisfaction level. Work on reliability and sensitivity to change is on the way for CAMS (Kagan, Simmons-Mackie, Victor, & Chan, in press), but has not yet been established for the KAQ©. This work appears to be lacking for most of the existing questionnaires, which target healthcare staff, including the professional and non-professional caregiver questionnaire used by Genereux et al. (2004) and its modification into the Communication Impairment Questionnaire (CIQ) used by McGilton and colleagues (2012).

Quantification or rating of interactional behaviour

Another proximal outcome in the exemplified generic CPT approach is CP’s behaviour and skill in use of different supportive techniques. Just as in the dyad-focused CPT approach, behaviour and skill are phenomena that may be directly observed by an independent rater in a video recording. This provides a more objective assessment of actual behaviours than asking CPs to report on their behaviour in a questionnaire. Specific well-defined behaviours may be counted, for example, the use of different supportive strategies. Alternatively, rating scales such as the MSC and MPC may be used (Kagan et al., 2001, 2004). One of the benefits of using a rating scale with established parameters is the possibility to use the same measure with several different participants and in different studies. However, a well-known problem with the use of rating scales for perceptual assessments of something as complex as conversational interaction is the challenge of obtaining acceptable reliability (Eriksson et al., 2014; Horton, Clark, Barton, Lane, & Pomoroy, 2016). Although the MSC and MPC has been shown to have a good reliability in some studies (Kagan et al., 2001, 2004), others studies have failed to establish reliability or raters have needed extensive training (Eriksson et al., 2014; Horton et al., 2016). As a consequence, several attempts have been made to adapt the MPC and MSC scales to allow an increased reliability. This has been done more or less successfully either by moving away from the original holistic measures by splitting up the original scales on several different parameters (Blom Johansson et al., 2013), or by combining the original scales to just one or two parameters and giving each minute of the recorded interaction a rating which is then summarized (Eriksson et al., 2014; Saldert et al., 2013).

Irrespective of whether the interaction assessed is between a PWA and their partner or healthcare staff, one important problem is the issue of comparing different conversational samples with each other. Natural interactions may be extremely variable in terms of content and balance in the participants’ contributions. Attempts have been made to control the context of the assessed samples by focusing on information transaction, for example, letting the PWA describe pictures or events not known to the CP (Hopper, Holland, & Rewega, 2002; Nykänen, Nyrkkö, Nykänen, Brunou, & Rautakoski, 2013). Another type of controlled task is to let the person with communication disorder and the CP discuss a specific, predetermined topic such as discussing recommendations meant for other people on where to go on holidays (Togher, Power, Tate, McDonald, & Rietdijk, 2010) or problem-solving tasks (Croteau et al., 2018). Still, these types of tasks and pre-set topics may form a context quite different from everyday natural interaction. An important issue is also how long a sample of conversation is needed to obtain a valid evaluation. Three to five minute segments (Correll, van Steenbrugge, & Scholten, 2010) or ten minute segments (Perkins, Whitworth, & Lesser, 1997) have been suggested depending on whether rating scales or qualitative assessment methods are used. The validity of the analysis of such limited samples is supposedly dependent on the content of the specific sample, though.

The main objective of the generic CPT approach is also related to a more distal outcome: Increased communicative participation for PWAs as patients in healthcare. The MPC (Kagan et al., 2004) provides an observational measure of how actively the PWA participates in a conversation with a specific CP. However, the concept of participation and how this may be measured has been under debate for a long time (Dijkers, 2010). Another way of conceptualizing participation is from the perspective of the stakeholders, including the PWA, and there is an increasing demand for measures of participation, which include self-report measurements (Brandenburg, Worrall, Rodriguez, & Bagraith, 2015; Eadie et al., 2006; Van de Velde et al., 2017). One rationale for this in relation to CPT is the fact that individuals with communication disorders may experience a high degree of satisfaction with their participation in a conversation, despite showing a minimum of observable behaviours like, for example, facial expressions, communicative initiatives, or topic elaborations.

Self-reported outcome in relation to behaviour and feelings/attitudes

In the application of the generic CPT approach, the main objective is the PWA’s participation, specifically in the planning and decisions about healthcare and effects on participation. However, this appears to be very distal to the primary focus of the intervention. Nevertheless, if this outcome is to be measured, a tool such as the short form of CPIB (Baylor et al., 2013) could be used. However, despite that the short form of the CPIB consists of ten questions concerning different aspects of conversation, only some of them would also be relevant in a healthcare context. Thus, the CPIB is likely to be lacking in responsiveness and to miss out on change actually present in the healthcare context. The patient version of the CAMS (Kagan et al., 2012) does address PWA’s perception of communication and participation in conversations with healthcare staff, and may perhaps turn out to be a sensitive tool to measuring change in this context from the perspective of the patient. However, this remains to be established.

Concerning the CP, a few of the items in KAQ© and CAMS concern feelings/attitudes concerning communication in relation to aphasia but otherwise there is a lack of established instruments to measure this outcome with healthcare staff, just as it is for family members or other significant others.

To sum up, despite the fact that various outcomes from CPT have been assessed for decades, there is no consensus on outcome measures suitable for measuring outcomes across different CPT approaches. There also seem to be gaps in the availability of established instruments for assessing both proximal and distal outcomes of CPT. Given that CPT approaches are context specific in their application and differ in specific intermediate goals or subgoals and intervention tasks, this is hardly surprising, since it is to be expected that measures of proximal study outcomes will differ between approaches. While these measures may very well be sensitive or responsive as well as valid and reliable, they do not appear to be suitable candidates for a shared outcome measure across CPT approaches. We shall therefore in the next section consider the possibilities of constructing a road map to arrive at such a measure.

Towards common tools for capturing key outcomes of communication partner training

As we have illustrated in the preceding sections, CPT is complex and an umbrella term for a range of intervention approaches with varied objectives and intermediate goals as well as diverse tasks and activities. However, all approaches and applications require behavioural changes in inter-related systems, within and between the participants as well as within the specific context, for example, a stroke unit or rehabilitation clinic. Accordingly, from a general research perspective, for example in healthcare, CPT can be described as a complex intervention (Craig et al., 2008). From a study or intervention-specific perspective it is therefore entirely appropriate and in agreement with Coster (2013) that the projected outcomes and the measurements chosen to assess these outcomes differ from one approach to another, or from one application of a specific approach to another. At a first glance, the need to measure outcomes which are well aligned with study goals does not appear reconcilable with the call in the field of aphasia for greater consensus on the outcome measures used in clinical trials; a call which is well motivated by the need to pool data and compare results across studies. However, as pointed out by Wallace et al. (2014), including one or more relevant evaluation tools from a shared set of outcome measures can be combined with the researcher’s assessment of more study-specific outcomes.

Consensus on outcome measures in aphasia rehabilitation and the relevance of these measures to communication partner training

A second part of the ROMA project has been to take steps towards obtaining consensus on the actual measurements to be used in aphasia trials (Wallace et al., 2016b), an endeavour shared by the European COST-funded network, Collaboration of Aphasia Trialists (http://www.aphasiatrials.org). Although the two groups have made different recommendations for the assessment of language functioning, there is concurrence in pointing to the Scenario test (van der Meulen, van de Sandt-Koenderman, Duivenvoorden, & Ribbers, 2010) or the Amsterdam–Nijmegen Everyday Language Test (ANELT) (Blomert, Kean, Koster, & Schokker, 1994) as suggested measures of functional communication (Visch-Brink, 2017).

These instruments may provide a direct, objective measure of performance in different types of communicative activity for a PWA who has participated in training, but the focus of both the Scenario Test and ANELT is on the communication of the PWA as an individual and based on his or her performance when interacting with the assessor in a scripted situation. Neither of these tools addresses constructs relating to change in partner behaviour or the consequences of such changes for the communicative participation of the PWA. Accordingly, they do not measure important proximal outcomes expected in CPT and are not optimal candidates for a shared outcome measure.

Consensus was also obtained in the ROMA project on the use of the Stroke and Aphasia Quality of Life Scale (SAQOL-39; Hilari et al., 2009) and the General Health Questionnaire-12 (GHQ-12; Goldberg, 1972) to evaluate quality of life and well-being. As mentioned previously, quality of life and general well-being are likely to be multi-determined outcomes, and as such they are likely to be distal and presumably not highly sensitive to the effects of interventions with a highly specified (as opposed to holistic) focus, including CPT.

It seems that for a group of interventions sharing a general purpose or main goal, such as different approaches to CPT, it ought to be possible to obtain consensus on measurement constructs, which are closer to the common goal of improving communication and participation in conversation for the PWA. A next step might be to develop appropriate measures for assessing these constructs, and then include such measures in addition to more study-specific measures in order to allow for comparisons of different approaches to CPT.

Core outcomes in aphasia rehabilitation

As mentioned earlier, Wallace and colleagues have carried out a series of studies elucidating what outcomes are of importance to different stakeholders, including people with aphasia and their families (Wallace et al., 2017a), service providers and managers in speech-language therapy (Wallace, Worrall, Rose, & Le Dorze, 2017b), and researchers in aphasia rehabilitation (Wallace, Worrall, Rose, & Le Dorze, 2016a). The results have uncovered constructs that seem important to measure from the perspective of CPT. For example, for people with aphasia findings from a nominal group technique showed that prioritized outcomes subcategories, such as to be able to help my communication partners communicate, to reduce communication breakdown and stress, to have complex conversations, and to be able to communicate independently and be understood by others (Wallace et al., 2017a, Table 3a, p. 1370). Similarly, for significant others themselves, there were prioritized subcategories such as for family to understand how to facilitate and support communication, to have tools to support communication, comprehension and cognition, to reduce communication breakdown, to have spousal conversation, and to have deeper conversation and in-depth discussion (Wallace et al., 2017a, Table 4, p. 1372). These subcategories seem well aligned with the goals of different approaches to CPT and therefore appear relevant as candidate constructs to be measured after intervention. What is missing, though, are outcomes more specific to the healthcare context and other professional CPs.

The research results of Wallace et al. (2017a), (2017b), 2016a) and the general methodology seems to be a great resource for initiating a further exploration of what are the desired outcomes viewed from different perspectives and adapted to different applications of CPT. For example, Mc Menamin, Tierney, and Mac Farlane (2015) explored which outcomes did the different stakeholders evaluated as important in relation to CPT involving trained SLT students’ regular visits with PWAs. Such an endeavour to establish core outcomes, which are important and meaningful in relation to CPT, may help establish the constructs, which should be measured in clinical trials. Unless such core outcomes are included in addition to more study-specific outcomes, significant change may be obtained on a measure in a trial, but the relevance and meaningfulness to stakeholders of the obtained results may be unclear.

Moving from underlying constructs to existing measurement tools, it is clear from the brief overview of outcome measures in the preceding sections that currently there is a lack of adequate measurement tools to assess many of these potential constructs. However, as pointed out by Hula, Fergadiotis, and Doyle (2014), this should not be a deterrent because the discussion should not necessarily be restricted to currently available outcome measurement tools, but include identification of gaps in available measurements and the need for development of new tools.

Inclusion of multiple outcome measures in studies of communication partner training

In research trials, it is sometimes recommended to have only one pre-specified primary outcome measure and to restrict the number of secondary outcome measures (Friedman, Furberg, DeMets, Reboussin, & Granger, 2015; Meinert, 2013). While these may be important concerns to keep in mind, the special case of complex interventions has been acknowledged within healthcare research (Craig et al., 2008) and expected outcomes of behavioural interventions are likely to be highly composite in nature. Consequently, it may be appropriate to use multiple measures to illuminate different perspectives, for example, the perspectives of different family members after systemic family therapy (De Los Reyes, Kundey, & Wang, 2011; Moran, 2017). In CPT, the perspectives of the PWA and the trained partner are both relevant in outcome evaluation, and their self-reported outcome is needed to complement a more objective rating or frequency count of behaviours carried out by the SLT or blinded judge.

Self-report measures as candidates for a shared outcome measure in communication partner training

Many of the measures reviewed in the previous section were objective assessments by an observer, for example, an SLT, who counts frequencies of specific behaviours or rates the interaction between the PWA and a trained partner. Other outcome evaluations were obtained by self-reports from the PWA or from the trained partner in questionnaires or interviews. Self-report measures from the PWA and/or significant other seem to be worth exploring as potential candidates for a common outcome measure in relation to CPT, since the overall goal of the intervention across different approaches is to facilitate communication for the PWA and his or her communication partners.

Within different fields of intervention, there is a strengthened focus on stakeholder views and the need to evaluate the meaningfulness of statistically significant changes obtained after an intervention to participants themselves (Staniszewska, Haywood, Brett, & Tutton, 2012; Wiering, de Boer, & Delnoij, 2017). Patient-Reported Outcomes Measurement Information System (PROMIS; (http://www.nihpromis.com) is an example of such an initiative, which aims to create and disseminate valid and reliable measurement instruments of patient-reported health and social well-being for different health conditions in adults and children.

Patient-reported or partner-reported measures in aphasia include well-established assessment tools such as the CETI (Lomas et al., 1989) and COAST/Carer-COAST (Long et al., 2009, 2008), which evaluate perceptions of functional communication in everyday life. However, as previously argued, the constructs measured in these instruments are quite distal or even unrelated to the goals and subgoals usually targeted in CPT.

Within the area of speech-language therapy, there have been several attempts to develop dynamic patient-reported outcome measures based on stakeholders’ experiences and appropriate for person-centered assessment of clinical trials, research, and clinical practice. One example of this is the CPIB (Baylor, Burns, Eadie, Britton, & Yorkston, 2011; Baylor et al., 2017, 2013; Baylor, Yorkston, Eadie, Miller, & Amtmann, 2009) and another, which is still in a more explorative phase, is the Aphasia Communication Outcome Measure (ACOM), measuring communicative functioning for persons with aphasia (Doyle, McNeil, Le, Hula, & Ventura, 2008; Hula et al., 2015). The stakeholders have been involved in the construction of the CPIB (people with communication disorders only) and the ACOM (both PWAs and their CPs) through surveys of candidate items to be included in the bank. The idea of an item bank, allowing for valid and person-centred assessment of different aspects of communication, seems to be a possible way forward in the construction of core outcome measures. Relevant constructs may address the transactional success or interactional well-being and engagement in both the PWA’s and significant other’s as well as, for example, healthcare staffs’ experiences, beliefs and wishes regarding their communication.

Further investigation is needed regarding the outcome constructs that are most meaningful for PWA and CPs. Based on this knowledge we may start to develop new outcome measures, which may be adapted to different CPT approaches, yet allow data pooling and comparison across different types of CPT interventions. The studies by Wallace et al. (2017a, 2017b, 2016a) have brought forward the field of aphasia by careful studies of outcomes, which are meaningful to stakeholders. A next step might involve further specification of constructs likely to be relevant to CPT with a large sample of people with aphasia and their families as well as healthcare staff and people with aphasia in institutional settings. Preferably, core outcomes across CPT approaches ought to be adaptable to individual preferences, different CPs and different contexts for communication. An important additional step in the development of new measures is to ensure the aphasia-friendliness of self-report instruments and to include the communicative support needed to use it with people with moderate-severe or severe aphasia.

However, issues remain with self-report measures. Even with such support, the validity of self-report measures may be questionable in severe aphasia. As mentioned earlier, another issue with self-report measures is that increase in knowledge and insights from CPT may affect self-ratings in a negative way (Rautakoski et al., 2008; Saldert et al., 2013; Shewan & Cameron, 1984; Sorin-Peters & Behrmann, 1995). Accordingly, observer-based or more objective assessments contribute necessary and complementary information to self-reported outcome assessments. Ideally, it would be optimal to assess similar constructs both from a self-reported perspective and from an observed perspective, as is possible within GAS (Eriksson et al., 2016; Kiresuk et al., 1994). Although there may be psychometric difficulties with such an undertaking, the possibility of creating item banks for GAS has been discussed within rehabilitation research (Kerckhofs, 2010; Tennant, 2007).

Clinical implications

Clinicians need to reflect on the alignment between objectives and included activities and tasks as well as choice of outcome measure for a specific CPT intervention. Both proximal and more distal outcomes should be measured and self-report measures for both the participating CP and the PWA have an evident value. However, as mentioned above, for people with severe aphasia, poor comprehension may present an obstacle to using self-report measures, despite provision of communicative support. This is a problem, since one of the advantages of CPT interventions is that they may be applied in clinical practice irrespective of type and degree of communication disorder. As in research, ideally, self-report measures should be supplemented with observational methods to provide both an internal and external perspective on change after CPT.

Many clinicians report on the use of only informal outcome measures for the assessment of outcome from CPT in clinical practice (Johansson, Carlsson, & Sonnander, 2011; Sirman, Beeke, & Cruice, 2017). From the proposed work on identification of relevant constructs to address and measure in CPT, new outcome measures may be developed and suitable evidence-based practices can be more readily implemented into the clinical realm. SLTs also need to develop new interventions, or revise existing ones, so that constructs important to stakeholders are adequately addressed.

Summary and conclusions

The overall purpose of CPT approaches is to facilitate communication between the PWA and relevant CPs through intervention including, or even exclusively, targeting the CP. This is one among several factors, which contribute to the complexity of measuring outcomes of CPT. It is also a factor, which makes it appropriate to assess the effects of CPT from several different perspectives, including both self-report and observational perspectives.

In assessing the effects of CPT, it is important to consider the alignment between the goals and objectives of a specific application of CPT with the tasks and activities used in the intervention (Coster, 2013). Furthermore, the proximal or distal relationship of projected outcomes with the activities and tasks need to be considered and the constructs to be measured as outcome needs to be identified. Although outcomes distal to the activities and tasks sometimes are more closely related to the main objective of the intervention, intervention effects are usually more likely to be demonstrated in proximal outcomes. Outcome from CPT has been assessed for decades and in numerous ways, but there remains a lack of appropriate measures suitable for evaluating outcomes from different CPT approaches. However, the identification of gaps in available measurements is an important step towards development of new tools. Item banks for GAS or for measuring communicative participation may be a promising way forward in developing new measurements based on key constructs related to CPT.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the Danish Council for Independent Research [Grant number DFF-4180-00046].