Feasibility, reliability and validity of a modified approach to goal attainment scaling to measure goal outcomes following cognitive remediation in a residential substance use disorder rehabilitation setting

Figures & data

Table 1. Modified GAS instructions and a hypothetical example.

Step		Instructions	Hypothetical example
1	Explanation	State: “Over the course of this study, we are interested in knowing whether your memory, thinking and self-regulation skills improve in day-to-day activities”.	E: “Over the course of this study, we are interested in knowing whether your memory, thinking and self-regulation skills improve in day-to-day activities”.
2	Goal selection	Hand the Goal Menu to the participant and explain: “Choose two goals from this goal menu that you would like to work on”	E: “Choose two goals from this goal menu that you would like to work on” P: Chooses goal 15, To remember_____ and specifies: “To remember to bring the things I need with me” (only one goal is exemplified here)
3a	Goal specification	State: “OK, so let’s make those goals really specific to you to make sure we are measuring real changes in your life” and ask about specific real-world examples for each of the selected goals.	E: “OK, so let’s make those goals really specific to you to make sure we are measuring real changes in your life. What is an example of a situation where you need to remember to bring the things you need?”
3b	Goal specification – documentation	Document the responses to questions in 3a for each selected goal	“To bring a pen and notepad to group sessions” is documented
4a	Establishing maximum realistic level	Refer to Maximum Realistic Level and Current Functioning Questions and ask the relevant MRL^d question for each selected goal	E: “In a typical week, how many group sessions do you attend?” P: “10”
4b	Maximum realistic level documentation	Document the responses to the questions in 4a	“10” is documented as the MRL^a
4c	Establishing current functioning	Refer to Maximum Realistic Level and Current Functioning Questions and ask the relevant CF^d question for each selected goal	E: “How many times per week do you remember to bring your pen and notepad to groups?” P: “5”
4d	Current functioning documentation	Document the responses to the questions in 4c	“5” is documented as the CF^b
5	Ensure five levels of measurement	Subtract 4d from 4b. If less than 4, double both 4d and 4b until difference is > = 4. Each time 4b and 4d are doubled, the measurement interval (denominator) should also be doubled. For example, “per week” doubled becomes “per fortnight” and “per fortnight” doubled becomes “per month”	10–5 = 5 (no need to double the values and denominator because 5 >  = 4)
6a	Calculation of the GAS^c levels/scale	Enter MRL^a and CF^b values from steps 4b and 4d (if difference is > = 4) or new values from step 5 (with difference > = 4) into the calculator at gas2.com.au	Enter the following values into the online calculator: MRL^a = 10, CF^b = 5
6b	Ensure goal achievability	Ensure the goal (“expected” outcome from the calculator output) is achievable. If not, reconsider the MRL^a and modify accordingly by repeating steps 4 and 5 with a more realistic MRL^a	The “expected” outcome of remembering to bring a pen and notepad to groups 8 times per week is achievable.
7a	Orientation to the GAS^c scale	Show the participant the calculator output on an electronic device or print or transcribe onto paper	+2 much better than expected 10 to 10 +1 better than expected 9 to 9 0 expected 8 to 8 −1 less than expected 7 to 7 −2 much less than expected 4 to 6
7b	Explanation of the GAS^c scale	Explain: “So, you’re currently [statement of behaviour] X out of a possible Y times per [interval]. However, it can be hard to motivate oneself to achieve something at the maximum realistic level when one is nowhere near that level currently. So, I suggest the target outcome be in between where you are currently functioning and that maximum realistic outcome. I have made some calculations to show you what I mean. You are currently [statement of behaviour] X times per [interval] and, as we discussed, the maximum realistic level is Y times per [interval]. So, a realistic goal for you might be to [statement of behaviour] Z times per [interval] (pointing to the GAS^c = 0 or ‘expected’ range). At the end of the trial we will be able to see whether you have achieved that goal, whether you made progress towards the goal but haven’t achieved it (pointing to the −1 GAS^c level), achieved more than the goal (pointing to the+1 GAS^c level), made no progress at all (pointing to the −2 GAS^c level) or achieved the maximum realistic level (point to the+2 GAS^c level)”	E: “So, you’re currently remembering to bring your pen and notepad to groups sessions 5 out of a possible 10 times per week. However, it can be hard to motivate oneself to achieve something at the maximum realistic level when one is nowhere near that level currently. So, I suggest the target outcome be in between where you are currently functioning and that maximum realistic outcome. I have made some calculations to show you what I mean. You are currently bringing your pen and notepad to groups sessions 5 times per week and, as we discussed, the maximum realistic level is 10 times per week. So, a realistic goal for you might be to bring your pen and notepad to groups sessions 8 times per week (pointing to the GAS^c = 0 or ‘expected’ range). At the end of the trial we will be able to see whether you have achieved that goal, whether you made progress towards the goal but haven’t achieved it (pointing to the −1 GAS^c level: bringing your pen and notepad 7 times per week), achieved more than the goal (pointing to the+1 GAS^c level: bringing your pen and notepad 9 times per week), made no progress at all (pointing to the −2 GAS^c level: bringing your pen and notepad 4 to 6 times per week) or achieved the maximum realistic level (point to the+2 GAS^c level: bringing your pen and notepad 10 times per week)”
8	Documentation of the GAS^c scale for target goals	Document the GAS^c scale in full, using the specified goal wording, ensuring a time-frame is stipulated. The SMART^d goal is represented by the expected outcome (GAS^c = 0) statement. This step can be completed later to save time during the goal setting process)	+2 I bring a pen and notepad to group sessions 10 times per week+1 I bring a pen and notepad to group sessions 9 times per week0 I bring a pen and notepad to group sessions 8 times per week-1 I bring a pen and notepad to group sessions 7 times per week-2 I bring a pen and notepad to group sessions 4 to 6 times per week
9a	Establishing maximum realistic level of control goal	Control goals should be set implicitly, so complete this and following steps after some interference task so as to dissociate the following questions from the goal setting process. Randomly choose a non-selected goal on the menu and ask the relevant MRL^a question for the chosen goal	Goal 6 on the Goal Menu is randomly chosen by the examiner, and the MRL^a question for that goal is asked, E: “In a typical week, how often do you need to be punctual?” P: “7” (only one control goal is exemplified here)
9b	Control goal maximum realistic level documentation	Document the responses to the questions in 9a for each selected control goal	“7” is documented as the MRL^a
9c	Establishing current functioning of control goal	Refer to Maximum Realistic Level and Current Functioning Questions and ask the relevant CF^b question for each selected control goal	E: “How many times per week are you currently punctual?” P: “4”
9d	Control goal current functioning documentation	Document the responses to the questions in 9c for each selected control goal	“4” is documented as the CF^b
9e	Ensure five levels of measurement of control goal	Subtract 9d from 9b. If less than 4, double both 9d and 9b until difference is > = 4). Each time 9b and 9d are doubled, the measurement interval (denominator) should also be doubled. For example, “per week” doubled becomes “per fortnight” and “per fortnight” doubled becomes “per month”	7–4 = 3. Doubled: 14–8 = 6 (the measurement interval is doubled from “per week” to “per fortnight”)
9f	Calculation of the GAS^c levels/scale	Enter MRL^a and CF^b values from steps 9b and 9d (if difference is > = 4) or new values from step 9e (with difference > = 4) into the calculator at gas2.com.au	MRL^a = 14, CF^b = 8 (per fortnight)
9 g	Ensure goal achievability	Ensure the goal (“expected” outcome from the calculator output) is achievable. If not, reconsider the MRL^a and modify accordingly by repeating steps 4 and 5 with a more realistic MRL^a	The “expected” outcome of being punctual 11 times per fortnight is achievable.
10	Documentation of the GAS^c scale for control goals	Document the GAS^c scale in full. This step can be completed later to save time during the goal setting process	+2 I am punctual 14 times per fortnight +1 I am punctual 12 to 13 times per fortnight 0 I am punctual 11 times per fortnight −1 I am punctual 10 times per fortnight −2 I am punctual 7 to 9 times per fortnight
11	Assessment of goal attainment	At the predetermined follow-up interval, ask about the frequency of the target and control goals over the relevant time period	E: “How often did you bring a pen and notepad to group sessions in the past fortnight?”; “How often were you punctual in the past fortnight?”

^aMRL = maximum realistic level.

^bCF = current functioning.

^cGAS = goal attainment scaling.

^dSMART = specific, measurable, achievable, relevant, time-based.

Table 2. Sample characteristics.

Characteristic		Control (n=12)		Intervention (n=13)		Overall (n=25)
		M	SD	M	SD	M	SD
Age		32.3	8.9	33.7	10.7	33	9.7
Education		11.8	1.6	11.7	2.5	11.8	2.1
TOPF^d		100.8	12.8	96.9	11.7	98.8	12.1
Years of regular use
	Alcohol	9.1	5.8	9.8	8.5	9.5	7.2
	Amphetamines^b	6.8	6.4	6.9	5.9	6.8	6
	Sedatives	3.8	5.8	2.8	5.6	3.3	5.6
	Cannabis	8.1	8	8.1	7.1	8.1	7.4
	Heroin	2	3.6	0	0	1	2.6
	Cocaine	1.4	3.4	1.3	4.4	1.4	3.9
Primary substance of misuse		%		%		%
	Methamphetamine	25		46.2		36
	Alcohol	41.7		23.1		32
	Sedatives	8.3		15.4		12
	Other Amphetamines	0		15.4		8
	Cannabis	16.7		0		8
	Heroin	8.3		0		4
Loss of consciousness after head injury		66.7		53.8		60
Hospitalised after head injury		58.3		30.8		44

^aTOPF = Test of Premorbid Functioning (Pearson, 2009).

^bincludes methamphetamine.

Table 3. Krasny-Pacini et al. (2016) criteria met in the current study.

Domain	Criterion	Criterion met	Comment
Content validity
	Collaborative goal setting	Yes	The first step of the GAS^b process involved giving the participant a choice of the goals they wanted to work on.
	Relevance/importance	No	Although selection of goals from the goal menu indicated relevance to the participant, an external judge was not asked to verify the relevance or clinical meaningfulness of the goals
	ICF^b classification of goal types	Yes	Documented in Results section
	Specificity	Yes	The 20 goals on the goal menu were related to the broad intervention target of improving executive functioning
Reliability of scale construction
	Equidistance of levels	No	Equality of levels was not verified by an external judge
	Preintervention performance	Yes	All goals had their baseline set at the −2 GAS^a level.
	Attainability/difficulty	No	Although an external judge did not verify the attainability/difficulty of the scales, the examiner and participant collaboratively appraised the attainability of the scale in step 6b of the method. An external judge would have reduced examiner bias
	Time-specificity	Yes	Time 3 (follow up) GAS^a data were defined as the outcomes.
Reliability of scale rating
	Interrater reliability	No	Outcomes were based on participant self-report
	Precise description of all levels	Yes	The calculator ensured all five GAS^a levels were clearly defined.
	Measurability	No	Outcomes were based on participant self-report
	Unidimensionality	Yes	The goal menu, calculator and overall method ensured only one variable was included per goal, and this was additionally confirmed by an external judge (first author)
	Context of measurement	No	Outcomes were based on participant self-report of their daily functioning
Other criteria
	Training	Yes	Training was provided to the examiner, who was given the opportunity to practice GAS^a scaling with corrective feedback prior to the trial
	Examiner bias	No	The examiner was involved in both setting/scaling of the goals and post-intervention scoring
	Statistical analysis	Yes	Non-parametric analyses were used to analyse the data and calculate an effect size
	Provision of a sample scale	Yes	Included three examples of full GAS^a scales

^aGAS = Goal attainment scale.

^bICF = International Classification of Functioning, Disability and Health.

Table 4. Examples of GAS scale across three activities and participation ICF subdomains.

GAS^a Level	GAS^a Descriptor	Learning and applying knowledge goal	General tasks and demands goal	Self care goal
+2	Much better than expected	To concentrate for 56 to 60 minutes in groups	To wake up at 6:30am 6 to 7 times per week	To practice mindful eating during 17 to 21 meals per week
+1	Better than expected	To concentrate for 52 to 55 minutes in groups	To wake up at 6:30am 5 times per week	To practice mindful eating during 13 to 16 meals per week
0	Expected	To concentrate for 47 to 51 minutes in groups	To wake up at 6:30am 3 to 4 times per week	To practice mindful eating during 9 to 12 meals per week
−1	Less than expected	To concentrate for 43 to 46 minutes in groups	To wake up at 6:30am 2 times per week	To practice mindful eating during 5 to 8 meals per week
−2	Much less than expected	To concentrate for 38 to 42 minutes in groups	To wake up at 6:30am 0 to 1 times per week	To practice mindful eating during 0 to 4 meals per week

^aGAS = Goal attainment scale.

Figure 1. BRIEF-A GEC (panel A) and target minus control goal outcomes (panel B) across the control and intervention groups.

Note: BRIEF-A GEC = Behaviour Rating Inventory of Executive Function – Adult version Global Executive Composite; T = Target; C = Control.

Pearson Assessment. (2009) . Advanced clinical solutions for WAIS-IV and WMS-IV: Clinical and interpretive manual. NCS Pearson Inc.

 Google Scholar

Krasny-Pacini, A., Evans, J., Sohlberg, M., & Chevignard, M. (2016). Proposed criteria for appraising goal attainment scales used as outcome measures in rehabilitation research. Archives of Physical Medicine and Rehabilitation, 97(1), 157–170. https://doi.org/10.1016/j.apmr.2015.08.424

 PubMed Web of Science ®Google Scholar

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request https://data.mendeley.com/datasets/3w3rb3stt2/1.