The feasibility of goal attainment scaling to measure case resolution in elder abuse and neglect adult protective services intervention

ABSTRACT

Objectives: This pilot study describes implementation procedures of goal attainment scaling (GAS) and examines the feasibility of using GAS to measure the multifarious intervention outcome of case resolution in elder mistreatment (EM) adult protective services (APS).

Methods: Substantiated EM victims (n = 27) were recruited prospectively from the State of Maine APS. An adapted GAS approach was implemented involving development of a pre-populated goal scale menu and web-based GAS application.

Results: The GAS menu comprised 18 goals and corresponding scales spanning several domains of case resolution: social support, service access, health/functioning, enhancing independence, and protective measures. The overall GAS process had mean length 33.8 min per case. The mean GAS summary t-score (54.3) aligned with theoretical expectations.

Discussion: Without a measure of case resolution, research cannot compare the effectiveness of different EM intervention models. Findings suggest that GAS is a feasible, client-centered strategy to measure the multifarious EM intervention case resolution outcome.

KEYWORDS:

• Adult protective services
• client-centered measurement
• elder abuse
• elder neglect
• elder self-neglect
• goal attainment scaling

Background and objectives

Elder mistreatment (EM) is recognized as a serious public health concern (Centers for Disease Control and Prevention [CDC], 2016) and top-priority aging issue among researchers, clinicians, and policy-makers (White House Conference on Aging, 2015). EM is broadly defined as an intentional act or lack of action by a person in a relationship involving an expectation of trust, which causes harm or risk of harm to an older adult (CDC, 2016). Several subtypes constitute EM, including physical abuse, emotional/psychological abuse, sexual abuse, financial abuse, and neglect by others (National Research Council, 2003). Approximately 9.5% of community-dwelling, cognitively intact adults age 60 years or older in the United States experience some form of EM each year (Pillemer, Burnes, Riffin, & Lachs, 2016). EM victimization is associated with serious consequences, such as premature mortality, poor mental and physical health, emergency service use, and hospitalization (Lachs, Williams, O’Brien, Pillemer, & Charlson, 1998; Yunus, Hairi, & Choo, 2017).

Although the EM literature has advanced in regard to basic EM science (e.g., prevalence, severity, risk factors, consequences), our knowledge of effective interventions remains limited (Pillemer et al., 2016). In particular, research available to inform practice in community-based EM interventions is scant (Ayalon, Lev, Green, & Nevo, 2016; Baker, Francis, Hairi, Othman, & Choo, 2016). Centralized state- or county-administered adult protective services (APS) represents the principal U.S. authority responsible for investigating and intervening on EM cases in the community. All states have developed an APS program that responds to EM, and all but one state has adopted mandatory reporting laws that require various populations to refer suspected EM cases to APS. APS programs respond to over 760,000 EM reports per year (National Adult Protective Services Association [NAPSA], 2012), and evidence would suggest that APS caseloads are rising (U.S. Government Accountability Office, 2011). Despite growing demand for APS, EM victims continue to be exposed to interventions that lack evidence of effectiveness (Ernst et al., 2014).

A major barrier constraining progress in EM intervention research is a lack of tools to measure intervention outcomes (Ernst et al., 2014; Stolee, Hiller, Etkin, & McLeod, 2012). Although EM screening/assessment instruments have been developed to indicate the presence of abuse/neglect (Beach et al., 2017; Gallione et al., 2017), the field lacks tools that measure the extent of case resolution over the course of intervention. Without a way to measure change in client or case status in response to the intervention, EM research cannot systematically compare the effectiveness of different intervention models.

Whether APS resolution of a case should be considered “a successful outcome” is difficult to measure due to its multifarious nature across cases (clients have different situations and wishes). Case resolution is defined by the core APS intervention-phase objectives to improve client safety and quality of life and to reduce the risk of EM revictimization (Administration for Community Living [ACL], 2016). The complexity arises in the implementation of those objectives. APS practice standards are grounded in a person-centered paradigm and guiding principles that include to

respect the integrity and authority of victims to make their own life choices; … take into consideration victims’ concepts of what safety and quality of life mean; and redefine success–success is defined by the victim, not what professionals think is right or safe. (NAPSA Education Committee, 2013, p.12)

Thus, “successful” case resolution is an intrinsically subjective and multifarious outcome, defined from the perspective of older adult EM victims and their varying definitions of appropriate solutions (Burnes, 2016). Indeed, very few clients choose to pursue absolute standards of case resolution defined by the elimination or cessation of EM revictimization risk or complete safety. Rather, the majority of EM victims choose to pursue a case plan that results in a reduction of revictimization risk – one that increases safety to some degree yet also preserves family relationships, maintains a sense of status quo, and/or does not expose a familial abuser to action by legal/justice systems. These client-generated formulations of case resolution often mean clients continue to be at some risk of revictimization upon case closure; such resolutions, carrying some ongoing degree of risk, are not perceived by clients or APS as failure (Burnes, 2016).

Standardized instruments are limited in their ability to measure multifarious outcome constructs, such as APS case resolution, since the underlying assumption of outcome construct uniformity across cases is violated. A standardized APS case resolution tool that implicitly evaluates all cases against some uniform benchmark of success (e.g., revictimization risk cessation, problem elimination, prosecution, separation from abuser) may not align with the construct of resolution generated by specific clients and would, therefore, have poor construct validity. Further, the nature of support (e.g., financial assistance, social support, health services, living placement, etc.) required to achieve case resolution varies widely across cases, depending on the unique needs and circumstances in a given EM situation (e.g., EM type, EM severity, victim–perpetrator relationship, living conditions, functional capacity, etc.) (Burnes, Pillemer, & Lachs, 2017; Burnes, Rizzo, Gorroochurn, Pollack, & Lachs, 2016). Therefore, indicators of the APS case resolution outcome construct are highly contextualized and uniquely constellated. A standardized tool that applies a fixed set of indicators to every case will conceivably miss salient case-specific factors (poor sensitivity) and/or lack responsiveness if meaningful change on a select few items is diluted by an absence of change on a host of other irrelevant, static items. Indeed, a systematic review of EM intervention research found that standardized outcome tools had difficulty detecting differences in client change across intervention groups (Ploeg, Fear, Hutchison, MacMillan, & Bolan, 2009).

Innovative measurement strategies that allow for client variability are necessary to measure APS intervention outcomes. The multifarious case resolution outcome construct requires a measurement strategy capable of tracking change on an individualized set of outcome indicators, which also accommodates varying standards of success across cases. Goal attainment scaling (GAS), a client-centered approach to measuring intervention outcomes, satisfies these measurement conditions (Burnes & Lachs, 2017) for the purpose of intervention research and/or tracking everyday APS practice.

GAS has been used extensively in other fields, including geriatric rehabilitation and community-based geriatric assessment settings. Most relevant to this study, it has demonstrated evidence of feasibility, reliability, validity, and responsiveness as an outcome measure for interventions that address complex, heterogeneous problems experienced by older adults (Rockwood et al., 2003; Stolee, Stadnyk, Myers, & Rockwood, 1999; Toto, Skidmore, Terhorst, Rosen, & Weiner, 2015). GAS has also been successfully employed as an outcome measure in randomized control trial studies seeking to identify effective intervention models targeting older adults in the community (Rockwood et al., 2003). With GAS, each case is assessed on a different, individualized set of goal items that align with a client’s specific needs and construction of success, yet a standardized summary t-score is generated that allows for comparisons across cases or groups (of cases) (Kiresuk, Smith, & Cardillo, 2015). Given its capacity to capture client-specific/salient items and accommodate outcome construct flexibility across cases, GAS represents a promising strategy to measure the central EM intervention outcome of case resolution (Burnes & Lachs, 2017). The current study sought to pilot the use of GAS in the EM APS intervention context. This article describes the GAS procedures adapted and developed for use in the APS context and examines the feasibility of implementing GAS as a measure of EM case resolution.

Research design and methods

Data collection

Data were collected from the state of Maine APS (MAPS). MAPS is a state-administered program within the larger Department of Health and Human Services. MAPS is responsible for investigating and intervening on EM cases toward the overall goal of client-driven case resolution. Eligible MAPS clients include adults age 18 or older who are incapacitated or dependent in any way upon others. A pilot sample (n = 27) of EM victims were recruited prospectively across six MAPS sites by their primary caseworker using an oral consent process. Study eligible participants were community-dwelling older adults age 60 or above with a case involving substantiated elder abuse (physical, emotional, sexual, financial) or neglect. This study also included substantiated cases of elder self-neglect since it is viewed as a closely related phenomenon (CDC, 2016) and considered a type of maltreatment by APS (ACL, 2016). The APS substantiation decision is based on a formal process incorporating evidence from client interviews, direct observation, physical signs/symptoms, third-party corroborating reports, and client history (ACL, 2016). Exclusion criteria were older adults living in institutional settings and those who refused APS intervention services.

GAS procedures

After APS investigation substantiated EM, and clients accepted APS services, GAS commenced. The APS practitioner and client collaborated to identify a mutually understood set of individualized goals toward the overall case resolution objectives of safety, quality of life, and revictimization risk alleviation. There was not a predetermined limit on the number of goals that could be set. In accordance with GAS methodology (Kiresuk et al., 2015), each goal was measured on a five-point scale (−2 to +2) reflecting varying expectations of success over the course of intervention, as defined collaboratively by the client–practitioner dyad. Each five-point GAS goal scale follows the same ordinal structure:

Table

Much less than expected	Somewhat less than expected	Expected client outcome	Somewhat better than expected	Much better than expected
-2	-1	0	+1	+2

For example, if victim social isolation represents a salient revictimization risk factor, then the following goal scale may be constructed to increase social engagement at the local senior center outside of the home: −2 (no attempt to attend community senior center programming), −1 (attends an activity at the community senior center once but chooses not to return), 0 (regularly attends one activity per week at the community senior center), +1 (regularly attends one activity type more than once per week at the community senior center), and +2 (regularly attends multiple activity types at the community senior center over multiple days of the week).

In the original GAS application, the client–practitioner dyad would develop five-point scales (e.g., wording for each scale level) from scratch for each client goal (Kiresuk, Smith, & Cardillo, 1994). However, this original application was found to be time-consuming in busy clinical settings (Mackay & Somerville, 1996). A commonly adapted GAS application involves the client–practitioner dyad selecting relevant goals and corresponding pre-populated five-point goal scales from a comprehensive menu of potential goals for specific client populations (Turner-Stokes, 2009; Yip et al., 1998). In this adapted version of GAS, the dyad only selects goals/scales from the menu that are relevant to a given client’s needs and circumstances. Once selected, there is flexibility to apply goal scales in their pre-populated template form or edit the wording to align with a client’s circumstances and expectations of success. If the client–practitioner dyad identifies a salient goal that is not on the menu, then a corresponding five-point scale must be developed from scratch. This adapted GAS application was adopted by the current study to enhance feasibility in the time-constrained APS context.

At MAPS, client cognitive capacity is assessed through formal evaluation by a clinical psychologist or, in some cases, the client’s primary care physician. Goal setting with clients assessed as lacking capacity followed a three-level protocol based on the client’s ability to participate in a goal-setting process and the availability of a trusted third-party substitute decision-maker. At the first level, clients assessed as maintaining the capacity to express preferences and engage in goal setting were directly involved in the GAS process of establishing goals and goal expectations. At the second level, in cases involving clients assessed as lacking the capacity to participate in goal setting, the practitioner collaborated with an available private guardian. Practitioners were comfortable working with a private guardian to set goals that were believed to be in the client’s best interest, unless a guardianship court study assessed the guardian as unfit. At the third level, in cases involving a client who lacked capacity and was under public (APS) guardianship without an external substitute decision-maker available, GAS became primarily APS driven. To enhance accountability and reduce conflict of interest, the practitioner collaborated with a third-party supervisor to establish goals, as opposed to working alone.

The APS practitioner scored the status of the case against each five-point goal scale at baseline (T1) and follow-up (T2). T1 was defined by the time when goal scales were initially established. Depending on specific client circumstances and client–practitioner rapport, the APS practitioner used professional judgment to determine when it was appropriate to engage in a process of identifying and discussing goals and, in turn, establishing goal scales. Since some APS cases remain open for long periods of time, follow-up T2 assessment occurred at the point of case closure or 6 months post-T1, whichever came first. Based on APS feedback, 6 months was anticipated as enough time for change to occur on client goals. At T1 and T2, each goal scale score was entered into the GAS formula as follows, which generates a standardized case summary t-score (Kiresuk et al., 2015):

,

where Wi is the weight assigned to the ith goal, Xi is the numerical score achieved on the ith goal, and ρ is the expected correlation of the goal scores. Barring a considerable clinical advantage to using weights, goal weighting has the potential of introducing bias to ordinal-level data computation (Tennant, 2007), and the GAS literature generally recommends using unweighted procedures. Thus, goals were weighted equally (Wi = 1). As the first integration of GAS in the EM APS context, this pilot study applied the conventional and widely adopted GAS assumption of ρ = 0.3 (Kiresuk et al., 2015).

To further enhance GAS feasibility in the APS context, the GAS process was implemented through web-based app technology that practitioners could access on a tablet or office computer. The app allowed APS practitioners to select pre-worded goals and corresponding scales from a menu without needing to carry or sift through several sheets of paper. It facilitated flexible and expedient goal scale editing/recording without having to write, cross-out, and rewrite edits on paper. This technology also calculated GAS t-scores.

Development of a pre-worded menu of goals and goal scales

Consistent with the adapted GAS version described above, a menu of goals and corresponding pre-populated scales was developed through successive stages. The initial stage involved a 2-day meeting spanning 12 hrs with eight practitioners from five MAPS sites to brainstorm common client-driven goals in EM cases. For each goal, the group collaborated and reached consensus on generating a corresponding, clinically relevant five-point scale. From this initial stage, a menu containing 14 pre-populated goals/scales emerged. As a second stage of input, the 14 goals/scales underwent review by an external MAPS group constituting a supervisor, program administrator, and director, which resulted in goal scale revisions and deletion of two goals/scales that were not viewed as plausibly client-driven. In a third stage of menu development, three MAPS staff (two practitioners, one program administrator) collaborated over a 1-day meeting to revise the goal/scale menu. During this meeting, five goals/scales were added to the menu with particular relevance to MAPS clients under some form of guardianship. Again, the updated goal/scale menu underwent review by the external MAPS group. As a final stage of development, the original eight MAPS practitioners were asked to field-test the menu of 18 goals/scales with incoming clients over a 2-month period. No additional goals were identified during this in vivo field-testing period. Having reached a stage in which no new goals were added, the menu was considered exhaustive and to contain a high degree of face validity. The study PI (DB) facilitated each stage of menu development.

Analytic plan

The developed goal menu was described in regard to goal domains and titles. GAS feasibility was examined using several indicators, including amount of time required to identify/discuss goals with clients and create and score the goal scales in the app; number of goals pursued with clients; extent of goal scale editing required; and proportion of goal scales that could be scored. A normal distribution of final (T2) GAS summary t-scores is expected with a mean of 50 and standard deviation (SD) of 10 (Kiresuk et al., 2015). Demonstrating that the sample mean T2 GAS t-score is close to 50 represents an acceptable feasibility check of practitioners’ ability to set achievable/realistic goals in a given context (Turner-Stokes, 2009).

Results

The sample (n = 27) was mostly female (66.7%) and predominantly Caucasian (96.3%) with a mean age of 75.9 years (range: 60–94). Most participants were widowed (40.7%) or separated/divorced (29.6%), while few were married (14.8%) or single/never married (7.4%). A minority of participants (11.1%) lacked capacity. Cases included substantiated self-neglect (63.0%), financial abuse (29.6%), emotional abuse (18.5%), neglect (18.5%), and physical abuse (11.1%), with 29.6% of cases having more than one subtype.

Table 1 presents the list of goal titles developed through the iterative process with APS stakeholders for the preexisting GAS menu, which are categorized into the following domains: social support (two goals), service access (two goals), managing health and functioning (six goals), enhancing independence (four goals), and protective measures (four goals). Goal scale template examples from the menu are provided in Supplementary Material. Of the 49 goals established across cases, the most common goals included accessing benefits or services (24.5%), placement in an alternative setting (22.4%), minimizing a perpetrator’s access to financial resources (12.2%), and accessing health care (8.2%).

Table 1. Frequency of goal types utilized from preexisting goal attainment scaling menu.

Goal menu	n (%)
Social support
Social engagement or community integration	2 (4.1)
Connecting with natural supports^a	0 (0)
Service access
Accessing “benefits or services” requiring an application^b	12 (24.5)
Accessing health care services	4 (8.2)
Managing health and functioning
Medication compliance	2 (4.1)
Substance abuse support	1 (2.0)
Maintaining independence with ADLs/IADLs	1 (2.0)
Placement in an alternative setting^c	11 (22.4)
Finding private guardian to help make decisions	0 (0)
Establishing end-of-life decisions	0 (0)
Enhancing independence
Enhancing personal guardianship and detaching from external guardianship	0 (0)
Gaining greater control over personal finances	0 (0)
Obtaining employment	0 (0)
Moving to a living setting with greater independence	0 (0)
Protective measures
Safety planning	1 (2.0)
Victim–perpetrator living arrangement separation – evict perpetrator	3 (6.1)
Minimizing perpetrator access to financial resources	6 (12.2)
Enhancing safety in living environment	3 (6.1)
Blank template	3 (6.1)

ADL = Activities of daily living; IADL = Instrumental Activities of daily living. Total number of goals established across 27 cases was 49.

^aNatural supports could include family, friends, neighbors, church community, etc.

^bGoal can be used for several benefits or services requiring an application, including assertive community treatment, case management, meals on wheels, food pantry, food stamps, Maine Care, veteran affairs, social security, general assistance, homemaker, visiting nurse, and legal services

^cAn alternative setting may include assisted living, nursing home, residential care, or a group home

APS practitioners spent a mean of 11.5 min (95% confidence interval [CI]: 7.3–15.6) to identify and discuss each goal with clients, 3.2 min (95% CI: 1.9–4.4) to create each goal using the app, 1.9 min (95% CI: 0.6–3.1) to score each goal in the app at T1, and 0.9 min (95% CI: 0.7–1.1) to score each goal at T2. The practitioner–client dyad created, on average, 1.8 (95% CI: 1.4–2.3) goals per case. The mean amount of time spent on GAS per case was 33.5 min (95% CI: 17.5–49.4, range: 5.0–202.5). Of the 49 goal scales established across cases, 30 (61.2%) were taken directly from the menu template without editing, 16 (32.7%) were taken from the menu with editing, and 3 (6.1%) were generated from scratch. Of the 49 goal scales set and scored at baseline, 47 were scored at follow-up; the two remaining goals became irrelevant and non-applicable to the client’s situation over time. The mean T2 GAS summary t-score was 54.1 (95% CI: 49.3–58.9, SD = 12.1). The distribution of T2 GAS summary scores was normally distributed (Kolmogorov–Smirnov test >0.05), as illustrated in a normal Q-Q plot (Figure 1), with slight negative skew (skewness = −0.6).

Figure 1. Normal Q-Q plot of goal attainment scaling summary t-scores at T2.

Discussion and implications

This study piloted the feasibility of implementing GAS as a client-centered strategy to measure the EM intervention outcome of case resolution in the APS context. GAS is capable of capturing varying constructions of success as defined by clients and tracking change on an individualized set of salient outcome indicators. This article described the implementation procedures of an adapted version of GAS in which goals and scales were selected from a preexisting/populated menu and the GAS process was facilitated using a web-based app.

Findings from this study suggest that GAS is a feasible measurement strategy to implement in the APS context. On average, the overall GAS process, including up-front goal identification/discussion, goal scale creation, and goal scoring, took just over half an hour with each case. Feedback from MAPS practitioners involved in the study indicated that this amount of time is reasonable within their overall scope of practice, especially since GAS compliments existing practice, rather than representing an additional task. Treatment planning and goal discussion are common components of existing everyday APS practice (ACL, 2016); GAS provides a framework to formally organize and measure progress on this work.

The adapted GAS application characterized by a preexisting menu of goals and pre-populated goal scales was a useful approach. Virtually all goals used in the GAS process were extracted from the menu. Practitioners used template goal scales directly or as a springboard to make case-specific edits. A limitation of the adapted GAS application, however, is that it departs from a pure individualized approach in which the practitioner and client have an opportunity to develop client-generated goals/scales from scratch. However, based on MAPS practitioner feedback during initial study design, it was anticipated that this original GAS application would be infeasible in the time-constrained APS context.

The sample distribution of T2 GAS summary t-scores aligned with broader GAS methodological expectations. In theory, given a large enough sample, the distribution of final GAS summary scores should be normally distributed around a mean of 50 and an SD of 10 (Kiresuk et al., 2015). A sample mean summary score less than 50 is indicative that practitioners have set goals that were unrealistic or too challenging. Conversely, a sample mean summary score greater than 50 is suggestive of goals that were too easy or of potential scoring inflation (Turner-Stokes, 2009). Even with a relatively small current pilot study sample size, the mean GAS summary score closely approximated the theoretically expected mean and SD. Findings suggest that APS practitioners were able to administer GAS in a clinically pragmatic and objective manner.

The level of client involvement in the GAS process requires further clarification. In the current study, the client–practitioner dyad engaged in a collaborative process to identify a mutually understood set of goals and goal expectations that aligned with the client’s objectives. However, best practice around whether or not to include EM victims in actual construction and scoring of five-point scales is unclear. MAPS practitioners tended to construct and score scales independent from clients. For some clients, learning the underlying ordinal scale structure or other scale construction intricacies was seen as unnecessarily burdensome or confusing. The scaling/scoring procedure was also perceived as potentially introducing a sense of judgment around traumatic or personal situations. Conversely, for some clients, further engagement in the GAS scaling process could result in a higher level of ownership and commitment over goals. Further research is required to clarify if and under what circumstances clients should be directly involved in the goal scaling/scoring process. As described above, despite tending to score goals independent from clients, the distribution of GAS summary scores was not indicative of self-scoring inflation/bias.

The GAS process itself may contain therapeutic value, in addition to serving as an outcome measure. Indeed, evidence suggests that formal goal-setting facilitates behavioral change (Locke & Latham, 2002). Thus, to balance the potential therapeutic effects of GAS across comparison groups in intervention research, studies should implement formal GAS training and quality assurance components to ensure exposure to similar GAS procedures across groups.

The current pilot study contained limitations. First, reflecting Maine’s older adult population (U.S. Census Bureau, 2015), the study sample was predominantly Caucasian. It is essential that further GAS research also occur in ethnically/racially diverse locations to understand the extent to which EM victims construct culture-specific goals and definitions of success. Second, for this initial study, the process of developing the GAS goal menu did not include direct input from EM victims. To truly construct a client-driven menu of goals/scales, further research is necessary that integrates information from EM victims themselves. Third, similar to many APS programs in the United States, MAPS does not work directly with suspected perpetrators. Therefore, the current GAS menu does not contain goals/scales reflecting direct practice with them. EM intervention theory recognizes a need to approach practice from an ecosystemic perspective that addresses vulnerabilities of the individual victim, the perpetrator, the victim–perpetrator relationship, as well as the surrounding environmental context (Burnes, 2016; Mosqueda et al., 2016). MAPS practitioners also report that victims often articulate a desire to obtain treatment/support for their familial perpetrator or others. Future GAS research should occur in APS settings that work with perpetrators or others in victims’ environments and introduce perpetrator-centric goals/scales to the menu. Fourth, research is required to examine GAS measurement validity, reliability, and responsiveness in the EM intervention context. Finally, in study exit interviews, MAPS workers reported interest in simplifying the pre-worded scales, using major category headings and options less limiting than fully pre-worded template options. To this end, future research with APS could examine the utility of simplified scales or other existing GAS adaptations in the literature (Turner-Stokes, 2009).

This pilot study described GAS implementation procedures in the APS context for the first time and provided evidence of feasibility and insights for how GAS might be better adapted to the APS context. It also represents one of very few prospective APS studies in the literature that recruits incoming EM victims and that does not exclude cases involving clients who lack capacity.

Acknowledgments

We would like to thank David Zimmerman and Kathryn Hester from the Elder Justice Foundation. We would like to thank the following partners from Maine Adult Protective Services for their considerable efforts: Ricker Hamilton, Sheryl Clark Nadell, Brian McKnight, Michael Parks, Joan Strandbygaard, Richard Keegan, Shannon Smith, Sherri Wakeling, Dena Kenney, Joanne Cookson, David Greenleaf, Elizabeth Crossman, Lori Chamot, Michael Libby, Diana Hopkins, and Jodi Liberty.

Additional information

Funding

This work was supported by the Elder Justice Foundation and the Social Sciences and Humanities Research Council (SSHRC) of Canada [430-2015-00785] to DB.