Prioritization of rehabilitation Domains for establishing spinal cord injury high performance indicators using a modification of the Hanlon method: SCI-High Project

Abstract

Objectives: To prioritize Domains of SCI Rehabilitation Care (SCI-Care) based on clinical importance and feasibility to inform the development of indicators of quality SCI-Care for adults with SCI/D in Canada.

Methods: A 17-member external advisory committee, comprised of key stakeholders, ranked 15/37 Domains of rehabilitation previously flagged by the E-scan project team for gaps between knowledge generation and clinical implementation. Priority scores (D) were calculated using the Hanlon formula: $\mathrm{D}=\left[\mathrm{A}+(2\times \mathrm{B})\right]\times C$, where A is prevalence, B is seriousness, and C is the effectiveness of available interventions. A modified “EAARS” (Economic, Acceptability, Accessibility, Resources, and Simplicity) criterion was used to rank feasibility on a scale of 0–4 (4 is high). The product of these two scores determined the initial Domain ranking. Following the consensus process, further changes were made to the Domain rankings.

Results: Despite a low feasibility score, Sexual Health was ranked as high priority; and, the Community Participation and Employment Domains were merged. The 11 final prioritized Domains in alphabetic order were: Cardiometabolic Health; Community Participation and Employment; Emotional Well-Being; Reaching, Grasping, and Manipulation; Self-Management; Sexual Health; Tissue Integrity; Urinary Tract Infection; Urohealth; Walking, and Wheeled Mobility.

Conclusions: The modified Hanlon method was used to facilitate prioritization of 11 of 37 Domains to advance the quality of SCI-care by 2020. In future, the Spinal Cord Injury Rehabilitation Care High Performance Indicators (SCI-High) Project Team will develop structure, process and outcome indicators for each prioritized Domain.

Keywords:

• Prioritization
• Rehabilitation
• Spinal cord injuries
• Health care quality indicator

Introduction

Concerns about the quality of care for individuals with spinal cord injury/disease (SCI/D) extend from injury onset, through emergency transfer, diagnosis, surgical consultation and early operative management, to post-acute care/tertiary rehabilitation and transition into community.¹ Considerable variation exists in care provision and outcomes across specialized post-acute and tertiary rehabilitation services.² Variation across settings in type, intensity and duration of rehabilitation services hinders identification of the optimal trajectories of recovery and the ability to monitor the quality of care across tertiary rehabilitation centers and health systems.³ After SCI, the occurrence of secondary health conditions⁴ results in multi-morbidity, premature aging,⁵ and impaired community participation⁶ over an individual’s lifespan. Thus, there are potent reasons to optimize Spinal Cord Injury Rehabilitation Care (SCI-Care) to enhance long-term health and wellbeing.

To date, there has not been a consensus-based understanding regarding what constitutes quality SCI-Care.^7,8 A key aspect of quality SCI-Care is to target the need for customized patient-centered care to improve the patient experience. Recently, there have been several initiatives to identify consumer priorities; however, they did not identify priority Domains for Quality Improvement and related indicators.^9,10 In other rehabilitation jurisdictions, pay for performance strategies have been shown to promote patient-centered care and impact the quality of care.¹¹ The quality of SCI-Care is not solely driven by patient priorities, and should reflect advances in care at the health system (macro) level, organizational (meso) level, and individual patient (micro) level. Learning health systems facilitate quality care by tracking indicators of quality care and providing feedback to a variety of stakeholders from diverse perspectives in order to establish frameworks for interpreting indicator data, and establishing benchmarks of quality care to identify exemplary performers^12,13 and embed these health standards into policy and routine practice.

To date, much about the quality of SCI/D rehabilitation has been driven by inpatient length of stay,¹⁴ heterogeneity in patient impairments,¹⁵ and medical co-morbidity,¹⁶ while neglecting: 1) the fate of patients following discharge from the inpatient rehabilitation environment and 2) patient perspectives on their lived experience and rehabilitation priorities.

Indicators are explicitly defined as measurable elements of practice performance for which there is evidence or consensus support.¹⁷ Indicators of quality care can be categorized as either structure, process or outcome indicators.^18,19 Structure indicators are defined by the properties of the setting in which the health care occurs.¹⁸ Process indicators describe what is actually done in giving and receiving SCI-Care, while an outcome indicator reflects the patient’s mortality, morbidity, health status, health-related quality of life or satisfaction within their overall SCI-Care.¹⁹ The current lack of comprehensive quality SCI-Care indicators for the SCI/D population results from a failure to prioritize measurement of the elements which are most important to advance quality care and reduce the gaps between current knowledge and practice implementation.¹⁷

A fundamental tenant of health system planning is to establish priorities for the allocation of available resources in order to achieve the greatest social impact.²⁰ To help articulate SCI-Care priorities, Craven et al. conducted a scoping review of Canadian SCI rehabilitation services utilizing a framework for describing the processes of SCI care delivery.²¹ Thirty seven Domains of SCI-Care were derived from the International Classification of Function, Disability and Health framework and opportunities for research, best practice and health policy implementation were identified. The enclosed work advances the E-Scan initiative by establishing community priorities based on importance and feasibility of SCI-Care Domains within the E-Scan rehabilitation framework.²²

Given the time required to implement practice change, resource constraints imposed by finite health systems funding, and the importance of addressing consumer’ priorities, the Spinal Cord Injury Rehabilitation Care High Performance Indicators (SCI-High) Project Team was formed to prioritize Domains for Quality Improvement. The SCI-High Project is a bold initiative to establish a comprehensive, yet succinct, set of indicators to facilitate uniform measurement and benchmarking of prioritized SCI-Care Domains within the first 18 months after tertiary inpatient rehabilitation admission among adult Canadians by 2020. This manuscript describes the process to prioritize and refine SCI-Care Domains for the adult Canadian SCI/D population.

Methods

The core SCI-High Team consists of three scientists with experience in rehabilitation (CC, MB, SH), health systems and implementation research, two of whom also have program leadership experience (CC and MB), one advance practice leader/implementation specialist (HF), one clinical epidemiologist (MA), and a research coordinator (FF), all with relevant expertize in SCI-Care. Based on the available SCI-High Project resources and feasibility estimates, the project Team’s aim was to develop, implement and evaluate one structure, process and outcome indicator for 10 SCI-Care Domains (30 indicators total).

The process for prioritization of SCI-Care Domains included three steps: 1) Narrowing of SCI-Care Domains for prioritization; 2) Convening an External Advisory Committee (EAC); and 3) Prioritization by utilizing a modified Hanlon Method followed by a facilitated consensus process.

Narrowing of SCI-care Domains for prioritization

The E-Scan project rehabilitation framework identified 37 Domains of SCI-Care.²² The SCI-High Project Team reviewed all of the Domains and then restricted the number of Domains for prioritization consideration to those with significant gaps between knowledge generation (research evidence) and clinical implementation (i.e. those Domains deemed to have opportunity to advance care by 2020).

The SCI-Care Domains with no new knowledge to drive clinical implementation were excluded (e.g. the bowel continence Domain demonstrated a persisting lack of new research).²³ Applying this “gap criterion”, 15 of the original 37 Domains were selected for Domain ranking including: Neuropathic Pain Amelioration; Urohealth; Cardiometabolic Health; Community Participation; Emotional Well-Being; Employment & Vocation; Independence in Breathing; Informed Self-Management; Sexual Health; Skeletal Integrity; Tissue Integrity; Reaching, Grasping, and Manipulation; Walking; Wheeled Mobility; and Urinary Tract Infection (UTI). UTI was not a Domain identified in the initial E-Scan, but was a priority for the Rick Hansen Institute (study sponsor) and many tertiary SCI/D rehabilitation hospitals for prevention of Hospital-Acquired Conditions, and thus, was added by the investigators to the SCI-Care Domains as Urinary Tract Infection for consideration in the prioritization process.

Convening the External Advisory Committee (EAC)

The EAC was comprised of 17 invited subject matter experts all Canadian from centers located in urban areas, including: scientists (n = 7), clinical experts in SCI-Care (n = 7), health-care administrator (n = 4), data expert (n = 2), representative from consumer and stakeholder organizations such as Accreditation Canada (n = 1), the Rick Hansen Institute (n = 1), the Ontario Neurotrauma Foundation (n = 1), the health policy maker (n = 2), and individuals living with SCI/D (n = 3), while each person might have more than one role in the EAC. The EAC was assembled on October 25th, 2015 to rank selected SCI-Care Domains using a modified Hanlon methodology for ranking of health care priorities. In this phase of the SCI-High Project, both a quantitative and consensus-based approach were used. The modified Hanlon Method²⁴ was used to individually rank the 15 Domains of SCI-Care.

Prioritization using the modified Hanlon Method

The “Hanlon Method for Prioritizing Health Problems” also called “Basic Priority Rating Model”, developed by J.J. Hanlon,²⁴ is an established technique to objectively and explicitly consider defined prioritization criteria and feasibility factors. We used a modified Hanlon Method to rank Domains of SCI-Care by means of a two-step prioritization and feasibility assessment. Each Domain of SCI-Care is rated on a scale of zero to ten, based on three specific considerations: 1) size or prevalence of the issue, 2) the seriousness of the Domain, and 3) the effectiveness of interventions for that Domain. A scale of 0–10 is a reliable rating scale to quantify qualitative questions and raters’ perception thereby increasing scoring variability and scoring differentiation.²⁵ To provide EAC members evidence to inform their scoring of the three criteria, data from the relevant E-Scan chapter, Spinal Cord Injury Research Evidence (SCIRE) systematic review.²⁶ Clinical Practice Guidelines,²⁷ (for each Domain) were collated for review and scoring. The priority score for each SCI-Care Domain was calculated using the formula, $\mathrm{D}=\left[\mathrm{A}+(2\times \mathrm{B})\right]\times C$, where D is the Priority Score, A is the score for size, B is the score for seriousness of SCI-Care Domain ranking, and C is the score for the effectiveness of interventions. The B or seriousness score is multiplied by two because it is considered twice as important, according to the original Hanlon technique.

Additionally, slight changes were made to the Hanlon feasibility assessment method. We added accessibility as a criterion for scoring of the ‘EAARS’ criteria (shown below) to derive a feasibility score:

Economics – Does it make economic sense to address the problem? Are there economic consequences if a problem is not carried out?

Acceptability – Will the SCI/D community accept the program? Is it wanted?

Accessibility – Are services available for inpatients and/or outpatients?

Resources – Is funding available or potentially available for a program?

Simplicity– How easy is it to implement the therapy/program?

The total score for each SCI-Care Domain was the product of the priority and feasibility scores.

An overview of the entire prioritization process is shown in (Fig. 1).

Figure 1 Prioritization process: (a) Domains of SCI-care with significant gaps between knowledge generation and clinical implementation; (b) description, size (incidence and/or prevalence), seriousness, and effectiveness of the rehab care/treatment/interventions for each Domain; (c) mock ranking using Modified Hanlon Method; and (d) Domain of SCI-care were ranked using Modified Hanlon Method.

The EAC met in October 2015 in a one day (8 h) workshop. The meeting was comprised of an orientation session, practice ranking using the modified Hanlon method and then the active ranking of SCI-Care Domains. To support this process, a booklet was provided to each EAC member prior to the meeting that contained the collation information on the most recent SCI research and best practice guidelines^26,28,29 information about the size (incidence and/or prevalence), seriousness, and effectiveness of treatment interventions for each SCI-Care Domain. A brief orientation document was provided outlining the process for prioritization of Domains using the modified Hanlon Method including sample ranking exercises unrelated to SCI/D. At the meeting, attendees were oriented to the materials to facilitate uptake of the ranking approach and processes. All EAC and the core SCI-High Team members then practiced the modified Hanlon Method by scoring and ranking three mock health issues: a) smoking cessation; b) flu immunization; and c) breast cancer. This process was to ensure all participants were familiar with the methodology and ranking process prior to applying it to ranking of the 15 SCI-Care (health issue/rehabilitation) Domains (Supplementary 1).

Once the orientation to the modified Hanlon Method and the mock ranking processes was completed, the individual EAC members and core SCI-High Project Team members were instructed to individually complete the ranking of the 15 SCI-Care Domains, without discussing or reviewing their calculations with the other EAC members in attendance. The product of the priority scores and feasibility scores were then calculated and the total scores for each Domain were posted for participants to review.

Two SCI-High Project Team leaders then led facilitated a discussion to achieve consensus as to the most important SCI-Care Domains.^30,31 Following the consensus process, all attendees signed a formal statement indicating their support for the outcome of the consensus process (supplementary 2).

Results

Table 1 shows the mean priority, feasibility, and total scores and their ranking for the 15 Domains derived from the modified Hanlon Method. There was substantial variation between the top five and bottom five items based on priority scores. The maximum total score belonged to Urinary Tract Infection (2183.82), and the minimum score belonged to Skeletal Integrity (702.82). The second priority ranking was Urohealth, which dropped down in the final ranking due to a low feasibility score. Sexual Health was ranked within the top 10 priorities, but was very low in feasibility score which resulted in the total score ranking of twelfth.

Table 1 Mean priority, feasibility and final scores for the 15 SCI Rehabilitation Care (SCI-Care) Domains in descending order by total score and rank.

Spinal Cord Injury Domain	Priority		Feasibility		Final Score
	Score	Rank	Score	Rank
Urinary Tract Infection	144.76	1	14.88	1	2183.82
Community Participation	120.82	3	13.53	3	1750.06
Wheeled Mobility	106.59	8	13.53	3	1738.00
Urohealth	121.76	2	13.00	5	1698.12
Tissue Integrity	117.88	4	14.12	2	1680.47
Informed Self-Management	109.71	6	12.35	8	1596.12
Cardiometabolic Health	114.29	5	10.94	12	1384.06
Employment	107.53	7	12.06	10	1371.65
Emotional Well-Being	102.29	9	12.47	7	1327.88
Walking	77.29	13	11.82	11	1164.41
Reaching, Grasping & Manipulation	79.76	12	12.48	6	1106.88
Sexual Health	93.00	10	9.41	14	1081.53
Independence in Breathing	82.41	11	10.53	13	944.06
Ameliorate Neuropathic Pain	65.00	14	12.12	9	843.24
Skeletal Integrity	61.47	15	9.29	15	702.82

As sexual health was deemed to be an issue of importance based on input from stakeholder organizations, recent national polls and the Hanlon priority scoring, the group consensus was to prioritize this Domain despite its low feasibility score. The group (EAC and project leaders) then decided to combine the Community Participation and Employment Domains into one single Domain as members felt these constructs were highly related. The EAC advised that the informed self-management Domain was a distinct focus of education during rehabilitation and warranted its own distinct indicator. Thus, self-management was not merged with community participation and employment/vocation Domains. Table 2 shows the final ranking of 11 SCI-Care Domains in alphabetic order.

Table 2 Final 11 SCI Rehabilitation Care Domains shown in alphabetic order defined by the modified Hanlon and consensus methods.

SCI-Care domains
Cardiometabolic Health
Community Participation and Employment^a
Emotional Well-Being
Informed Self-Management
Reaching, Grasping, and Manipulation
Sexual Health
Tissue Integrity
Urinary Tract Infection
Urohealth
Walking
Wheeled Mobility

^aMerged Domains following RAND/UCLA method.

Discussion

This manuscript presents the first study using mixed quantitative and qualitative methods for deriving priorities for Quality Improvement and knowledge translation in SCI/D Rehabilitation Care. The Hanlon Method offered quantitative support for prioritizing one SCI-Care Domain over another. The Team believed it was helpful to modify the Hanlon feasibility scoring to include accessibility criteria when ranking. The objectivity of the method increased the transparency and face validity of the results, and permitted stakeholders including health policy makers, researchers, clinicians and patients and other stakeholders to identify explicit factors driving the healthcare priorities. The ranking and consensus processed identified 11 SCI-Care Domains for selecting, implementing and evaluating quality care indicators in SCI-Care from the time of rehabilitation admission to the first 18 months post inpatient rehabilitation admission.

Among the 15 ranked SCI-Care Domains, prevention of UTI was ranked as the top priority. This result is not surprising, as UTI is the most common reason for inpatient rehabilitation service interruption,³² re-hospitalization, emergency department visits,³³ sepsis requiring hospital admission³⁴ and adverse longitudinal rehabilitation outcomes.³⁵

There is a variety of prior published evidence supporting the face and content validity of the other identified SCI-Care priorities; the functions that have received greatest attention in the published SCI literature are: walking, hand, and bladder function.^36,37 In a survey by Anderson return of arm and hand function was the highest priority for people with tetraplegia.³⁸ A systematic review aimed at identifying health and life priorities among individuals with SCI, showed that relationships, restoration of motor function, bowel, bladder and sexual function were ranked as the most important areas for research into neurorecovery.³⁹ Similarly, we sought to address the priority gaps between knowledge generation and clinical practice implementation, identifying similar priorities. Arm and hand function are associated with independence in self-care and instrumental activities of daily living. Frequently, upper extremity impairments can compound difficulties in other SCI-Care Domains, specifically bowel and bladder management.⁴⁰ Each of these three aforementioned priorities (Walking; Reaching, Grasping, and Manipulation; Urohealth were present within the top ranked SCI-Care Domains).

The World Health Organization defines community participation as involvement in life situations.⁴¹ The prioritized SCI-High Domains intersect with one another within the E-scan rehabilitation framework including, but not limited to: Community Participation, Employment/Vocation, and Informed Self-Management. The Domain of Community Participation was ranked second; however, during the consensus process the EAC members discussed at length the interrelated nature of these SCI-Care Domains. A decision was made to merge Community Participation with the Employment/Vocation Domain due to overlap in the constructs and the potential to link or merge indicators of quality care in these Domains as the SCI-High Project proceeds.

Readers may note that Bowel Continence was not selected as an SCI-Care Domain to score using the modified Hanlon Method as this Domain was lacking in research and new knowledge. In contrast, Walking was a Domain where there was a large volume of new knowledge and evidence of a gap between knowledge generation and clinical practice. Ambulation recovery and walking depend strongly on a variety of factors including the neurological level of injury, severity of injury (AIS), sensory preservation, proprioception, lower extremity muscle strength, locomotion mechanics, and spasticity and rehabilitation intensity.⁴² Ambulation outcomes are considered an important marker of rehabilitation effectiveness and quality of life among individuals with incomplete SCI. Recent trends in the epidemiology of SCI showing increasing numbers of individuals with incomplete paraplegia and a favorable prognosis for community ambulation, supporting the importance of the related Domains of Reaching, Grasping, and Manipulation; Walking; and Wheeled Mobility.⁴³

Given the biopsychosocial nature of sexuality and its profound impact on individuals with SCI/D and their interpersonal relationships, this SCI-Care Domain was flagged for priority inclusion despite low feasibility scores. Individuals with SCI have rated their sexual functioning as either their first or second priority, with respect to quality of life, on a needs survey.⁴⁴ Anderson identified sexual function as the most important Domain for quality of life of individuals with paraplegia.³² The current gaps in the area of sexual functioning, discomfort of patients and regulated health care professionals in dealing with this Domain, in concert with reduced inpatient rehabilitation length of stay over the last decade, have all resulted in limited attention to the Sexual Health Domain within Canadian rehabilitation service delivery models, with rare exceptions. This reality accounts for the low Sexual Health Domain feasibility scores. The decision by the EAC consensus process to include Sexual Health as one of the SCI-Care priorities has clinical and face validity; however, challenges in implementing indicators are anticipated.

A limitation of this study is that the methodology transparently coalesces inputs from multiple experts on each criterion, and by its nature, is subject to review and challenge by an alternate group of stakeholders at a later date. We invited a diverse group of experts and stakeholders from a cross-discipline and across the country as members of our EAC in an attempt to minimize bias within the constraints of project funding. Although the EAC members were primarily from urban areas, considering the simplicity of the created care indicators we believe the indicators will lead to higher quality of care in both rural and urban areas. Moreover, SCI-Care Domain selection was restricted to the 15 Domains where significant gaps between knowledge generation and clinical application were identified according to the E-Scan report cards at the end of each chapter. During the course of E-Scan development, data validation and data synthesis processes; important themes and specific care gaps were identified by participating sites across provinces.^22,44 With the completion of the prioritization process, all 11 selected SCI-Care Domains will be considered of equal importance during the phase II processes for identifying, developing and implementing the indicators.

In phase II, national Working Groups containing content experts and patients with lived experience will identify, develop and implement at least three indicators (one structure, one process, and one outcome indicator) for each SCI-Care Domain. The Investigative Team plans to engage consumers actively during the indicator development process in order to avoid unintended consequences. In addition, the Investigative Team plans to link the development of SCI-Care indicators to the Accreditation Canada (AC)⁴⁵ and Health Standards Organization (HSO)⁴⁶ standards for SCI rehabilitation. Requirement for institutions to demonstrate for accreditation purposes, how they use data to inform SCI-Care within their programs, represents a unique and golden opportunity to address our desire for equitable and optimal care nationwide, through implementation of prioritized indicators linked to the AC/HSO standards and principles of patient centered care.

Conclusion

This method for prioritization of Domains could be applied in other rehabilitation and health care environments. The list of 11 priority SCI-Care Domains derived by the modified Hanlon Method narrowed the focus from a large number of disparate Domains (n = 37) to a feasible number where gaps still exist between knowledge and clinical implementation. This restricted list will be used to develop and implement related Domain specific structure, process and outcome indicators to advance quality of SCI-Care. We anticipate that the SCI–High Project will launch a knowledge translation process that ultimately changes clinical behaviors in favor of effective SCI-Care through benchmarking by 2020.

Disclaimer statements

Contributors None.

Declaration of interest Dr. Mark Bayley acknowledges support from the Saunderson Family Chair in Acquired Brain Injury Research, the Toronto Rehab Chair in SCI Rehabilitation, and the Toronto Rehab Foundation. Dr. B. Catharine Craven acknowledges support from the Toronto Rehab Foundation as the Toronto Rehabilitation Institute Chair in Spinal Cord Injury Rehabilitation and receipt of consulting fees from The Rick Hansen Institute.

Conflicts of interest None.

ORCID

S. Mohammad Alavinia http://orcid.org/0000-0002-5503-9362

Sander L. Hitzig http://orcid.org/0000-0002-9139-9250

Farnoosh Farahani http://orcid.org/0000-0002-3937-7708

Heather Flett http://orcid.org/0000-0001-9444-8006

Mark Bayley http://orcid.org/0000-0001-7860-9463

B. Catharine Craven http://orcid.org/0000-0001-8234-6803

Additional information

Funding

This work is embedded in the larger SCI-High Project funded by the Rick Hansen Institute (CA) (Grant #G2015-33), Ontario Neurotrauma Foundation (ONF; Grant #2018 RHI HIGH 1057), and Toronto Rehab Foundation. Open access publication fees were kindly provided by Coloplast. Toronto Rehab Foundation.