Compliance with standards of practice for health-related rehabilitation in low and middle-income settings: development and implementation of a novel scoring method

Abstract

Purpose: To (A) develop a method for measuring compliance with standards, and (B) implementation of the method in 12 rehabilitation centers in six low and middle-income countries (LMICs).

Methods: In part A, existing standards were compiled and operationalized into scores, organized into 5 ‘scorecards’ and 15 ‘sub-scorecards’, then tested and refined in an iterative process. In part B, 12 rehabilitation centers in 6 countries implemented the standards using the new method, revealing relative performance between centers, and across different standards. Internal consistency of scores within domains was computed using Chronbach’s alpha.

Results: A standardized method for scoring compliance with standards for rehabilitation was developed. The method evaluated compliance with standards in five domains of practice: user focused approach, service outputs, finances, staff, and general management. Multiple standards within domains were strongly related, with Chronbach’s alpha >0.80 for all but the equipment and supplies domain. Overall, in the 12 rehabilitation centers examined, 36% of standards were met or exceeded. Compliance within each scoring domain was 56% (user-focused approach), 38% (service outputs), 27% (financial management), 30% (staff management), and 33% (general management). Two out of 12 (17%) of centers met more than two-thirds of the standards, 3 (25%) met more than one-third of standards, while the remaining 7 (58%) met less than one-third of standards.

Conclusions: A new, standardized method for measuring performance of rehabilitation services in LMICs was developed. The method examines standards in five rehabilitation practice domains, and can be used to understand barriers to quality performance, particularly in resource-constrained settings. Implementation of the method demonstrated that current compliance with standards is modest. Ongoing interest in new standards for rehabilitation practice should be accompanied by measures to ensure they are used to strengthen quality in an emerging rehabilitation sector.

Implications for rehabilitation

• We developed a method for measuring compliance with standards for rehabilitation, and implemented the method in 12 rehabilitation centers in low and middle income countries.

• We demonstrate modest compliance with an adapted list of known standards of practice.

• New standards, scoring methods and evidence of current performance may assist service providers and policy makers to implement standards, and methods to strengthen rehabilitation services.

• Modest compliance with current standards suggest new emphasis on quality performance of health-related rehabilitation is needed

• New emphasis on standards for rehabilitation should be accompanied by consideration of how performance against those standards can be measured and improved.

Keywords:

• Rehabilitation
• rehabilitation centers
• standard of care
• quality
• health-related rehabilitation
• LMICs

Background

Rehabilitation is a ‘set of measures that assist individuals who experience, or are likely to experience, disability to achieve and maintain optimal functioning in interaction with their environments’ [1]. In low and middle-income countries (LMICs), rehabilitation is usually provided by a mix of family, community volunteers and professionals or nonprofessional personnel, either in community or rehabilitation centers. Citing increased years lived with disability (YLDs) in the last decades, and profound unmet needs for rehabilitation, the World Health Organization’s Rehabilitation 2030 ‘Call for Action’ highlighted the pressing need to strengthen and extend access to rehabilitation, particularly in LMICs [2].

Health-related rehabilitation is sparsely represented in public policy in low income and post-conflict settings [3,4] and is often not a priority for government. This is despite the recognition of rehabilitation as a critical part of health systems and as a right for persons with disability under the United Nations Convention on the Rights of Persons with Disabilities (CRPD), which 157 countries have ratified since 2008. The CRPD requires signatories to ‘…better organize, strengthen and extend’ health-related rehabilitation [5,6]. Rehabilitation has been recognized as an essential part of primary health care in the Declaration of Alma Ata [7], but its implementation has been slow, and largely separate from reforms in other healthcare sectors. This is especially concerning given the ageing population and forecasted needs for chronic care for persons with a range of non-communicable diseases including musculoskeletal diseases, which are both a cause and consequence of reduced mobility [8].

Centre-based, health-related rehabilitation is only one model of rehabilitation service delivery, but an area that requires good policy development and monitoring. Even with recent policy push factors such as World Health Assembly resolutions calling for better support for rehabilitation, there are still many implementation challenges. These include a lack of recognized standards that are simple to implement and monitor [9], and might be used to drive reforms [10]. This has led to current efforts to develop global guidelines for health-related rehabilitation [11]. The utility of these new guidelines will depend on the extent to which they are used and lead to positive changes [12]. Evaluation of health service quality and accreditation has become more common in high income settings [13–16], but has received less attention in low income settings, where more emphasis is placed on sustainability than quality [3,17,18] or accessibility [19].

The case of prosthetics and orthotics, a subset of health-related rehabilitation, is illustrative. In 2006, a multi-stakeholder collaboration developed consensus-based guidelines for prosthetics and orthotics projects and programs [20]. While these guidelines were developed in consensus with more than 35 international agencies, their uptake has not been evaluated in the decade since their release.

Despite their potential, the utility of instruments for understanding compliance with standards has not been reported in rehabilitation services in LMICs. There are new calls to strengthen and extend rehabilitation to respond to changing population demographics and the neglect of rehabilitation in health and social systems reform. It is therefore timely to examine the implementation of existing standards.

Statement of problem

Rehabilitation service reform is de-emphasized in health systems strengthening and lacks clear, globally accepted standards and targets. Implementing and improving compliance with quality standards is an essential starting point to strengthen and extend health-related rehabilitation. There is currently interest in developing new standards for rehabilitation, but limited evidence for good practice in applying those standards to policy and practice.

The objectives of this paper are therefore (A) to describe development of a method for evaluating one type of health-related rehabilitation services based on existing consensus-based standards and (B) to report on current compliance with process standards using this method in 12 rehabilitation centers in six countries (Cambodia, Nepal, Palestine, India, Sri Lanka, and Lao PDR).

Methods

The research program was conducted in two parts. Part A involved development and refinement of a tool for systematic appraisal of performance against existing standards. Part B implemented the new process in 12 rehabilitation centers to determine current compliance with existing standards, and explored the internal consistency of groups of standards, to inform further refinement of the methods.

Part A: developing a process for evaluating compliance with consensus-based standards

This work arose from a need to examine compliance with process standards and guidelines. We therefore drew on the Prosthetics and Orthotics Program Guide [21] as a starting point, because they represented the clearest set of standards developed specifically for rehabilitation services in LMICs. To develop the evaluation method, we drew from the application of assessment rubrics in health services evaluation and accreditation [16,22,23], and developed ‘compliance statements’ for each of the standards.

Table 1 summarizes the scoring method. For each standard, the evaluation rubric included 4 ‘statements’, each corresponding to a numerical score between 0 and 3. A statement corresponding to a score of 0 describes no progress towards the standard or ‘critical issues’; a statement corresponding to a score of 1 describes progress towards the standard without full compliance, ‘some issues’, or limitations in documentation; a score of 2 describes the standard being met with objectively verifiable evidence; finally, a statement corresponding to a score of 3 describes the standard having been exceeded.

Table 1. Scoring criteria and examples of scoring statements.

Score	Score basis	Description	Example (standard 1.3.2 – physical access to buildings)
0	‘Critical issues’	No progress against the standard, or risk or recent history of specific adverse events	‘There are clear accessibility limitations for some users or potential users OR no audit has been conducted’
1	‘Some issues’	Basic progress against the standards not likely to create direct risk to individuals including staff, services users and the community, or weak evidence of the standard being met	‘Some user spaces are not accessible for some people, but there is an up to date audit and basic plans to improve accessibility’
2	‘Standard met’	Standard is consistently met. The service clearly documents the standard and processes to achieve it in policy or other documentation, and attainment of the standard is demonstrable through monitoring	‘Audits have been conducted but they are infrequent or not followed up OR Access audits identify some accessibility limitations, and there are realistic plans in place to address them’
3	‘Standard exceeded’	The standard is exceeded. The policy processes to attain it and results are reliably documented	‘An accessibility audit is conducted routinely. All user spaces are accessible to wheelchair users, persons with low vision and those who use orthopedic equipment. Documentation takes low vision or low-literacy users into account’

These standards were then organized according to the criteria proposed for balanced scorecards (BSCs) [24,25], then further into logical ‘sub-cards’. The initial set of standards were implemented in four sites: two each in Nepal and Sri Lanka by one researcher, together with center management and staff. In addition to scoring, participants reported on each of the standards’ relevance, clarity of wording, usefulness in their center, and were invited to propose additional standards or refinements to the format of the scorecards.

During this phase of the research program, we adapted scorecards to align more closely with the World Health Organization’s Health Systems Building blocks approach [26]. Sub-cards were developed by clustering-related standards and were refined during further testing through discussion, clarification, and agreement between the researchers and center staff. Once standards and statements had been developed, a spreadsheet-based tool was prepared to facilitate data entry, reporting, and planning responses based on the results.

The revised scorecards and standards were then implemented in these sites again, with an additional five sites in Nepal, Sierra Leone and Liberia. Results and experiences using this method were presented at three regional workshops (Burkina Faso, Nepal, and Cambodia). The workshops were attended by members of the research team, representatives of each of the rehabilitation centers, external rehabilitation experts, officials and NGO staff, and resulted in further refinements, including revised standards and clearer methods to develop recommendations for service changes based on scoring results.

Part B: measuring compliance with standards

Selection of centers

After initial development of the standards and method, all physical rehabilitation services operating in South Asia and in current collaborations with the international Non-Government Organization (NGO), Humanity and Inclusion (HI) were initially considered for inclusion in this phase of the research program. HI is an international independent organization specialized in disability and vulnerability. It provides support to local organizations and government to strengthen rehabilitation, or provides services directly if necessary. The centers in this study were not run directly by HI, but received technical and financial support.

Inclusion criteria were that centers offered comprehensive rehabilitation services including orthotics and prosthetics, and occupational therapy, physical therapy or both. We excluded satellite services with emphasis on one discipline, or with limited human resources, which were outside the intended scope of the standards. Two additional services not supported by HI implemented the scoring system and their results are included in this analysis. Fifteen centers were included in the initial implementation. Of those, 12 provided the necessary consent and data. The centers examined here therefore represents a substantial proportion of NGO-supported rehabilitation services in six LMICs. Characteristics and outputs of the rehabilitation centers are summarized in Table 2.

Table 2. Characteristics of rehabilitation services in the sample.

		Centre code (Type^1, setting^2)
		C01 (B, I)	C02 (A, II)	C03 (B, II)	C04 (A, II)	C05 (B, II)	C06 (B, II)	C07 (B, II)	C08 (B, II)	C09 (C, II)	C10 (B, II)	C11 (B, I)	C12 (B, II)
	Country	Nepal	Cambodia	Palestine	Palestine	Nepal	Nepal	India	Nepal	Nepal	Sri Lanka	Lao PDR	Lao PDR
Clients per year	Total	1437	2245	456	1748	700	1710	471	1141	724	500	1800	600
	Women	400	292	69	110	246	525	221	397	283	280	800	280
	Men	606	785	112	47	454	839	250	744	441	150	800	245
	Girls < 18	199	483	156	897	132	165	43	172	113	40	100	40
	Boys < 18	238	685	117	694	215	207	58	321	114	30	100	35
	Total sessions^3	5283	9774	3070	11,494	2977	5905	2487	4421	1997	1300	23,000	4500
Assistive products outputs	Basic push wheelchairs^4	17	0	100	0	22	0	0	0	3	6	480	16
	Active wheelchairs^5	34	131	0	33	6	65	31	26	10	0	120	6
	Orthoses	133	442	0	0	173	555	74	126	97	318	503	94
	Prostheses	65	248	0	0	67	82	29	10	53	103	384	79
	Special seats	6	21	0	8	52	13	2	26	3	0	15	0
	Total walking aids^6	150	400	65	7	107	634	46	47	44	95	205	23
	Total repairs^7	62	541	0	0	126	868	46	52	42	560	284	43
Staff	Women	10	8	4	27	10	15	12	5	4	4	102	10
	Men	12	19	3	26	10	3	30	11	7	3	81	6
	Percentage of staff with disability	18	19	0	4	25	39	7	0	9	0	10	0
Budget	Total budget p/a^8	$92,767	$215,000	$62,750	$709,000	$92,767	$92,767	$298,000	$72,046	$79,246	$90,000	$975,000	$120,000
Percentage of compliance	User perspective	7	40	20	47	47	7	47	73	93	87	93	0
	Service outputs	6	25	13	6	19	19	25	69	69	88	94	0
	Staff	21	43	14	7	21	7	36	79	36	71	71	21
	Finances	30	50	38	38	20	30	50	70	30	40	90	20
	Management	18	14	45	36	9	18	36	59	64	77	57	5
	OVERALL^9	16	27	26	24	19	14	33	60	55	74	78	8
Overall compliance^10		Low	Low	Low	Low	Low	Low	Medium	Medium	Medium	Good	Good	Low

Explanatory notes. 1 – Centre types: A, Standalone rehabilitation center with emphasis on physical rehabilitation; B, center with physical rehabilitation and additional services including social work, audiology, speech and language therapy; C, Rehabilitation services within tertiary hospital. 2 – I, National/provincial capital; II, regional. 3 – Total recorded or estimated episodes of care across all rehabilitation services. 4 – Hospital type, folding wheelchair usually attendant propulsion. 5 – Active, custom wheelchair for user or attendant propulsion. 6 – Including all types of crutches, canes and walkers. 7 – Repairs of all assistive products combined. N/A – not available. 8 – USD, 2016. 9 – percentage of all standards met within center. 10 – <33% compliance – ‘low’; 33–66% = ‘medium’; >66% = ‘good’.

The scoring process

One researcher facilitated the management team in each service to score against the criteria at each of the test sites. This involved approximately half a day of training on rehabilitation standards, reviewing any national standards, introducing the new method, and a further two to three days of implementation. There was some variation between centers. Some preferred to score a few scorecards at a time, while others sought to score all the standards together. Our intention was to align the process with typical management procedures rather than to impose an entirely new approach. If a standard was considered irrelevant or out of scope by the team, we discussed options to refine or ignore the standard. In these instances, subsequent descriptive statistics are adjusted accordingly. While this may introduce some variability in results, flexible implementation was an important design criterion.

To conduct the scoring, participants were first advised to test the statement corresponding to a score of 2, which describes the standard being met. If the statement describes the practice in the center being scored, additional tests are conducted to determine if there is objectively verifiable evidence from records, evaluation, reports, or similar. If a score of 2 is verified, the statement corresponding to score ‘3’ is tested to determine if the standard is exceeded. If this statement is true, a score of 3 is awarded. If the statement corresponding to score 2 is false, the standard is not attained, and statements corresponding to score 1 – ‘some limitations’, and score 0 – ‘critical limitations’ are evaluated and the score is awarded according to the statement best describing the center’s practice. For each score, a narrative justification and related documentation or other evidence are noted. Disagreements were resolved by reexamining statements and mutual discussion. If no consent could be reached, the lower score was awarded, and areas of disagreement noted for follow-up if necessary.

Data analysis

To examine current performance against known standards, the percentage of standards met or exceeded was computed for each of the five domains represented by the scorecards. This value was defined as the quotient of the total number of scores above 1 (i.e., score 2 or 3) and the total number of standards, expressed as a percentage. If the scoring team agreed a standard was irrelevant, it was omitted, and the score corrected by expressing the total compliance rate only of standards actually scored. This score is computed for individual centers and for all centers combined, and for each sub-scorecard, overall scorecards and all scorecards combined. This allows comparison of an individual center’s performance against the other centers for each scorecard and sub-scorecard.

To understand whether standards within sub-scorecards were related, internal consistency of sub-scorecards and scorecards was examined by computing Chronbach’s Alpha on individual center scores for each scorecard and sub-scorecard.

To group centers into logical overall performance categories, we draw from Dimitriadis et al. [16] who reported a rating scale for a rehabilitation quality assurance system in Greece. Their scale required at least 90% of 110 criteria to be achieved, to be recommended for accreditation. A score between 80% and 90% required ‘Small review’, between 60% and 80% required major review, and where less than 60% of criteria were achieved, the facility ‘cannot be accredited’. In our study, we recognized the complex and often poorly resourced working environments of the centers, and that the purpose of this work was not accreditation, but to understand areas for improvement by implementing standards and a method to review performance. As such, we used a much less stringent cutoff and categorized center performance as ‘good’ where more than two-third of standards were met, ‘medium’ where more than one-third of standards where met, or ‘low’ where more less than one-third of standards were met.

Ethics approval

Ethics approval for the use of scoring and service descriptive data was obtained from the University of Melbourne Human Ethics Sub Committee (1646788.1). Rehabilitation center managers were provided with full information about the use and disclosure of information and gave written consent.

Results: part A

The final scorecards comprised 84 total standards. The final rubric, organized in scorecards and sub-cards is provided in Table S1. The scorecards and sub-cards are summarized in Table 3.

Table 3. Score-cards and sub-cards, internal validity measures.

Scorecard	Sub-card	n Standards	Overall compliance(scorecard)	α (sub-card)	α (card)
User perspective	User focused approach	4	56%	0.83	0.91
	Client experience	8		0.80
	Accessibility and access	3		0.58
Service outputs	Evidence-based practice	4	38%	0.90	0.84
	Service model	3		0.82
	Planning, monitoring, and evaluation	3		0.93
Staff	HR Policy	3	30%	0.75	0.81
	Performance, recognition, and salary	8		0.66
	Staff management	9		0.77
Finances	Cost calculation and recovery	5	27%	0.90	0.87
	Budgeting procedures and accountability	2		0.66
Internal management	Facility management	14	33%	0.60	0.81
	Priority setting	4		0.55
	OH&S	5		0.68
	Governance	9		0.76

Piloting provided a further opportunity to refine scorecards and revealed three main areas to strengthen the method. First, introduction of new standards needed to be aligned with current management processes, while providing clear scoring criteria and step-by-step methods. Second, our method was strengthened by developing an understanding of practical issues such as the time taken to implement scoring. This ranged from one day for centers on subsequent implementations, to three days for new centers, including training, scoring, analysis of the results, and development of plans. Third, piloting highlighted the need to reconcile the requirement for standardized scoring with flexibility in conducting the evaluation. The background of center staff and contexts in which they worked varied widely. As such, our method sought to be as flexible and adaptable as possible, while maintaining the fidelity of carefully developed standards and scoring method. These findings were used to inform part B.

Results: part B

Return rate

Twelve of 15 initial sites are included in the present analysis. One site was excluded because the management team did not wish to continue after piloting. Another two sites did not submit full datasets after repeated contact. We estimate the final sample represents 80% of all rehabilitation services offered in Nepal, 60% in Sri Lanka, and a smaller sample of remaining countries.

In participating centers, 94% of all standards were scored. Individual standards were not scored when the center team and researchers agreed that the standard was outside the scope of practice at the center. Examples include provision of community outreach services, where those services were provided by a separate agency. Out of scope standards in the 12 centers varied, but two trends emerged. First, there was some difficulty in deciding whether standards that lay outside the remit of the center but with higher authorities (such as a regional office or Ministry) should be scored. Decisions such as remuneration structure and recruitment processes, at least in public settings, are usually determined by government policy. This was handled by scoring the standard if possible (or ignoring it if not), noting the barriers to effecting change at the center level. Secondly, standards that related to the scope of practice often appeared to depend on the relative size of the service. That is, some standards might be applicable only in larger centers. In these situations, we again encouraged scoring of the standard as it was written, but noting the limitations of the center in meeting that standard given the dimension and limitations.

For all cases where the standard reflects a currently missing but necessary service element, a score of 0 was awarded. This ensures final scoring is a true reflection of performance against standards, with very minor exclusion of standards based on different service scope.

Service characteristics

Table 2 summarizes the main output, staffing and financial characteristics of the 12 rehabilitation services examined. Services were mostly run by national NGOs, but one private facility and three government facilities are represented. All receive some technical support from international NGOs. The size, staffing levels, budget and outputs of services vary widely. Emphasis on services ranges from physiotherapy and a small selection of assistive mobility products, to comprehensive mobility device services. Some services provide outreach services, social work, or speech and language therapy.

Current compliance with standards

Of the 94% of standards that were scored, 36% of standards were met or exceeded (4% exceeded, 32% met). 46% had some issues, and 11% critical issues. The ‘user’-related standards were strongest overall (56% met or exceeded) and finances the weakest (27%). Relative compliance of the centers is presented in Figure 1.

Figure 1. Total compliance with standards-all centers.

Two out of 12 (17%) centers met more than two-thirds of the standards, 3 (25%) centers met more than one-third of standards, while the remaining 7 (58%) centers met less than one-third of standards.

Internal consistency of standards

Using Chronbach’s alpha to compute internal consistency within scorecards demonstrated good to excellent agreement between standards (α = 0.81–0.91) for all but the equipment and supplies scorecard (α 0.49), perhaps reflecting the smaller number of standards (3) in that scorecard. Individual center performance and Chronbach’s alpha for scorecards are reported in Table 3. There is higher variability within sub-scorecards, with Chronbach’s alpha as low as 0.55 for the priority setting sub-scorecard.

Discussion

This study reports the development and implementation of a method to score compliance with consensus-based standards for rehabilitation services. The method included standardized scoring, supported by scorecards and sub-cards, a spreadsheet based data-entry tool, and demonstrated good internal consistency within most scorecards. Using the methods and standards revealed modest compliance. Results suggest that despite investment in developing standards, even with technical assistance from international NGOs, compliance with standards is poor or average in most of the centers studied.

Previous efforts to understand performance of rehabilitation services have explored particular dimensions of the overall rehabilitation sector such as self-defined measures of sustainability [3], stakeholder connectedness [17,18], or reflection of rehabilitation in national policy [27,28], rather than standards for practice at the level of a typical rehabilitation center. Research on standards and governance for rehabilitation in higher income settings has typically considered specific clinical situations such as trauma [29], stroke care [30] or cardiac rehabilitation [31], or been targeted at a national level, such as recent work to introduce rehabilitation quality assurance in Greece [13,16]. The method developed and implemented here builds on previous work by including standards across all dimensions of center performance, including a simple scoring and interpretation method, being purposively aligned to LMIC settings, and encouraging flexible implementation.

To our knowledge, the only comparable work on standards for quality assurance system for rehabilitation tool in 15 Greek rehabilitation centers [16], which uses a checklist of 110 questions in 11 sections. Like in our study, Dimitriades and colleagues report a wide range of compliance between centers, with between 11% and 100% compliance for the 15 centers researched. While the methods and standards differ, these results offer an interesting contrast to our findings. In the present study ‘Evidence Based Practice’ and ‘planning, monitoring & evaluation’ sub-cards are similar to the ‘results oriented’ (47% of centers achieved the standard) and ‘continuous improvement’ (49%) categories reported by Dimitriades. The high-performing criteria concerning ‘ethics’ and ‘participation’ reflect the standards of user focused approach, which was relatively poor in our study (31% total compliance). Further comparison between studies is difficult, but highlights the potential utility of comparable data on quality of rehabilitation service across contexts.

By using stakeholder consultations, review and refinement of standards, the method and standards developed here reflect the real-world experience of providing rehabilitation services in LMICs. Our method appeared to satisfy the requirement of developing user-friendly standards for rehabilitation and methods for their implementation. This builds on previous work to define outcome measures, indicators or standards specific to practice areas or geographical contexts, and offers the possibility of a simple method to introduce in emerging rehabilitation services.

The wide range of results across the different centers may reflect different operating conditions such as budget, staff profile, and external technical support. Further, during the scoring process, we observed a wide range of outcomes-based indicators in use, which may also contribute to the range of results. Quality targets often fluctuate with current funding and donor requirements, and were reported as being burdensome by some staff in these centers. This implies careful prioritization of different reporting and accountability processes is needed. The World Health Organization has promoted the Balanced Scorecard approach, which informed our method, in other areas of health services in low income settings [25], partly to simplify the analysis of complex, multiple dimensions of health services. The method used here examined process guidelines within service delivery, rather than the specific quantitative outputs of the service that are normally included in balanced scorecards [32], but drawing from BSCs helped rationalize standards into manageable groups.

In this study, we found performance in the ‘user focused approach’ scorecard to be the highest, with 56% compliance across the centers. This scorecard comprises standards concerning direct interaction with the users, such as confidentiality, dignity, participation, and user feedback. These standards are often a focus of efforts to improve service quality, which could explain the stronger performance relative to the other scorecards, even if overall performance is poor.

Similarly, ‘service outputs’ (38%) comprises standards on the clinical services, which are usually emphasized in training, continuous development, and evaluations. Within this scorecard, evidence-based practice (EBP) represented a weak performance area, perhaps reflecting the relative novelty of EBP the studied contexts. Challenges to implementing EBP in rehabilitation in resource-constrained settings is poorly researched, but is known to be a challenge and constraint to service improvement in other healthcare domains [33,34]. As service outputs depend on availability of outreach, follow-up, access to community services and referrals, performance is often dependent on other services, and outside of the control of individual centers. This may explain some of the poor compliance, and reinforces the need for rehabilitation services to understand and promote coordination with other services.

Across all centers, 30% of ‘staff’-related standards were met. These standards include performance management, salary structure, and professional development. In many settings, staff members are appointed without annual review or clear performance management. As such, performance in these criteria was modest, and may highlight opportunities to target workforce incentivisation and performance management in strategies for strengthening quality of care.

Despite the obvious need for and emphasis on financial management in most centers, performance in ‘finances’-related indicators was the lowest of the domains, with 27% of standards met across all centers. These standards include budgeting, review, accountability measures and related areas. The performance here may reflect the absence of review processes, clear documentation and accountability reflected in the standards, rather than an absence of financial management practice. Nonetheless, there is evidence that addressing financial management may be necessary.

Internal management standards include planning, review, community liaison, and policy review. All-site performance in this domain was 33%. We find that many simple performance standards are not addressed in routine practice. Examples include external referrals. Good performance on that standard requires linkages with external service provided, which is often seen as out-of-scope for some rehabilitation centers. These findings highlight the need to consider quality practice comprehensively, including basic management procedures, rather than targeting clinical quality alone.

Good internal consistency between the different standards within scorecards implies reasonable correlation between the scores within a scorecard. While further work is needed to understand the reliability of the standards and the process we have described, our results describe a potentially reliable quality assessment method. It also suggests some redundant items, which could be addressed in further refinement of the standards.

One possible interpretation of the modest compliance found in these centers is the tendency towards external evaluation of rehabilitation centers, often within the confines of development or aid financing accountability [35]. Developing management and quality assurance capabilities within the rehabilitation service providers may lead to better uptake of standards and center performance. Further, rehabilitation in low-resource settings is a complex intervention requiring input from a range of professionals, often in challenging environments with insufficient resources. There is often limited and variable financing for the necessary hardware, consumables and difficulties with recognizing rehabilitation professionals in national healthcare cadres [4]. There can be limited knowledge of protocols and capacity to implement them [36], and of overall stewardship of rehabilitation in LMICs.

The process standards developed and explored here could be useful to evaluate accessibility of services, which could support measures to improve overall coverage of rehabilitation in health care. The sub-scorecard ‘accessibility and access’ was comprised of three standards: affordability, physical access, and outreach or community level services. McIntyre et al. [19] proposed availability, affordability and acceptability as three dimensions of ‘access’. In the present rehabilitation centers, there was a wide range of performance in the accessibility and access sub-scorecard; three centers satisfied all these standards, while four satisfied none of the standards.

This work has some important limitations. Ten of the 12 centers examined had been supported by one technical agency. Each rehabilitation service has different priorities, strengths, limitations and operational contexts which is reflected in the varied scores between centers. Standards should be carefully refined and adapted to suit the specific location, while maintaining the value of consistency across sites.

Improving access to rehabilitation will require understanding rehabilitation as part of a multisectoral system in which rehabilitation services are located, and recognizing their relative novelty in health systems and public health discourse. Additional research would help understand how interventions can improve performance over time, and how prioritization and context influences service quality.

Conclusions

A rehabilitation management system comprising standards, descriptors and systematic scoring rules was developed, and demonstrated moderate compliance with standards in 12 rehabilitation centers in Nepal, Cambodia, Palestine, India, Sri Lanka, and Lao PDR.

The new method developed is a useful adjunct to existing management techniques, and might be useful in efforts to strengthen and extend health-related rehabilitation. Evidence from this study suggests limitations in the uptake of standards, and therefore current management processes of rehabilitation services. Limitations may be associated with variable funding sources and operating in complex environments, and varied experiences with implementing quality management approaches. Emphasis on developing standards and guidelines should be balanced with understanding how they are used, and with specific technical and financial investment in improving stewardship capabilities and quality management practices in rehabilitation service providers and decision-makers.

Acknowledgements

The authors wish to thank USAID for initial financing of the rehabilitation management system upon which this research is based, and the rehabilitation center staff, management, and clients, for and with whom this work was conducted.

Disclosure statement

No potential conflict of interest was reported by the authors.