Use of patient-reported outcomes in low-resource settings — lessons from the development and validation of the Zimbabwean Caregiver Burden Scale

Background: why patient-reported outcome measures (PROMs) in caregivers of children with cerebral palsy?

With the advent of family-centred approach to care and the drive towards evidence-based practice (EBP), there is a growing impetus towards the use of patient reported measurements (PROMs) [1–4]. The use of PROMs is even more pertinent in the evaluation of the mental health of caregivers of children with life-long disabilities such as cerebral palsy (CP). Several systematic reviews have demonstrated that caring for a child with functional limitations often leads to caregiver burden [5–8]. Caregiver burden can be defined as “strain or load borne by a person who cares for a family member with a disability” [9]. Consequently, there is a growing thrust towards development of interventions for improving caregivers’ mental health in addition to improving the functional outcomes and health-related quality of life (HRQoL) of children with life-long disabilities [6,8,10–12]. However, the development of context-specific interventions is reliant upon the availability of psychometrically sound PROMs which are essential for baseline needs assessment and for the evaluation of the efficacy of the caregiver well-being interventions [1,3,8]. Furthermore, caregiver burden is a context-specific, latent variable which is dependent on socialisation and socio-economic background among other variables [5,10,13,14] which underscores the importance of developing culturally sensitive caregiver burden PROMs.

Caregiver burden is likely to be greater in caregivers residing in low resourced settings [4,15–18]. This warrants an evaluation of the mental health of caregivers in these settings. Unfortunately, most caregiver burden PROMs have been developed in high income settings [4,19] and their applicability in low resourced settings such as Zimbabwe may be limited. Additionally, several systematic reviews have highlighted methodological flaws in PROMs development and validation [20–22]. We, therefore, set to fill this ‘methodological gap’ by applying a robust methodology in the development of a context-specific PROM.

Purpose

This paper outlines the methodology and our experiences in the development and validation of the Zimbabwean Caregiver Burden Scale (ZCBS).

Methods and results: development and validation of the ZCBS

Outlined in Figure 1 is the schematic diagram for the development of the ZCBS. We will explain the rationale of the steps, challenges and recommendations.

Figure 1. Schematic presentation of the development of the ZCBS.

Systematic review

Systematic reviews (SRs) and meta-analysis are considered the highest level of evidence [23]. In order, not to ‘reinvent the wheel’, it was important to perform a SR to identify already existent caregiver burden outcome measures and hopefully extract candidate items from PROMs with demonstrated sound psychometrical properties [19]. It was challenging to perform the SR for the following reasons: poor internet connectivity, and a lack of technical support and experience in conducting SRs. Unfortunately, most of the validation studies were of poor methodological quality when judged according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN). Consequently, we had to adopt a pragmatic approach and extracted candidate items from tools with a seemingly high face validity.

Structured interviews

Given that caregiver burden is a subjective and complex construct, it was important to gain a contextualised meaning of this phenomenon from the perspective of Zimbabwean caregivers through structured interviews. We utilised the thematic approach to analysis. This phase of the study was particularly challenging as we are predominantly quantitative researchers but important given the growing significance of qualitative research in understanding psycho-social phenomenon in healthcare research [24,25].

Panel of experts

We synthesised findings from the SR and structured interviews to come up with the initial list of candidate items which were rated for face and content validity by a panel of experts. They reviewed the relevancy, clarity and appropriateness of each item. The panel consisted of clinicians, and academics in the fields of rehabilitation, nursing, paediatrics, neurology, psychology, social work and psychiatry. The experts independently rated the items, and the ratings were then collated by the principal investigator. The panel then convened afterwards to synthesise the findings. It proved difficult to get all experts to meet jointly to discuss the discrepancies. Most of the panellists had dual roles of being consultant clinicians and lecturers and thus, it was not possible to have all the panellists present for the consensus meeting. This was unsurprising given the well-documented shortage and work overload of healthcare specialists in Sub-Saharan Africa [26,27]. Video conferencing could have been an alternative to the physical meeting, however, due to connectivity issues this was not possible. Secondly, it was difficult to provide monetary incentives for the panellists. Some of the potential panellists did not consider co-authorship of the papers emanating from the developing of the ZCBS as an adequate incentive.

Cognitive debriefing

The output of the panel of experts’ consensus meeting was the first draft of the ZCBS. This was administered to caregivers to assess the relevancy and item clarity. We used convenient sampling to select caregivers of varying educational and socio-economic backgrounds. For caregivers who were illiterate, the teach back method (TBM) technique was employed to increase the validity of outcomes [28]. The researchers read out the questions and caregivers were asked to repeat the question in their own words before providing their response. The TBM approach has been demonstrated to yield more reliable responses as it enhances the learning process. This phase was essential as the importance of incorporating the views of the target users in PROMs development cannot be over emphasised [29].

Field testing

Feedback from caregivers during the cognitive debriefing was considered during the production of the second draft of the ZCBS which was administered to a larger cross section of caregivers. We made use of pictorial illustrations to further simplify the response options. This strategy has been previously employed in Uganda and Malawi and has been demonstrated to enhance participants’ understanding of the scoring instructions [30,31]. Further, items on the ZCBS were presented in both English and Shona (a Zimbabwean native language) to improve clarity of the items as most Zimbabweans are bilingual.

It was difficult to incentivise prospective participants to taking part in the study. This challenge has also been reported in other Sub-Saharan African countries where participation in research is always associated with monetary benefits [32,33].

In addition to the ZCBS, we also administered the EQ-5D, Shona Symptoms Questionnaire, The Multidimensional Scale of Perceived Social Support and Caregiver Strain Index to evaluate the construct validity of the ZCBS. Considering that the ZCBS had 45-items, this could have potentially resulted in respondent burden. It was thus important to devise strategies to ensure internal validity. Firstly, we included five negatively worded questions on the ZCBS to avoid stereotypical responses [34]. In addition, questionnaires were interviewer-administered for caregivers with nine or less years of education. Finally, research assistants were trained to check for consistency in responses, and where appropriate, seek clarity from respondents who had recorded multiple responses for one item.

Psychometric evaluation

We then performed exploratory factor analysis (EFA) and Rasch Analysis as part of psychometric evaluation of the ZCBS. The shortage of Biostatisticians was the greatest obstacle in the validation of the ZCBS. This problem is not uncommon in the Sub-Saharan region [35–37]. Although, learning the analytical techniques was challenging, it has proved invaluable for the principal researcher who had no expertise in psychometric evaluation.

Discussion and conclusions: future directions/recommendations/reflections on the clinical use of PROMs in low resource settings

Considering our experiences in the development of the ZCBS and clinical practise, we recommend the development of context-specific PROMs for use in low resourced settings. However, before developing any PROMs, authors should translate, adapt and validate already existing PROMs to avoid ‘reinverting the wheel’. Systematic reviews investigating the psychometric properties of the existent PROMs are of paramount importance as they would inform researchers whether it is necessary to develop a new PROM. Furthermore, researchers should also take advantage of already existent methodological protocols such as the COSMIN guidelines [38,39] in developing and evaluating psychometric properties of PROMs. We also recommend the fostering of a culture of EBP through routine use of PROMs. This could be cultivated through continuous professional activities, including PROMs in training curriculum and integrating the use of PROMs in clinical practice. This could also be facilitated through the training of healthcare practitioners in applying the PROMs and making the PROMs readily available. In low resourced settings, providing paper-based versions of the PROMs could increase the uptake. An effort should be made in developing shorter versions of the PROMs to decrease respondent burden as well as increasing the feasibility of usage of the PROMs for routine clinical work. For example, the SF-6D which is a six-itemed HRQoL outcome measure, has demonstrated psychometric robustness when compared to the 12-item (SF-12) and 36-item (SF-36) versions [40,41]. After structural validity assessment, our next objective is to validate a shorter version of the ZCBS for routine clinical use.

There is also need for a greater collaboration between clinicians and academics. That interface would propagate the culture of integration of PROMs in routine clinical practice and the development of clinically relevant and context-specific PROMs. As clinicians are responsible for the clinical supervision of trainee physiotherapists, their use of PROMs is critical. Academics can offer the technical aspect to increase the validity and reliability of the PROMs. Lastly, we propose the creation of a PROMs bank/repository to further enhance the usage of PROMs in low resourced settings. In conclusion, although development and validation of a PROM may seem like a daunting task, it is a worthy challenge given the need of developing context-specific outcome measures.

Acknowledgements

Special thanks go to the caregivers for their voluntary participation. This extended abstract was presented at the World Confederation of Physical Therapy (WCPT) Congress, 2–5 July 2017, Cape Town, South Africa. The principal author would wish to extend his gratitude to Zimbabwe Physiotherapy Association (ZPA) and AMARI for funding travelling and registration costs, respectively.

Funding

The development and validation of the ZCBS is part of the primary authors’ PhD work at the University of Cape Town. The work is being funded by The African Mental Health Research Initiative (AMARI). AMARI is a consortium of four African universities whose overall goal is to build excellence in leadership, training and science amongst African scholars in mental, neurological and substance use (MNS) research in Ethiopia, Malawi, South Africa and Zimbabwe. AMARI, at the University of Zimbabwe College of Health Sciences (UZCHS) secured funding from the Wellcome Trust through the Developing Excellence in Leadership and Science (DELTAS) Africa initiative. The DELTAS Africa Initiative is an independent funding scheme of the African Academy of Sciences (AAS)’s Alliance for Accelerating Excellence in Science in Africa (AESA) and supported by the New Partnership for Africa’s Development Planning and Coordinating Agency (NEPAD Agency) with funding from the Wellcome Trust [DEL-15-01] and the UK government.

Disclosure statement

The views expressed in this publication are those of the author(s) and not necessarily those of AAS, NEPAD Agency, Wellcome Trust or the UK government.