Using GRADE as a framework to guide research on the sexual and reproductive health and rights (SRHR) of women living with HIV – methodological opportunities and challenges

ABSTRACT

In March 2016, WHO reviewed evidence to develop global recommendations on the sexual and reproductive health and rights (SRHR) of women living with HIV. Systematic reviews and a global survey of women living with HIV informed the guideline development decision-making process. New recommendations covered abortion, Caesarean section, safe disclosure, and empowerment and self-efficacy interventions. Identification of key research gaps is part of the WHO guidelines development process, but consistent methods to do so are lacking. Our method aimed to ensure consistency and comprised the systematic application of a framework based on GRADE (Grading of Recommendations, Assessment, Development and Evaluation) to the process. The framework incorporates the strength and quality rating of recommendations and the priorities reported by women in the survey to inform research prioritisation. For each gap, we also articulated: (1) the most appropriate and robust study design to answer the question; (2) alternative pragmatic designs if the ideal design is not feasible; and (3) the methodological challenges facing researchers through identifying potential biases. We found 12 research gaps and identified five appropriate study designs to address the related questions: (1) Cross-sectional surveys; (2) Qualitative interview-driven studies; (3) Registries; (4) Randomised controlled trials; and (5) Medical record audit. Methodological challenges included selection, recruitment, misclassification, measurement and contextual biases, and confounding. In conclusion, a framework based on GRADE can provide a systematic approach to identifying research gaps from a WHO guideline. Incorporation of the priorities of women living with HIV into the framework systematically ensures that women living with HIV can shape future policy decisions affecting their lives. Implementation science and participatory research are appropriate over-arching approaches to enhance uptake of interventions and to ensure inclusion of women living with HIV at all stages of the research process.

KEYWORDS:

• HIV
• GRADE
• sexual and reproductive health and rights
• research gaps
• WHO

Introduction

Development of normative guidance is a key role of the World Health Organization (WHO); such guidance informs the development and revision of national policies and guidelines as well as the development of programmes. The process of WHO guideline development includes identification and documentation of gaps in the available evidence base, notably in the balance of benefits and harms of an intervention (World Health Organization, 2014). Identification of these gaps is intended to guide focused research to inform and strengthen future WHO recommendations.

At a three-day meeting in March 2016, WHO formulated new evidence-based recommendations on sexual and reproductive health and rights (SRHR) of women living with the human immunodeficiency virus (HIV). The consolidated guideline brings together relevant, existing recommendations from WHO guidelines, and also includes new recommendations that were developed with the help of a Guideline Development Group (GDG) of experts, including people living with HIV. The GDG reviewed evidence from systematic reviews in order to formulate new consensus-based clinical recommendations and good practice statements (World Health Organization, 2017). The clinical recommendations were specific to interventions not previously addressed in other WHO guidelines and covered the following four topics: Caesarean section, mode of abortion, safer disclosure of HIV serostatus, and interventions to enhance self-efficacy and empowerment. The good practice statements included guidance on psychosocial support, sexuality, economic empowerment and resource access (including food security), and integrated service delivery, and were not further included in this study.

Research gaps were proposed by the GDG informed by the evidence from systematic reviews and from a global survey of the SRHR priorities of women living with HIV. The survey was commissioned by WHO specifically for this process to be led by and for women living with HIV (Narasimhan et al., 2016; Salamander Trust, 2014). The community survey was led by a global core group of 14 women living with HIV and comprised an online, solution-focused survey of 832 women living with HIV from 94 countries, supplemented by focus groups with 113 women living with HIV from seven countries.

In this article, we present a systematic approach to the prioritisation of the research questions identified during the guideline process, identify optimal study designs, and propose alternative designs and strategies to overcome practical or methodological challenges which may arise. We expanded the methods used in a prior WHO guideline and employed the Grading ofRecommendations, Assessment, Development and Evaluation (GRADE) approach as a framework to guide this process (Siegfried, Beanland, Ford, & Mayer, 2015). We aimed to demonstrate the feasibility and utility of using a structured process within a WHO guideline context to identify gaps in the research of SRHR of women living with HIV.

The GRADE approach and WHO

WHO has adopted the GRADE system for the development of global clinical, programmatic and public health guideline recommendations (Guyatt et al., 2008; World Health Organization, 2014). GRADE provides guideline developers with a transparent, systematic approach to aid their decision-making (Andrews, Guyatt et al., 2013). Each recommendation is formulated in response to an a priori clinical or programmatic question regarding the effectiveness of an intervention. Questions are structured following the PICO format (specifying the Population, Intervention, Comparator, and Outcomes of interest for the review) and are drafted by a WHO-led Steering Group in advance of the final GDG meeting. A systematic review is conducted for each question prior to the GDG meeting and the overall quality of evidence is rated as high, moderate, low or very low, dependent on the risk of bias, precision, consistency, and directness of the results. WHO recommendations are then formulated by the GDG in response to the quality of the evidence combined with an assessment of benefits and harms, resource utilisation, user values and preferences, feasibility, ethics, equity and human rights (World Health Organization, 2014). These domains are summarised in an Evidence to Decision Table (See Table 1). Recommendations are categorised by the GDG as strong or conditional. In general, strong recommendations are made when the quality of evidence is high and the benefits of an intervention outweigh the harms, whereas conditional recommendations recognise that the quality of the evidence is low or that specific country contextual factors will determine the uptake of a recommendation (Andrews, Schunemann et al., 2013).

Table 1. Key domains that require consideration when formulating WHO recommendations.

Factor	How the factor influences the direction and strength of a recommendation
Quality of the evidence	The quality of the evidence across outcomes critical to decision-making will inform the strength of the recommendation. The higher the quality of the evidence, the greater the likelihood of a strong recommendation.
Values and preferences	This describes the relative importance assigned to health outcomes by those affected by them; how such importance varies within and across populations; and whether this importance or variability is surrounded by uncertainty. The less uncertainty or variability there is about the values and preferences of people experiencing the critical or important outcomes, the greater the likelihood of a strong recommendation
Balance of benefits versus harms	This requires an evaluation of the absolute effects of both benefits and harms (or downsides) of the intervention and their importance. The greater the net benefit or net harm associated with an intervention or exposure, the greater the likelihood of a strong recommendation in favour or against the intervention
Resource implications	This pertains to how resource-intense an intervention is, whether it is cost–effective and whether it offers any incremental benefit. The more advantageous or clearly disadvantageous the resource implications are, the greater the likelihood of a strong recommendation either for or against the intervention
Priority	The problem’s priority is determined by its importance and frequency (i.e., burden of disease, disease prevalence or baseline risk). The greater the importance of the problem, the greater the likelihood of a strong recommendation.
Equity and human rights	The greater the likelihood that the intervention will reduce inequities, improve equity or contribute to the realisation of one or several human rights as defined under the international legal framework, the greater the likelihood of a strong recommendation
Acceptability	The greater the acceptability of an option to all or most stakeholders, the greater the likelihood of a strong recommendation.
Feasibility	The greater the feasibility of an option from the standpoint of all or most stakeholders, the greater the likelihood of a strong recommendation. Feasibility overlaps with values and preferences, resource considerations, existing infrastructures, equity, cultural norms, legal frameworks, and many other considerations.

Note: Reproduced from the World Health Organization (2014).

Methods

We adopted a similar approach to future research formulation used in a previously published WHO guideline for HIV post-exposure prophylaxis (Siegfried et al., 2015). The GRADE system provides a framework to determine research gaps based on the strength of the recommendation and quality of the evidence with conditional recommendations and low quality evidence clearly identifying when further research is required. We expanded upon this to include a component specific to the values and preferences of the community the recommendations intend to serve – in this case, women living with HIV. Following the GDG meeting, we tabulated the following for each a priori PICO question, and for additional research questions identified from the global survey:

1. The clinical, programmatic or values-based research question identified by the GDG or the survey;

2. The related recommendation formulated by the GDG or a record that no recommendation was made or that the question was not part of the current GDG process;

3. The strength of each recommendation where applicable; and

4. The quality of evidence underpinning the recommendation where applicable

From the above, a research gap was identified when one or more of the following conditions was met: (1) the recommendation was conditional; or (2) the quality of evidence was low or very low (even in the presence of a strong recommendation); or (3) no recommendation was made and the GDG or survey had articulated a research gap. The authors then considered and tabulated the following:

(5)	The most appropriate and most robust study design to answer the future research question(s);
(6)	Conceptualisation of a pragmatic alternative study design if the ideal design was not feasible; and
(7)	Consideration of the methodological challenges of the study design with the potential bias(es) identified.

Results

Table 2 (See Supplemental data) summarises the 12 research gaps identified specifically around the four main PICO topic areas newly covered in the guidelines: Caesarean section, mode of abortion, safer disclosure of HIV serostatus, and interventions to enhance self-efficacy and empowerment. Five distinct study design formats were identified as feasible and appropriate to address the 12 research questions:

1. Cross-sectional surveys

2. Qualitative interview-driven studies;

3. Establishment of a global registry, or national registries;

4. Randomised controlled trials of comparative effectiveness; and

5. Medical record audit.

We identified selection bias as the primary methodological challenge in mixed methods studies incorporating cross-sectional surveys and qualitative key informant interviews. Healthcare providers and women who choose to participate in such studies may exhibit similar characteristics and be qualitatively different from those who choose not to participate, limiting the generalizability of the results. In some circumstances this may be desirable such as when exploring the promotion of healthy sexuality of women living with HIV as women who have considered this issue will have rich experiences and observations to share which can guide the development of health promotion tools.

Additional methodological challenges identified included confounding in cross-sectional surveys and registries limiting causal inference, and measurement bias arising when blinding is not possible in RCTs or when evaluation tools are not validated to measure highly sensitive outcomes in surveys. Registries are dependent on the quality of data collected and as such are susceptible to misclassification bias, especially if women choose not to disclose their HIV status or do not agree to anonymised testing. Contextual bias is an important consideration in generalising from RCTs of assisted partner notification or disclosure strategies as culture and context will likely play a significant role in the effectiveness of such strategies.

Discussion

Application of a GRADE framework to a WHO guideline on the SRHR of women living with HIV enabled the systematic identification of 12 questions for future research. Our framework comprises the strength of relevant WHO recommendations, the quality of the current supporting evidence base, and a strong values and preferences component to ensure SRHR priorities and desires of women living with HIV are included directly into the research prioritisation process. Five study designs and associated methodological challenges were identified for each of the 12 research questions.

Randomised controlled trials remain the gold standard for evaluating effectiveness where equipoise exists and provide the highest quality evidence for formulating WHO recommendations. However, several of the identified research questions require evaluation of complex, multi-faceted, and often multi-sectoral interventions which may be better suited to evaluation within an implementation research paradigm. Implementation research seeks to explore effects in real-world conditions, mindful of context and user needs (Peters, Adam, Alonge, Agyepong, & Tran, 2013). Outcomes can include acceptability, feasibility and costs in addition to conventional measures of effectiveness. Collaboration with those who will implement and receive the intervention in the design and conduct of the research is a recognised strength of implementation research (Peters et al., 2013). As a paradigm it fits well with the desire of women living with HIV to be equal participants in the research endeavour as articulated in the global survey (Narasimhan et al., 2016).

The global survey of women living with HIV demonstrated that many women continue to experience stigmatising attitudes and behaviours within healthcare settings and from many cadres of healthcare providers when accessing sexual and reproductive health or HIV care (Salamander Trust, 2014). This was noted to be experienced across diverse legal and policy environments. A 2015 systematic review identified an absence of peer-reviewed studies of implementation of rights in healthcare programming and called for urgent research to better integrate rights into healthcare interventions (Kumar, Gruskin, Khosla, & Narasimhan, 2015). To ensure provision of rights-based care and better uptake of services, community participation is an essential element in the success of SRHR and HIV programmes. Negative behaviours of healthcare providers need to be clearly identified and described in future research methodologies, aiming: (1) to provide supportive evidence of the need to implement awareness-raising and training regarding appropriate caring and non-discriminatory behaviours among healthcare providers; (2) to guide development of focused interventions and accountability mechanisms; and (3) to ensure community participation in order to reduce community-based barriers to service uptake and retention in care. Research methods include surveys supplemented with participatory structured qualitative interviews which can be shaped, delivered and analysed by peers. Where the human rights of women living with HIV are shown to be infringed, participatory research using quantitative and qualitative methods may also be appropriate to further explore the role of, and need for, accountability mechanisms within the healthcare sector (Jagosh et al., 2015; Raab & Stuppert, 2014). Participatory research processes may be unfamiliar to many scientists. They take time, funds and expertise in participatory methodologies. However they are increasingly recognised as critical to understanding the psychosocial challenges of patient-oriented care (Jagosh et al., 2015; Peters et al., 2013; Salamander Trust, 2014).

In addition to the time-limited and participant-dependent study designs outlined during the prioritisation process, the team also identified two passive means of data collection and analysis: medical audit and establishment of registries. In situations where active data collection is expensive or not feasible, both methods can provide useful indications of association and trends providing data integrity is ensured, and suitable adjustment is made for potential confounding during the analysis stage. For example, a global termination of pregnancy (TOP) registry would shed light on whether reasons for, and consequences of, TOP differ for women living with HIV. Databases of TOP exist in several countries and provide insight into the scale, reasons, methods, and outcomes of TOP (Justad-Berg, Eskild, & Strom-Roum, 2015; McCall, Flett, Okpo, & Bhattacharya, 2016; Svensson et al., 2014). Monitoring of TOP indications and outcomes would allow for evaluation of accountability mechanisms for ensuring that TOP services are equitably offered regardless of HIV status. Importantly, a TOP registry can enumerate and explore reasons for repeat TOP which may, for example, indicate a failure of effective contraceptive services and/or inaccurate guidance from healthcare providers regarding the SRHR of women living with HIV. Understanding and addressing ineffective contraception services was raised by the survey participants as a priority for women living with HIV with a call for women to be at the centre of all decision-making regarding service provision which affects their lives (Narasimhan et al., 2016; Salamander Trust, 2014). Newer models of registry development adopt a patient-led approach and align data collection with patient priorities (Nelson et al., 2016). Women with HIV should ideally be included in the design, oversight and use of a future TOP registry.

Previous researchers have collated and reviewed the evidence base in the field of SHRH of women living with HIV and have identified a comprehensive set of research gaps (Kumar et al., 2015; Loutfy, Sonnenberg-Schwan, Margolese, & Sherr, 2013). Our paper is limited in scope as the gaps identified are specific to the new recommendations included in the consolidated WHO guideline, and are not intended to be a comprehensive future research agenda for all SRHR issues of women living with HIV. However, our method expands prior research prioritisation approaches as we involved women living with HIV throughout the process and incorporated their voices directly into the GRADE framework.

Transparency is a key strength of the GRADE approach which has here been expanded to identify research gaps based on the strength of the recommendation and the quality of the evidence rating. However, reliance on the strength and the rating of quality alone is insufficient for guiding future research as gaps may remain even when recommendations are strong or quality is high. Paradoxically there are instances when the quality of evidence is low, but it is acceptable to make strong recommendations, such as in life-threatening situations (Alexander et al., 2014). In these circumstances, while more research may be desirable, it may be ethically impossible to conduct more studies. Guideline developers should therefore continue to apply their critical judgement when applying the GRADE framework to identify research gaps.

Conclusions

Use of the modified GRADE framework to establish a focused research agenda for SRHR of women living with HIV is feasible and contributes towards the goal of WHO to collaborate with those most affected by guidelines (World Health Organization, 2014). A systematic approach to identifying research gaps and the necessary study designs to address these provides researchers, including women living with HIV, and funders with the necessary information to conduct appropriate, high-quality research required for future revisions of the guidelines. Adoption of participatory and implementation research will ensure maximal involvement of women living with HIV at all stages of the research process and ultimately, in SRHR decision-making which directly affects their lives.

Acknowledgements

We thank Avni Amin (WHO, Department of Reproductive Health and Research) for her helpful comments on the final draft. We would like to acknowledge the valuable inputs of the WHO Guideline Development Group for the Guideline on SRHR of women living with HIV: Pascale Allotey (School of Medicine and Health Sciences, Monash University Malaysia), Elham Ahmed Abdulla Atalla (Al Hoora Health Center, Arad Health Center, Ministry of Health, Kingdom of Bahrain), Susan Bewley (Women’s Health Academic Centre, King’s College London, London, United Kingdom), Shalini Bharat (School of Health Systems Studies, Tata Institute of Social Sciences, Maharashtra State, India), Elizabeth Anne Bukusi (Research Care Training Program (RCTP), Center for Microbiology Research, Nairobi, Kenya) (Co-Chair), Caroline Phiri Chibawe (Mother and Child Health, Ministry of Health, Lusaka, Zambia), Sophie Dilmitis (International Community of Women Living with HIV, Harare, Zimbabwe), Raoul Fransen-Dos Santo (International Civil Society Support, Amsterdam, The Netherlands), Asha George, (University of Cape Town, South Africa), Selma Hajri (African Network for Medical Abortion (ANMA), Tunis, Tunisia), Maurice Tomas Hiza (Reproductive and Child Health, Ministry of Health, Dar es Salaam, Tanzania), Mona Loutfy (Women’s College Research Institute, Women’s College Hospital, University of Toronto, Toronto, Canada), Chisale Mhango (Royal College of Obstetrics and Gynaecology, College of Medicine, Blantyre, Malawi), Glen Douglas Liddell Mola (School of Medicine and Health Services, Papua New Guinea), Rejoice Nkambule (Public Health Services, Ministry of Health, Kingdom of Swaziland), Vera SilviaFacciolla Paiva (PST & NEPAIDS, University of Sao Paulo, Brazil), Trang Thi Thu Quach (Center for Creative Initiatives in Health and Population (CCIHP), Ha Noi, Viet Nam), Paul Van Look (Independent Consultant, Belgium and Switzerland) (co-Chair), Alice Welbourn (Salamander Trust, London, United Kingdom), Anandi Yuvaraj (Independent Community Health Activist, Gender, SRH and HIV, Uttar Pradesh, India). The following additional WHO staff were part of the Steering Group: Avni Amin (Department of Reproductive Health and Research), Rachel Baggaley (Department of HIV), John Beard (Department of Aging and Life Course), Ana Pilar Betran (Department of Reproductive Health and Research), Francoise Bigirimana (AFRO Regional Office), Sonja Caffe (AMRO/PAHO Regional Office), Rebekah Bosco Thomas (Gender, Equity and Rights Team), Nathalie Broutet (Department of Reproductive Health and Research),Tarun Dua (Department of Mental Health and Substance Abuse), Shaffiq Essajee (Department of HIV), Jane Ferguson (Department of Maternal, Neonatal, Child and Adolescent Health), Mario Festin (Department of Reproductive Health and Research), Bela Ganatra (Department of Reproductive Health and Research), Claudia García Moreno (Department of Reproductive Health and Research), Fahmy Hanna (Department of Mental Health and Substance Abuse), Rajat Khosla (Department of Reproductive Health and Research), Gunta Lazdane (EURO Regional Office), Ramez Mahaini (EMRO Regional Office), Leopold Ouedraogo (AFRO Regional Office), Razia Pendse (WHO Regional Office for South-East Asia), Sheryl van der Poel (Department of Reproductive Health and Research), Joanna Vogel (Department of Maternal, Neonatal, Child and Adolescent Health).

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This paper was funded by the World Health Organization.