A Review of Initiatives that Link Provider Payment with Quality Measurement of Maternal Health Services in Low- and Middle-Income Countries

Abstract

Abstract—To reduce maternal and newborn morbidity and mortality, health care payers are experimenting with ways to better align incentives to promote high-quality maternal health services. This review examined 26 recent initiatives of health care payers in 16 low- and middle-income countries to pay for quality, and not solely quantity, of maternal health services. Payers measured quality by assessing availability of structural inputs (24 of 26 cases), adherence to processes (25 of 26 cases), and observation of key outputs of health facilities (14 of 26 cases). Two payers sought to also assess quality through observed patient outcomes. In 25 of the initiatives, payers used the quality assessment to adjust facility payments; in the remaining initiative, the payer used the quality assessment to adjust payments to provincial governments, which in turn pay facilities. The recent growth in such payment systems suggests more health care payers have identified ways to link quality measurement with provider payment mechanisms. Eleven impact evaluations of systems documented changes in provider behavior consistent with various elements of quality; however, only three evaluations reported effects on maternal or newborn morbidity and mortality and do not conclude whether the design or flaws in how it was implemented led to the results. Implementation fidelity—the degree to which the initiative was implemented as designed—was not widely addressed and is an area for future research. Furthermore, although payers in low- and middle-income countries have identified ways to operationalize a payment system that adjusts payments based on some measure of quality, the complexity and level of resources required to operationalize them raise concerns about sustainability.

Keywords:

• maternal health
• provider payment
• quality
• results-based financing
• strategic purchasing

INTRODUCTION

Although use of facility-based maternity services has increased, maternal mortality in developing nations remains unacceptably high at 232.8 per 100,000 live births in 2013. The majority of these deaths occur during labor, delivery, and the immediate postpartum period, with obstetric hemorrhage being the main medical cause of death.¹ Maternal morbidities such as anemia, fistula, uterine rupture and scarring, and genital and uterine prolapse also represent significant global burdens.²

To reduce maternal and newborn morbidity and mortality, high coverage of maternal health interventions needs to be matched with overall improvements in quality of maternal health care. Health facilities and their staff play a key role in this effort. Sustainable Development Goal 3 makes the link between improved outcomes and an increase in the proportion of births attended by skilled health personnel.³ A facility-based intrapartum care strategy has been identified by experts in the field as critical to achieving reductions in maternal mortality.⁴ However, high rates of facility-based deliveries are not enough. A 2013 study using data from 29 countries compared the burden of complications related to pregnancy with the coverage of key maternal health interventions in facilities and concluded that high coverage alone of essential maternal health interventions did not correlate with reduced maternal mortality if the interventions provided were of poor quality.⁵

Organizations or governments paying for health care services—henceforth referred to as purchasers—can influence the quality of services produced within the health system. How providers are paid matters for encouraging availability of key inputs for high-quality maternal health services.⁶ Dysfunctional incentives for health workers, or inadequate mechanisms to support, supervise, and hold workers accountable, can also stymie provision of high-quality care.⁷ Provider payment initiatives can be designed to promote results-oriented behaviors and better align incentives with desired outputs.⁸

This review identifies and describes recent initiatives in low- and middle-income countries that aim to link provider payment with quality measurement of maternal health services. We discuss the implications of the findings, identify gaps in the available literature, and propose areas for further research.

BACKGROUND

Poor-quality services provided at health facilities are increasingly of concern for maternal health as more women in developing countries choose to deliver at health facilities. Substandard care by a health worker accounted for two thirds of avoidable factors contributing to maternal or perinatal deaths in a systematic review of mortality audits (the other one third included patient-oriented factors such as patient delay, administrative/supply factors such as medication shortage, and transport/referral factors).⁹ A 2013 review found examples of suboptimal provider performance that contribute to low quality of care, including absenteeism; the “know–do gap” (failure to do in practice what a provider knows to do in principle); providing unnecessary or incorrect services; and failing to provide recommended preventive and outreach services.¹⁰ Real or perceived poor quality of care at facilities can lead to public mistrust of the system, resulting in lower demand for services.¹¹ Facility-based maternal health services have been found to be cost-effective in studies that measured efficacious services delivered by skilled professionals, but services provided in real-world settings by less-skilled professionals may not be effective at all, let alone cost-effective.¹²

Although providers in resource-poor settings face immense challenges and barriers beyond their immediate control, there is room for quality improvement at the point of care within current resource constraints. Health worker adherence to high-quality clinical practice guidelines, when combined with simulation-based training, can improve providers' clinical skills, attitudes, and respectful care.¹³ Multiple conceptual models of quality of care identify health workforce motivation and provider actions as key inputs to quality at the point of care.^14-17

Financial incentives have been used to improve provider motivation and adherence to clinical guidelines in higher-income countries for many years, including for maternal health services. Yet until recently, few health care purchasers in low- and middle-income countries linked provider payments to quality. A 2012 Cochrane Systematic Review of performance-based financing studies that reported on performance or patient outcomes identified nine payment interventions, and only one linked payments to quality of care.⁶ The United States Agency for International Development's (USAID) 2012 Maternal Health Evidence Summit reviewed the literature on how financial incentives enhance the quality and uptake of maternal health care; reviewers found that few studies explicitly discussed whether quality was incentivized in the programs they evaluated, and few reported effects on quality measures.¹⁸ A 2013 systematic review of the effect of health insurance on maternal and neonatal health found that few studies focused on the relationship between health insurance and the quality of maternal health services.¹⁹

Since the 2012 Cochrane Systematic Review, several initiatives have begun linking provider payment for maternal health services with quality of those services. Some of these initiatives fall under the heading of performance-based financing or results-based financing because payment is partially or fully contingent on delivering services that meet a predetermined standard of quality; however, it is important to note that not all initiatives using these terms link payment with quality. A recent study described how quality of care assessments factored into 32 performance-based financing schemes²⁰; the broad scope of that study did not allow much discussion on quality of maternal health services specifically. This review summarizes literature produced since the 2012 Cochrane Systematic Review. We review and discuss recent provider payment initiatives by health care payers in low- and middle-income countries to incentivize high quality of maternal health service delivery.

METHODS

Inclusion Criteria

We searched for provider payment initiatives in which one of the design objectives was to improve quality of care, including maternal health care. For a case to be eligible for inclusion, one or more maternal health care quality indicators must be regularly measured as part of the system, and at least part of the provider payment must be based on the quality of maternal health care indicators. Payment initiatives that do not specifically measure and link payment to quality of maternal health care indicators were excluded. Payment initiatives that pay providers solely on measures of volume of services or access to services, regardless of the provider's quality at the point of care, were excluded.

Search Strategy

We conducted keyword and free text searches in the following electronic reference libraries to identify potential cases: PubMed, ProQuest, World Bank's RBFhealth.org Database, Google, and Google Scholar. Keywords included combinations of “quality,” “maternal health,” “provider payment,” “performance-based financing,” “results-based financing,” and “strategic purchasing.” We considered French- and English-language published articles from peer-reviewed journals and published and unpublished program reports that included details on how a specific provider payment system linked maternal health quality measurement to provider payment and, if available, evaluation results of the system. We also performed detailed examination of cross-references and bibliographies of available publications to identify additional sources of information and drew on author and other experts' knowledge.

The search identified 74 peer-reviewed articles and program reports covering information from 30 low- and middle-income countries in PubMed, ProQuest, and Google Scholar. In addition, 36 payment initiatives that link provider payment with quality measurement in 31 countries were identified through the RBFhealth.org database and cross-referenced with the initiatives identified through peer-reviewed articles. Additional initiatives were identified through cross-references and bibliographies. We excluded initiatives that did not meet the inclusion criteria specified above, which resulted in 26 cases from 16 countries.

We stratified initiatives by the following payment recipient types: provincial governments, referral facilities, and primary care facilities. Purchasers that paid both referral facilities and primary care facilities were considered two separate cases. Purchasers that paid the same type of recipient using two different methods were also considered two separate cases.

Data Extraction

We extracted data from the peer-reviewed articles and program reports identified through the search and from two databases in the public domain. After the study team reviewed all of the articles and program reports identified during the search for relevance, 33 articles and program reports were included for data extraction. We extracted standardized information on the chosen payment mechanisms, how they linked to quality, and the approaches used to assess quality. We obtained each country's income category from the World Development Indicator database for calendar year 2015.²¹ We accessed the Multi-Country Performance Based Incentives Quality Checklist Database to extract information about elements of quality assessed and assessment methods used by any provider payment initiative included in both this study and in the database (15).²² The database lists and categorizes quality of care indicators used in some performance-based financing initiatives and was developed under the USAID Translating Research into Action (TRAction) Project and published in August 2016. For the nine provider payment initiatives included in this study but not included in the database, we extracted the information from within the articles and program reports.

RESULTS

The literature search identified 26 provider payment initiatives that linked payment to quality of maternal health care services in 16 low- and middle-income countries. Initiatives were launched between 2004 and 2015. Two countries were designated upper-middle income, six countries were designated lower-middle income, and eight countries were designated low income in calendar year 2015.

The 26 provider payment initiatives shared basic design features by virtue of the search strategy and scope of the review. In each initiative, an entity (a government agency, a donor, or another risk-pooling entity such as an insurance agency) acted as a purchaser of health services. This purchaser's role was to finance health care delivery by paying providers to deliver health services to a population. This purchaser–provider arrangement is quite common; it exists anywhere that health care is not solely financed by user fees paid by patients at the point of care. What makes these 26 initiatives unique from the common purchaser–provider arrangement is that under each initiative, the purchaser opted to implement a quality assessment process and use the results of the assessment to adjust payments to the providers.

Although sharing those basic design features, the 26 provider payment initiatives varied in several ways. These design variations include the type of entity that acted as the purchaser; the type of health care provider that was the payee; the elements of health care quality assessed; the quality assessment method; the quality assessment frequency; the way in which the payee's payment was modified; and the payment frequency. These variations are explored below.

Types of Purchasers and Payees

In most cases, the purchaser included a donor partner and the country's Ministry of Health. The Health Results Innovation Trust Fund (HRITF), a multidonor trust fund administered by the World Bank, was the primary funding source for 16 of the 26 cases. Five cases were funded through other World Bank funding mechanisms, two by USAID, one by the Chinese Ministry of Health, one by the Department for International Development, and one jointly funded by the governments of Norway and Germany. National ministries of health were reported as the sole implementing organization for most schemes. Co-implementation by another sub-national or external organization with the national Ministry of Health was reported for six of the 26 cases. Only one case did not list any government agency as an implementing organization. Appendix A summarizes the purchaser (including the primary funding source and the implementing organization) and the payee (payment recipient) of the 26 initiatives.

The payee in each of the 26 cases fell into one of three categories: sub-national governments, referral facilities, and primary care facilities. One initiative linked payment to sub-national governments with quality of maternal health services. This initiative was included because the quality-adjusted payment to the sub-national government directly affected provider payment. Nine initiatives linked payments to hospitals with quality of maternal health services, and the remaining 16 initiatives linked payments to primary care facilities to quality of maternal health services.

Purchasers' Strategies to Purchase Quality Maternal Health Services

Purchasers employed a variety of strategies to purchase good quality in general, including good quality maternal health services. Appendix B shows the five design elements of the purchaser's strategy in each of the 26 cases to purchase quality: elements of quality assessed, assessment method, assessment frequency, payment modification, and payment frequency.

Purchasers assessed maternal health service quality using indicators of outcomes, outputs, processes, or structural inputs. Table 1 shows an example to illustrate each type of quality indicator. Outcomes were measured in only two of the 26 cases. In slightly over half the cases, purchasers assessed outputs as a way of measuring quality. In most cases, purchasers assessed processes and structural inputs of providers.

TABLE 1. Illustrative Indicators of Maternal Health Service Quality, by Element of Quality Measured. VDRL = Venereal Disease Research Laboratory, RPR = rapid plasma reagin, IPT = intermittent preventive treatment, ANC = antenatal care, RBF = results-based financing, P4P = performance-based financing

Element of Quality Measured	Indicator Related to Quality of Maternal Health Services
Outcome	Five-minute Apgar scores of over six
Output	The following have been noted for all visits: (five points) (1) hemoglobin, (2) venereal disease research laboratory (VDRL) / rapid plasma reagin (RPR), (3) blood pressure checks, (4) intermittent preventive treatment (IPT) for malaria (if pregnancy over 20 weeks), (5) tetanus vaccine administered accordingly, (6) fetal heart rate/lie/presentation, (7) ferrous sulfate/folic acid
Output	Antenatal care with defined quality parameters—starting before 16 weeks, four visits or more, including provision of tetanus vaccination and malaria prevention, with appropriate measures for the prevention of mother-to-child transmission of HIV
Process	Correct prescription of (1) iron and folic acid, (2) mebendazole, (3) insecticide-treated net, (4) compliance with sulfadoxine pyrimethamine protocol (refer to ten ANC records)
Process	Analysis of 10 randomly selected partograms: (1) partogram filled out according to the rules, (2) decision made/documented if alert line is passed within one hour, (3) delivery by qualified staff (at least a nurse, midwife, clinical officer, doctor)
Structural input	Available and functional equipment and supplies: (1) consultation table, (2) blood pressure cuff, (3) stethoscope, (4) tape measure, (5) scale with height gauge, (6) fetoscope, (7) unused and nontorn surgical gloves
Structural input	Privacy: Curtains or painted windows, room divider (if shared room), doors that close, running water (tap or bucket with tap), three buckets for infection prevention, labeled

Purchasers employed multiple methods to assess quality. In 23 cases, purchasers used on-site checklists combined with one or more other methods (direct observation, patient record review, patient/household survey, register review, and staff interview). In the remaining three cases, purchasers did not use on-site checklists but analyzed provider-reported electronic data without on-site verification of those data (Argentina's Plan Nacer) or assessed quality through household surveys (India's Karnataka Health System Development Project). Quality assessments occurred quarterly in 22 of the 26 cases.

Purchasers also employed a variety of methods to modify payments based on the quality assessment. In most of the cases, the purchaser paid providers on a fee-for-service basis for the quantity of services provided during the assessment period and then adjusted the total facility payment proportional to the composite quality score. These facility payments were often, but not always, referred to as bonus payments. In some cases, no bonus payment was made unless the facility met or exceeded a predetermined target for each of the indicators individually or for the composite quality score.

Two cases allocated payments based on the expected volume of care in a catchment area. In one scheme in China, part of the facility's global budget (determined by the number of patients in the catchment area) was initially withheld and later paid based on the quality assessment. A global budget-derived bonus was the payment mechanism employed in Rwanda's hospital scheme. Appendix C identifies the methods used by purchasers or their surrogates in each of the 26 cases to measure quality of care.

How the Provider Payment Initiative Improved Quality of Maternal Health Services and Maternal and Newborn Health Outcomes

External evaluations were completed on 11 of the 26 initiatives. These evaluations reported findings related to one or more of the following: the effects of the initiative on the quality of maternal health services, the effect of the initiative on maternal and newborn health outcomes, and the effect of the initiative on provider (individual or facility) behavior. Appendix D summarizes evaluation findings on the effect of the initiative on quality of maternal health services and the effect of the initiative on maternal and newborn health outcomes. The external evaluations were of mixed quality: some were conducted as randomized control trials and relied on data sources independent of the initiative, whereas others relied on data collected through the initiative itself. Several evaluations reported qualitative findings collected on a small, nonrepresentative sample.

Three external evaluations identified changes in facility management that were associated with improvements in quality of maternal health services. Three evaluations reported that basing provider payment on quality promoted better management. The evaluation of Burundi's initiative reported improved monitoring systems at all facility and district levels, improved governance structures to analyze and hold service providers accountable for results, development of verification activities and evaluations to measure the effects at household level, and introduction of guidance to institutionalize changes at facilities.²³ In Rwandan hospitals, the payment initiative helped to clarify the responsibilities and roles of all parties involved in the production, monitoring, and evaluation of health services, although it was unclear whether this clarification led to an improvement in quality.²⁴ The evaluation of the payment initiative in Senegal reported strengthened leadership of health post directors, increased involvement of community health workers, more transparent financial management of the facility, and improved recording and monitoring of services provided.²⁵ These evaluations did not seek to measure the direct correlation between improvements in patient outcomes with such observed management results, so results should be interpreted with caution.

Two evaluations reported that the payment initiative promoted better care processes that are typically associated with higher quality at the point of care. In Zimbabwe, qualitative research conducted in five districts found that improved teamwork, facilitated by the team-based incentives and more regularly received structured supervision and feedback, improved health worker performance and enhanced community participation.²⁶ The initiative in Senegal was found to improve communication and promote better division of labor among facility staff, improve working conditions (including hygiene, infrastructure, and availability of equipment), and improve monitoring of drugs stocks and procurement.²⁵

One evaluation in Uganda found that the outcomes observed after basing provider payment on quality were not uniform across facilities. The evaluation found that the financial incentives introduced through the provider payment initiative increased quantity and quality of health services provided in facilities that had more transparent communication between management and clinical staff more than in facilities without this observed characteristic. However, when a staff incentive was anticipated but not provided, staff demotivation and, in some cases, boycotts were observed.²⁷

One of the evaluations of the Rwanda hospital initiative discussed the operational difficulties in and considerations for institutionalizing quality measurement. First, hospitals and evaluators had difficulty understanding some indicators and their composite criteria. An operations manual was not available, and peer evaluators had to rely heavily on technical assistance from central government staff. Second, gathering of information and uniform interpretation of data were difficult due to the lack of standardization of medical files and forms in the hospitals, and the evaluation tool was initially very complex. Aside from these difficulties, the evaluation found that hospitals were able to achieve high-quality scores quickly, which prompted two revisions of the evaluation grid over three years in order to make the evaluation criteria more specific, precise, and measurable and to adapt to changing needs observed at hospitals.²⁴ This finding may be helpful for designers and managers of future quality measurement initiatives to consider when rolling out a new system that can promote continuous quality improvement.

Other evaluations noted additional implementation challenges. The program in China brought additional patients into the participating hospitals, but hospitals' efforts to reorganize staff lagged behind the increased workload.²⁸ The program in Uganda encouraged private facilities to better retain staff given the effort required to train staff in providing higher-quality care. However, the evaluation found that turnover was a repeated challenge as staff in the participating private facilities continued to transition to public-sector positions. There was a perception that the workload in participating private clinics was considerably higher than at public facilities in the area, though salaries were lower. As a result, gains in quality of care that had been achieved were often lost when a health worker left the facility.²⁷

Authors of three evaluations concluded that the size of the provider payment—in both absolute terms and relative to payments for other services—is an important factor in improving quality at the point of care. In all of the cases, the payment that is tied to quality is considered a subsidy, or a payment above and beyond the cost of inputs required to provide care. This subsidy is intended to promote the provider behaviors that lead to high quality at the point of care and ultimately to improvements in utilization. The authors of the Burundi evaluation compared the experience in Burundi to Rwanda and found that the probability of institutional delivery improved in Rwanda but not in Burundi. They posit that this finding could be explained by the relatively low subsidy for institutional deliveries in Burundi compared with other services, unlike in Rwanda, where the subsidy for institutional deliveries was higher.²⁹ The authors of the China evaluation suggested that the incentive payments in the program—although not tied to maternal health—might not have been large enough to induce behavioral change among providers managing chronic illnesses.²⁸ Authors of the evaluation in Rwandan hospitals found that subsidies were considered inadequate for the requisite efforts made.²⁴

DISCUSSION

This review identified at least 26 cases where purchasers in low- and middle-income countries are experimenting with paying providers based on quality of maternal health services in an effort to improve the quality of maternal health services. This apparent increase in cases in recent years suggests that more health care purchasers recognize the potential to design payment systems to incentivize quality improvement and have identified ways to operationalize the approach. Details of the payment models and evaluation results indicate that conditioning provider payment on quality can improve facility management and enhance care processes that are associated with higher levels of care. Challenges that impede the potential of linking payment to quality include staff turnover and unreliable payment.

These cases indicate that payment mechanisms that reward quality have potential to stimulate improvements in management processes at facility level. By specifying quality measures and holding facility teams accountable for achieving them, facility managers and staff monitor and report on achievements and are stimulated to improve the quality that is rewarded through the payment system. Cases report that information, drug and financial management systems, and the use of those systems are improved. In addition, cases document increased supervisory visits that may contribute to improved quality.

These cases also provide evidence of better care processes that are associated with higher levels of quality. For example, incentives that reward facilities stimulate teamwork. Regular and structured supervision with feedback strengthens care delivery. Cases document improved working conditions that lead to improved care processes such as better monitoring of drug stocks, timely procurement, improved hygiene, and enhanced availability of equipment.

In settings with frequent staff rotation, turnover is a challenge to realizing quality improvements associated with payment systems that reward quality because new staff need to be retrained and team work may weaken. Cases that documented irregular or unreliable payment found that staff became unmotivated. In addition, getting the payment levels at a high enough level to stimulate change was a challenge in some cases.

In most cases, quality is assessed through on-site monitoring by purchasers or their surrogates (e.g., district health officials) using on-site checklists combined with one or more other methods (direct observation, patient record review, patient/household survey, register review, and staff interview). This approach appears popular particularly in lower-income countries. A minority of cases do not use on-site evaluations. One example is Argentina's Plan Nacer, which measures quality by analyzing facility-reported electronic data and does not regularly conduct on-site verification of those data. This may be because self-reported health facility data in Argentina are generally complete and accurate. Incorporating quality measurement into a payment system in a country with widespread data incompleteness and inaccuracy appears to require on-site verification. This raises questions about the long-term sustainability of such initiatives, given that an in-person verification process can be quite resource intensive.

This review has several limitations. We only included provider payment initiatives in low- and middle-income countries that had documentation available in the web-based public domain. There are likely other initiatives being tried by purchasers in low- and middle-income countries that were not included due to lack of available documentation. It is likely that strategies employed by private health care payers in particular would not have been identified through this review. Second, a large proportion (20 of the 26 cases) received financing through the World Bank, through either the Health Results Innovation Trust Fund (15 cases) or another financial vehicle (5 cases). The authors note that the Trust Fund cases have similar designs, which limits our ability to compare and contrast a wide variety of approaches. A strategy applied in a majority of study cases does not necessarily mean that different payers have converged on that strategy, and majority findings should be interpreted with caution. Finally, less than half of the cases had external evaluations, so this study is not able to conclude which purchasing strategies are most effective in improving quality maternal health services or improving maternal and newborn health outcomes.

Available impact evaluations reported the provider response to the payment system intervention and effects on health outcomes (findings presented in Appendix D). Most evaluations discuss contextual details surrounding the results, which we are not able to do so in this article. Of note, our review found that implementation fidelity—the degree to which the initiative was implemented as designed³⁰—is not widely addressed in impact evaluations. This finding is consistent with a 2013 literature review of evaluations of performance-based financing initiatives in low- and middle-income countries.³¹ Implementation fidelity is important to consider when interpreting evaluation results, because the initiative's impact on quality of maternal health services and maternal and newborn health outcomes will be affected not only by the appropriateness of the design but also by the degree to which the initiative was implemented as designed. For example, the available literature includes very little discussion of how well purchasers or their surrogates were able to measure quality of care and communicate results to health workers. Future implementation research could shed light on whether the impact of an initiative was likely muted by poor implementation and provide lessons learned for health care payers seeking to implement similar programs.

Overall, findings from this review can help other payers in low- and middle-income countries seeking to improve quality of maternal health services identify models for potential replication or adjustment, as well as identify available documentation and evaluations of different models to assist with that process.

DISCLOSURE OF POTENTIAL CONFLICTS OF INTEREST

No potential conflicts of interest were disclosed.

ACKNOWLEDGMENTS

The authors are grateful to the maternal health quality and health financing experts who reviewed the study findings and provided valuable technical comments.

Additional information

Funding

This manuscript was funded by the U.S. Agency for International Development (USAID) as part of the Health Finance and Governance project (2012-2018), a global project working to address some of the greatest challenges facing health systems today. The project is led by Abt Associates in collaboration with Avenir Health, Broad Branch Associates, Development Alternatives Inc., the Johns Hopkins Bloomberg School of Public Health, Results for Development Institute, RTI International, and Training Resources Group, Inc. This material is based upon work supported by the United States Agency for International Development under cooperative agreement AID-OAA-A-12-00080. The contents are the responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government.

APPENDIX A

: CASES MEETING INCLUSION CRITERIA, ALPHABETIZED BY COUNTRY FOR EACH TYPE OF PAYEE

Table

						Purchaser	Payee
Case No.	Country	Country Income Level	Program/Scheme Name	Year Launched	Primary Funding Source	Implementing Organization	Payment Recipient
Payee: Sub-national government
G-1	Argentina	Upper-middle	Plan Nacer	2004	World Bank	Ministry of Health, Argentina	Provincial governments
Payee: Referral facilities
R-1	Burundi	Low	Performance-based financing	2006	Government of Burundi; HRITF	Government of Burundi	Hospitals
R-2	Cameroon	Lower-middle	Performance-based financing initiative in Cameroon	2012	HRITF	Performance Purchasing Agency contracted by the Ministry of Health, Cameroon	Hospitals
R-3	Kyrgyz Republic	Lower-middle	Health and Social Protection Project: Results-based financing (RBF) pilot for hospitals	2012	HRITF	Kyrgyz Ministry of Health	Rayon (district) hospitals
R-4	Nigeria	Lower-middle	Nigeria States Health Program Investment Project (Nasarawa and Ondo states as well as in all northeastern states)	2014	HRITF	Federal Ministry of Health, national and state primary health care development agencies	Secondary care providers
R-5	Rwanda	Low	Rwanda national PBF scheme for district hospitals	2006	World Bank	Ministry of Health, Rwanda	District hospitals
R-6	Sierra Leone	Low	Reproductive and Child Health Project	2011	HRITF	Ministry of Health and Sanitation	Hospitals
R-7	Tanzania	Low	Tanzania results-based financing system	2015	World Bank	Ministry of Community Development, Gender and Children	District hospitals
R-8	Zambia	Lower-middle	Zambia results-based financing scheme	2012	HRITF	Ministry of Health	District hospitals
R-9	Zimbabwe	Low	Health Sector Development Support Project	2011	HRITF	Ministry of Health and Child Care; Cordaid	District hospitals
Payee: Primary care facilities
P-1	Benin	Low	Benin results-based financing, Health System Performance Project	2011	HRITF	Ministry of Health, Benin; independent verification agency	Health centers
P-2	Burundi	Low	Performance-based financing	2006	Government of Burundi; HRITF	Government of Burundi	Health centers
P-3	Cameroon	Lower-middle	Performance-based financing initiative in Cameroon	2012	HRITF	Performance Purchasing Agency contracted by the Ministry of Health, Cameroon	Health centers
P-4	China	Upper-middle	Separation of revenue and charges scheme	2005	Ministry of Health, China	Ministry of Health, China	Community health centers and rural mutual health care village clinics
P-5	India	Lower-middle	Karnataka Health System Development Project	2013	HRITF	Ministry of Health and Family Welfare, government of Karnataka	Rural, private obstetric care providers
P-6	India	Lower-middle	Karnataka Health System Development Project	2013	HRITF	Ministry of Health and Family Welfare, government of Karnataka	Rural, private obstetric care providers
P-7	Malawi	Low	RBF4MNH Initiative	2012	Government of Norway, government of Germany	Malawi Ministry of Health	Health centers
P-8	Malawi	Low	Support for service delivery—integration	2014	USAID	Malawi Ministry of Health	Health centers
P-9	Nigeria	Lower-middle	Nigeria States Health Program Investment Project (Nasarawa and Ondo states as well as in all northeastern states)	2014	HRITF	Federal Ministry of Health (via the National Primary Health Care Development Agency); State Primary Health Care Development Agency	Primary care providers
P-10	Rwanda	Low	Rwanda national performance-based financing (P4P) scheme for primary health centers	2005	World Bank	Ministry of Health, Rwanda	Primary health centers
P-11	Senegal	Lower-middle	Performance-based financing	2012	USAID	Ministry of Health and Social Action	Health centers and health posts
P-12	Sierra Leone	Low	Reproductive and Child Health Project	2011	HRITF	Ministry of Health and Sanitation	Peripheral health units
P-13	Tanzania	Low	Tanzania results-based financing system	2015	World Bank	Ministry of Community Development, Gender and Children	Health centers
P-14	Uganda	Low	Northern Uganda Health Programme	2011	UK Aid	Health Partners International	Private not-for-profit facilities
P-15	Zambia	Lower-middle	Zambia results-based financing scheme	2012	HRITF	Ministry of Health	Rural health centers
P-16	Zimbabwe	Low	Health Sector Development Support Project	2011	HRITF	Ministry of Health and Child Care; Cordaid	Health centers

APPENDIX B:

COMPARISON OF QUALITY MEASUREMENT MECHANISMS

Table

		Elements of Quality Measured (related to Maternal Health Services and Others)				Assessment Method
Case No.	Country	Outcomes	Outputs	Process	Structural Inputs	Analysis of Electronic Data without On-site Verification	On-site Checklist	Direct Observation	Patient Record Review	Patient/Household Survey	Register Review	Staff Interview	Assessment Frequency
G-1	Argentina	X	X	X	X	X							Every four months
R-1	Burundi		X	X	X		X		X		X	X	Quarterly
R-2	Cameroon		X	X	X		X		X		X	X	Quarterly
R-3	Kyrgyz Republic		X	X	X		X	X	X	X		X	Quarterly
R-4	Nigeria		X	X	X		X	X	X		X	X	Quarterly
R-5	Rwanda		X	X	X		X	X	X		X		Quarterly
R-6	Sierra Leone			X	X		X						Quarterly
R-7	Tanzania		X	X	X		X		X	X			Quarterly
R-8	Zambia			X	X		X	X	X		X		Quarterly
R-9	Zimbabwe			X	X		X						Quarterly
P-1	Benin		X	X	X		X		X		X	X	Quarterly
P-2	Burundi			X	X		X		X		X		Quarterly
P-3	Cameroon		X	X	X		X		X		X		Quarterly
P-4	China		X	X	X		X						Annually
P-5	India	X								X			One-time measurement
P-6	India			X						X			One-time measurement
P-7	Malawi		X	X	X		X		X		X		Quarterly
P-8	Malawi			X	X		X	X	X	X		X	Quarterly
P-9	Nigeria		X	X	X		X	X	X		X	X	Quarterly
P-10	Rwanda		X	X	X		X	X	X		X		Quarterly
P-11	Senegal			X	X		X						Quarterly
P-12	Sierra Leone			X	X		X				X		Quarterly
P-13	Tanzania		X	X	X		X		X	X	X		Quarterly
P-14	Uganda			X	X		X			X			Quarterly
P-15	Zambia			X	X		X	X	X		X		Quarterly
P-16	Zimbabwe			X	X		X						Quarterly
Total		2	14	25	24	1	23	8	16	7	14	7	—

APPENDIX C:

COMPARISON OF PURCHASING STRATEGIES TO PURCHASE QUALITY—PAYMENT MODIFICATIONS

Table

Case No.	Country	Payment Modification	Payment Frequency
G-1	Argentina	Budget transfer from national to provincial governments based on enrollment and quality measurement	Monthly and quarterly^a
R-1	Burundi	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
R-2	Cameroon	Payment for quantity of services provided in the quarter, adjusted for quality if minimum threshold reached (if minimum threshold reached for each indicator)	Quarterly
R-3	Kyrgyz Republic	Bonus payments totaling up to 10% of a hospital's maternal and newborn health budget for demonstrated quality improvements	Quarterly
R-4	Nigeria	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
R-5	Rwanda	Global prospective budget-derived bonus	Quarterly
R-6	Sierra Leone	Payment for quantity of services provided in the quarter, adjusted for quality if 50% quality score achieved	Quarterly
R-7	Tanzania	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
R-8	Zambia	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
R-9	Zimbabwe	Payment for quantity of services provided in the quarter, adjusted for quality if predefined performance threshold surpassed	Quarterly
P-1	Benin	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
P-2	Burundi	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
P-3	Cameroon	Payment for quantity of services provided in the quarter, adjusted for quality (if minimum threshold reached for each indicator)	Quarterly
P-4	China	Payout of withheld funds of the facility's global budget	Annually
P-5	India	Bonus payment based on decreasing linear function of incidence of four adverse maternal and neonatal health outcomes	One-time payment
P-6	India	Bonus payment based on increasing linear function of quality	One-time payment
P-7	Malawi	Payment for number of services provided or quality assurance actions completed in the quarter	Quarterly
P-8	Malawi	Payment for quantity target achieved, adjusted for quality	Quarterly
P-9	Nigeria	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
P-10	Rwanda	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
P-11	Senegal	Payment for quantity of services provided in the quarter, adjusted for quality if targets for output indicators are met	Quarterly
P-12	Sierra Leone	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
P-13	Tanzania	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
P-14	Uganda	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
P-15	Zambia	Payment for quantity of services provided in the quarter, adjusted for quality	Quarterly
P-16	Zimbabwe	Payment for quantity of services provided in the quarter, adjusted for quality if predefined performance threshold surpassed	Quarterly

^a 60% of the per capita value monthly based on enrollment; 40% of the per capita value every four months based on tracer indicators.

APPENDIX D:

LIST AND SUMMARY OF QUALITY-BASED PROVIDER PAYMENT INITIATIVE EFFECTS ON MATERNAL HEALTH SERVICES FROM IMPACT EVALUATIONS

Table

Case No.	Country	Reported Effect on Provision of Maternal Health Services	Reported Effect on Maternal and Newborn Health Outcomes	Citation
G-1	Argentina	• At facilities enrolled in the program:	• Overall impact of clinic adopting the program is 22% reduction in neonatal mortality (regardless of individual beneficiary status)	Gertler et al.^32
		• Increase in the number of prenatal care visits	• The impact on Plan Nacer beneficiaries specifically is a 74% reduction in neonatal mortality
		• Increase in the share of mothers who receive the tetanus toxoid vaccine
		• Reduction in the number of births delivered by caesarian
R-1; P-2	Burundi	• Significant rise in the likelihood of blood pressure measurement and anti-tetanus vaccination during antenatal care visit	• Did not improve population coverage of ANC	Bonfrer et al.^29
		• Improvements found in types of care that require a behavioral change of health care workers when the patient is already in the clinic	• Greater effects on institutional deliveriesamong higher income people
		• Little effect on services that require effort from the provider to change patients' utilization choices
P-10	Rwanda	• Reduction in the gap between provider knowledge and actual practice of the appropriate clinical procedures by approximately 20%	• Increase of 0.53 standard deviations in the weight-for-age of children zero to 11 months and 0.25 standard deviations in the height-for-age of children 24–49 months	Gertler and Vermeersch^33
			• No increase in prenatal care usage
P-10	Rwanda	• Increase of 0.157 standard deviations (95% confidence interval, 0.026–0.289) in prenatal quality (i.e., compliance with Rwandan prenatal care clinical practice guidelines; measured through patient exit interviews and household surveys performed for the impact evaluation)		Basinga et al.^34
P-10	Rwanda		• Significant increase in the proportion of women delivering in facilities	Sherry et al.^35
			• No impact on antenatal care utilization
P-10	Rwanda	• Program achieved efficiency gains by inciting health care providers to focus on the easier to reach (the less poor). Equity remains an issue		Lannes et al.^36
R-5	Rwanda	• Behavioral change observed in the studied sites: (1) introduction of mechanisms to take initiatives resulting in better performance (appropriate archiving, additional staff recruitment, improved welcoming conditions for patients) and/or (2) or development of new services (e.g., installation of a dental surgery, physiotherapy services)		Janssen et al.^24
P-11	Senegal	• 42% of facilities met target for postnatal care attendance		El-Khoury et al.^25
		• 7% met target for skilled birth attendance
P-14	Uganda	• Monitoring progress in labor using a partograph was uncommon at baseline but improved substantially over time in both intervention and control group and more so in the intervention group		Health Partners International^27
R-8; P-15	Zambia	• Health workers in facilities with the new payment mechanism spent significantly more time during consultations with their patients as compared to control health facilities	• Women residing in districts with the new payment mechanism were significantly more likely to list several out of the 12 danger signs during pregnancy	Chansa et al.^37
		• Timing of the first Antenatal care (ANC) visit was earlier by two weeks under the new payment mechanism as compared to control groups	• The new payment mechanism contributed to some important health gains over business as usual, but several of the gains were also achieved in the districts that received additional financing without direct incentives for quality
		• Number of deliveries by skilled providers improved in facilities that received additional funding and facilities that used the new payment mechanism, relative to the control group that received no additional funding
		• Quality of the delivery room was higher in districts with the new payment mechanism
R-9; P-16	Zimbabwe	• 12 percentage point increase in postnatal care coverage		Friedman et al.^26
		• 13 percentage point increase in the in-facility delivery rate due to the Zimbabwe results-based financing (RBF)
		• A significant increase in the rate of pregnant women receiving a full package of ANC services includingurine and blood tests and tetanus shots
		• Not all indicators show relative improvement in RBF districts during the pilot (both indicators where baseline coverage levels were already quite high):
		• No increase in ANC service coverage
		• Small gain in use of modern contraceptives