The Quality of Primary Care in Cambodia: An Assessment of Knowledge and Effort of Public Sector Maternal and Child Care Providers

ABSTRACT

Improving the quality of primary care is essential for achieving universal health coverage in low- and middle-income countries. This study examined the level and variation in primary care provider knowledge and effort in Cambodia, using cross-sectional data collected in 2014–2015 from public sector health centers in nine provinces. The data included clinical vignettes and direct observations of processes of antenatal care, postnatal care, and well-child visits and covered between 290–495 health centers and 370–847 individual providers for each service and type of data. The results indicate that provider knowledge and observed effort were generally low and varied across health centers and across individual providers. In addition, providers’ effort scores were generally lower than their knowledge scores, indicating the presence of a “know-do gap.” Although higher provider knowledge was correlated with higher levels of effort during patient encounters, knowledge only explained a limited fraction of the provider-level variation in effort. Due to low baseline performance and the know-do gap, improving provider adherence to clinical guidelines through training and practice standardization alone may have limited impact. Overall, the findings suggest that raising the low quality of care provided by Cambodia’s public sector will require multidimensional interventions that involve training, strategies that increase provider motivation, and improved health center management. The authors reported there is no funding associated with the work presented in this article.

KEYWORDS:

• Cambodia
• health system reform
• know-do gap
• maternal and child care services
• quality of care

Introduction

As low- and middle-income countries (LMICs) move toward universal health coverage, substantial efforts have been made to enhance the capacity of public sector primary care facilities and reduce the financial barriers to essential health services.^1–3 As a top policy priority, access to maternal, newborn, and child health services, including antenatal care, skilled birth attendance, and postnatal and well-child care, has continued to rise across LMICs and reached near-universal coverage in some countries.⁴ Despite this significant expansion of coverage, there is still substantial preventable maternal and newborn morbidity and mortality,^5,6 which is suggestive of gaps in the quality of care and performance of primary care providers.^7–10 While limited clinical knowledge of providers remains a significant constraint,¹¹ there is growing concern over low effort exerted by providers, as evidence from a number of LMICs shows that primary care providers often fail to do what they know are the best practices in actual patient encounters (the know-do gap).^12–15

Since the early 1990s, the Cambodian government has undertaken a series of initiatives to expand access to primary care and strengthen the performance of public sector providers, with a specific focus on routine maternal, newborn and child health services.¹⁶ A national network of over 1,100 health centers has been established to provide basic health services to 8,000–12,000 people each.¹⁷ Health centers, together with referral hospitals and health posts (which serve remote areas), are organized into Operational Districts (OD), which form the lowest level of Cambodia’s public health system. The scope of practice for health centers is set forth in the Guidelines on Minimum Package of Activities, which includes maternal, newborn, child and reproductive services, services for certain communicable and non-communicable diseases, and health education and health promotion.¹⁸

Cambodia has also experimented with performance-based financing (PBF) to increase provision of certain health services, particularly maternal and child health care, at public health facilities.¹⁹ In 2009, the government started converting some ODs, selected primarily based on their readiness for management autonomy, into Special Operating Agencies (SOA). SOAs use performance-based internal contracting to enhance provider accountability and help their health centers with annual operational planning and performance review.^20,21 To reduce the burden of out-of-pocket costs, the country introduced Health Equity Funds (HEF) in the early 2000s that cover user fees for services received by eligible beneficiaries, primarily low-income individuals, at public health facilities.^22,23 HEFs were rolled out to health facilities gradually over time but the scale-up was accelerated during 2015. By the end of 2015, HEFs had been rolled out to all public health facilities,^24,25 currently covering more than 3 million beneficiaries.²⁶

As a result of these initiatives, access to skilled maternal and newborn care in Cambodia has increased considerably. For example, in 2014, over 90% of women with a pregnancy in the previous 5 years received antenatal care from a trained provider, up from about 38% in 2000, and 83% delivered in a health facility, as compared to 10% in 2000. In addition, public health facilities accounted for over 80% of institutional deliveries in 2014.^27,28

As utilization at public sector facilities continues to grow, achieving further improvement in maternal and child health increasingly depends on the quality of care and thus the competence and effort of frontline health care workers. This includes, for example, providing adequate antenatal care to identify and manage existing health conditions of pregnant women and pregnancy complications to prevent maternal deaths, maintaining and increasing coverage for childhood vaccines, and providing growth monitoring and counseling to address malnutrition that afflicts about a third of Cambodian children under the age of five.²⁹

In this study, we assessed the quality of basic maternal, newborn, and young child health care services provided at health centers in Cambodia by examining what providers at these health centers know and what they do. Specifically, we focused on provider knowledge of clinical guidelines of antenatal, postnatal and well-child care and their effort in delivering these services at health centers in nine provinces of Cambodia. Our analysis relied on cross-sectional data from clinical vignettes and direct observations of clinical encounters. We created measures of provider knowledge based on the vignettes, and measures of provider effort in actual practice (in absolute terms and relative to knowledge) using the direct observations. We analyzed factors associated with provider knowledge and effort and decomposed variances in knowledge and effort into operational district, health center, and provider-level components using mixed effects models. This approach can shed light on which levels of the health system should be targeted by future quality improvement strategies. We also simulated the impact of several hypothetical policies to explore the extent to which process quality may be improved through conventional initiatives, such as provider training and quality benchmarking.

Materials and methods

Data and Sample

Our primary data included vignette and direct observation data collected from health centers under the “Level 2” Quality Assessment (L2), a part of the Cambodian government’s recent efforts to strengthen the performance of the public health system. While an earlier “Level 1” assessment focused on structural quality of public health facilities such as facility infrastructure, supplies and staffing, the goal of L2 was to assess baseline process of care quality and provider competency.³⁰ L2 collected data on provider knowledge and practice of guideline-recommended practices for services routinely offered at public health facilities. We obtained L2 data collected from all health centers in nine provinces of Cambodia by the University Research Co., LLC, (URC) for the Cambodian Ministry of Health (MOH).³¹ The bulk of the data used for this study was collected from 2014 through early 2015, with a small percentage of data collected in late 2012 and 2013. URC developed the assessment instruments based on existing MOH protocols, clinical guidelines, and standard operating procedures.³² The nine provinces covered in our data accounted for nearly 49% of Cambodia’s total population per the latest census (see Appendix Table A1 for province-level characteristics).³³

For health centers, the L2 assessment focused on services in accordance with the Minimum Package of Activities. Each L2 assessment included a knowledge assessment using a clinical vignette and a process-of-care assessment using direct observation of provider effort during actual patient encounters. A provider was scored based on the number of checklist items that they completed during a vignette or patient encounter as observed by the enumerators. Most providers were assessed once for a given vignette, but they could have multiple direct observations for the same service depending on the number of patients they saw during the time when the enumerators were present.

For this analysis, we examined data collected for antenatal care normal visit (ANC), postnatal care normal visit for mothers and newborns between 0 and 6 weeks after birth (PNC), and well-child visit for children under 2 (WCH). The clinical vignettes for these three services were designed to represent normal visits without health concerns presented at the time of the visit or uncovered during the visit. The contents of the vignettes and direct observations are listed in Appendix Tables Table A2–A4. We excluded just below 4% of the full sample due to missing values for 30% or more of the checklist items included in a L2 instrument or due to missing provider identifiers. Some assessments may appear to be “incomplete” because of new questions being added later to the L2 instrument; we kept these observations in the analytic sample. Sections concerning documentation, staff attitudes and hygiene in the direct observations were excluded from the analysis, as they were not specific to a particular clinical service and were not part of the vignette assessment. We also dropped checklist items regarding vaccination and prophylaxis from the process-of-care assessments if the patients did not receive vaccine and/or prophylaxis or if we could not tell from the data whether they did.

We supplemented the L2 assessment data with: 1) delivery volume at the health center level from Cambodia’s Health Information System, and information on 2) whether a health center was covered by HEF, 3) whether an operational district was designated as an SOA, and 4) the total population and poverty rate at the administrative district level.³⁴

Outcome Measures

Following existing literature, we considered vignettes as measuring provider clinical knowledge (hereinafter “knowledge score”) and direct observations as measuring provider effort (hereinafter “effort score”).^12,35 We constructed summary scores in two ways—a percentage score which equals the percentage of correct responses for each individual vignette or direct observation, and a standardized z-score derived from the sample mean and standard deviation of the percentage score. We relied on the percentage score for summary statistics and used the standardized score as the dependent variable in regression analysis given ease of interpretation. As mentioned above, we retained observations with missing values for less than 30% of checklist items in each L2 assessment. If a checklist item was missing, the score was calculated out of the total number of items observed. Less than 1% of the analytic sample had missing values.

Statistical Analysis

Our main analysis used linear mixed effects models with nested random effects to decompose the variance in knowledge and effort scores into operational district, health center, and provider components. We summarized the contributions to variance in the knowledge and effort scores by fixed and random effects using statistics akin to R-squared, based on Nakagawa et al. (2013).³⁶

We estimated two mixed effects models. The first model examined variations in the knowledge scores and effort scores separately using the full analytic sample. To decompose the variances, we included random intercepts at the operational district and health center levels for knowledge scores, and random intercepts at the operational district, health center and individual provider levels for effort scores. We did not include provider random intercepts in the knowledge score regressions, as most providers completed only one vignette for a given service and few health centers had more than one provider completing a vignette. As a result, in the knowledge score regressions, the health center random effects component would capture some of the provider-level variances, and the residual variances would capture the provider-level variances unaccounted for by the model.

Although random effects models can accommodate small size of clusters (health centers or individual providers) provided that there is a large number of clusters,^37,38 including both health center and provider random intercepts in the effort score regressions may not be informative if many health centers had only one or few observed individual providers, which was the case for PNC and WCH. Since there was some overlap in the checklist items between PNC and WCH, primarily regarding physical exam on the child and growth monitoring, we also pooled together data from these assessments and ran an additional set of regressions of knowledge and effort scores based on those common child care items. Doing so increased the number of health centers with more than one observed provider and allowed us to better estimate the relative contribution to variance by health centers and individual providers.

In the second model, we analyzed variations in the effort scores using random intercepts at the operational district and individual provider levels. Moreover, we included the knowledge scores as a predictor to assess the relative importance of provider knowledge in explaining variation in the effort scores. For this model, we restricted the sample to providers with data from both vignette and direct observation for the same assessed service and removed checklist items that were not included in both assessment tools. The restricted sample contained about 45% to 64% of the providers with direction observation data. If a provider completed multiple vignettes for the same assessed service, we kept their highest knowledge score. Because most health centers in the restricted sample had only one provider with direct observations of process of care, we did not include health center random intercepts in this model.

In all regressions, we adjusted for a health center’s HEF status and a series of binary variables for health center delivery volume, an operational district’s SOA status, and total population and poverty rate at the administrative district level. In the first model that analyzed knowledge and effort scores separately based on the full analytic sample, we also adjusted for province indicators. We did not adjust for province in the second model to avoid overfitting, as the restricted sample was much smaller.

Finally, we simulated the impact of several policy scenarios on process of care quality to explore how much improvement may be possible in this context. We began with two scenarios featuring practice standardization and benchmarking, using the full analytic sample:

Scenario 1: Standardize performance such that providers’ effort scores equal the highest knowledge or effort scores observed in their own health center.

Scenario 2: Standardize performance such that providers’ effort scores equal the highest knowledge or effort scores observed in their own operational district.

We then simulated three scenarios using the subset of providers with both vignette and direct observation data for the same assessed service. For these scenarios, we built on the work of Ibnat et al. (2019) and considered changes in both the size of the gap between knowledge and effort scores (“know-do gap”) and the slope of the relationship between the two.

Scenario 3: Increase effort of all providers with a positive know-do gap (i.e., knowledge > effort), assuming knowledge remains the same but increasing the slope of the relationship between knowledge and effort.

Scenario 4: Increase knowledge uniformly by 20% points, assuming the slope of the relationship between knowledge and effort remains the same.

Scenario 5. A multidimensional approach that first increases knowledge uniformly by 10% points (which in turn increases effort based on the observed knowledge-effort curve) and then shifts effort uniformly up by 5% points, assuming the slope of the relationship between knowledge and effort remains the same.

Ethical Clearance

The protocol for this study was approved by the Harvard University Institutional Review Board (IRB21–1533) and the National University of Singapore Institutional Review Board (NUS-IRB-2021-976). The analysis of the L2 assessment data for this work was undertaken in consultation with the Cambodian Ministry of Health.

Results

Summary Statistics

Table 1 reports the summary statistics and the number of observations at each level in the full analytic sample and the restricted sample consisting of providers with both vignette and direct observation data. The sample size varied across assessments, as a provider may not be present or there might not have been any visits for a specific service during the time when the enumerators visited the health center. The full sample and the restricted sample were similar in the available characteristics.

Table 1. Summary statistics of the analytic sample, mean(sd).

	Antenatal care (ANC)			Postnatal care (PNC)			Well-child visit (WCH)
	Vignette (knowledge)	Direct observation (effort)		Vignette (knowledge)	Direct observation (effort)		Vignette (knowledge)	Direct observation (effort)
Panel A: full analytic sample
Assessment score (%)	44.8 (12.9)	31.6 (12.1)		13.8 (7.9)	7.3 (6.3)		21.8 (9.9)	11.9 (6.8)
Health center level characteristic
Covered by HEF (%)	58.4 (49.3)	58.9 (49.2)		60.5 (48.9)	63.4 (48.2)		58.0 (49.4)	62.5 (48.5)
OD converted to SOA (%)	41.8 (49.4)	42.5 (49.5)		45.0 (49.8)	39.7 (49.0)		42.0 (49.4)	39.2 (48.9)
Delivery volume	180.3 (164.0)	182.5 (165.9)		177.8 (169.3)	186.8 (137.4)		178.5 (166.5)	189.7 (180.3)
Administrative district population (000s)	113.9 (43.8)	114.4 (43.3)		114.3 (44.7)	113.5 (44.0)		113.5 (43.9)	112.5 (45.9)
Administrative district poverty rate (%)	20.0 (4.1)	19.9 (4.1)		20.0 (4.2)	20.1 (4.2)		19.9 (4.2)	20.3 (4.2)
Sample size
Observations	516	2777		465	744		502	1388
Providers	514	847		463	370		501	473
Health centers	495	475		438	290		471	360
Operational districts	38	38		38	38		38	38
Panel B: providers with vignette and direct observation data for the same assessed service
Assessment score (%)	44.9 (13.0)	33.8 (10.9)		13.4 (6.4)	6.4 (4.5)		20.4 (9.0)	12.9 (6.8)
Know-do gap (percentage point)		11.1 (12.3)			6.5 (6.5)			8.2 (9.0)
Health center level characteristic
Covered by HEF (%)	60.0 (49.1)	60.0 (49.1)		62.4 (48.5)	62.4 (48.5)		58.9 (49.3)	58.9 (49.3)
OD converted to SOA (%)	42.1 (49.4)	42.1 (49.4)		40.4 (49.2)	40.4 (49.2)		37.3 (48.5)	37.3 (48.5)
Delivery volume	184.5 (176.4)	184.5 (176.4)		188.4 (141.9)	188.4 (141.9)		193.8 (193.0)	193.8 (193.0)
Administrative district population (000s)	112.0 (43.7)	112.0 (43.7)		111.6 (44.1)	111.6 (44.1)		112.4 (45.0)	112.4 (45.0)
Administrative district poverty rate (%)	20.0 (4.1)	20.0 (4.1)		20.0 (4.2)	20.0 (4.2)		20.0 (4.1)	20.0 (4.1)
Sample size
Observations	380	1557		216	463		301	999
Providers	379	379		216	216		300	300
Health centers	375	375		213	213		292	292
Operational districts	38	38		38	38		38	38

HEF = Health Equity Fund; OD = Operational District; SOA = Special Operating Agency. The table above reports the mean and standard deviation (in parentheses) of the assessment scores and health center level characteristics, as well as the sample size at each level by service and type of assessment. The performance scores represent the percentage of completed checklist items out of all observed checklist items. “Know-do” gap was calculated by taking the difference between knowledge score and effort score and averaging the difference across observations.

Overall, both knowledge and effort scores were low across all three services. Providers scored the highest on ANC assessments, completing an average of 44.8% and 31.6% of the checklist items in vignettes and direct observations, respectively. They scored the lowest on PNC assessments, completing only 13.8% and 7.3% of the checklist items in vignettes and direct observations, respectively. Knowledge scores were in general higher than effort scores in both the full sample and the restricted sample. Based on the restricted sample, the average know-do gap, i.e., the average difference between knowledge score and effort score across observations, ranged from 6.5 (PNC) to 11.1 (ANC) percentage points.

Figure 1 and Appendix Figures A1–A3 show the distributions of knowledge and effort scores overall and by domains of care. Several findings emerge. First, there were substantial variations in both knowledge and effort scores, especially for ANC. Second, for PNC, providers performed poorly across domains of care but particularly so for domains focusing on the mother’s health. Third, while most providers scored low on these assessments, there were a few providers that performed well, but even these high-performing outliers rarely scored over 60% in total on PNC and WCH.

Figure 1. Distributions of knowledge scores from vignettes and effort scores from direct observations.

Histogram of knowledge and effort scores for each assessed service area. Distribution of knowledge and effort scores for each domain of care within a service area is reported in Appendix Figures A1–A3\.

Figure 2 indicates that knowledge scores and effort scores were positively correlated in all three assessed services, but the correlation was weaker for PNC and WCH. Among providers with roughly the same knowledge score, there was considerable variation in their effort scores, suggesting that many factors in addition to provider knowledge likely contributed to observed quality of care. Moreover, most providers’ effort scores fell below their knowledge scores, indicating wide-spread presence of the know-do gap.

Figure 2. Relationship between provider knowledge score and average effort scores.

Sample restricted to providers with vignette and direct observation data for the same assessed service. Provider knowledge score is on the x-axis: most providers were assessed once for a given vignette; in the event of multiple knowledge scores for the same assessed service, the highest score was retained. Average effort score at the provider level is on the y-axis: this was calculated by averaging the effort scores across all available direct observations for a given provider. Checklist items not included in both the vignette tool and the direct observation tool were removed.

Regression Results

Table 2 presents the results from the mixed effects models that analyzed knowledge and effort scores separately based on the full analytic sample. Of the three predictors of interest, an operational district’s SOA status was most consistently associated with higher provider knowledge and effort across all three services. The estimates indicate that providers in districts with SOA designation scored higher on ANC, PNC, and WCH vignettes by 0.51, 0.32 and 0.42 standard deviations, respectively. However, providers at high-volume health centers—a rough proxy for experience—did not perform better than those at low-volume health centers on any assessments except WCH knowledge.

Table 2. Results of linear mixed effects models of knowledge and effort scores (Z-scores) based on full sample.

		Antenatal care (ANC)		Postnatal care (PNC)		Well-child visit (WCH)
		Knowledge score	Effort score	Knowledge score	Effort score	Knowledge score	Effort score
Fixed effects
	Health center covered by HEF	0.369**	0.046	0.291*	−0.224	0.131	0.434**
		(0.160)	(0.134)	(0.172)	(0.245)	(0.169)	(0.216)
	OD converted to SOA	0.511***	0.364**	0.316**	0.282	0.418***	0.143
		(0.147)	(0.168)	(0.160)	(0.192)	(0.151)	(0.174)
	Delivery volume: 100-199	−0.023	0.009	0.024	0.158	0.086	0.038
		(0.103)	(0.086)	(0.115)	(0.148)	(0.110)	(0.125)
	Delivery volume: 200-299	−0.076	0.059	−0.089	0.300*	0.075	0.084
		(0.126)	(0.102)	(0.142)	(0.169)	(0.137)	(0.153)
	Delivery volume: 300-399	−0.248	−0.060	−0.163	−0.176	−0.047	−0.021
		(0.180)	(0.135)	(0.211)	(0.239)	(0.194)	(0.197)
	Delivery volume: ≥ 400	−0.008	−0.136	0.026	0.032	0.466**	0.116
		(0.204)	(0.145)	(0.228)	(0.240)	(0.215)	(0.205)
Random effects
	Operational district	0.039	0.100	0.035	0.002	0.029	0.028
	Health center	0.112	0.129	0.747	0.272	0.601	0.248
	Provider		0.346		0.621		0.547
Pseudo-R^2
	Fixed effects	0.155	0.098	0.086	0.071	0.094	0.063
	Random effects	0.152	0.559	0.773	0.781	0.631	0.738
	Residual	0.693	0.343	0.142	0.149	0.275	0.199
N
	Observations	516	2777	465	744	502	1388
	Providers		847		370		473
	Health centers	495	475	438	290	471	360
	Operational districts	38	38	38	38	38	38

The table above reports the results of linear mixed effects models based on the full analytic sample. HEF = Health Equity Fund; OD = Operational District; SOA = Special Operating Agency. The dependent variables were standardized z-scores. The fixed effects coefficients indicate the change in sores in standard deviation associated with a unit change in a specific predictor. For example, an estimated coefficient of 0.1 means an increase in performance score by 0.1 standard deviation. The reference category for delivery volume is 0–99 deliveries. The random effects report the amount of variance at each level. R² report the share of variance explained by each component. Standard errors in parentheses. Each model also adjusted for province indicators, and poverty rate and total population at the administrative district. *** p < 0.01; ** p < 0.5; * p < 0.1.

The conditional random effects estimates show the extent to which an individual unit’s intercept differs from the overall intercept at a certain level. For knowledge scores, the variability in the intercepts was lower at the operational district level and higher at the health center level, suggesting greater variations in knowledge across health centers than operational districts. But, as noted, part of the health center random effects estimates might represent provider-level variations in knowledge, as the samples for knowledge scores were too small to include provider random effects in the regressions. For effort scores, the most variability was observed at the individual provider level, followed by health centers and operational districts.

The pseudo-R-squared indicates that, compared with the random effects, the fixed effects predictors explained only a small share of the total variation in provider performance—ranging from 6.3% for WCH effort to 15.5% for ANC knowledge (Table 2). Of the random effects, larger shares of the variances were either at the health center level or at the provider level, whereas the operational district intercepts captured only a small portion of the total variances (Figure 3a,b). Nonetheless, a substantial portion of the variances in ANC knowledge and ANC effort scores remained unaccounted for. Similar findings were reached when we ran the regressions using the bigger pooled sample of PNC and WCH on shared checklist items (Table 3).

Table 3. Results of linear mixed effects models of knowledge and effort scores (Z-scores) based on pooled data from postnatal care (PNC) and well-child visit (WCH) on common checklist items.

		Knowledge scores	Effort scores
Fixed effects
	Health center covered by HEF	−0.127	0.313*
		(0.153)	(0.190)
	OD converted to SOA	0.452***	0.335*
		(0.138)	(0.172)
	Delivery volume: 100-199	−0.175*	0.119
		(0.100)	(0.110)
	Delivery volume: 200-299	−0.193	0.056
		(0.125)	(0.133)
	Delivery volume: 300-399	−0.067	−0.068
		(0.178)	(0.180)
	Delivery volume: ≥ 400	0.139	0.039
		(0.198)	(0.184)
Random effects
	Operational district	0.025	0.048
	Health center	0.534	0.181
	Provider		0.563
Pseudo-R^2
	Fixed effects	0.066	0.088
	Random effects	0.556	0.664
	Residual	0.378	0.248
N
	Observations	967	2132
	Providers		610
	Health centers	476	415
	Operational districts	38	38

The table above reports the results of linear mixed effects models based on pooled PNC and WCH data on common items related to child physical exam and growth monitoring. HEF = Health Equity Fund; OD = Operational District; SOA = Special Operating Agency. The dependent variables were standardized z-scores. The fixed effects coefficients indicate the change in sores in standard deviation associated with a unit change in a specific predictor. For example, an estimated coefficient of 0.1 means an increase in performance score by 0.1 standard deviation. The reference category for delivery volume is 0–99 deliveries. The random effects report the amount of variance at each level. R² report the share of variance explained by each component. Standard errors in parentheses. Each model also adjusted for visit type (i.e., PNC or WCH), province indicators, and poverty rate and total population at the administrative district. *** p < 0.01; ** p < 0.5; * p < 0.1.

Table 4 reports the mixed model estimates based on the restricted sample of providers with both vignette and direct observation data. As expected, provider knowledge scores were positively correlated with effort scores. An increase in knowledge scores by one standard deviation was associated with an increase in effort scores by 0.44, 0.26 and 0.29 standard deviations for ANC, PNC and WCH, respectively. Including provider knowledge as covariate in the regression increased the share of variances explained by fixed effects considerably; it also reduced the size of the coefficient estimates on SOA status, which is consistent with the fact that SOA status and knowledge scores were positively correlated (as suggested in Table 2). However, the coefficient estimates on SOA status remained relatively large. Lastly, despite the significant association between knowledge and effort scores, much of the variance in effort scores was still captured by the provider random effects instead of the fixed effects (Figure 3c).

Figure 3. Percentage of variance explained by fixed effects and random effects components included in each model.

The figures above are based on results from three distinct models. Each reports the percentage of variance in the knowledge scores and effort scores that was explained by fixed effects and random effects at the operation district, health center and provider level, as well as the percentage of variance that remained unexplained (i.e., residual). ANC = Antenatal Care; PNC = Postnatal Care; WCH = Well Child Visit. Figure 3a,b report the results from models that analyze knowledge scores and effort scores separately based on the full sample. 3(c) reports the results from models based on the sample restricted to providers with both vignette and direct observation data for the same assessed service. Of each pair of the stacked bars in Figure 3c, the top bar reports the results of regressions without adjusting for knowledge score, the bottom bar reports the results of regressions adjusted for knowledge score.

Table 4. Results of linear mixed effects models of effort scores (Z-scores) based on providers with both vignette and direct observation data for the same service.

		Antenatal care (ANC)		Postnatal care (PNC)		Well-child visit (WCH)
		(1)	(2)	(1)	(2)	(1)	(2)
Fixed effects
	Knowledge scores		0.437***		0.253***		0.288***
			(0.040)		(0.063)		(0.054)
	Health center covered by HEF	0.193	0.026	−0.006	−0.057	0.396**	0.291**
		(0.124)	(0.100)	(0.173)	(0.167)	(0.155)	(0.142)
	OD converted to SOA	0.369***	0.184*	0.483***	0.402**	0.284*	0.189
		(0.143)	(0.106)	(0.175)	(0.170)	(0.163)	(0.146)
	Delivery volume: 100-199	0.080	0.059	−0.112	−0.110	0.089	0.060
		(0.108)	(0.095)	(0.163)	(0.158)	(0.137)	(0.131)
	Delivery volume: 200-299	0.068	0.058	−0.058	−0.079	0.205	0.125
		(0.128)	(0.112)	(0.184)	(0.178)	(0.166)	(0.160)
	Delivery volume: 300-399	−0.052	−0.027	−0.545*	−0.433	−0.070	−0.050
		(0.171)	(0.148)	(0.293)	(0.283)	(0.217)	(0.208)
	Delivery volume: ≥ 400	−0.220	−0.159	−0.221	−0.237	0.432*	0.267
		(0.202)	(0.176)	(0.254)	(0.244)	(0.227)	(0.220)
Random effects
	Operational district	0.107	0.041	0.086	0.079	0.100	0.064
	Provider	0.425	0.305	0.644	0.588	0.665	0.612
Pseudo-R^2
	Fixed effects	0.064	0.221	0.125	0.179	0.094	0.143
	Random effects	0.517	0.347	0.665	0.610	0.685	0.628
	Residual	0.419	0.432	0.210	0.211	0.221	0.229
N
	Observations	1557	1557	463	463	999	999
	Providers	379	379	216	216	300	300
	Operational districts	38	38	38	38	38	38

The table above reports the results of linear mixed effects models based on the sample restricted to providers with both vignette and direct observation data for the same assessed service. HEF = Health Equity Fund; OD = Operational District; SOA = Special Operating Agency. The dependent variables were standardized z-scores. The fixed effects coefficients indicate the change in sores in standard deviation associated with a unit change in a specific predictor. For example, an estimated coefficient of 0.1 means an increase in performance score by 0.1 standard deviation. The reference category for delivery volume is 0–99 deliveries. The random effects report the amount of variance at each level. R² report the share of variance explained by each component. Standard errors in parentheses. Standard errors in parentheses. Each model also adjusted for poverty rate and total population at the administrative district level; we did not adjust for province indicators to avoid overfitting given the smaller sample size. If a provider had multiple knowledge scores for the same service, the highest score was retained. *** p < 0.01; ** p < 0.5; * p < 0.1.

Policy Simulations

Table 5 presents the results from our policy simulation exercise, with additional visualizations in Appendix Figures A4–A6. The largest improvements were achieved when provider effort was increased to the same level as the best performing providers in the same operational district (Scenario 2). Scenario 2 might be possible through a combination of training, continuous performance monitoring, and interventions to motivate low-performing providers. Due to low baseline knowledge and the know-do gap (or the weaker relationships between knowledge and effort) for PNC and WCH, none of the other policy scenarios achieved more than a marginal improvement in quality. By contrast, ANC quality improved considerably under the two most realistic scenarios (Scenarios 4, 5) and was only marginally worse than that under the ideal scenario of eliminating the know-do gap (Scenario 3).

Table 5. Effort score in percentage as observed and as simulated under various policy scenarios, mean (sd).

	Antenatal care (ANC)	Postnatal care (PNC)	Well-child visit (WCH)
Panel A: based on full analytic sample
As observed	31.6 (12.1)	7.3 (6.3)	11.9 (6.8)
Scenario 1: Standardize performance such that providers’ effort scores equal the highest knowledge or effort scores observed in their own health center	48.9 (11.7)	15.0 (8.5)	23.6 (9.0)
Scenario 2: Standardize performance such that providers’ effort scores equal the highest knowledge or effort scores observed in their own operational district	66.9 (6.6)	34.6 (12.0)	43.6 (11.6)
Panel B: based on sample restricted to providers with both vignette and direct observation data
As observed	33.8 (10.9)	6.4 (4.5)	12.9 (6.8)
Scenario 3: Improve effort such that the know-do gap is eliminated for all providers with effort scores < knowledge scores; knowledge stays the same	46.2 (11.8)	13.3 (6.2)	21.9 (8.1)
Scenario 4: Increase knowledge uniformly by 20 ppts; slope of the relationship between knowledge and effort stays the same	41.8 (10.9)	10.4 (4.5)	18.1 (6.8)
Scenario 5: Increase knowledge and effort uniformly by 10 and 5 ppts, respectively; slope of the relationship between knowledge and effort stays the same	42.8 (10.9)	13.4 (4.5)	20.5 (6.8)

The table above reports the results of the policy simulation exercise. ppts = percentage points.

Discussion

Using the L2 quality assessment data, we found that providers working at health centers in nine provinces of Cambodia had low knowledge of and exerted limited effort in delivering antenatal care, postnatal care, and well-child visits. Performance was exceptionally poor for postnatal care, particularly in domains of care related to the mother’s health.

We also observed that knowledge and effort were quite variable. Results from our linear mixed models indicate that much of the variance was at the health center or the individual provider level. An operational district’s SOA designation was an important predictor for knowledge scores and in some cases effort scores, which may explain why the amount of variance at the operational district level was relatively small. Although knowledge was positively associated with effort, a substantial fraction of variance in effort score was still captured by the provider random effect even after we accounted for provider knowledge in the regressions.

Our findings of poor provider knowledge and effort at public health centers are consistent with those documented in LMICs.^11,39 In particular, we observed a know-do gap, suggesting that providers failed to adhere to clinical guidelines despite (partially) knowing what to do. However, because the knowledge scores were also low, improving provider effort to close the know-do gap would only generate modest quality improvement, as highlighted in our policy simulation. In addition, our simulation suggested that improving provider knowledge alone would not increase the quality to a satisfactory level for postnatal care and well-child visits because of the dual issues of insufficient knowledge and the know-do gap. The very low baseline performance that we observed would require a considerable change to both provider knowledge and their practice of care.

Moreover, our analysis suggested that providers were inconsistent in the quality of care they delivered across patients. A recent study on the quality of care during labor and delivery in India, Kenya and Malawi found that health facilities contributed to a greater share of the observed variation in quality of care provided during labor and delivery, as compared with individual nurses within those facilities.⁴⁰ In contrast, we found that individual providers played a relatively more important role in contributing to the variation in quality of care than health centers. It is possible that in situations where care is delivered by teams of providers (which is often the case for labor and delivery), more variation in service quality is at the facility level as norms, peer effects, and productivity spillover may help to reduce variation in practice across providers within a facility. But in cases where care is delivered by a single provider, such as the three services we examined here, individual providers’ own decision-making might have a greater influence on quality.

Our study highlighted the considerable challenge facing Cambodia to improve quality of care at public health centers. In this context, improving provider adherence to guidelines likely requires multidimensional interventions with a particular focus on individual provider behavior and decision making. Since we observed that knowledge was a key constraint especially in postnatal and well-child care, training is an important first step and research suggests that training programs with features such as supervision (e.g., routine supervision, benchmarking, audit and feedback) and group problem solving have the potential to achieve meaningful improvement in provider performance.⁴¹ Pre-service training could be important as well, as recent graduates can exhibit substantial gaps in their clinical performance,⁴² but our data did not allow us to differentiate recent graduates from experienced clinicians. Nonetheless, due to the know-do gap and wide variation in provider practice, increasing knowledge alone is likely insufficient.

Another—possibly concurrent—approach to improving quality could be to strengthen the model of Special Operating Agencies and expand it to more operational districts. The SOA model combines incentives and administrative accountability through performance-based internal contracting and performance monitoring. Within each SOA, health facilities selected for internal contracting have a part of their payments being tied to performance; within facilities, performance-based contracts are established between the head of facility and individual staff members.^20,24 The SOA model also intends to give health facilities more managerial authority and flexibility in enforcing performance contracts. Although we cannot establish a causal relationship, our finding that SOA designation was associated with higher provider knowledge and, in some cases, with better provider effort suggests that the SOA mechanism may partly work by selecting and retaining more qualified and higher-performing providers. However, a recent qualitative study found that SOA managers faced capacity and funding constraints in monitoring the performance of SOA and there were concerns over the reliability of the performance monitoring data.²¹ Moreover, the afforded flexibility and insufficient incentives may have contributed to more dual practice among staff,²¹ which could have contributed to the variation in provider performance that we observed in this study.

Finally, Cambodia has been piloting and evaluating a new PBF program that rewards provider competence as measured by a quality checklist.²⁴ Unlike the SOA model, this new PBF program will be implemented in all operational districts regardless of their SOA status. Alongside this new PBF program, the government has instituted periodic electronic data collection from health facilities, as part of the National Quality Enhancement Monitoring Program (NQEMP) that commenced in mid-2019. The current evidence on the effectiveness—and cost-effectiveness—of financial incentives to improve quality of care and health outcomes remains mixed,⁴³ and an evaluation of earlier PBF initiatives in Cambodia (prior to the SOA model and the more recent PBF program implemented since 2016) shows that PBF shifted institutional births from private to public facilities but did not measurably improve neonatal mortality or child vaccination rates.¹⁹ Yet the lack of conclusive evidence on PBF may in part due to the wide variation in how these programs are designed and implemented.⁴⁴ The explicit focus of Cambodia’s recent PBF program on provider knowledge and effort is somewhat unusual and may generate better results, compared to programs that target structural quality (e.g., supplies and equipment) and/or the quantity of services. The implementation of NQEMP will also lay the critical foundation for more rigorous future evaluations of health care quality and provider performance.

Our analysis has several limitations. First, our data were cross-sectional and only captured a snapshot of provider clinical knowledge and process quality, but to our knowledge there was no other data collection of this nature undertaken after 2015. Second, our data did not contain any provider or patient characteristics and only a limited set of health center characteristics. The knowledge scores presumably captured critical aspects of provider qualification and experience, but our analysis cannot fully account for the difference in effort scores due to, e.g., differences in provider characteristics not reflected in the vignettes and in the patient case mix. To minimize the impact of unobserved provider qualifications and patient case mix, we focused on normal ANC, PNC, and WCH visits, which have clear, standardized guidelines and are usually delivered by the same type of providers per the Minimum Package of Activities, i.e., secondary or primary midwife licensed by MOH for antenatal and postnatal care.¹⁸ While we did not observe health center infrastructure and staffing, health centers were designed to be somewhat homogeneous across these dimensions.¹⁸ Our health center-level covariates, such as SOA and HEF status, were related to incentive contracts and reimbursement arrangements, which may be more predictive of performance. Lastly, direct observation data are potentially sensitive to Hawthorne effects, which could result in higher observed performance relative to usual performance when observers are not present.⁴⁵

Conclusion

After decades of effort to rebuild and strengthen the country’s public health care system and expand financial coverage for health services, Cambodia has achieved considerable progress in increasing coverage of care by trained professionals and in improving population health. As highlighted by our findings, however, more attention needs to be directed toward improving process quality by enhancing public sector provider knowledge and effort. Traditional approaches focusing on provider training, while essential, are insufficient when providers fail to do what they know is the best practice or perform inconsistently from patient to patient in domains of care with well-established guidelines. Narrowing this know-do gap will likely require a multidimensional approach combining incentives, routine performance monitoring and feedback, and strategies leveraging norms and peer effects to increase provider effort. Research on providers’ decision-making process could shed light on additional behavioral drivers of poor adherence to clinical guidelines and inform policy design.

Contributions statement

DH and SB conceived of the research question and methodology. SN and SB supported the data acquisition and oversaw the work. DH conducted analyses and drafted the manuscript. SN and SB reviewed several versions of the manuscript and approved the final manuscript.

Ethical approval

The protocol for this study was approved by the Harvard University Institutional Review Board (IRB21–1533) and the National University of Singapore Institutional Review Board (NUS-IRB-2021-976). The analysis of the L2 assessment data for this work was undertaken in consultation with the Cambodian Ministry of Health.

Acknowledgements

We would like to thank the World Bank team for Health, Nutrition and Population based in Phnom Penh, Cambodia, as well as colleagues at the Cambodian Ministry of Health and University Research Co, LLC in Phnom Penh. This work was supported by the World Bank. The views expressed in the article do not necessarily reflect the views of the World Bank, the Executive Directors of the World Bank or the governments they represent.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data underlying the results presented in the study are held by the Cambodian Ministry of Health. Interested researchers may contact the Ministry to obtain more information regarding data availability at www.moh.gov.kh or physically at No: 80, Samdach Penn Nouth Blvd (289), Sankat Beoungkak 2, Tuol Kork District, Phnom Penh, Cambodia.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Appendix A

Table A1. Population characteristics per the 2014 Cambodia demographic and health surveys.

Province	Wealth quintile (%)			Antenatal care (ANC, %)		Place of delivery (%)				First postnatal care (PNC, %)			Vaccinations in children 12-23 months (%)
	Lowest two quintiles	Highest two quintiles		Received ANC from a skilled provider		Public sector health facility	Private sector health facility	Home		PNC in the first two days after birth	Received PNC from a skilled provider		All basic vaccines	No vaccinations
Panel A: provinces included in the sample (data collection led by URC)
Banteay Meanchey	18.1	59.3		98.9		70	17.9	11		10.7	89.3		91.3	1.6
Kampong Cham	46.2	28.5		96.9		61	23.5	15.2		17.3	80.2		56.8	4.7
Kampong Speu	41.7	32.6		97.8		66.9	17.2	15.5		8.6	85.3		66.5	1.4
Prey Veng	50.2	22.9		99		69.4	20.5	10		1.3	98.7		61.7	4.8
Pursat	63.8	20.2		94.7		72.6	5.8	21.6		4.2	93.3		79.7	3.0
Siem Reap	56.2	28		96.1		87	4.6	8.4		12.1	87.8		78.6	1.0
Battambang/Pailin	22.7	56.5		97.1		81.3	8.9	9.6		4.5	95.5		89.2	0.0
Panel B: remaining provinces (data collection led by Cambodian MOH)
Kampong Chhnang	61.1	21.1		99.5		96	1.2	2.9		0	100		74.6	0.0
Kampong Thom	63.4	21.5		95.6		67.5	6.9	25.7		0	85.8		70.9	2.4
Kandal	18.6	53.9		96.3		63.6	17.2	19.2		14.3	85		64.5	0.0
Kratie	64.5	20.8		72.8		40.3	5.9	53.7		8.2	63		65.1	7.7
Phnom Penh	1.6	95.4		98.5		65.2	30.7	4.1		0	98.2		89.1	0.0
Svay Rieng	53.4	21.8		98.1		74.3	8.2	17.2		9.9	89.2		82.7	5.6
Takeo	29.9	42.9		97.6		80.1	12	6.9		3.6	95.5		87.8	0.2
Otdar Meanchey	50.8	29.4		96.7		85.9	2.5	11.1		5.8	89.8		75.0	2.6
Kampot/Kep	51.7	24.7		93.9		72.2	8.7	18.2		6.1	90.6		72.0	2.0
Preah Sihanouk/Koh Kong	17.3	71.8		97.6		68.6	20.4	10.2		5.1	94.2		82.6	2.0
Preah Vihear/Stung Treng	74.4	12.7		85.5		49.3	1.8	48.8		31.8	56.7		55.6	1.2
Mondul Kiri/Ratanak Kiri	56.9	31.6		76		39.3	11.9	48.1		60.8	38.1		43.9	10.6

Data from the 2014 Cambodia Demographic and Health Surveys (DHS) Final Report. Battambang and Pailin were grouped together in the DHS and both provinces were included in our study sample. One province in our sample, Tbong Khmum, was formed in 2014 by splitting Kampong Cham province in two, which is not reflected in the 2014 DHS. Skilled provider includes doctor, nurse, and midwife. URC = University Research Co., LLC. MOH = Ministry of Health.

Table A2. Checklist items included in the antenatal care (ANC) vignette and direction observation data collection tools.

Section	Antenatal care vignettes	Antenatal care direct observation
	Checklist items	Checklist items
1. History taking	reason for visit; ask for her ANC book; how old is she; when was her last period; calculate expected delivery date; has she had a baby before; has she had an abortion; has she ever had a miscarriage; does she have any chronic illness such as TB, epilepsy	ask for her ANC book; tell/remind woman of her estimated date of delivery; go over the woman’s file; when was her last period; how old she is; has she had a baby before; number of previous pregnancies; has she had an abortion; has she had a miscarriage; has she had stillbirth or newborn death; has she had cesarean, forceps, vacuum extraction; calculate the estimated delivery date
2. Physical exam	height; weight; blood pressure; temperature; respiratory rate; heart rate/pulse; conjunctival pallor; palmar pallor; is she getting tired during routine tasks; ask if she felt any fetal movement; fetal heartbeat; fundal height; fetal presentation; check for goiter; any pain in her upper abdomen; any abnormal vaginal discharge or bleeding since last visit; check for edema	weight; blood pressure; temperature; was the height taken; respiratory rate; heart rate/pulse; was BMI calculated; check the eyes; check for goiter; check lymph nodes; listen to the heart; listen to the lungs; check the breasts; examine the skin; check the hands, feet; ask about vaginal discharge; check fundal height; check fetal presentation; ask if she felt any fetal movement; listen to fetal heartbeat; look for cesarean/other abdomen surgery scar
3. Preventive measures	administer TT1; give 60 doses of iron/folate; give 1 mebendazole 500 mg; ask if she and her children are sleeping under insecticide treated bednets; reason for taking iron/folate; the reason for the vaccination; what vaccination was given	check that the woman received TT, if due; check woman’s supply of the prescribed dose of iron/folate; explain to the woman the importance of taking iron/folate; give mebendazole 500 mg once
4. Paraclinic tests	HIV test; syphilis test; hemoglobin; proteinuria	check urine—for proteinuria; check blood for anemia or refer woman to a facility where hemoglobin test is available; check mother’s health book for syphilis test during this pregnancy or give the test; check the woman was tested for HIV during this pregnancy or refused, or refer woman to a facility where HIV test is available; test for blood group
5. Maternal care counseling	Maternal nutrition: the importance of good nutrition for the mother postpartum/while breastfeeding; eat more foods to help her gain strength; eat a variety of foods; address food taboos; use iodized salt; get enough rest and sleep; use filtered or boiled water for drinking; take only medication prescribed in health facility; avoid drinking alcohol; avoid smoking Pregnancy danger signs: severe headaches; blurred vision; high fever and too weak to get out of bed; fast or difficult breathing; severe abdominal pain; convulsions; bleeding or spotting; swelling	Maternal nutrition: the importance of good nutrition for the mother postpartum/while breastfeeding; eat more foods to help her gain strength; eat a variety of foods; address food taboos; use iodized salt; get enough rest and sleep; use filtered or boiled water for drinking; take only medication prescribed in health facility; avoid drinking alcohol; avoid smoking Birth preparedness: on the important of birth in a facility is recommended; determine in advance who should go with her to the health facility when labor begins; determine in advance health facility to be taken to when labor begins; arrange ahead of time the transport to health facility, for when labor begins; negotiate ahead of time the cost of transport to health facility when labor begins; prepare money in advance for delivery expenses; on labor signs; take more than two sarongs for herself; take two clean clothes for keeping the newborn clean; if in 3rd trimester, on the benefits of breastfeeding; if in 3rd trimester, on essential newborn care; how to take care of breasts Pregnancy danger signs: severe headaches; blurred vision; high fever and too weak to get out of bed; fast or difficult breathing; severe abdominal pain; convulsions; bleeding or spotting; swelling
6. Feedback & advice on follow-up visit	feedback on findings; return for any problems, even if it’s before the scheduled follow up; date to return for her next checkup; carry her medical records with her always, and bring them to all visits to health facility; ask if she has any questions	feedback on findings; if 3^rd trimester, make sure she receives information family planning/birth spacing methods; on when to return for her next checkup; that she should carry her medical records with her always, and to bring to all visits to health facility

Table A3. Checklist items included in the postnatal care (PNC) vignette and direction observation data collection tools.

Section	Postnatal care vignettes	Postnatal care direct observation
	Checklist items	Checklist items
1. History taking: newborn	check the child’s yellow card; how old is the baby	check medical records for immunization status; check medical records to see when the child last had a health checkup; review the newborn’s yellow card or other medical record to guide the consultation
2. History taking: mother	check mother’s medical records; ensure that the woman took her postpartum iron folate; ensure that the woman’s tetanus vaccine is current; ensure she took a deworming pill after delivery	check medical records for immunization status e.g., mother’s pink book, TT card etc.; review the mother’s pink book or other medical record to guide the consultation; check medical records to see if mother received all her postpartum iron/folate tablets; check the mother’s TT card to make sure her tetanus vaccine is current; make sure she took a deworming tablet recently
3. Newborn physical exam	height/length; weight; plot weight for age; temperature; respiratory rate; pulse; check the fontanel; check the eyes; check the ears; check the nose; check muscle tone; check reflexes; check umbilical cord—redness, swelling, pus, odor; listen to the lungs; listen to the heart; examine/palpate abdomen; check the hands; check the feet	temperature; respiratory rate; heart rate/pulse; weight; check the fontanel; check the eyes; check the ears; check the nose; check mouth—problem with sucking reflex; check lymph nodes; listen to the heart; listen to the lungs; examine/palpate abdomen; umbilical cord—redness, swelling, pus, odor; examine the skin; examine the extremities; excessive sleepiness
4. Maternal physical exam	temperature; respiratory rate; pulse; weight; blood pressure; check the eyes; check for goiter; check the breast; check the hands; check the feet; check the lymph nodes; listen to the lungs; listen to heart; examine the skin; massage/examine the uterus to see if it is contracted; examine perineum for bleeding, tear, swelling, pus and lochia	weight; blood pressure; temperature; respiratory rate; heart rate/pulse; was BMI calculated; check the eyes; check for goiter; check lymph nodes; listen to the heart; listen to the lungs; check the breasts; examine the skin; massage/examine the uterus to see if it is contracted; examine perineum e.g. for bleeding, tear, swelling, pus and lochia; check the hands, feet; vaginal bleeding; abdominal pain, especially if severe
5. Newborn care counseling	provide immunization Breastfeeding: ask how she is feeding her baby; tell her that breastfeeding is good; the benefits of breastfeeding; breastfeeding exclusively for the first 6 months is healthy for her and the baby; the baby should breastfeed day and night; anything other than breastmilk for the first 6 months is unnecessary and can be harmful for the baby; are there any difficulties with breastfeeding; place the baby skin-to-skin; monitor newborn’s condition Hygiene: bathe the baby, using warm water; thoroughly dry baby, dress and cover after bath Newborn danger signs: fever or feel cold; bleeding; pus; diarrhea; yellow skin; poor feeding; fast or difficult breathing; convulsion	provide any needed immunizations Growth monitoring: plot weight-for-age; show the caregiver the newborn’s growth chart on the yellow card; explain the newborn’s growth; explain the newborn’s current nutritional status; provide tailored feeding advice based on the newborn’s growth and mother’s feeding questions Newborn danger signs: fever or feels cold; bleeding e.g., from cord stump; pus e.g., from eyes or from cord stump; diarrhea; yellow skin; poor feeding; fast or difficult breathing; convulsions Breastfeeding: observe a breastfeeding session for a few minutes; assist the mother improve positioning, attachment or the newborn’s suckling as needed; advise that the baby should breastfeed more than 8–12 times or every 2–3 hours; advise that the baby should breastfeed day and night; advise against using bottles; advise against giving food or drink other than breastmilk; advise the mother on how to deal with any breastfeeding difficulties; advise that the baby should breastfeed exclusively for the first 6 months; use counseling materials to explain feeding advice to the caregiver Hygiene: keep the baby warm and close to the mother, day and night; special care for a low birth weight baby; advise on how to bathe the baby; advise on how to monitor her newborn’s condition; cover the stump with clean dry clothes, but not wrap it; not to put or apply anything on the cord; advise to wash hands before and after touching the cord; keep stump clean and dry; bathe the baby; when she bathes the baby she should make sure to use warm but not hot water; thoroughly dry baby, dress and cover after bath
6. Maternal care counseling	Nutrition: the importance of good nutrition while breast feeding; eat more foods; eat a variety of fruits vegetables and protein; use iodized salt; take only medicine prescribed in health facility; avoid drinking alcohol; avoid smoking; do not practice “roasting” Hygiene: wash hands before eating; bathe regularly; wash hands after using the bathroom; keep perineal area clean Danger signs: vaginal bleeding; abdominal pain; convulsion; fever and too weak to get out of bed; urine dribbling or pain on passing urine; pain or draining pus in perineum; foul smelling lochia Family planning: general benefits of birth spacing methods; POP method of contraception; methods available at the health center and referral hospitals	Danger signs: convulsions; diarrhea; fever and too weak to get out of bed; urine dribbling or pain on passing urine; pain or draining pus in perineum; foul-smelling lochia; breasts swollen, red/tender, or nipples are sore Nutrition: the importance of good nutrition for the mother postpartum/while breastfeeding; eat more foods to help her gain strength; eat a variety of foods; address food taboos; use iodized salt; get enough rest and sleep; eating a variety of fruits vegetables and protein will not harm the breastfeeding baby; use filtered or boiled water for drinking; take only medication prescribed in health facility; avoid drinking alcohol; avoid smoking Family planning: general benefits of family planning; advise the mother on pop method of contraception; she is not more than 6 months postpartum; her menstrual cycle has not returned; she is breastfeeding 8 or more times a day not more than 4 hrs apart; provide a birth spacing method to the mother according to her choice; counseled on methods available at the health center; IUD; implants; tubectomy; vasectomy; made a separate appointment for the mother to come back for a birth spacing visit later—either at the mother’s request or because this is her final PNC at 6 weeks postpartum Hygiene: wash her hands before eating and after going to the bathroom; wash her perineum area regularly
7. Feedback & advice on follow-up visit	explain the vaccination schedule; return in 4 wks. for the baby’s next vaccination; bring the baby’s medical records to every PNC checkup; ask if the mother has any questions	she/any caregivers should know where to go if she or her child develops any danger signs; arrange ahead of time for transportation in case of emergency; arrange ahead of time the fee for transportation explain the reason for taking the medicine/prophylaxis; tell the mother the name of the medicine; tell mother what kind of medicine it is e.g. antibiotic; instructions for taking the medicine; explain the side effects to be expected; explain how to manage the side effects; when to return for her next checkup; told to bring her baby with her at the same time so that both can receive their checkups; advise the mother of her PNC schedule; that she should bring all medical records with her to all visits to health facility

Table A4. Checklist items included in the well-child visit (WCH) vignette and direction observation data collection tools.

Section	Well-child visit vignettes	Well-child visit direct observation
	Checklist items	Checklist items
1. History taking	check the child’s yellow card; how old is the baby	check medical records for immunization status; check medical records to see when the child last had a health checkup; review the client’s yellow card; was a deworming tablet given if needed; was vita given if needed
2. Physical exam	height/length; weight; plot weight for age; temperature; respiratory rate; pulse; check the eyes; check the ears; check inside the mouth; check lymph nodes; listen to the heart; listen to the lungs; examine/palpate abdomen; check the hands; check the feet	temperature; respiratory rate; heart rate/pulse; take the child’s weight; check the eyes; check the ears; check inside the mouth; check lymph nodes; listen to the heart; listen to the lungs; examine/palpate abdomen; examine the skin; examine the extremities
3. Assessment of mother’s health	is she sick; did she take a deworming tablet recently; is she getting family planning/birth spacing service; check/ask if her tetanus vaccine is current	ask if the mother is sick; ask if she took a deworming tablet recently; ask if she is getting family planning/birth spacing services; check the mother’s TT card/ask if her tetanus vaccine is current
4. Young child care counseling	provide immunization Breastfeeding: ask how she is feeding her baby; are there any difficulties with breastfeeding; tell her that breastfeeding is good; the benefits of breastfeeding; breastfeeding exclusively for the first 6 months is healthy for her and the baby; the baby should breastfeed day and night; anything other than breastmilk for the first 6 months is unnecessary and can be harmful for the baby; Growth monitoring: tell her the child is growing normally; show the child’s growth chart on yellow card; explain the child’s current nutritional status; Danger signs: persistent vomiting; persistent diarrhea or bloody diarrhea; weight loss; fever or feel cold; stiff neck; unconscious or lethargic; skin rash; bleeding from nose, gums, mouth, anus; abdominal pain and irritability; swollen feet and leg; convulsion	provide any needed immunizations Growth monitoring: show the caretaker the child’s growth chart on the yellow card; explain the child’s growth; provide tailored feeding advice based on the child’s growth and mother’s feeding questions; explain the child’s current nutritional status; Danger signs: persistent vomiting; persistent diarrhea or there is bloody diarrhea; weight loss or severe wasting; coughing with difficulty or fast breathing; fever or feels cold; stiff neck; skin rash; bleeding from nose, gums, mouth, anus; abdominal pain and irritability; swollen feet or legs; convulsions; unconscious or lethargic Breastfeeding and nutrition: If <6 months: ask if the baby gets food or drink besides breastmilk; ask how often has her baby breastfed in the past day and night; ask about position/attachment/how well the baby sucks; ask if there are difficulties with breastfeeding; observe breastfeeding to check on position/attachment and sucking; assist the mother to position/attach the baby to the breast; advise that the baby should breastfeed exclusively for the first 6 months; advise that the baby should breastfeed more than 8–12 times or every 2–3 hours; advise that the baby should breastfeed day and night; advise against using bottles; advise against giving food or drink other than breastmilk; advise the mother on how to deal with breastfeeding difficulties; use counseling materials to explain feeding advice to the caretaker If 6–24 months: ask if the child eats solids/semi-solid foods; ask how often the child is fed each day; ask how much the child eats at each meal; advise on giving the child healthy snacks like fruits, vegetable; advise on giving the child healthy snacks many times a day; ask who feeds the child; advise on continued breastfeeding; advise on breastfeeding until at least 2 years old; advise to breastfeed day and night; assist in resolving any breastfeeding problems; advise on good food hygiene practices to prepare meals in the house; use counseling materials to explain feeding advice to the caregiver Hygiene: sleep under insecticide-treated bednets; using filtered or boiled water for drinking; washing the child’s hands before eating and after going to the bathroom
5. Feedback & advice on follow-up visit	what vaccination was given; the reason for the vaccination; side effects to be expected; how to manage the side effect(s); ask if the mother remembers the vaccination schedule; explain the vaccination schedule; return in 4 weeks for the next vaccination; bring the baby’s medical records at every visit to health facility; ask if the mother has any questions	any problems identified during the consultation; when to return for the next checkup; come immediately back to the health center or go to the hospital if the child develops any danger signs; bring medical records on all health checkups; if referred, why s/he needs to be referred; the reason for taking the prophylaxis or medicine; the name of the medicine; the type of the prophylaxis or medicine e.g. DPT-Hep B, iron/folic acid etc.; instructions for taking the medicine; side effects to be expected; how to manage side effects

Figure A1. Distributions of knowledge scores and effort scores: antenatal care (ANC).

Figure A2. Distributions of knowledge scores and effort scores: postnatal care (PNC).

Figure A3. Distributions of knowledge scores and effort scores: well-child visit (Wch).

Figure A4. Policy simulation results: antenatal care (ANC).

ppts = percentage points.

Figure A5. Policy simulation results: postnatal care (PNC).

ppts = percentage points.

Figure A6. P, Massachusetts

ppts = percentage points.