Does courtesy bias affect how clients report on objective and subjective measures of family planning service quality? A comparison between facility- and home-based interviews

Abstract

Purpose

Despite a general understanding that exit interviews being conducted at service providers’ facilities may influence clients’ responses favorably to health professionals, there is very little evidence available that demonstrates the extent to which this problem exists. This study aimed at assessing and comparing clients’ perceptions of the quality of family planning services and their satisfaction levels between facility- and home-based interviews.

Methods

A cross-sectional survey was conducted among clients receiving family planning services across three service delivery channels – nongovernmental organization (NGO) clinics, social franchise (SF) centers, and outreach camps. The survey took place from December 2015 to January 2016 in 70 districts across all four provinces of Pakistan. A total of 2,807 clients were interviewed, of whom 1,404 clients were interviewed at health facilities after receiving services and 1,403 were interviewed at their homes within 3 days of method uptake.

Results

Overall, we found no significant differences between the characteristics of study participants interviewed at health facilities or at home. The findings suggested that experiences reported in exit surveys at facilities were strongly biased positively. This was true for both experiential (service quality) and perception-based (satisfaction) questions in the context of SF centers, while at NGO clinics the interview location only affected clients’ responses regarding service quality. However, in outreach settings, clients are more likely to share bad experiences in exit interviews than in home-based interviews on objectively asked questions (service quality).

Conclusion

Our study indicates signs of courtesy bias and possibly the Hawthorne effect in exit interviews. Program implementers could opt for home-based interviews for women receiving services at NGO clinics or SF center, whereas exit interviews could be used in outreach settings.

Keywords:

• Pakistan
• exit interview
• contraception
• social franchise
• outreach camp

Introduction

The core strategy used to achieve millennium development goals (MDGs) was the expansion of effective health intervention coverage.¹ However, focusing solely on health care coverage may not be sufficient to meet Sustainable Development Goal 3 unless the critical issue of quality of health care is addressed.^1,2 In the context of family planning (FP), ensuring high-quality, rights-based FP services is critical to achieving sustained uptake of a broad mix of contraceptive methods and to helping men and women realize their desired fertility outcomes.^3–5 The reinvigorated focus on “quality of care” has accelerated efforts to define and develop measures of service quality.⁶

The measurement of quality of FP care is guided by the framework developed by Bruce in 1990 which outlined six components of quality.⁷ The International Planned Parenthood Federation later proposed 10 dimensions of quality, expanding on Jain’s framework.⁸ More recently, rights-based frameworks for FP programming have incorporated the availability, accessibility, acceptability, and quality framework and human rights to broaden the understanding of quality in FP programs.⁹ Several tools are developed to translate framework into defined indicators including service provision assessment,¹⁰ service availability and readiness assessment,¹¹ quick investigation of quality,¹² situation analysis,¹³ client-oriented provider-efficient services,¹⁴ and continuous quality improvement.¹⁵

Although useful in certain settings, these previously developed tools may be too intricate, time-consuming, and expensive for small service providers to utilize.¹⁶ Exit interview, a component of some of the aforementioned tools, is deemed an easy, cost-effective, and practical approach to gather clients’ experiences of care in a less resource-intensive manner.^16,17 Further, exit interviews do not threaten the confidentiality of clients receiving services in secrecy. The exit interview methodology is commonly practiced around the world, especially in the field of FP by nongovernmental organizations (NGOs), to evaluate the performance.^16,18,19 However, this approach has inherent limitations such as structured questioning and an unwillingness to participate from clients who experience lengthy wait times at the facility. The main shortcoming of this approach is the likelihood of “courtesy bias” affecting clients’ responses whereby they may be reluctant to share negative experiences due to the proximity of service providers.^20–22 This may result in a high level of reported satisfaction with services,^23,24 which limits the utility of survey results for program improvement. Moreover, health care providers are usually aware of exit interviews, which may result in bias due to the Hawthorne effect where providers pay more attention to their treatment and care of clients under surveyors’ observations.^18,25,26

Despite a general understanding that exit interviews conducted in proximity of service providers may bias clients’ responses positively, scarce evidence is available to demonstrate the extent to which this problem exists. Limited research studies have examined the potential effect of courtesy bias in FP programs using different methodological approaches, but have found mixed results.^18,27,28 Some of these observed discrepancies may be attributable to methodological differences. To the best of the authors’ knowledge, few studies have assessed the differences between exit surveys conducted in facility- versus home-based settings.^17,29,30 These studies found that clients’ scores are higher in exit interviews conducted at the facility, but none focused on the differences between experiential and perceived satisfaction of FP services. To contribute to the existing body of knowledge, our study aimed at assessing and comparing clients’ experiences of the quality of FP services and their satisfaction levels between facility- and home-based interviews.

Marie Stopes Society (MSS), an affiliate of Marie Stopes International in Pakistan, provides services through three different service delivery channels including social franchising centers,³¹ mobile outreach camps,³² and static clinics (hereafter referred to as NGO facilities).³³ All these three service delivery channels have distinct facility infrastructures, follow different operating procedures, and cater to different segments of the population based on their location (urban, peri-urban, or rural). Therefore, to study the effect of courtesy bias in each service delivery channel, we ran segregated analysis.

Country context

Pakistan is among the developing countries that have failed to achieve MDG 5.³⁴ The modern contraception rate has remained steady at ~26% with female sterilization (8.7%), condom (8.8%), and withdrawal (8.5%) being the most commonly used modern methods. Currently, the total fertility rate is 3.8, and 20% of couples have an unmet need for FP. Numerous supply and demand-side barriers are responsible for low contraceptive use including the quality of care.^35–37 Heavy reliance on less effective contraceptive methods (short-term and traditional) and the poor quality of information provided to the clients may be one of the reasons for the high rate of discontinuation within the first year (37%).³⁵

MSS service delivery channels

NGO clinics

NGO clinics are located in urban or peri-urban areas and offer a comprehensive set of high-quality and affordable FP services including short-term, long-term, and permanent methods. These clinics are run by a team of eight individuals including a service provider (medical doctor), two paramedics, a counselor, a receptionist, a center manager, an office attendant, and a driver. The facility is comprised of five to six rooms including the reception area, counseling room, waiting hall, procedure room, changing area, and recovery room. Every visiting client follows a defined path to see the primary care provider. The average client flow is two per day with a range of one to 10 clients daily.

Social franchise (SF) centers

SF centers are primarily based in rural areas. The SF model is based on a partnership between MSS and local private-sector service providers for the provision of quality contraceptive services. The facilities are usually located in partitioned houses and are comprised of one or two rooms. Franchised centers are led by mid-level providers who are married and aged between 30 and 40 years. These providers are assisted by an attendant. The average FP client volume is one or two per day.

Outreach camps

These camps target poor and underserved women to provide a wide range of contraceptive services in rural and hard-to-reach areas. A typical outreach team comprises of six individuals: a doctor, two nurses, a counselor, a health care assistant, and a driver. The team arranged a camp at existing public or private health facility (rarely in school or even in a tent or a van, if no facility is available), preceded by sensitization and demand generation among the target population through its community health workers. The average client flow per camp is 30–50.

Methods

Design and setting

This was a cross-sectional survey conducted among clients receiving FP services across the three service delivery channels – 41 NGO clinics, 151 SF clinics, and 20 outreach camps. The survey took place during December 2015–January 2016 in 70 districts across all four provinces of Pakistan. A total of 2,807 clients were interviewed, of whom 1,404 clients were interviewed at health facility while they exit, and 1,403 were interviewed at their homes within 3 days of method uptake.

Eligibility criteria

Women of reproductive age (15–49 years), receiving any modern contraceptive method or counseling service (irrespective of other non-FP services) and who gave voluntary informed consent, were eligible to participate in the survey. To ensure consistency in the construction of outcome measures for satisfaction and service quality, we excluded 79 cases (2.8%) where women only received counseling service and did not use any contraceptive method. Consequently, analyses were performed on the remaining 2,728 cases.

Sampling strategy

MSS conducts exit interview survey annually to gauge experiences of its clients for multiple projects. In order to have sufficient statistical precision, the sample is estimated separately for each project and service delivery channel that allows conducting segregated analysis for performance monitoring of each project.³⁸ In general, the assumptions used for sample calculation were to detect 15% point difference¹⁷ (from 76% in exit interview to 61% in home interview) in outcome variable with 80% power and 5% level of significance; the estimated sample was then adjusted for design effect and nonresponse. The total sample (summing up all projects and service delivery channels) was 2,805, and to have optimal study power, the sample was split into half for facility- and home-based interviews.

We employed multistage sampling technique with stratification for the selection of clients. The first stage included the selection of health facilities which revolved around social franchising – the largest service delivery channel. A total of 151 SF centers were randomly selected from a list frame, stratified by project; thereafter, within the selected districts, all static clinics and outreach camps were included in the survey. The second stage included the selection of clients: for exit interviews, we employed a consecutive sampling technique whereby all clients visiting the study sites were invited to participate in the survey till the desired sample is achieved. In contrast, the enumerators, as per the predefined quota, sequentially selected clients (using the medical records in backward direction) who had already received FP services in the preceding dates. The sample was proportionally distributed across health facilities in accordance with the client volume for each service delivery channel.

Instrument

We used a structured exit interview questionnaire developed by Marie Stopes International with slight modifications deemed necessary in view of the country context. The questionnaire for exit and home interviews was identical; however, few minor grammatical tweaks were made in household questionnaire to account for different timeframes of receiving services. The questionnaire broadly covered the following sections: 1) socioeconomic and demographic characteristics; 2) service use; 3) counseling or perception of quality; 4) sources of information; and 5) client satisfaction.

Data collection and management

The data were collected by an independent third-party research firm who had hired and trained local enumerators to conduct face-to-face interview in privacy on paper-based forms. Service providers were not given prior information about the survey, and in order to minimize potential bias of the Hawthorne effect (from service providers), home-based interviews preceded the exit interviews. At each study site, the enumerators selected clients using the medical records for home-based interviews (as described in the Sampling strategy section). After the completion of home-based interviews, they were stationed at the health facility to conduct exit interviews with new clients meeting eligibility criteria. On average, each interview took 15–20 minutes. The data were double-entered in EpiData Version 3.1.

Measures

Dependent variables

The survey contained 10 questions to capture clients’ perceived satisfaction with the services. The questions were classified into two broad dimensions of quality as per Donabedian framework³⁹ structural care: facility opening hours, cleanliness, waiting time, and service prices; and procedural care: staff attitude, friendliness and respect from the service provider, privacy, length of time spent with the service provider, the quality of advice or information, and procedure they underwent. The respondents were asked to rate their answers on a scale from 1 (strongly dissatisfied) to 5 (strongly satisfied). For the construction of composite measures, all 10 satisfaction items were dichotomized by coding strongly dissatisfied, dissatisfied, or neutral as “0” and “1” for satisfied or very satisfied. Thereafter, by adding up the ratings for participants’ responses, we constructed two separate composite measures for structural and procedural care – ranging from 0 to 4 and 0 to 6, respectively. Finally, to ensure sufficient cell count, we merged initial two (0 and 1) and three (0, 1, and 2) categories of structural and procedural measures, respectively.

Service quality was measured by asking client’s experiences through objectively phrased questions. All questions could get only a binary response: “1” if yes and “0” if no. The coding of negatively phrased questions was reversed for consistency. The categories of Bruce’s framework⁷ were reworked, and the questions of service quality were classified according to the following three categories: interpersonal relations (welcome on arrival, did not misbehave, maintained privacy, and comfort clients to ask questions), information given to clients (talk about potential side effect, potential benefits, what to do in case of side effect, and when to return for follow-up), and choice of methods (discuss fertility intentions, offered range of method, ask client’s method preference, and show sample of contraceptive methods). Separate composite indices were created for each of the thematic category, with an ordinal scale, by summing the responses of questions pertinent to respective category. Due to low cell counts, the initial three categories (0, 1, and 2) were merged to prevent the potential influence of outliers on estimates.

Independent variables

The key independent was place of interview which was binary: interviews conducted at facility were coded as “1” and home-based interviews were coded “0”.

Control variables

The analysis was adjusted for participants’ socioeconomic and demographic characteristics: age (continuous), education (none, some primary, completed primary or above), number of children (discrete), and wealth quintile (poorest, poorer, medium, wealthy, or wealthiest); and health-seeking behaviors: prior use of contraceptive (no or yes), ever been to this health facility (study site; no or yes), travel time to health facility (≤15 minutes or >15 minutes), interaction with community health worker before visiting the health facility (no or yes), and the type of contraceptive method (short-acting, long-acting, or permanent). The construction of wealth quintile was similar to the approach internationally used by the Demographic and Health Surveys.⁴⁰

Statistical analysis

The statistical analyses were performed in Stata Version 14.0 (StataCorp LP, College Station, TX, USA). Percentages, averages, and standard errors were used to describe the characteristics of study participants. For categorical variables, Pearson’s χ² was used to test crude differences in sample characteristics between home- and facility-based groups, whereas two-sample t-test was used for continuous variables. The descriptive analysis (Table 1) presents the overall results; each service delivery channel is given equal weights to account for varying sample sizes. The differences in clients’ responses (between facility- and home-based interviews) for indicators pertaining to quality and perceived satisfaction were assessed by using ordered logit regression that was used with a proportional odds assumption.⁴¹ A more appropriate technique would have been ordered probit model, but after trying both the models and not finding any differences between them (see the results of probit model in Table S1), we preferred to present the results obtained with the ordered logit method for ease of interpretation. The possible correlation among the participants receiving services from the same health facility was controlled by estimating robust standard errors.

Table 1 Characteristics of study participants

Indicators	Exit interview (n=1,369) %	Home interview (n=1,359) %
Socioeconomic and demographic
Participants’ age, mean (SE)	31.9 (0.3)	31.2 (0.3)
Education
None/no formal/some primary	53.9	52.9
Complete primary or higher	46.1	47.1
Living children, mean (SE)	4.4 (0.2)	4.2 (0.2)
Wealth quintile
Poorest	11.4	9.4
Poor	19.3	17.9
Medium	21.7	28.6
Wealthy	33.0	29.7
Wealthiest	14.6	14.1
Behaviors pertaining to contraception
Ever been to index health facility (study site)	55.0	48.2
Prior use of any modern contraceptive in the last 3 months*	60.7	47.1
Travel time of health facility
≤15 minutes	34.5	39.3
>15 minutes	65.5	60.7
Meeting with community health worker before visiting health facility	59.4	66.0
Type of contraceptive method received
Short-acting (oral pill, condom, injectable)	39.6	35.1
Long-acting (intrauterine device, implant)	36.8	44.1
Permanent (female sterilization)a	23.6	20.9

Notes:

* p<0.05.

a No case of male sterilization was observed during the survey. Data presented as percentage unless otherwise mentioned.

In Equation 1$\log \left[\frac{\mathrm{P}({\mathrm{y}}_{\text{ij}}\le \mathrm{k})|\mathrm{x}}{\mathrm{P}({\mathrm{y}}_{\text{ij}}>\mathrm{k})|\mathrm{x}}\right]={\alpha }_{\mathrm{k}}-{\mathrm{x}}^{\mathrm{T}}\mathrm{\beta },$(1) , cumulative probabilities are expressed as a function of the explanatory variables. A key assumption is that the odds ratios (ORs) are identical for each of the “k” cumulative probabilities, hence the name “proportional odds model.” The following empirical specification was estimated in the analysis for each of the five outcomes variables about clients’ satisfaction and dimensions of service quality: $\log \left[\frac{\mathrm{P}({\mathrm{y}}_{\text{ij}}\le \mathrm{k})|\mathrm{x}}{\mathrm{P}({\mathrm{y}}_{\text{ij}}>\mathrm{k})|\mathrm{x}}\right]={\alpha }_{\mathrm{k}}-{\mathrm{x}}^{\mathrm{T}}\mathrm{\beta },$(1) where i and j denote participant and health facility, respectively; y represents the outcome variables in ordinal form; α_k represents the intercept term, which is ordered to reflect the cumulative probabilities; x is a vector containing covariates that included participants’ characteristics and “place of interview,” which is the dummy variable with exit interview coded as “1” and “0” for home-based interview; and b is a parameter vector containing regression coefficient associated with covariates in x. The OR was obtained by exponentiating the log odds coefficient. The parameter of interest was b that reports difference in outcome variables for exit interview relative to home-based interview. A value >1 reflects clients’ favorable ratings to health professionals indicating the presence of courtesy bias or the Hawthorne effect whereby service providers perform better in the presence of survey enumerator at the facility.

The proportional odds assumption was violated for some indicators; therefore, we used gologit2⁴² to estimate unconstrained proportional odds. Because outcome variables used in this study were innocuous and we found no difference in the conclusion derived from both the models (constrained and unconstrained proportionality), we presented the results of constrained proportionality model.

Ethical consideration

The study protocol was approved by the Ethics Review Committee of Marie Stopes International (reference number: 014-14-E-Am15-2) and National Bioethics Committee (NBC) Pakistan (reference number: 4-87/12/NBC-193/RDC/1555). All survey participants provided a written informed consent. All the participants were aged >18 years.

Results

Table 1 presents the characteristics of study participants separately for each survey type. Overall, no statistically significant differences were found between the characteristics of respondents interviewed at facility and home. The only exception was prior use of modern contraceptive method in the last 3 months: participants in exit interview group reported higher use compared with those who were interviewed at home (60.7% vs 47.1%). Overall, the mean age of the participants was 31 years; they had four living children, approximately half of the participants had completed primary or higher level of education, and nearly one-third belonged to lowest two wealth quintiles.

Half of the participants had been to the facility (study site) before, and it took them >15 minutes to get to the health facility (study site). Nearly 60% of the participants had interacted with community health worker before visiting the site (on the day of interview). Of the participants interviewed, the majority received long-acting and reversible contraceptive, followed by short-acting and permanent methods (Table 1; refer these results segmented by service delivery channels in Table S2).

Bivariate analysis

Table 2 summarizes crude effect of survey type on measures of service quality and perceived satisfaction by service delivery channels. In NGO clinics, no significant differences were found in client’s responses regarding satisfaction and service quality; the only exception was interpersonal communication, which was inflated positively in favor of the provider in the exit interview (p=0.0231). In the context of social franchising, respondents tend to report higher satisfaction in exit interviews with both structural (p=0.0453) and procedural (p=0.0219) dimensions than in home-based interviews. Unlike NGO clinics and SF centers, we found that the respondents tend to report better experiences in home-based interview as opposed of exit interview. Client’s reports of “interpersonal communication” and “information given to client” were substantially negative in exit interviews compared with home-based interviews (p≤0.0001 and 0.0001, respectively).

Table 2 Client’s responses regarding the quality of services and satisfaction by the type of survey and service delivery channel

Indicators	NGO clinics		Social franchise centers		Outreach camps
	Exit interview (n=230) %	Home interview (n=216) %	Exit interview (n=976) %	Home interview (n=983) %	Exit interview (n=163) %	Home interview (n=160) %
Satisfaction
Structural dimension	p=0.3587		p=0.0453		p=0.6856
Satisfied with less than 3 dimensions	3.5	8.3	7.8	12.4	13.5	8.1
3	15.2	18.1	12.5	13.3	12.9	15.6
4	28.7	29.2	19.4	19.3	16.0	10.6
Satisfied with all 4 dimensions	52.6	44.4	60.4	54.9	57.7	65.6
Procedural dimension	p=0.2494		p=0.0219		p=0.5590
Satisfied with less than 3 dimensions	3.0	6.9	4.5	7.4	11.0	6.3
3	2.6	5.1	3.8	5.5	5.5	10.0
4	13.9	9.3	8.5	11.6	7.4	15.6
5	18.7	24.5	13.0	12.3	12.3	11.9
Satisfied with all 6 dimensions	61.7	54.2	70.2	63.2	63.8	56.3
Service quality
Interpersonal communication	p=0.0231		p=0.2911		p=<0.0001
Met 2 or less standards	5.7	13.4	11.3	13.5	39.3	8.1
Met 3 standards	22.6	31.5	29.0	25.6	44.8	15.6
Met all 4 standards	71.7	55.1	59.7	60.8	16.0	76.3
Information given to clients	p=0.1269		p=0.4006		p=0.0001
Met 2 or less standards	1.3	1.9	4.2	5.5	31.3	1.3
Met 3 standards	5.7	10.7	10.9	12.0	15.3	6.9
Met all 4 standards	93.0	87.5	84.9	82.5	53.4	91.9
Choice of methods	p=0.5344		p=0.6557		p=0.1352
Met 2 or less standards	8.2	4.6	6.9	6.3	29.5	6.9
Met 3 standards	25.7	31.5	29.9	27.5	40.5	42.5
Met all 4 standards	66.1	63.9	63.2	66.3	30.1	50.6

Note: p-values in bold denote statistically significant differences.

Abbreviation: NGO, nongovernmental organization.

Multivariate analysis

Table 3 presents ordered logistic regression estimates for the following indicators by service delivery channels: satisfaction with structural care, satisfaction with procedural care, experiences of interpersonal communication, information given to clients, and choice of methods. All models were controlled for respondents’ socioeconomic and demographic characteristics and contraceptive behavior.

Table 3 Ordered logit: aOR for client’s perceived satisfaction and service quality overall and by service delivery channel

Independent variable	Perceived satisfaction				Service quality
NGO clinics	Structural care		Procedural care		Interpersonal relations		Information given to clients		Choice of methods
	aOR	(95% CI)	aOR	(95% CI)	aOR	(95% CI)	aOR	(95% CI)	aOR	(95% CI)
Place of interview (home)	Ref		Ref		Ref		Ref		Ref
Exit interview	1.35	(0.75, 2.43)	1.27	(0.58, 2.77)	2.14*	(1.18, 3.85)	2.07*	(1.01, 4.24)	0.96	(0.49, 1.85)
Social franchise centers
Place of interview (home)	Ref		Ref		Ref		Ref		Ref
Exit interview	1.33*	(1.03, 1.72)	1.43***	(1.12, 1.84)	1.07	(0.83, 1.38)	1.56*	(1.12, 2.18)	0.98	(0.72, 1.32)
Outreach camps
Place of interview (home)	Ref		Ref		Ref		Ref		Ref
Exit interview	1.27	(0.34, 4.73)	2.43	(0.49, 11.97)	0.09***	(0.03, 0.36)	0.10*	(0.02, 0.68)	0.45	(0.1, 1.95)

Notes:

* p<0.05;

*** p<0.001; adjusted for respondent’s age, education, number of children, wealth quintile, prior use of contraceptive in the last 3 months, ever been to index health facility, travel time to health facility, type of method received, and interaction with community health worker before visiting the health facility.

Abbreviations: aOR, adjusted odds ratios; CI, confidence interval; NGO, nongovernmental organization; Ref, reference category.

In NGO clinics, interview location had no effect on client’s perceived satisfaction. However, clients are 2.14 (95% confidence interval [CI] =1.18, 3.85) and 2.07 (95% CI =1.01, 4.24) times more likely to report better experiences about interpersonal relations and information given to them, respectively. We found that clients served through SF centers are more likely to report being satisfied with structural (aOR =1.33, 95% CI =1.03, 1.72) and procedural care (aOR =1.43, 95% CI =1.12, 1.84) in exit interviews. Furthermore, in exit interviews, clients are 1.56 (95% CI =1.12, 2.18) times more likely to respond positively regarding information given to them, compared with home-based interviews.

Finally, scores of perceived satisfaction among outreach clients were higher in favor of the provider in exit interviews – however, the association was statistically insignificant. Interestingly, for the measures of service quality, the odds of interpersonal relation and information given to client were 0.09 (95% CI =0.03, 0.36) and 0.10 (95% CI =0.02, 0.68) in exit interviews compared with home-based interviews – indicating that clients are more likely to report bad experiences in exit interviews than in home-based interviews.

Discussion

This study assessed and compared clients’ perceived satisfaction and experiences of FP services between facility- and home-based interviews. Our results showed that, when interviewed at health facility, clients report substantially higher satisfaction with both structural care and procedural care in SF settings. The results partially substantiate the inference from the earlier research that demonstrated that client’s responses (of perception-based questions) tend not to be biased for location in exit interviews,¹⁷ whereas in our case, location affected both structural and procedural measures of care in SF settings.

The direction of estimates (Table 3) for both the measures of satisfaction across all service delivery channels also indicates signs of courtesy bias or possible Hawthorne effects, in exit interviews. The magnitude of the estimated coefficients was similar across service delivery channels, but the relationship was statistically insignificant for NGO clinics and outreach camps, which may be due to smaller sample sizes. Moreover, the magnitude of ORs was greater (in SF and outreach camp) for procedural care compared with structural care – suggesting higher scores in favor of care provider than health facility in exit interviews.¹⁷

Service quality was measured through objectively phrased questions covering the experiential aspect of care. In NGO clinics, clients gave higher scores on interpersonal relations and information given to them; however, in the context of social franchising, high scores were observed only for procedural care. It is pertinent to note that to assess service quality, clients were asked the range of questions about providers’ behavior – whether she talked about side effects, when to return for follow-up, etc.; therefore, there may be some room for recall bias in home-based interviews. We found no significant differences in interpersonal relations among social franchising clients, which was contrary to expectations, because (MSS) franchised centers are usually run by a single provider, and at times, in small franchised health facilities, clients may be overheard easily by others (including service provider) – which could easily influence them to give positive feedback. We see two possible reasons for this finding: 1) franchised service providers have good interpersonal relations with clients, but are not adequately conveying information regarding FP methods to clients; and 2) clients visiting franchised centers came from adjacent vicinities (results not shown), and in rural areas (small villages), providers are usually well acquainted with community members – consequently, that personal acquaintance may refrain clients from sharing negative feedback about service providers even in home-based interviews.⁴³

Clients who received services from outreach camps reported considerably negative experiences regarding service quality (interpersonal relations and information given to them) in exit interviews than in home-based interviews. These results seemed counterintuitive – however, from a broader perspective, we see a combination of factors that together may be responsible for these results. For instance, the average client volume at a camp is 40 with the camp lasting from 6 to 8 hours, and the majority of the clients choose long-acting or permanent contraceptive methods that are procedurally more time-consuming than short-acting methods; this overcrowding could possibly affect information exchange and interpersonal care in an undesirable manner.⁴⁴ Moreover, since the majority of the outreach clients receive tubal ligation, which requires administration of anesthesia, it may be possible that these clients had difficulty recalling their discussions with providers due to momentary forgetfulness. Moreover, in contrast to SF and NGO clinics, outreach clients may have seen the providers for the first time, and because the majority of them received permanent contraceptive methods, they may not have worried about future care, as they are less likely to come in contact with the same provider, and therefore they may have felt more empowered to report negative feedback at the time of their exit. It was unexpected, however, to observe that in outreach settings, clients’ scores of service quality contradict with their perceived level of satisfaction; this warrants further inquiry for better understanding.

Limitations

There are some inherent limitations to our study: the clients interviewed at health facility and home were not the same – although, the observed characteristics of both the samples were not statistically dissimilar. In order to control for unobserved factors, it may be a better approach to conduct exit and home-based interviews on same clients. Moreover, our analyses did not account for a range of facility or provider characteristics that may affect client’s satisfaction.^17,45 Clients interviewed after some time (2–3 days) of service uptake may have recall bias. The questionnaire used in this study was completely structured; exploring clients’ experiences through some unstructured questions could have enhanced understanding regarding facility and home environment.

Conclusion

Overall, our study indicated some signs of courtesy bias and possibly the Hawthorne effect in exit interviews. This was true for both experiential (service quality) and perception-based (satisfaction) questions in the context of SF, and in NGO clinics, the location of interview only affected clients’ responses regarding service quality. On the contrary, in outreach settings, clients are more likely to share bad experiences in exit interviews than in home-based interviews on objectively phrased questions. Program implementers could opt for home-based interviews for static clinics (NGO clinics and SF), whereas exit interviews could still be used for outreach setting. However, home-based interviews are generally more complex and resource-intensive, especially cost; hence, exit surveys could be confined to certain questions that are least affected by the facility environment.

Acknowledgments

This study received grants from the Department for International Development, and the US Agency for International Development during the conduct of the study. The present paper includes the collective views of group of experts, and does not necessarily represent the decisions or the stated policy of the Department for International Development, United States Agency for International Development, or Marie Stopes Society. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Supplementary materials

Table S1 Ordered probit: adjusted coefficients for client’s perceived satisfaction and service quality overall and by service delivery channel

Independent variable	Perceived satisfaction				Service quality
	Structural care		Procedural care		Interpersonal relations		Information given to clients		Choice of methods
NGO clinics
Place of interview (home)	Ref		Ref		Ref		Ref		Ref
Exit interview	0.21	(−0.14, 0.55)	0.16	(−0.31, 0.63)	0.45*	(0.11, 0.78)	0.33¥	(−0.01, 0.68)	−0.05	(−0.44, 0.34)
Social franchise centers
Place of interview (home)	Ref		Ref		Ref		Ref		Ref
Exit interview	0.17*	(0.02, 0.33)	0.22***	(0.07, 0.36)	0.05	(−0.1, 0.2)	0.22*	(0.05, 0.39)	−0.02	(−0.2, 0.16)
Outreach camp
Place of interview (home)	Ref		Ref		Ref		Ref		Ref
Exit interview	0.05	(−0.68, 0.77)	0.37	(−0.42, 1.15)	−1.36***	(−2.02, −0.7)	−1.3***	(−2.14, −0.46)	−0.48	(−1.28, 0.32)

Notes:

* p<0.05;

*** p<0.001;

¥ p<0.10 (marginally significant); adjusted for respondent’s age, education, number of children, wealth quintile, prior use of contraceptive in the last 3 months, ever been to index health facility, travel time to health facility, and interaction with community health worker before visiting the health facility.

Abbreviations: NGO, nongovernmental organization; Ref, reference category.

Table S2 Characteristics of study participants by sites and by survey

Indicators	NGO clinics		Social franchise centers		Outreach camps
	Exit interview (n=230) %	Home interview (n=216) %	Exit interview (n=976) %	Home interview (n=983) %	Exit interview (n=163) %	Home interview (n=160) %
Socioeconomic and demographic
Participants’ age, mean (SE)			p=0.037
	32.1	31.2	30.3	30.9	33.1	32.6
Education
None/no formal/some primary	32.6	45.4	50.5	51.5	79.1	61.9
Complete primary or higher	67.4	54.63	49.5	48.5	20.9	38.1
Living children, mean (SE)	3.8	3.8	3.6	3.8	5.0	5.8
Wealth quintile
Poorest	2.2	3.7	4.1	5.2	28.2	19.4
Poor	4.8	4.2	14.9	12.8	38.7	36.9
Medium	16.9	18.5	29.8	29.3	18.4	36.9
Wealthy	43.9	40.3	41.3	42.2	13.5	6.9
Wealthiest	32.2	33.3	9.8	10.5	1.2	0.0
Behaviors pertaining to contraception
Ever been to index health facility (study site)	66.9	62.9	69.6	66.9	28.2	15.0
Prior use of any modern contraceptive in the last 3 months					p=0.0071
	61.3	62.9	68.8	63.7	52.2	15.0
Travel time of health facility
≤15 minutes	29.1	38.4	51.3	47.4	19.0	28.8
>15 minutes	70.9	61.6	48.7	52.6	81.0	71.2
Meeting with community health worker before visiting health facility	p=0.0304
	24.4	37.0	75.8	76.0	79.1	83.8
Type of contraceptive method received			p<0.0001
Short-acting (oral pill, condom, injectable)	45.7	43.1	61.4	40.9	11.7	21.3
Long-acting (intrauterine device, implant)	49.1	42.6	36.5	58.1	24.5	31.2
Permanent (female sterilization)a	5.2	14.4	2.1	1.1	63.8	47.5

Notes: p-values are only shown for differences that were statistically significant.

a No case of male sterilization was observed during survey.

Abbreviation: NGO, nongovernmental organization.

Disclosure

The authors report no conflicts of interest in this work.