Agreement between patient-reported and provider-reported choice of contraceptive method among family planning patients in New York City: implications for public health

Abstract

National data on choice of contraceptive method and subsequent use are critical for monitoring progress toward meeting public health goals in reducing unintended pregnancy in the US. Yet few studies have focused on the reliability of clinically-reported or patient-reported measures of choice of contraceptive method for the range of available contraceptive methods. Among 1,844 women receiving reproductive health care at two federally funded centers in New York City, choice of contraceptive method at the end of the visit from two data sources was compared, ie, patient self-report, and provider-report as recorded in the clinical-administrative database. Agreement between the two data sources was assessed for the sample. Sociodemographic predictors of agreement were assessed using logistic regression. Agreement between the data sources was also assessed on a method-by-method basis using positive specific agreement. Participants were predominantly Latina (69%), foreign-born (76%), and low-income (99% with incomes <200% federal poverty level). Agreement of patient-reported and provider-reported contraceptive choice was highest for hormonal methods (positive specific agreement 94.0%) and intrauterine devices (89.9%), and lowest for condoms (53.5%). In the logistic regression model, agreement was lower among teens aged 16–19 years compared with women aged 25+ years (odds ratio 0.74; 95% confidence interval 0.55–0.99). Because teens are more likely to rely on condoms, the logistic regression model was repeated, adjusting for provider report of condom choice; after adjustment, no sociodemographic differences in agreement were observed. National data sources or studies relying on provider-reported method choice to derive estimates of contraceptive prevalence may overestimate choice of condoms. Our findings raise the question of whether condom choice can be accurately assessed by a single open-ended measure of choice of contraceptive method.

Keywords:

• contraception
• condoms
• methodology
• service providers

Introduction

Unintended pregnancy is a serious public health concern in the US, because it has been found to result in adverse health outcomes for infants and women,^1,2 including preterm delivery^3,4 and increased rates of abortion.⁵ In the US, it is estimated that each year unintended pregnancies result in $4.6 billion in direct medical costs, half of which is attributable to imperfect contraceptive use.⁶ Consistent with other health indicators, the burden of unplanned pregnancy in the US is not evenly distributed across the population, with young,^7–9 low-income,^10,11 and minority women^10,12 being disproportionately affected. These disparities in unintended pregnancy reflect disparities in contraceptive use, with young women, women of low socioeconomic status, and black women (compared with women of other races) being found to be less likely to use contraception or more likely to rely on less effective methods, such as condoms.^5,10,13–16

Consistent use of effective contraceptive methods prevents unintended pregnancy; this ongoing behavior requires an individual to first make a choice among the available contraceptive methods.^17–20 National data on choice of contraceptive method and its subsequent use, such as those reported in the National Survey of Family Growth, are critical to understanding the scope of unintended pregnancy as a public health issue, to estimating numbers of women at risk of unintended pregnancy, and to monitoring progress toward meeting public health goals in reducing unintended pregnancy.^16,21–25 National data on choice of contraceptive method, as reported in clinical-administrative records, are also used to monitor and guide decisions about the allocation of funding for publicly supported family planning programs, such as the federal Title X program, which served 5.2 million women in the US in 2010.²⁶

In spite of the clinical and public health implications, few studies have focused on the reliability or validity of either clinically-reported or patient-reported measures of choice of contraceptive method for the range of available methods. A recent review of the literature concluded that information on the validity or reliability of self-reported measures of oral contraceptive choice or use was scarce.²⁷ To address this gap, this study assessed agreement between patient-reported and provider-reported choice of contraceptive method at family planning clinic visits. Due to lack of a “gold standard” necessary to assess the validity of either patient-reported or provider-reported choice of contraceptive method, the aim of this study was to assess agreement, as a form of reliability, between the two sources of report.

Materials and methods

Study design and setting

The data used for this study were collected as part of ongoing quality assurance activities within a randomized controlled trial of a contraceptive assessment module at two clinics in New York City receiving Title X federal family planning funding (as reported elsewhere).²⁸ All patient recruitment activities were carried out by two trained bilingual (Spanish/English) project assistants between April 2008 and August 2010. Participants were recruited at time of visit and screened for eligibility. English-speaking or Spanish-speaking women age 16 years and over who had a family planning visit on the date of recruitment and were capable of providing informed consent were eligible for participation. Exclusion criteria included women at the clinic for a walk-in pregnancy test and women who were pregnant, seeking pregnancy, surgically sterilized, with a partner who was surgically sterilized, or those who had started menopause. The protocol was approved by the Public Health Solutions institutional review board. All study participants received family planning services according to existing standards of care.

In total, 2,448 women consented to participate in the randomized controlled trial. Of those women, 465 did not complete the end-of-visit survey that included patient report of choice of contraceptive method, and 139 were missing provider-reported contraceptive choice data in the clinical-administrative database, leaving a final sample of 1,844 women for these analyses.

Collection of patient-reported data

Eligible and consenting participants were given a touch screen laptop loaded with audio-computer-assisted self-interviewing (ACASI) software. All participants were asked ten basic demographic questions using the ACASI computer prior to their clinic visit. All study participants completed a seven-question interviewer-administered survey immediately following the clinical visit. The survey asked “Which birth control method(s) did you get today?” All materials, including the ACASI survey and the end-of-visit survey, were available in both English and Spanish. In total, 11 patients reported choice of more than one method, with ten of these women reporting choice of condoms with another method; for these participants, the contraceptive method chosen was categorized as the most effective method reported, consistent with National Survey of Family Growth methodology.¹⁶

Collection of provider-reported data

Sociodemographic data, insurance status, and provider-reported choice of contraceptive method at the end of the visit were exported from the existing clinical-administrative database of the participating family planning provider network. Per clinic protocols, choice of contraceptive method was documented by the clinical provider at the time of visit on a machine-readable form and then imported into the clinical-administrative database. The form included a list of all available contraceptive methods. Following completion of contraceptive counseling, the method(s) documented by the provider was that which the patient chose to either continue on or switch to, regardless of whether the provider was able to physically provide the patient with their chosen method at the time of visit or if a plan was put in place for the patient to obtain the method at a later time (eg, scheduled visit for insertion of an intrauterine device). The provider-reported choice of contraceptive method as documented in the clinical-administrative database was used for these analyses; subsequent data cleaning was done for the efficacy analyses in the main trial.²⁹ Consistent with the patient-reported data, in instances in which the provider reported choice of more than one method (n = 17; 16 of these instances involved choice of condoms with another method), the choice of method was categorized as the most effective method recorded.¹⁶

Outcome 1: positive specific agreement

Positive specific agreement was calculated to assess the level of agreement between the two information sources (ie, patient and provider) for each individual contraceptive method, as presented in Figure 1. Positive specific agreement measures the conditional probability that a source (eg, patient) will identify a positive outcome (eg, choice of a particular method) given that another randomly selected source (eg, provider) has also identified the same case as positive (ie, indicated choice of the same method).²⁹ In other words, it is a measure of chance-corrected agreement that estimates the proportion of positive cases that were agreed upon (denoted as a in Figure 1). Of relevance to this study, positive specific agreement can be used to approximate traditional measures of reliability, such as the k statistic, that require an accurate count of negative cases. In this study, the negative case count refers to instances where neither the provider nor the patient reported choice of the particular method for which agreement is being assessed (denoted as d in Figure 1); these negative cases are not well defined or clinically relevant.

Figure 1 Calculation of positive specific agreement for each method.

Outcome 2: simple agreement

For the entire sample and for selected sociodemographic subgroups, simple agreement was calculated as a proportion, ie, the number of instances where the provider and patient both reported the same choice of contraceptive method (or both reported no choice of method) divided by the total number of participants. This outcome measure was used for logistic regression models to compare overall agreement across subgroups.

Statistical analyses

Data analyses were done using Statistical Package for the Social Sciences version 20.0 software (SPSS Inc, Chicago, IL, USA). For each participant, choice of contraceptive method as reported by the patient and choice of contraceptive method as reported by the provider was cross-tabulated, and simple agreement was calculated. For each contraceptive method, positive specific agreement was calculated. Logistic regression analyses were conducted to examine each sociodemographic characteristic as a predictor of simple agreement. The logistic regression model was repeated, controlling for condom choice (as reported by the provider), to examine whether lower levels of simple agreement observed among some subgroups was explained by differing prevalence of condom choice.

Results

As shown in Table 1, the 1,844 participants were predominantly Latina (69%), foreign-born (76%), and low-income. The mean age of participants was 28 years. Over half (59%) of the sample did not have health insurance and 40% were covered by public insurance. According to income eligibility standards for public insurance and the provider network’s sliding fee scale for uninsured patients, 99% of the sample had incomes below 200% of the federal poverty level.³⁰

Table 1 Socio-demographic data of sample participants (n = 1,844)

Characteristic	Total (n = 1,844)
	n	%
Age at interview
16–19	265	14.4%
20–24	402	21.8%
25–29	458	24.8%
30–34	366	19.8%
35 and older	353	19.1%
Race/ethnicity
Hispanic, any race	1,260	69.3%
Non-Hispanic black	256	14.1%
Non-Hispanic, other race	301	16.6%
Module language
English	812	44.0%
Spanish	1,032	56.0%
Educational attainment
Less than high school	488	26.9%
High school/GED	732	40.3%
Some college or higher	595	32.8%
Nativity
US-born	438	23.9%
Foreign-born	1,397	76.1%
Insurance status
Public insurance	740	40.1%
Private insurance	16	0.9%
Uninsured	1,088	59.0%

Abbreviation: GED, general equivalency diploma.

Positive specific agreement for each contraceptive method, as detailed in Table 2, varied widely. For some methods, positive specific agreement was high, ie, 82.8% for the hormonal patch, 89.9% for intrauterine devices, and 93.4% for oral contraceptives. Positive specific agreement was lowest for condoms (53.5%) and no method (31.2%). In the 358 instances in which the provider reported that condom use was the method choice, the patient reported that condom use was the method chosen in only 149 instances, while the patient reported that no method was chosen in 152 instances. The remaining 57 instances without agreement were evenly distributed across seven other methods.

Table 2 Positive specific agreement between provider report and patient report, by contraceptive method (n = 1,844)

Contraceptive method	Method was reported by both provider and patient (n = 1,406)	Method was reported by provider (n = 1,844)	Method was reported by patient (n = 1,844)	% of patients agreeing with provider-reported method	% of providers agreeing with patient-reported method	Positive specific agreement
IUDa	360	406	395	88.7%	91.1%	89.9%
Hormonal methods	828	883	919	95.9%	92.2%	94.0%
Depo-provera	166	181	187	91.7%	88.8%	90.2%
Nuvaring	52	55	70	94.5%	74.3%	83.2%
Patch	12	16	13	75.0%	92.3%	82.8%
Pill	598	631	649	94.8%	92.1%	93.4%
Condom	149	358	199	41.6%	74.9%	53.5%
No method	69	142	322	48.6%	21.4%	31.2%
Abstinence	0	28	1	–	–	–
Sterilization or vasectomyb	0	3	1	–	–	–
Fertility awareness method	0	3	2	–	–	–
Female barrier	0	0	5	–	–	–
Spermicide	0	3	0	–	–	–
Seeking pregnancyb	0	17	0	–	–	–
Pregnantb	0	1	0	–	–	–

Notes:

a Hormonal and copper intrauterine devices;

b participants who were pregnant, seeking pregnancy, who had had tubal ligation (sterilization) or a partner with vasectomy were not eligible for the study. The 18 participants who were reported by the provider to be pregnant or seeking pregnancy reported, in the eligibility screening instrument, that they were neither pregnant nor seeking pregnancy. The 3 participants who were reported using sterilization by the provider and the 1 participant who reported choosing sterilization at the end of the visit also met all eligibility criteria at the start of the visit.

Abbreviation: IUD, intrauterine device.

For the entire sample of 1,844 participants, there was simple agreement in 1,406 instances (76.2%). Simple agreement was lower among some subgroups: among teens aged 16–19 years simple agreement was 71.7% compared with 77.5% among women aged 25 years and over (odds ratio 0.74; 95% confidence interval 0.55–0.99), and 71.9% among non-Hispanic black women compared with 78.4% among Hispanic women who completed participation in Spanish (odds ratio 0.71; 95% confidence interval 0.52–0.97). Non-Hispanic black women were also more likely to have the provider report that they chose condoms (32.8%) than Hispanic women who completed participation in Spanish (15.8%). Because these subgroups (both in this sample and in national data reports) are more likely to rely on condoms, and because positive specific agreement for condoms was very low, the logistic regression model was repeated, adjusting for provider report of condom choice. In the adjusted model, these observed sociodemographic differences in simple agreement no longer remained (Table 3).

Table 3 Logistic regression models predicting simple agreement by sociodemographic subgroup, unadjusted and adjusted for provider report of condom choice (n = 1,844)

Characteristic	n	Simple agreementa		Unadjusted	Provider reported condom choice	Adjusted for provider report of condom choice
		n	%	OR (95% CI)	%	OR (95% CI)
Entire sample	1,844	1,406	76.2%	–	19.4%	–
Age
16–19	265	190	71.7%	0.74 (0.55–0.99)	26.8%	0.90 (0.65–1.26)
20–24	402	304	75.6%	0.90 (0.69–1.18)	21.4%	1.00 (0.75–1.34)
25 and older [ref]	1,177	912	77.5%	–	17.1%	–
Race/ethnicity
Non-Hispanic, black	256	184	71.9%	0.71 (0.52–0.97)	32.8%	1.05 (0.74–1.49)
Non-Hispanic, other	301	222	73.8%	0.78 (0.57–1.05)	19.9%	0.83 (0.60–1.16)
Hispanic, English module	336	251	74.7%	0.82 (0.61–1.09)	19.0%	0.86 (0.63–1.18)
Hispanic, Spanish module [ref]	924	724	78.4%	–	15.8%	–
Educational attainment
Less than high school [ref]	488	384	78.7%	–	18.4%	–
High school/GED	732	553	75.5%	0.84 (0.64–1.10)	19.5%	0.84 (0.62–1.13)
Some college or higher	595	444	74.6%	0.80 (0.60–1.06)	20.5%	0.81 (0.59–1.10)
Nativity
US-born	438	322	73.5%	1.21 (0.94–1.54)	27.2%	0.94 (0.72–1.24)
Foreign-born [ref]	1,397	1076	77.0%	–	17.0%	–
Insurance status
Public insurance	740	563	76.1%	0.98 (0.79–1.22)	19.9%	1.00 (0.78–1.26)
Private insurance	16	11	68.8%	0.68 (0.23–1.97)	18.8%	0.62 (0.20–1.99)
Uninsured [ref]	1,088	832	76.5%	–	19.1%	–

Note:

a Simple agreement: patient and provider both reported the same contraceptive method choice (or both reported no contraceptive method choice).

Abbreviation: GED, general equivalency diploma; ref, reference group.

Discussion

In this study, agreement of patient-reported and provider-reported contraceptive choice varied significantly by contraceptive method, and was highest for those methods most effective at preventing pregnancy, such as hormonal methods (positive specific agreement, 94.0%) and intrauterine devices (89.9%). The high agreement in reporting of oral contraceptive choice is similar to that found in another study comparing patient-report and provider records of lifetime history of oral contraceptive use, which reported high agreement on the specific brand name for the most recent oral contraceptive used.³¹ Agreement was lowest for methods with lower typical use effectiveness, particularly condoms; in almost half of the instances in which the provider reported that the patient chose condoms, the patient reported choosing no method of contraception (152/358; positive specific agreement for condoms, 53.5%). This low agreement between patient and provider is consistent with other research looking at self-reported condom use.^32–36

Few recent studies in the US, with one exception,³⁶ have examined the reliability of reported contraceptive method choice or use across the range of available methods, or have compared patient self-report with the clinical–administrative data sources that inform national data estimates. In our sample, the level of simple agreement (ie, the total amount of agreement between patients and providers for all methods combined) differed significantly among sociodemographic subgroups. However, these observed differences could all be explained by the low reliability of condom report rather than patient characteristics. Previous research using provider-reported data in similar clinical settings has found that non-Hispanic black adults and adolescents are more likely to choose condoms or other barrier methods than hormonal methods of contraception.^14,37 Our findings suggest that providers may be over-reporting condom choice for these groups.

The findings of this study should be interpreted with consideration of some limitations. Slightly different question wording was used to collect provider-reported and patient-reported choice of contraceptive method after the clinic visit, and this variation in data collection methods could have contributed to low levels of simple agreement and positive specific agreement. Specifically, the wording of the survey used to measure patient-reported contraceptive method choice after the clinical visit (“Which method did you get today?”) may have contributed to lower positive specific agreement for intrauterine devices in cases where the device was inserted prior to the clinic visit on the date of recruitment. It is plausible that in such instances the patient might have reported choice of no method of contraception and the provider might have reported intrauterine device as the method. Of the 404 instances in which the provider reported an intrauterine device, the patient also reported an intrauterine device in 360 instances (positive specific agreement, 89.9%), and in only 36 instances the patient reported choosing no method.

Similarly, it is possible that the wording of the survey question used to measure patient-reported choice of contraceptive method contributed to the low positive specific agreement for condoms (53.5%). However, at the time of this study, standard of care at the participating family planning clinics was to distribute condoms and provide relevant counseling to all patients at risk for acquiring a sexually transmitted infection regardless of the contraceptive method chosen. Although we do not have specific data on whether patients actually received condoms at each visit, given the wide availability of condoms in the clinics, it seems unlikely that confusion related to question wording would result in significant under-reporting of condom choice by patients.

Additionally, as noted in other studies, medical data captured in the use of administrative databases has low reliability,³⁸ and provider-reported choice of method may have been subject to data entry errors in the clinical-administrative database. Because assessing validity would have required us to consider the provider report (recorded in the clinical-administrative database) as a gold standard, our study examined agreement as a measure of reliability rather than validity. We did not have data on provider characteristics and were therefore unable to discern how individual provider behavior, which has been found to affect clinical outcomes in prior research,³⁹ may have influenced the contraceptive method reported. Lastly, the study was conducted among a population of primarily foreign-born, low-income Latina women seeking clinical services at publicly funded family planning clinics in New York City, which limits the generalizability of these findings to other populations and settings. Women in our study sample were more likely to be Hispanic, foreign-born, Spanish-speaking, uninsured or publicly insured, younger, and with lower educational attainment than the general female population in New York City.⁴⁰

Despite these limitations, the consistency of our choice of method data with US national data for publicly funded family planning centers suggests that these findings are likely to be of considerable relevance to providers and public health policymakers, both in the US and elsewhere, who rely on similar national data sources.⁴¹ Distribution of provider-reported choice of method after the clinic visit was compared with a national provider-reported primary contraceptive method mix, in the Family Planning Annual Report.⁴² The prevalence of condom choice in our sample as reported by the provider (19%) was consistent with that reported in the 2009 Family Planning Annual Report for women who are not pregnant, seeking pregnancy, or relying on sterilization (17%).

This similarity suggests that the providers in this study were not over-reporting condom choice as compared with a national sample of family planning providers. In our sample, in almost half of instances in which the provider reported condom choice, the patient reported receiving no method (152/358). Applying the positive specific agreement percentage for condoms found in this study to the number of women in the US reporting condom choice as reported in the 2009 Family Planning Annual Report, national family planning providers may be inaccurately reporting choice of male condoms for over 300,000 women in the US who have actually not chosen any method of contraception.

Accordingly, extrapolating national data on choice of contraceptive method to estimate use of contraception use merits additional caution. Contraceptive choice is not equivalent to contraceptive use, and should be conceptualized and measured as a distinct construct. Studies that have looked at both choice and use of methods have found that the proportion of participants actually using a particular method is significantly less than the proportion of patients who have chosen the same method.^43,44 Data sources or studies that rely on provider-reported method choice to derive estimates of contraceptive prevalence may overestimate choice of condoms, and therefore may further overestimate use of condoms.

Additionally, the low reliability in the reporting of choice of method observed in our study suggests that attention should be given to the role that patient-provider interactions play in the reporting of choice of contraceptive method. Our findings raise the question of whether or not condom choice can be accurately assessed by a single open-ended measure of choice of contraceptive method, as is currently standard of practice. Future research, using more detailed assessments to establish a gold standard in reporting, is needed to determine the validity of provider-reported choice of contraceptive method, particularly for condoms.

Acknowledgments

This project was made possible by grants from the National Campaign to Prevent Teen and Unplanned Pregnancy, the Bridge the Gap Foundation, and a private foundation. The authors wish to thank the study participants; Diana Castillo, Sarina Jean-Louis, and the staff at MIC Women’s Health Services; Melissa Kottke, and Robert Hatcher, and Peggy Goedken from Emory University; our reviewers, Roberta Scheinmann, and Sabina Hirshfield; and our interviewers, Rosanna Martinez, M Gabriela Pelaez-Ruiz, and Elizabeth Verdesoto.

Disclosure

The authors report no conflicts of interest in this work.