Estimating Causal Effects With Propensity Score Models: An Evaluation of the Touch Condom Media Campaign in Pakistan

Abstract

Rapid population growth in Pakistan poses major risks, including those pertinent to public health. In the context of family planning in Pakistan, the current study evaluates the Touch condom media campaign and its effects on condom-related awareness, attitudes, behavioral intention, and behavior. This evaluation relies on 3 waves of panel survey data from men married to women ages 15–49 living in urban and rural areas in Pakistan (N = 1,012): Wave 1 was March 15 to April 7, 2009; Wave 2 was August 10 to August 24, 2009; and Wave 3 was May 1 to June 13, 2010. Analysis of variance provided evidence of improvements in 10 of 11 condom-related outcomes from Wave 1 to Wave 2 and Wave 3. In addition, there was no evidence of outcome decay 1 year after the conclusion of campaign advertising dissemination. To help compensate for violating the assumption of random assignment, propensity score modeling offered evidence of the beneficial effects of confirmed Touch ad recall on each of the 11 outcomes in at least 1 of 3 time-lagged scenarios. By using these different time-lagged scenarios (i.e., from Wave 1 to Wave 2, from Wave 1 to Wave 3, and from Wave 2 to Wave 3), propensity score modeling permitted insights into how the campaign had time-variant effects on the different types of condom-related outcomes, including carryover effects of the media campaign.

Notes

¹ In terms of fertility rates for women ages 40–49, there is variance across wealth quintiles: first quintile, 6.5%; second quintile, 6.2%; third quintile, 6.0%; fourth quintile, 5.4%; and fifth quintile, 4.5% (National Institute of Population Studies, 2013). Among married women ages 15–49, contraception practices increase by wealth quintile: first quintile, 20.8%; second quintile, 29.7%; third quintile, 38.2%; fourth quintile, 41.5%; and fifth quintile, 45.8%. Finally, the percentages of married women ages 15–49 experiencing unmet need in terms of family planning are as follows: first quintile, 24.5%; second quintile, 23.2%; third quintile, 19.0%; fourth quintile, 18.8%; and fifth quintile, 15.3%.

² Confirmed Touch ad recall was calculated using a hybrid approach (for a more detailed explication, see Beaudoin & Stephenson, in press). In Step 1, respondents were asked the following unaided question: “During the last 3 months, have you seen any advertisements on television about contraceptive methods or reproductive health services?” In Step 2, respondents who answered “yes” to the previous question were then asked, “What did you see in the advertisements?” Respondents were coded as having confirmed ad recall if their responses were consistent with at least one of the 12 main elements of the Touch condom advertising. In Step 3, in an aided approach, respondents who said “no” to the initial unaided ad awareness question were then asked (constituting aided ad awareness) the following question specific to contraceptive methods or reproductive health services: “Did you see an advertisement in the last 3 months in which a young couple meet, get married, and have a happy married life?” In Step 4, respondents who answered “yes” were then asked to complete six jingles from the campaign ad. Respondents were coded for having aided confirmed ad recall if they could properly complete at least one jingle. An overall group for confirmed Touch ad recall was composed of respondents coded as unaided confirmed ad recall (see Step 2) or aided confirmed ad recall (see Step 4).

³ The eight covariates were included in the estimation of propensity scores because they could explain variance in confirmed Touch ad recall, as well as the outcomes, but were not themselves caused by the campaign or the outcomes (Yanovitzky et al., 2005).

⁴ PSM was implemented to match observations with other observations, thus creating a sample of the control group (i.e., respondents without confirmed Touch ad recall) to match the treatment group (i.e., respondents with confirmed Touch ad recall). As a result, any differences between recall and nonrecall groups should have resulted from the Touch media campaign.

⁵ Although ANOVA specified significant changes in embarrassment in purchasing condoms and decreased pleasure by condom use, more refined differences were nonsignificant via Scheffé post hoc tests.