An Interrupted Time Series Evaluation of the Testing Makes Us Stronger HIV Campaign for Black Gay and Bisexual Men in the United States

Abstract

Black gay, bisexual, and other men who have sex with men (BMSM) are the subpopulation most disproportionately affected by HIV in the United States. Testing Makes Us Stronger (TMUS), a communication campaign designed to increase HIV testing rates among BMSM ages 18 to 44, was implemented in the United States from December 2011 through September 2015. We used interrupted time series analysis (ITSA) to compare pre- and post-campaign trends in monthly HIV testing events among the priority audience in six of the implementation cities from January 2011 through December 2014. In the 11 months prior to the launch of TMUS, HIV testing events among BMSM in the six campaign implementation cities decreased by nearly 35 tests per month (p = .021). After the introduction of TMUS, the number of HIV testing events among BMSM in the same cities increased by more than 6 tests per month (p = .002). ITSA represents a quasi-experimental technique for investigating campaign effects beyond underlying time trends when serial outcome data are available. Future evaluations can be further strengthened by incorporating a comparison group to account for the effects of history and maturation on pre- and post-campaign trends.

Acknowledgments

This work was funded by the Centers for Disease Control and Prevention under a contract with RTI International.

The authors would like to acknowledge and thank Jacqueline Rosenthal, Jennie Johnston, Donata Green, Nancy Habarta, Guoshen Wang, NaTasha Hollis, George Khalil, Natasha Vazquez, and Sidney Holt for their contributions to the Testing Makes Us Stronger campaign development and evaluation.

The findings and conclusions in this paper or those of the authors do not necessarily represent the views of the Centers for Disease Control and Prevention or RTI International.

Notes

^1. Cities were geographically delineated by Core Based Statistical Area, as defined by the United States. Office of Management and Budget.

^2. Philadelphia implemented the TMUS campaign on the local level.

^3. A valid testing event includes all National HIV Prevention Program Monitoring & Evaluation (NHM&E) HIV testing records for which a test result (positive or negative) was reported. A single HIV test event could include multiple tests that were administered to the same person to make a final determination of the test result. The NHM&E data do not contain unique individual identifiers and are not de-duplicated. Therefore, a test event is unique to one person, but that person may be represented more than once in the dataset. State, territorial and local health departments and CBOs report HIV test event-level data that includes information about assigned sex at birth, current gender, age, race/ethnicity, vaginal or anal sex with males or females and use of injection drugs. The behaviors used to calculate the target population of BMSM 18-44 include anal sex with males, race/ethnicity, and age. The Health Department Monitoring and Evaluation Team in the Division of HIV/AIDS Prevention’s Program Evaluation Branch reviewed the completeness and quality of the HIV test event-level data submitted by grantees to determine data for inclusion in this report.

4. Before selecting a final model, we conducted a sensitivity analysis comparing the standard OLS model results to a variety of models correcting for autocorrelation and/or heteroscedasticity. Overall, these corrections had very minimal impact on the estimates; all standard errors were within a few tenths of a point and none of the p values deviated by more than five-hundredths of a point across models. Compared to the estimates from the standard OLS model, correcting for heteroscedasticity yielded slightly higher standard errors and p values. In effect, by choosing to report results from an adjusted regression model with standard errors that are robust to heteroscedasticity, we erred against committing Type I error. Ultimately, though, our choice of model had no impact on interpretation.

5. Estimates for the post-campaign slope, or trend, are produced by the linear combination of the coefficients for Time and Time×TMUS (i.e., 6.22 = −34.96 + 41.17).