Projecting the Health and Economic Impact of Road Safety Initiatives: A Case Study of a Multi-country Project

Abstract

Objective: The Road Safety in 10 Countries (RS-10) project will implement 12 different road safety interventions at specific sites within 10 low- and middle-income countries (LMICs). This evaluation reports the number of lives that RS-10 is projected to save in those locations, the economic value of the risk reduction, and the maximum level of investment that a public health intervention of this magnitude would be able to incur before its costs outweigh its health benefits.

Methods: We assumed a 5-year time implementation horizon corresponding to the duration of RS-10. Based on a preliminary literature review, we estimated the effectiveness for each of the RS-10 interventions. Applying these effectiveness estimates to the size of the population at risk at RS-10 sites, we calculated the number of lives and life years saved (LYS) by RS-10. We projected the value of a statistical life (VSL) in each RS-10 country based on gross national income (GNI) and estimated the value of the lives saved using each country's VSL. Sensitivity analysis addressed robustness to assumptions about elasticity, discount rates, and intervention effectiveness.

Results: From the evidence base reviewed, only 13 studies met our selection criteria. Such a limited base presents uncertainties about the potential impact of the modeled interventions. We tried to account for these uncertainties by allowing effectiveness to vary ±20 percent for each intervention. Despite this variability, RS-10 remains likely to be worth the investment. RS-10 is expected to save 10,310 lives over 5 years (discounted at 3%). VSL and $/LYS methods provide concordant results. Based on our estimates of each country's VSL, the respective countries would be willing to pay $2.45 billion to lower these fatality risks (varying intervention effectiveness by ±20 percent, the corresponding range is $2.0–$2.9 billion). Analysis based on $/LYS shows that the RS-10 project will be cost-effective as long as its costs do not exceed $5.14 billion (under ±20% intervention effectiveness, the range = $4.1–$6.2 billion). Even at low efficacy, these estimates are still several orders of magnitude above the $125 million projected investment.

Conclusion: RS-10 is likely to yield high returns for invested resources. The study's chief limitation was the reliance on the world's limited evidence base on how effective the road safety interventions will be. Planned evaluation of RS-10 will enhance planners’ ability to conduct economic assessments of road safety in developing countries.

Keywords:

• Road safety
• Cost of injury
• Value of statistical life
• Economic evaluation
• Trauma

INTRODUCTION

Road traffic injuries impose a serious—and often unrecog- nized—disease burden around the world. The 2004 update of the Global Burden of Disease Project counted 1.30 million road-related deaths and about 47.8 million injuries yearly (World Health Organization [WHO] 2008). This burden is projected to grow; though road deaths are currently ranked as the ninth cause of death around the world, by 2030 they will climb to fifth place (WHO 2009).

In light of rising mortality and research gaps, Bloomberg Philanthropies launched the Road Safety in 10 Countries (RS-10) project. With a $125 million initial disbursement over 5 years (WHO 2011b), RS-10 is the largest international road safety initiative undertaken to date. The initiative, launched in 2009, aims to “support road safety projects in 10 low- and middle-income countries (LMICs) and monitor progress at a global level” (WHO 2011b). The participating countries—Brazil, Cambodia, China, Egypt, India, Kenya, Mexico, the Russian Federation, Turkey, and Vietnam—account for almost half (48%) of global traffic deaths. RS-10 interventions will be implemented between 2011 and 2016 and are grouped in 4 main types: drink-driving prevention, motorcycle helmets, seat belts, and speed management. These interventions have demonstrated cost-effectiveness in high-income countries and were proposed for worldwide rollout by the WHO (2009). Because RS-10 also includes a strong evaluation component, the project is expected both to save lives and provide evidence for stronger road interventions in the developing world.

The overall goal of this study is to estimate how many lives RS-10 can potentially save in selected sites. Additionally, the article identifies the maximum investment RS-10 can incur to save those lives before its costs outweigh the health benefits. In doing so, this article tests the methods to do such estimations within a multi-country project in LMICs. As a result, it comprises a case study that offers potential for informing similar work in other settings. Recognizing that different methods may yield different estimates, this article uses 2 different economic approaches: investment per life-years saved ($/LYS) and value of statistical life (VSL). $/LYS calculations reflect traditional cost-effectiveness estimates. VSL calculations, however, estimate how much society would be willing to pay to achieve the risk reductions promised by the RS-10 interventions. Similar findings from both methods strengthen the conclusions of this evaluation.

METHODS

The current study adopts a societal perspective to evaluate the potential impact and return on investment of RS-10. Our research attempts to answer the following related questions: (1) How many lives is RS-10 expected to save in the selected sites? and (2) What is the monetary value of the health benefits potentially generated by RS-10? Our calculations assume that RS-10 will be implemented over a timeframe of 5 years, which corresponds to the stated duration of the project.

The first step was to derive an estimate of the effectiveness of each RS-10 intervention. We reviewed published evaluations of road safety interventions in developing countries. In the search, conducted with a specialized librarian, we consulted Pubmed, Scopus, Web of Science, Global Health Library (WHO Library), and Scirus. Key terms were included for each of the following categories: study type, intervention type, and developing countries. In addition, in order to be included, studies must have been published between January 3, 2003, and January 3, 2011. Main reasons for a study's exclusion were not including the main outcome of interest (i.e., deaths averted), absence of a clear intervention, and evaluation of an intervention outside of RS-10's scope of work (i.e., engineering interventions such as road redesign were excluded). As explained in the Appendix, each effectiveness estimate was calculated as the simple arithmetic average of estimates from the literature's corresponding studies. These interventions will be implemented in the future; there is uncertainty about their potential effectiveness. As suggested in the literature (Schackman et al. 2004), we account for this uncertainty by allowing each effectiveness estimate to vary by ±20 percent in our sensitivity calculations. The ±20 percent range facilitates readers efforts to extrapolate away from the baseline levels of intervention effectiveness. This range does not express statistical uncertainty about the ultimate effectiveness of the interventions in each country. Given so few empirical estimates and the number of factors that could alter implementation, it is impossible to assert a meaningful range for effectiveness.

The types of interventions in RS-10 were grouped into 4 main categories: speed management, drink-driving, motorcycle helmets, and seat belts. Within each category, separate estimates were developed for enforcement vs. social marketing interventions (e.g., speed management–enforcement; speed management–social marketing). Cognizant of the fact that some RS-10 countries will implement multifaceted interventions, we also developed estimates for categories containing multicategory enforcement or social marketing.

The effectiveness estimates we reviewed typically reflect relative reduction in risk expressed as (RR = Rate_{Pre-Intervention} /Rate_{Post-Intervention}). With baseline estimates of death rates (M_Baseline) and baseline population (P_Baseline) at each of the RS-10 sites (see companion papers in this special issue), we were able to calculate projected deaths, pre-intervention, as M_Baseline × P_Baseline. The number of lives saved by the project (LS) results from subtracting deaths postintervention from deaths pre-intervention as follows: Lives Saved = (1 − RR) × M_Baseline × P_Baseline. Most RS-10 countries are implementing 2 interventions; the total number of lives saved was computed as the simple sum of the lives saved by each intervention. Given that these are community-wide interventions, the chance that we double count an averted death of a helmeted motorcyclist who later is saved by wearing a seat belt, etc., is negligible but not zero. Following convention, we discounted the lives saved over the 5 years of the timeframe at 3 percent. At the end of the 5-year time horizon, we stopped calculating benefits from RS-10 due to uncertainty about the extent to which the interventions will endure and be sustained after 2015 in the absence of renewed investments in enforcement and behavioral change promotion.

In order to compute LYS at the country level, we multiply the estimate of deaths averted times LYS per death averted, which is the discounted value of the average length of life remaining for a crash victim in each country. The length of remaining life was based on life table data from each country (WHO 2011a) and RS-10 baseline data on the average age of road deaths and the life expectancy at such age. Following convention (Sachs 2002), LYS were also discounted at 3 percent. Further, we assume that RS-10 is a cost-effective endeavor as long as its costs do not exceed 3 times the country's gross national income (GNI). This same threshold was proposed in international guidelines for calculations using similar measures, such as disability-adjusted life years (DALYs; Sachs 2002).

VSL is sometimes misunderstood as the dollar value of “somebody's” life. It is actually a way to express the monetary value of a small reduction in the risk of dying for a population (Viscusi 2008). If someone is willing to pay no more than $1 to avoid a 1/million risk of dying, we can say that a population of a million of exactly similar individuals would be willing to pay $1 million to save one statistical life. There is no claim that any single person would trade his or her life for a million dollars—the VSL simply summarizes the population's average willingness to pay money for the reduction of health risks. The RS-10 interventions offer the populations being served a reduction in health risks. If we knew how much people in those countries routinely paid out of their own resources to reduce the health risks they face in their daily lives, we could estimate what they would pay to receive benefits of the same magnitude as RS-10. This would provide a monetary value for the lives saved by the RS-10 project.

Our VSL calculations use the approach published by Ozawa et al. (2011) and Laxminarayan et al. (2009), Ozawa et al. (2011) calculated the value of a statistical life through the following benefits transfer formula: VSL _j = VSL_USA × (GNI _j /GNI_USA) × ξ where subscript j denotes country j and ξ is an elasticity term estimated from the literature. Relying on previous road safety literature, we estimate the VSL of road injuries in a developed country like the United States to be $3.58 million (De Blaeij et al. 2003). The country-specific GNIs were obtained from the World Bank (2011). The elasticity measure reflects the percentage change in VSL per percentage change in GNI. Our midpoint estimation uses an elasticity of 1.5, based on a review by Ozawa et al. (2011) that compared the income elasticity of the value of statistical life between the United States with that in low-income countries. We calculated the present value in 2010 of the lives saved by running project from 2010 to 2015 as the product of VSL _j × Total Discounted Deaths Averted _j . In order to test the sensitivity to key assumptions, we have also conducted alternative analyses varying the discount rate between 0 and 6 percent, the elasticity between 1 and 2, and intervention effectiveness at ±20 percent.

RESULTS

Literature Review on Intervention Effectiveness

The literature review was conducted until the point of saturation. Overall, although over 800 abstracts were reviewed, there was significant overlap among databases, and the vast majority of studies included epidemiologic data but lacked effectiveness estimates. In fact, the majority of the epidemiologic studies found were conducted in a few countries (notably China, Iran, and Mexico). The search process yielded 26 effectiveness evaluations of road safety interventions in developing countries. Of these, 10 studies explored the effectiveness of interventions not covered by the RS-10 project; 5 of these assessed the effectiveness of road redesign interventions (Afukaar 2003; Arreola-Rissa et al. 2008; Ayati and Shahidian 2007; WHO 2009; Yang 2010) and another 5 reviewed the effectiveness of emergency medical services to road injury victims (Arreola-Risa et al. 2004, 2007; Husum, Gilbert, and Wisborg 2003; Husum, Gilbert, Wisborg, et al. 2003; Jayaraman et al. 2009). Of the 16 articles that met our inclusion/exclusion criteria, one was excluded due to high participant attrition (45% in Bacchieri et al.'s [2010] study) and another 2 were excluded because the effectiveness estimates included only intermediate outcomes (Hidalgo-Solorzano et al. 2008; Williams et al., 2007). Consequently, 13 studies (Table I) constituted the body of evidence for the effectiveness estimates. Of these, 7 (Bishai et al. 2008; Maffei de Andrade et al. 2008; Poli de Figueiredo et al. 2001; Rahimi-Movaghar 2010; Salvarani et al. 2009; Soori et al. 2009; Stevenson et al. 2008) evaluated the effectiveness of interventions across categories. The other half provided estimates for individual interventions. These studies focused mainly on police enforcement, helmets (Espitia-Hardeman et al. 2008; Ichikawa et al. 2003; Law et al. 2005; Passmore et al. 2010), seat belts (Harris and Olukoga 2005; Williams et al. 2007), drink-driving (Guanche Garcell et al. 2008), and speed (Poli de Figueiredo et al. 2001). Hence, the number of studies used to develop each estimate was limited due to the lack of evaluative studies in LMICs.

Table I Literature review of effectiveness studies for selected road safety interventions

Type of intervention (1st is primary intervention)	Source	Intervention	Main findings	Country
Speeding
Speed-enforcement; drink-driving + speeding enforcement	Poli de Figueiredo, L. F., S. Rasslan, et al. (2001). “Increases in fines and driver licence withdrawal have effectively reduced immediate deaths from trauma on Brazilian roads: first-year report on the new traffic code.” Injury 32(2): 91–4.; Liberatti, C. L, S. M. Andrade, et al. (2001). “The new Brazilian traffic code and some characteristics of victims in southern Brazil.” Inj Prev 7(3): 190–3.	New Traffic Code for Brazil: increased fines, point system and raised speed limits.	Poli paper reduces deaths by 25% reduces admissions by 33% so in 10,000 popGet .512 lives saved and 4.549 YLDs saved	Brazil
Speed-enforcement; drink-driving + speeding enforcement; seatbels enforcement; enforcement-general	Maffei de Andrade S, Soares DA, Matsuo T, Barrancos Liberatti CL, Hiromi Iwakura ML. “Road injury-related mortality in a medium-sized Brazilian city after some preventive interventions”. Traffic Inj Prev. 2008 Oct;9(5):450–5.	Speed control, seatbelts, new traffic code and prehospital attention for road traffic victirms in Londrina (Brazil)	Seatbelts, radar (end of 1995 and 1996), and introduction of prehospital attention (starting in June 1996). Small impact on mortality among victims of road traffic injuries, which continued at + 35 per 100,000 population. In 1999, the year after a new national road traffic code had been implemented, a larger reduction in mortality levels was observed (to 27.2 per 100,000). However, this downward trend was not maintained over subsequent years, with mortality levels continuing at 23 to 29 per 100,000 population.	Brazil
Speed-enforcement; drink-driving + speeding enforcement; enforcement-general	Bishai D, Asiimwe B, Abbas S, Hyder AA, Bazeyo W. “Cost-effectiveness of traffic enforcement: case study from Uganda”. Inj Prev. 2008 Aug;14(4):223–7.	Enforcement of traffic laws (speed through scale up of police resources (20 policemen, radars, and 4 mobile units) in Kampala	17% drop in road deaths after the intervention. $603 per death averted or $27 per life year saved discounted at 3%	Ghana
Drink Driving
Drink enforcement	Guanche Garcell H, Su´rez Enriquez T, Gutiérrez García F, Martínez Quesada C, Peña Sandoval R, S´nchez Villalobos J. “Impact of a drink-driving detection program to prevent traffic accidents (Villa Clara Province, Cuba)”. Gac Sanit. 2008 Jul–Aug;22(4):344–7.	Alcohol testing checkpoints during weekends at Villa Clara province (pop 833,424)	Comparing to previous year (2002), accidents were reduced by 29.9%, deaths by 70.8%, and injuries by 58.7%.	Cuba
Helmets
Helments-enforcement	Espitia-Hardeman V, Vélez L, Muñoz E, Gutiérrez-Martínez Ml, Espinosa-Vallín R, Concha-Eastman A. “Impact of interventions directed toward motorcyclist death prevention in Cali, Colombia: 1993–2001”. Salud Publica Mex. 2008;50 SuppM:S69–77.	Series of laws that made mandatory a) helmet in motorcycle drivers; b) helmet in passengers; c) forbade motorcycle traffic during Christmas holidays; and d) use of reflecting vests. The study also examined the impact of a reduction in police personnel.	Between 1993 and 2001, 52% reduction in mortality rates amongst motorcycle drivers from 9.7 to 3.6. According to the ARIMA (adj) model, helmet in drivers reduced rate by 3.8, and helmet in passengers by 3.3 (mortality rates seem to be per 100,000 inhabitants).	Colombia
Helments-enforcement	Ichikawa M, Chadbunchachai W, Marui E. “Effect of the helmet act for motorcyclists in Thailand”. Accid Anal Prev. 2003 Mar;35(2):183–9.	National helmet law of Thailand	41% reduction in head injuries and 21% reduction in deaths	Thailand
Speeding
Helments-enforcement	Passmore J, Tu NT, Luong MA, Chinh ND, Nam NP. “Impact of mandatory motorcycle helmet wearing legislation on head injuries in Viet Nam: results of a preliminary analysis”. Traffic Inj Prev. 2010Apr;11(2):202–6. PubMed PMID: 20373241	National helmet law, requiring all motorcycle riders and passengers to wear helmets at all times.	16% reduction in road traffic injuries and 18% reduction in road traffic deaths (both significant at p<0.05).	Vietnam
Seatbelts
Seatbelts-Enforcement	Harris GT, Olukoga IA. “A cost benefit analysis of an enhanced seat belt enforcement program in South Africa”. Inj Prev. 2005 Apr;11(2):102–5	Seatbelt law	Increase of seatbelt usage by 16%; reduction of injuries by 9.5%. 13.6 million rand in societal savings. However, 9.5% reduction estimate results assumes increases seatbelt usage and the impact of such usage similar to those from meta-analyses from US.	South Africa
Seatbelts-Enforcement; seatbelts-social marketing; enforcement-general	Soori H, Royanian M, Zali AR, Movahedinejad A. “Road traffic injuries in Iran: the role of interventions implemented by traffic police”. Traffic Inj Prev. 2009. Aug;10(4):375–8.	The four interventions simultaneously put into place in 2005 were (1) enforcement of laws on the mandatory fastening of seat belts, (2) enforcement of the laws on use of motorcycle helmets, (3) enforcement of general traffic laws, and (4) mass media educational campaigns on national radio and television.	Significant decrease in RTI-related death and morbidity rates in Iran, after intervention (P < 0.001). Death rate decreased from 38.2 per 100,000 in 2004 to 31.8 in 2007 (OR = 0.83, 95% Cl = 0.82–0.85). The death rate per 10,000 vehicles also showed a significant decline from 24.2 to 13.4 (OR = 0.56, 95% Cl = 0.55–0.57). Similar reductions were seen among nonfatal RT	Iran
Seatbelts-Enforcement; seatbelts-social marketing; enforcement-general	Stevenson M, Yu J, Hendrie D, Li LP, Ivers R, Zhou Y, Su S, Norton R. “Reducing the burden of road traffic injury: translating high-income country interventions to middle-income and low-income countries”. Inj Prev. 2008 Oct;14(5):284–9.	Enhanced police training and enforcement, social marketing, and health educaiton. Intervention rolled out Sept 2005–August 2006.	12% increase (from 50% to 62%) in intervention city, and considerable narrowing of confidence intervals. Based on demonstrated effectiveness of seatbelt usage on probability of death (=0.45), the authors calculate 7% death reduction based on simple formula. Authors estimate also $418 per DALY saved.	China
Seatbelts-social marketing; speed-social marketing; social marketing-general	Salvarani CP, Colli BO, Carlotti Junior CG. “Impact of a program for the prevention of traffic accidents in a Southern Brazilian city: a model for implementation in a developing country”. Surg Neurol. 2009 Jul;72(1):6–13	Brazilian adaptation of “Think First”, an educational program targeted at overall popuolation (particular focus on adolescents and young adults). Key themes are alcohol, helmets, speed and seatbelt. The intervention lasted for 1 year and was implemented at a variety of places.	During implementation year, total nr. of accidents increased by 1.6%, but the number of fatal injuries was reduced by 23.6% .	Brazil
Social marketing
Social marketing-general	Rahimi-Movaghar V. “Controlled evaluation of injury in an international Safe Community: Kashmar, Iran”. Public Health. 2010 Apr;124(4):190–7.	Intervention is Safe Community model from WHO. Therefore, road-safety intervention seems a bit unclear.	No effect was found of Safe Community intervention on road safety .	Iran
Helmets-social marketing	Law TH, Umar RS, Zulkaurnain S, Kulanthayan S. “Impact of the effect of economic crisis and the targeted motorcycle safety programme on motorcycle-related accidents, injuries and fatalities in Malaysia”. Int J Inj Contr Saf Promot. 2005 Mar;12(1):9–21.	Intervention aimed at modifying use of motorcycle user behavior in: helmet use, lack road conspicuity, and speed	27% reduction in motorcyle-related deaths	Malaysia

Estimate of Lives Saved

The Appendix explains the methodology used to develop the effectiveness estimates of the RS-10 interventions. As explained, we developed one estimate for each type of intervention. These estimates are expressed in terms of one minus relative risk for mortality. Our calculations (available by request) show that RS-10 interventions are expected to reduce road safety deaths by 23 percent on average, varying between 11 percent (seat belt enforcement) and 30 percent (drinking enforcement). However, these estimates must be taken with caution, because the number of studies used to develop each was quite limited. For example, some estimates are based on 4 studies (i.e., seat belts enforcement and police enforcement), whereas others—such as drink-driving—are based on just one study. Therefore, the results from this projection cannot be taken as definitive but rather are directional.

The global effectiveness estimates were then used to project the number of lives that RS-10 can save by country over the next 5 years (Table II). Assuming a 3 percent discount rate, RS-10 could save about 10,310 lives. The table shows wide variations in the size of the populations at risk and those receiving the intervention. For example, the average population size receiving the intervention is 1.89 million, ranging from 380,211 (Cambodia) to 19.5 million (Egypt). The data also reflect wide variation of baseline death risks across sites. Across all intervention sites, the risk of dying on the road was 19.9 per 100,000 population (mean = 19). This figure resembles national averages of middle-income countries such as Brazil (18.3) and Thailand (19.6; WHO 2004). However, the average figure masks a wide variety of baseline risks, ranging from 6.6 (Kenya) to 29.7 (Mexico). In addition, sites are implementing different types of interventions, ranging from specific (speed, seat belts, etc.) to multicategory interventions. Hence, cross-country comparisons must be made with caution.

Table II Estimate of lives saved and their value for RS-10 sites

Country	Population receiving intervention (persons)	Risk of injury death (per 100,000)	Lives saved over 5 years, discounted	Value of these life years @ 3 × GNI /LYS($)	Value of these saved lives using VSL ($)
Brazil	7,510,000	24.4	1857	1,378,615,088	697,378,540
Cambodia	6,932,000	12.1	980	119,038,820	27,908,701
China	14,606,000	15.4	1975	926,689,883	414,234,667
Egypt	19,482,000	14.7	2411	1,018,868,595	378,369,689
India	3,618,000	25.8	1568	308,942,625	89,304,298
Kenya	1,377,000	6.6	135	9,033,373	3,075,128
Mexico	4,400,000	29.7	995	1,037,213,269	605,578,089
Russia	1,275,000	22.8	181	209,205,483	164,841,888
Turkey	1,352,000	13.3	120	118,760,697	69,000,141
Vietnam	380,000	19.0	88	17,582,567	4,749,056
Total	60,934,000	19.9	10,310	5,143,950,399	2,454,440,197

The last two columns in Table II indicate the value of the estimated health gains produced by RS-10. These are obtained through 2 different approaches: the 3 × GNI per LYS criterion and the VSL estimation. The table shows that, not surprisingly, both methods yield different estimates. According to the maximum investment calculations, for it to be true that the costs exceed benefits of saving 10,310 lives (in terms of 3 × GNI/LYS), RS-10 would need to cost over $5.2 billion. The VSL method, on the other hand, estimates that society values these lives at $2.4 billion. Hence, RS-10 would need to cost over $2.4 billion for it not to be a worthwhile enterprise. Both methods convey that if RS-10 rises to its potential to save 10,310 lives, then it will be worth more the planned investment of $125 million by a factor of 10.

Finally, we conducted sensitivity analysis for both methods. The analysis shows that maximum investment and VSL calculations are vulnerable to different assumptions. Maximum investment calculations are more influenced by discount rates (range = $2.3–$10.7 billion) than by intervention efficacy assumptions (range = $4.1–$6.2 billion). Figure 1 shows that VSL variation due to efficacy assumptions (range = $2.0–$2.9 billion) is comparable to that from the maximum investment calculations. However, the VSL approach is much less vulnerable to varying discount rates (range = $2.0–$3.0 billion). In addition, VSL calculations are most vulnerable to elasticity assumptions (range = $1.1–$5.6 billion)—a construct that only applies to this approach and is therefore not comparable to maximum investment calculations.

DISCUSSION

If the RS-10 interventions have similar effectiveness to the results obtained in past studies, our analysis predicts that they could save 10,310 lives over 5 years. Our VSL calculations reveal that if 10,310 lives are actually saved as a consequence of RS-10, its implementation costs would have to be 19.6 times higher than RS-10 investment cost before they exceed the value of the statistical lives saved ($2.4 billion). The maximum investment calculations (3 × GNI/YLS) yield even more striking results. RS-10 costs would need exceed 41.6 times the investment cost before they surpass the conventional 3 × GNI/YLS cost-effectiveness threshold. In addition, sensitivity analysis shows that varying assumptions will have a significant impact on our estimates but not on our conclusions: the lowest effectiveness estimand (VSL, 2% elasticity) found the VSL of the lives saved to be $1.1 billion. This is still almost 9 times the grant from the Bloomberg Philanthropies.

The findings from this study cannot be compared to similar accounts, because this is the first study of its nature. Though previous reviews (Ameratunga et al. 2006; Quistberg et al. 2010; Schmucker et al. 2010; Sharma 2008) point to the general lack of attention to injury research in the developing world, they did not attempt to develop effectiveness estimates using these methods.

The main limitation of this study is the weak effectiveness evidence available from LMICs. The articles we reviewed for our effectiveness estimates exhibit weaknesses in 4 main areas. These are (1) not focusing on a specific intervention, (2) not having specific baseline data, (3) not having a control group (i.e., counterfactual), and (4) not adjusting for potential confounders. For example, one study assessed interventions that had not been implemented yet (Harris and Olukoga 2005), and another assessed an intervention that was not clearly definable (Rahimi-Movaghar 2010). Though all studies compared pre-study with post-study outcomes, only 2 studies included a control group for comparison (Rahimi-Movaghar 2010; Stevenson et al. 2008), and only one study (Law et al. 2005) controlled for potential confounders. Of the studies that used a time time-series approach, only 3 (Bishai et al. 2008; Espitia-Hardeman et al. 2008; Law et al. 2005) corrected for the autocorrelation of the outcome over time. Finally, these studies used different data sources, including self-reports, official public sector reports, and secondary data (Harris and Olukoga 2005).

In addition, the methods we used in our estimation may face some potential challenges. RS-10 will implement more than one intervention in some sites; we have assumed first that there is no overlap of interventions. Therefore, there is a chance that our calculations may double count a limited number of lives that were saved twice through 2 separate interventions in the same place. It is also possible that concurrent interventions will have multiplicative or additive effects. However, no literature exists in this regard. Third, we have also assumed that all interventions are fully implemented and that interventions are equally effective across sites and countries. Finally, we estimate that the effectiveness estimate for each intervention is an arithmetic average of the estimates across relevant studies. Given that the studies found varied considerably in the populations and methods used, this may not be the case. In sum, the direction of such biases is unknown, and we have tried to help readers extrapolate away from the baseline estimates through sensitivity analysis,

The tornado diagram (Figure 1) shows that even when intervention efficacy was 20 percent lower, the RS-10 program generated health benefits worth over $2 billion. Intervention efficacy in RS-10 countries would need to be less than one tenth the reported efficacy to threaten the conclusion that benefits exceed the projected $100 million cost of the interventions. Therefore, RS-10 is a highly worthwhile enterprise, because the lives saved and their value lie well beyond the projected investment.

Figure 1 Tornado diagram for road safety estimates, VSL calculations (color figure available online).

An additional assumption concerns the cost-effectiveness threshold ($/LYS), which we fixed at 3 times the GNI. As discussed, this arbitrary benchmark is similar to international thresholds for other measures. However, because our analysis does not adjust for prevented disability, using the same threshold may make us underestimate the benefit of RS-10. On the other hand, the disability care and economic losses (cost of treatment, property damage, and reduced production due to traffic delay) induced by RS-10 are not part of our cost calculations. It is unclear whether over- or underestimation prevails; both effects may cancel out.

In sum, in spite of the methodological challenges, it is striking that our 2 different approaches—investment per LYS and VSL—conclude that RS-10 benefits must be 20-fold less than those projected for its costs to outweigh the benefits. Given such a high margin, it is very likely that RS-10 will be worth the money invested. This finding is strengthened by the fact that the 2 methods used (LYS and VSL) reflect conceptually different aspects of the benefits of injury prevention. Though LYS calculations reflect the maximum cost to save 10,310 lives while staying cost-effective, VSL estimates how much society would be willing to invest to save those lives. Projections like the ones used in this study are important to predict the impact of interventions as well as define the worthiness of investments—both important for road safety in LMICs. The lack of effectiveness and costing data from LMICs ought to be of concern to both researchers and decision makers; the evidence base needed for the wise use of resources is missing from the countries that need it most. The evidence generated by research activities like those planned in conjunction with RS-10 should be a routine part of road safety initiatives to inform future policies and save more lives in the developing world.

APPENDIX: CALCULATIONS OF THE GLOBAL EFFECTIVENESS ESTIMATES

This evaluation has calculated effectiveness estimates pertinent to the RS-10 interventions. First, relevant studies were grouped by intervention type (speeding, drinking and driving, helmets, seat belts). Two additional groups were created for cross-cutting interventions, which focused on overall police enforcement and social marketing. These emphasized 2 or more RS-10 interventions at once. Hence, there were 6 main groups of interventions (speed, drinking and driving, helmets, seat belts, speeding, police enforcement, and social marketing). The first 4 groups were divided into 2 main subgroups: enforcement versus social marketing. Overall effectiveness estimates were calculated by averaging studies in each relevant subgroup. To account for uncertainty, intervention effectiveness was allowed to oscillate ±20 percent from the calculated mean.

ACKNOWLEDGMENTS

This work was conducted as part of the Road Safety in 10 Countries project funded by the Bloomberg Philanthropies.