Reliability and validity of cross-national homicide data: a comparison of UN and WHO data

ABSTRACT

Data reliability and validity are methodological concerns in cross-national analyses of crime, but there is little agreement on which source of data provides the most reliable estimates. Moreover, few studies have examined the potential threat to validity posed by unclassified deaths. The current study aims to (1) assess the reliability of cross-national homicide data from the United Nations (UN) and the World Health Organization (WHO); and (2) investigate the impact of unclassified deaths on the validity of WHO data. Findings indicate that UN and WHO homicide rates (n=56) differ in magnitude, but produce similar outcomes. The UN data produce more robust results and statistical models with less error. The WHO data are more stable and reliable over time, and better suited for longitudinal analyses. Analyses drawing on WHO data should not disregard unclassified deaths because their inclusion provides a more accurate estimate of the true number of homicides.

KEYWORDS:

• Cross-national
• homicide
• unclassified deaths
• reliability
• validity
• World Health Organization
• United Nations

Notes

1. Countries in UN and WHO samples (n = 56): Argentina, Armenia, Australia, Austria, Belarus, Bermuda, Bulgaria, Canada, Colombia, Costa Rica, Croatia, Czech Republic, Denmark, Dominican Republic, El Salvador, Estonia, Finland, France, Georgia, Germany, Greece, Guatemala, Guyana, Hungary, Ireland, Israel, Italy, Japan, Kazakhstan, Kyrgyzstan, Latvia, Lithuania, Mauritius, Mexico, Netherlands, New Zealand, Nicaragua, Norway, Panama, Poland, Portugal, Puerto Rico, Republic of Moldova, Romania, Saint Kitts and Nevis, Saint Vincent and Grenadines, Singapore, Slovakia, Slovenia, South Africa, Spain, Sweden, Switzerland, Ukraine, United Kingdom, and Venezuela (Bolivarian Republic of).

2. HDI is scored between 0 and 1. Because of issues of multicollinearity with another key variable (i.e., poverty; VIF > 10), the HDI was recoded into a dummy variable, using cut-offs determined by the UN (Human Development Report, 2014). A value of 0 corresponded to low/medium-level HDI (<.799), and a value of 1 denoted high-level HDI (>.800).

3. An explanatory factor analysis revealed that all the WGI items (i.e., voice and accountability, political stability and absence of violence, government effectiveness, regulatory quality, rule of law, and control of corruption) loaded onto a single factor and reflected the same dimension. A reliability test on the quality of governance factor provided an alpha of .961, indicating high reliability.

4. The Breush–Pagan/Cook–Weisburg test revealed that two of the three models (UN and WHO) failed to meet the OLS assumption.

Additional information

Notes on contributors

Catrin Andersson

Catrin Andersson graduated with a Ph.D. in Criminal Justice from the Graduate Center/John Jay College of Criminal Justice in 2015. She is currently a lecturer in Criminology at Sheffield Hallam University in the UK. Her research interests include violence at the macro and micro levels, substance use and recovery, comparative research and research methods.

Lila Kazemian

Lila Kazemian is an Associate Professor at John Jay College of Criminal Justice, City University of New York. She earned her Ph.D. from the Institute of Criminology, University of Cambridge, in 2006. Her research work has focused on changes in criminal career patterns across various periods of the life course, the process of desistance from crime, prisoner reentry, and comparative criminology. Her work has been published in the Journal of Quantitative Criminology, the Journal of Research in Crime and Delinquency, Punishment & Society, and the European Journal of Criminology.