Police research, officer surveys, and response rates

ABSTRACT

In recent years, policing scholars have increasingly used survey methods to gain insight into officers’ attitudes and behaviours. Yet, surprisingly, methodological research analysing surveys of police officers is rare. We analysed the extent and correlates of response rates in police surveys, providing insights about the survey design features and study characteristics associated with higher rates of officer participation. We examined the response rates to 497 police surveys reported in 390 articles published in 15 journals from 2008 to 2017. Findings included the following: (1) the average response rate was 64%, but there was a great deal of variation, (2) in-person surveys achieved substantially higher response rates, (3) inviting a greater number of officers to participate in surveys was associated with lower response rates, and (4) response rates have declined over time (though primarily among surveys not administered in-person). Given the weight of the evidence suggesting response rates are typically a poor predictor of nonresponse bias, we argue that a low response rate on its own is an insufficient reason to dismiss a study’s merit. Furthermore, we recommend minimally acceptable reporting standards and discuss avenues for future research.

KEYWORDS:

• Policing
• law enforcement
• survey research
• response rates
• unit nonresponse

Acknowledgement

The authors would like to thank John MacDonald, Kyle McLean, and two anonymous reviewers for their helpful comments on an earlier draft.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Justin Nix http://orcid.org/0000-0002-3812-8590

Notes

1 Groves and Peytcheva (2008) reported nonresponse rates. The figures we report are the inverse of the nonresponse rates they reported.

2 Note that some articles involved multiple surveys and reported response rates for each (e.g. Gaub et al. 2016). In these instances, we treated each survey as a separate case in our dataset.

3 Top coding is a common strategy in our field for reducing the influence of outliers (e.g. Anwar and Loughran 2011, Slocum et al. 2013). However, substantively identical results were obtained when models were estimated without top coding and using robust regression.

4 The four journals were Policing & Society, Journal of Criminal Law & Criminology, Journal of Interpersonal Violence, and Youth Violence and Juvenile Justice. The resulting formula used to impute a Sorensen score for British Journal of Criminology was (0 + .09 + .09 + .03)/4 = .05.

5 While we include a measure of each journal’s impact factor in our analyses, we recognise the limitations of such metrics (Baker 2015). Perhaps we need to revise our concept of ‘impact’. Currently, it refers to who cites which paper, and in which journal. This means that impact operationally measures our willingness to cite each other, and is influenced by how often the journal is published, and how often it publishes articles in a particular field, among other factors. Currently, it is not a measure of impact on the world that is being studied.

6 We used robust standard errors in order to account for heteroskedastic error terms (Hayes and Cai 2007).