![Sample our Law journals, sign in here to start your FREE access for 14 days]({"type":"image" , "src" : "/sda/5942/TOC-Subject-Sample-2021-Law.jpg"})
Abstract
The errors associated with measuring the number of militia and patriot groups may cast doubt on conclusions drawn from prior studies of the spatial variation of these movements. Most studies of militias have been qualitative investigations of a single group, state, or region. A growing number of studies, however, have used quantitative techniques to assess the hypothesis that the number of militia groups by state covaries with structural and cultural forces. We outline a number of concerns with the validity of the counts, conducted by the Anti‐Defamation League and the Southern Poverty Law Center, used by these studies. We re‐estimate models from previous studies using the four alternative measures of these groups employed in prior studies. We find that many inferences drawn for identical theoretical models differ based upon the measure used. These discrepancies apply not simply to tangential control variables but to indicators of key theoretical constructs. In other words, the decision as to whether or not a particular theoretical framework receives empirical support often depends upon which measure of the dependent variable is used. This suggests that the inconsistent findings in prior research may be due to measurement error and makes it difficult to assess the validity of the conclusions drawn from these studies. It is important to be aware of these weaknesses since scholars studying political crimes and related phenomena often use information from similar sources, making this specific example relevant to a more general area of research.
Acknowledgments
We thank Donald Haider‐Markel for providing the data necessary to replicate his study and we thank Richard Spano, Valli Rajah, Craig Rivera, and Bert Useem for helpful critiques of earlier drafts. The second author also thanks the Davis Center at Harvard University, where he was a Research Fellow when carrying out this research.
Notes
1. While the ADL (Citation1995) ranked the 50 United States from 0 to 3 in terms of their level of militia‐related activity, it did not define militia‐related activity, describe how the information was collected, or explain how the measure was coded.
2. For instance, the Federal Bureau of Investigation’s Uniform Crime Reports, which are based upon a highly developed counting system with uniform definitions, caution that its counts may not be internally consistent and as a result warns against comparing crime rates across states.
3. Chermak’s (Citation2002, pp. 125–126) analysis of newspaper coverage of the militia movement found that “some of the SPLC’s estimates, published in various newspapers … about the militia threat grew dramatically in a relatively short period of time … after the [Oklahoma City] bombing, the estimate grew to 224 groups in 39 states by June, and 340 groups by September … [and] the number grew to 950 in all 50 states by the end of the year.”
4. For example, the ADL Citation1994 and Citation1995 reports both offered descriptions of militia activity in Arizona and Michigan. However, O’Brien and Haider‐Markel used the 1995 count for Arizona (seven groups) as opposed to information from the 1994 report (which appeared to indicate two groups). They used the 1994 count for Michigan (66 groups), as opposed to information from the 1995 report (which, while unclear, appeared to indicate fewer groups), even though the ADL (Citation1994, p. 14) itself conceded that the count of 66 was likely incorrect.
5. Since the SPLC numbers indicate that the number of groups almost trebled from 1995 to 1996, the decision of which year to examine may represent a theoretical issue (Freilich & Pridemore, Citation2005). The 1995 count may have been lower since the militia movement was still emerging, while by 1996 it had greatly expanded in size. Since factors that influence the initial appearance of a social movement may be different from those that fuel its rapid rise, the choice of year may depend upon whether researchers wish to investigate the emergence or the surge of the paramilitary right.
6. O’Brien and Haider‐Markel’s final model is in fact a reduced model with only four independent variables. To evaluate each of their original hypotheses we used their full model. In their data, Hawaii and Alaska had missing observations on mass polarization and Nebraska on Democratic representation. The discussion of the data in their article does not mention whether and/or how these observations were replaced. Thus we did not replace them and dropped these cases. Further, they provided the number of Gulf War and Vietnam veterans but we did not have the 1991 state population data (the year they used) to create rates. Instead we used population counts from the 1990 Census. Even with these potential differences, we replicated the models from their table 2 nearly exactly. Before using their data for our purposes we undertook exploratory data analysis and found that the distributions of the measures of registered voters, Vietnam veterans, NRA members, stolen explosives, and population density were all positively skewed (the latter three especially so, with skew statistics several times their standard errors). When the natural logarithm of these values was taken, the positive skews were reduced substantially and we therefore used these logged values in our analysis (since several states had a value of zero for pounds of stolen explosives, we added 1 to all values before making the transformation). When the logged values were used to estimate their reduced Model 3 in table 2 using OLS regression, three of the four measures were still significant (explosives was non‐significant).
7. In the original Freilich data, the minority presence and paramilitary culture measures were positively skewed. In each case, however, taking the natural logarithm of the values did not aid in normalizing their distributions. Thus the original values were used in the models presented here.
8. Since we are modeling counts, the population at risk must be used to control for exposure. We used the white male population since militia members are overwhelmingly white males.
9. We also estimated O’Brien and Haider‐Markel’s reduced “final” model with each of the four dependent variables. The outcome (not presented here) was similar (i.e., the inferences drawn from each of these reduced models were sensitive to the dependent variable employed).
10. The Freilich and Pridemore measures of social disorganization and farm job loss were different from the Freilich study.
11. Similarly, the point of our study is also not to attack either the ADL or the SPLC. Both of these organizations have their own reasons for and ways of measuring militia groups that may not include the precise definitions and estimates that scholars require when testing theories that attempt to explain variation.