The Discriminant Validity of Perceptual Incivility Measures

Abstract

Researchers have been unable to determine whether individuals’ estimates of incivilities can be separated from constructs such as crime and victimization. Accordingly, survey respondents’ perceptions of incivilities were compared first to their perceptions of neighborhood crime and then to their personal victimization experiences. Exploratory and confirmatory factor analyses revealed that perceptions of incivilities (of the physical variety) loaded on factors separate from personal victimization. It was not clear, however, whether perceptions of incivilities were independent from perceived crime. Even so, the findings provide some evidence that estimates of incivilities represent a construct separate from estimates of crime, thereby increasing the likelihood that certain perceptual incivility measures enjoy discriminant validity.

Keywords:

• crime
• social incivilities
• physical incivilities
• discriminant validity
• convergent validity

Acknowledgments

The author would like to thank Steve Tibbetts and Alex Piquero, for their helpful thoughts and suggestions.

Notes

1. Specifically, there is no agreed‐upon list of indicators or appropriate number of such indicators. See, for instance, Piquero (1999, p. 805), Taylor (1999, p. 78), and Spelman (2004, p. 73).

2. Taylor (1998, p. 81) made this observation: “In short, discriminant validity for survey‐based items appears acceptable, but not so for assessment‐based items.” In fairness, though, others have argued for assessment‐based measures over survey‐based measures (see, e.g., Brown, Perkins, & Brown, 2004).

3. There is also a wealth of literature concerning empirical support for the thesis. Examples include Brown et al. (2004), Covington and Taylor (1991), Harrell and Gouvis (1994), Lewis and Maxfield (1980), Perkins et al. (1992), Robinson, Lawton, Taylor, and Perkins (2003), Rountree and Land (1996a, 1996b), Sampson and Raudenbush (1999, 2001), Skogan and Lurigio (1992), Taylor (2001), and Worrall (2003, n.d.).

4. For some implementations of the Campbell and Fiske approach, see Bagozzi and Yi (1991), Golding and Seidman (1974), and Marsh (1993).

5. For critical reviews of the multitrait–multimethod matrix approach see Schmitt, Coyle, and Saair (1977) and Schmitt and Stults (1986).

6. The cities were Chicago, IL; Kansas City, MO; Knoxville, TN; Los Angeles, CA; Madison, WI;, New York, NY; San Diego, CA; Savannah, GA; Spokane, WA; Springfield, MA; Tucson, AZ; and Washington, DC.

7. There were some missing data, but mostly because of screening questions. For example, of the 13,918 completed interviews only 4,217 people reported that they were aware of crimes committed in their neighborhood. Only then were they asked whether each of several specific crimes was committed. Taking into account screening questions, the data were‐on average‐over 90 percent complete for the 26 indicators used in the analysis. See Table 1.

8. The indicators of serious crime that were excluded from the analysis were “people openly selling drugs” and “people openly using drugs.”

9. Vandalism victimization included person and property targets.

10. See Taylor (1998), for a description of how he operationalized community structure.

11. The only identifiers in the data set were for the city in which the respondent resided.

12. The author thanks an anonymous reviewer for pointing this out.

13. Factor analyses of incivilities alone revealed that they clearly loaded on two factors. The results from these analyses are available from the author on request.

14. Three eigenvalues exceeded one when two factors were forced for the physical incivilities analysis, two exceeded one when one factor was forced for the physical incivilities analysis, and two exceeded one when one factor was forced for the social incivilities analysis. This illustrates that the “forced” factor structure was not necessarily the same as the “true” factor structure, but there is some debate as to whether one is the proper cutoff. The author thanks a reviewer for pointing this out.

15. Only one eigenvalue exceeded one for any of the exploratory factor analyses in Table 3.

16. Measurement models are the components of structural equation models. The latter specify the relationships between unobserved latent variables. The former specify the relationships between unobserved latent variables and observed indicators.

17. Because the indicators were dichotomous, distributional assumptions of structural equation modeling may have been violated. Accordingly, an asymptotically distribution‐free estimator was used.

18. A reviewer pointed out that a three‐factor model, one including perceived incivilities, perceived crime, and reported victimization, should have also been estimated. The author estimated such a model, but the results are not reported for two reasons (they are, of course, available from the author on request). First, a three‐factor model introduces a new research question, namely, Can people distinguish between perceived crime and reported victimization? Second, the three‐factor model did not fit very well (CMIN/DF = 8.027, AGFI = 0.888, RMSEA = 0.079; Critical N = 171 for the physical incivilities model; CMIN/DF = 10.155, AGFI = 0.851, RMSEA = 0.090, Critical N = 132 for the social incivilities model). This can surely be explained by the poor fit of the two‐factor models shown in Table 4.

19. It is conceivable that a higher‐order factor model could fit the data better, in either of the results reported in Tables 4 and 5 (indeed, a saturated model‐one with as many parameters as data points‐would fit perfectly). In the interest of parsimony, though, such models (with the exception of the three‐factor model described in the previous note) were not estimated as part of this study.

Additional information

Notes on contributors

John L. Worrall

John L. Worrall is an associate professor in the Crime and Justice Studies Program at the University of Texas at Dallas. He received his Ph.D. in political science from Washington State University in 1999. His research interests are crime control policy, prosecution, and legal issues in policing. He is the author of six books, the most recent of which is Crime Control in America: An Assessment of the Evidence (Allyn & Bacon, 2006). His work has also appeared in Evaluation Review, Justice Quarterly, and Social Science Research, among other journals.