Further Evidence on the Discriminant Validity of Perceptual Incivilities Measures

Abstract

Exploratory factor analysis tested the extent to which measures of incivilities and measures of both crime perceptions and victimization had distinct factor loadings in one‐ and two‐factor models. Confirmatory factor analysis tested the fit of one‐ and two‐factor structural equation models. Exploratory and confirmatory factor analysis showed that perceptual incivilities measures and victimization reports tended to load on distinct factors, offering evidence of the discriminant validity of perceptual incivilities measures relative to victimization reports. Exploratory and confirmatory analysis of perceived incivilities measures and measures of perceptions of crime provided equivocal results. In exploratory factor analysis, perceived incivilities measures and measures of crime perceptions did not always load on distinct factors and confirmatory factor analysis models did not meet the specified thresholds for good model fit across all fit criteria.

Keywords:

• incivilities
• discriminant
• validity

Introduction

The potential link between public incivilities and crime was popularized by Wilson and Kelling in their seminal work Broken Windows. Subsequently, the hypothesized link between incivilities and crime has had a major influence on both popular thinking about the causes of crime, and public policy intended to address crime (Kelling, 1997; Skolnick, 1997). Concomitant with this influence, public incivilities have been an important part of theory and research on both ecological explanations of crime and explanations of citizen fear of crime.

Incivilities are typically divided into two primary dimensions: social and physical. Physical incivilities often include such things as broken windows and vacant lots, while social incivilities include loitering, loud parties, drug sales and prostitution. Within contemporary theories highlighting the importance of incivilities, social and physical incivilities are unique characteristics of neighborhoods causally linked to a number of important outcomes including crime (Taylor, 2001; Wilson & Kelling, 1982), fear of crime (Hunter, 1978; Lewis & Salem, 1986), and neighborhood decline (Skogan, 1990).

Recently, the research incorporating measures of incivilities have been called into question. Taylor (1999) notes that the discriminant validity of measures of public incivilities has not been established in the literature. Despite this important challenge only two studies have directly addressed the discriminant validity of incivilities measures. These include Taylor’s (1999) own work and a recent test by Worrall (2006). While each of these studies represent an important contribution, the discriminant validity of incivilities measures remains at issue. Here we contribute to this limited body of research by offering an additional test of the discriminant validity of physical and social incivilities measures.

The Relationship between Incivilities, Crime, and Fear of Crime

Public incivilities or disorders have been linked to a number of outcomes including fear of crime, crime and neighborhood decline. Early work emphasizing incivilities argued that citizen fear of crime was partially determined by incivilities (Wilson, 1975) or a sense of urban unease (Garafalo & Laub, 1978). Hunter (1978) theorized that signs of incivilities and crime have a reciprocal relationship with each in turn influencing fear of crime. The interactive influence of incivilities and crime and fear of crime was further emphasized by Lewis and Salem (1986) who argued that citizen fear of crime will be particularly pronounced when both crime and incivilities are high, and low when either fear of crime or crime itself is low.

Other theoretical models ascribing an important causal influence to incivilities have focused on the explanation of crime. In their Broken Windows theory, Wilson and Kelling (1982) argue that public incivilities have an important influence on crime through informal social control. Incivilities are thought to lead to citizen withdrawal from public spaces, reducing informal social control and emboldening offenders. This in turn leads to fear of crime, additional withdrawal and an increase in serious offenses. Extending the scope of outcomes influenced by incivilities, Skogan (1990) incorporates incivilities in an explanation of urban decline. In this model, incivilities influence urban decline though three primary mediating processes: (1) informal social control, (2) community morale, contributed to by concerns about neighborhood safety or fear of crime, and (3) instability in the housing market.

Not all scholars agree that incivilities cause crime. In work largely devoted to developing and argument against order maintenance policing, Harcourt (2001) summarily rejects the link between incivilities and crime specified by Wilson and Kelling. In a more moderate dissenting voice, Sampson and Raudenbush (1999) acknowledge incivilities have important implications for neighborhood development, while arguing both incivilities and crime are a result of the same explanatory process. In this process, community structural characteristics, including concentrated disadvantage and residential instability, impact neighborhood collective efficacy, a mix of informal social control and social cohesion. This general process then influences both incivilities and crime.

The Validity of Incivilities Measures

Support for a link between incivilities and crime and the link between incivilities and fear of crime can be found in a growing number of studies (reviews and recent studies include Brown, Perkins, & Brown, 2004; Covington & Taylor, 1991; Harrell & Gouvis, 1994; Perkins, Meeks, & Taylor, 1992; Robinson, Lawton, Taylor, & Perkins, 2003; Rountree & Land, 1996a, 1996b; Sampson & Raudenbush, 1999, 2004; Skogan, 1990; Taylor, 2001). However, there is some question as to the validity of incivility measures used in studies testing the link between incivilities and crime.

Taylor (1999) argues that theories specifying a link between incivilities and crime “propose that incivilities represent a construct separate from other related features of the individual, street block and neighborhood. But researchers have not yet examined the discriminant validity of incivilities indicators” (p. 65). Discriminant validity, popularized through Campbell and Fiske’s (1959) multitrait‐multimethod matrix technique for assessing construct validity, holds that measures of multiple constructs taken with a single measurement method should be empirically distinguishable from each other.

Its not hard to imagine why it might be difficult to distinguish measures of incivilities from measures of fear of crime or measures of crime itself. As it is typically defined in the literature, incivilities have much in common with crime. Some of the indicators of incivilities such as drug sales and prostitution, typically incorporated in incivilities measures, are less serious forms of crime. In theoretical models it is anticipated that these less serious forms of crime, which are conceptualized as an element of a larger incivilities construct, will lead to serious crime. While conceptually distinct, it remains to be seen of indicators of incivilities, including forms of less serious crime can be empirically distinguished from measures of serious crime. It may be that perceptions of incivilities and perception of crime are both driven by a more global impression of one’s neighborhood.

Concerns regarding discriminant validity of incivilities measures are particularly relevant when measures of crime and incivilities are measured with a single survey. When addressing research on the relationship between incivilities and fear of crime, Taylor (1999) notes that the strong association between fear and incivilities may be in part attributable to shared method variance. Shared method variance occurs when two constructs are measured with the same method, and this shared measurement method increases the strength of relationship between these constructs (Campbell & Fiske, 1959). Taylor’s (1999) concerns regarding the influence of shared survey method variance on the relationship between incivilities and fear can be extended to the relationship between incivilities and crime, further highlighting the need to establish the discriminant validity of incivilities measures.

A handful of studies have addressed the discriminant validity of perceptual incivilities measures. Perceptual incivilities measures typically refer to incivilities measures based on citizen perceptions of incivilities through a survey. Studies providing results relevant for a discussion of the validity of perceptual incivilities measures include work testing the association between incivilities and other non‐crime neighborhood characteristics and work with a direct focus on testing the discriminant validity of perceptual incivilities measures by assessing the extent to which incivilities measures can be empirically distinguished from perceptual crime measures.

Work exploring the relationship between incivilities measures and other non‐crime neighborhood characteristics provides mixed evidence regarding the discriminant validity of perceptual incivilities measures. Taylor (1995) found that the racial composition of a neighborhood had a heavy influence on residents’ perceptions of graffiti prevalence. Other work by Taylor, Koons, Kurtz, Greene, and Perkins (1995) has found perceptions of incivilities loaded on separate factors than land‐use features, but were not distinct from measures of territorial signage and defensible space.

Sampson and Raudenbush (2004) explored the neighborhood characteristics that influence resident perceptions of incivilities using data describing approximately 500 block groups from 196 census tracts in the city of Chicago. Among potential predictors of perceived incivilities, the authors included measures of socio‐demographic characteristics and an observation measures of incivilities (for a full description of this measure see Sampson & Raudenbush, 1999). Results showed that the observational measure of incivilities predicted perceived incivilities, but “racial and economic context matter more” (p. 319). Collectively the work of Sampson and Raudenbush and that of Taylor show that it is difficult to discriminate between citizen perceptions of incivilities and other neighborhood characteristics including indicators of race, economic context, territorial signage and defensible space.

Recently two studies have offered direct assessments of the discriminant validity of incivilities relative to crime. These studies are particularly important as they test the validity of measures of the incivilities construct. The incivilities construct has played a central role in causal models that have served as the theoretical underpinning of order maintenance policing. Should the results of these studies show that measures of incivilities are indistinguishable from measures of crime it would undermine weight that may be given the causal models that have been used to support order maintenance policing.

As part of a broad assessment of theory measurement and policy regarding incivilities, Taylor (1999) assessed the relationship between perceived incivilities and official measures of crime using neighborhood level data from five cities. Indicators of incivilities included vandalism, rowdy teens and abandoned buildings. Crime measures included rates of robbery and assault. Exploratory factor analysis found incivilities measures and official crime rate measures loaded on separate factors. These results suggest that at the neighborhood level incivilities measures may be discriminable from official crime rate measures.

Worrall (2006) offers the sole test of the discriminant validity of measures of perceived incivilities relative to measures of perceived crime and reports of victimization. For this test, analyses were based on survey data from approximately 14,000 respondents across 12 cities where police departments were practicing community policing. ¹ Exploratory and confirmatory factor analyses provided mixed results. Survey respondents were able to distinguish their perceptions of physical incivilities from their reports of victimization; however, they were unable to distinguish their perceptions of social incivilities from their reports of victimization.

Results for the discriminate validity of incivilities measures relative to respondents’ perceptions of crime did not support the discriminant validity of incivilities measures. Confirmatory factor analysis found that crime measurers and incivilities measures did not always load on distinct factors. Exploratory factor analysis based on structural equation models yielded model fit statistics that indicated that both factor models provided a poor fit to the data. Worrall (2006) concludes “Thus additional research is called for before it can be known for sure whether people can separate perceptions of crime from perceptions of incivilities” (p. 379).

Method

Despite its importance for ecological theories of crime and for police policy, relatively little attention has been paid to the discriminant validity of measures of incivilities used in tests of neighborhood characteristics associated with crime. In the current work we extended the limited literature on the discriminant validity of perceptual incivilities measures by building on the work of Worrall (2006). Our analysis was based on data from a major Southwestern city. Importantly these data have a key methodological feature distinguishing them from the data used by Worrall (2006). The data used herein include indicators of perceived incivilities and perceived crime gathered with ordinal response categories, while indicators of incivilities, and crime used in Worrall’s (2006) study were dichotomous. We anticipate that this difference might lead to different results. Ordinal response categories preserve information lost in dichotomous response categories. This additional variation may make it possible to discriminate between responses to two closely linked but conceptually distinct constructs such as incivilities and crime.

Sample

Analyses are based on data from Mesa, Arizona. Mesa has a relatively low level of unemployment (2.2%) and high median household income ($33,676), although a number of its residents still live below the poverty line (10%). Mesa experienced substantial growth between 1990 and 2000. In 2000, about 405,000 individuals resided in the city compared with 288,000 in 1990. The ethnic composition of the community is diverse, with Caucasians making up approximately 78.9% of the community; Hispanics, 15.5%; African Americans, 2.3%; Asians, 1.6%; and American Indians, 1.2%. Mesa’s crime rate is somewhat higher than that of some Southwestern cities (e.g., Las Vegas, Los Angeles, San Diego), but lower than that of others (e.g., Albuquerque, Phoenix).

Data were gathered with a survey of citizens residing in Mesa, Arizona. The survey sample consisted of 800 randomly selected respondents. ² Survey data were collected using a telephone survey conducted in September and October 2002. Households included in the sample were drawn at random, with any member over the age of 17 eligible to complete the interview. The survey took approximately 20 minutes to administer. It included questions about the respondents’ perceptions of physical and social incivilities, perceptions of crime, and reports of victimization. Sample characteristics are presented in Table 1.

Table 1 Sample characteristics

Variable	Characteristic	Percentage
Ethnicity	White	87.7
	Non‐white	12.3
Gender	Male	42.5
	Female	57.5
Home ownership	No	21.9
	Yes	78.1
Education	Below high school	5.7
	High school	25.1
	Some college	30.9
	College degree	23.3
	Some postgraduate	4.9
	Masters degree	9.0
	Doctorate	1.1
Income	Less than $5,000	1.8
	$5,000 to < $10,000	2.5
	$10,000 to < $15,000	4.5
	$15,000 to < $20,000	4.7
	$20,000 to < $25,000	4.9
	$25,000 to < $30,000	7.6
	$30,000 to < $35,000	8.2
	$35,000 to < $40,000	10.6
	$40,000 to < $50,000	18.7
	$50,000 to < $60,000	11.9
	$60,000 or more	24.5
	Mean	Standard deviation
Age	48.99	18.25
Tenure	10.93	11.40

Variables

Data included indicators of respondents’ perceptions of incivilities and crime as well as respondent reports of victimization.

Perceived incivilities

Respondents were asked to indicate the extent to which 14 indicators of incivilities were prevalent in their neighborhood. Responses were: “not a problem,” “somewhat of a problem,” and “big problem.” Incivilities measures included seven indicators of physical incivilities and six indicators of social incivilities. Indicators of physical incivilities included broken windows, unkempt property, vandalism/graffiti, vacant lots, vacant houses/buildings, abandoned cars and trash. Indicators of social incivilities included groups of loitering teenagers, drunks or tramps, harassment, people fighting or arguing, noisy neighbors, and drug dealing.

Perceived crime

Respondents’ perceptions of crime in their neighborhood were based on responses to questions regarding the prevalence of four crimes: burglary, car theft, robbery and assault. Response categories were: “not a problem,” “somewhat of a problem,” “a big problem.” For each crime type respondents were asked two questions. The first addressed respondents’ perceptions regarding non‐gang related crime, the second addressed respondent perceptions regarding gang related crime. For example, the first question addressing respondents’ perceptions of robbery in their neighborhood asked “in your neighborhood is non‐gang related robbery, not a problem, somewhat of a problem, or a big problem.” The second question asked how much of a problem gang related robbery was. To create a single indicator of respondent perceptions of each crime type, responses to both questions addressing a single type of crime were summed and then returned to their original metric.

Victimization

Victimization measures included general indicators of direct personal victimization and indirect vicarious victimization. Personal victimization was measured with a question asking the respondent to indicate whether or not in the past 12 months they have been “verbally harassed, physically intimidated, assaulted, robbed or had their property vandalized or stolen.” Indirect victimization was measured by asking respondents to indicate whether or not someone in their household or a neighbor had been “verbally harassed, physically intimidated, assaulted, robbed or had their property vandalized or stolen.”

Analysis

Following Worrall (2006), we conducted a two‐stage test of the discriminant validity of measures of perceived incivilities. The first stage was an exploratory factor analysis based on traditional factor analytic techniques. The second stage was a confirmatory factor analysis based on structural equation measurement models. This approach is widely used in fields with an emphasis on the demonstration of the discriminant validity of key constructs (for examples and key studies, see Anderson, 1987; Kenny & Kashy, 1992; Kumar & Dillon, 1990; Marsh, 1989; Marsh & Bailey, 1991). Prior to our test of discriminant validity we split the overall sample into two randomly selected subsamples (Hayduk, 1987). Exploratory and confirmatory factor analyses were each based on a distinct subsample.

In the first stage of analysis, traditional factor analytic techniques were used to determine whether or not measures of incivilities loaded on a factor distinct from measures of perceived crime and reports of victimization. In this stage, measures of perceptions of incivilities have discriminant validity relative to measures of perceived crime when each set of measures loads on a distinct factor.

In the second stage of analysis, structural equation measurement models were used to confirm the results of exploratory factor analysis. Fit statistics from structural equation models evaluate the overall fit of distinct factor structures. In this stage, discriminant validity is indicated when a model specifying two distinct factors, for example perceived incivilities and perceived crime, has adequate model fit and represents an improvement in fit over a one‐factor model.

In each stage of our data analysis, we use analytical techniques that can accommodate ordinal and dichotomous measures. Traditional factor analysis and structural equation modeling techniques assume that indicators are continuous. Our measures of perceived incivilities and perceived crime were ordinal and our measures of reported victimization were dichotomous. When data are measured at the ordinal level, factor analysis and structural equation models can be based on the polychoric correlation matrix. Similarly, when data are measured at the dichotomous level, factor analysis and structural equation models may be based on the tetrachoric correlation matrix. ³ Both the tetrachoric and polychoric correlations approximate the Pearson correlation that one would obtain if data were measured continuously. For the current analysis, tetrachoric and polychoric correlations were obtained using LISREL 8.80.

An initial review of the data revealed that across all measures of respondents’ perceptions of crime in their neighborhood just over 22% of the sample did not provide a valid response to at least one of the crime measures. Across the other types of measures included in our data—social and physical incivilities and reports of victimization—missing data were much less common ranging from 3.25% to 8.36%. Among measures of respondents’ perception of crime missing data were largely attributable to responses of “don’t know” to queries regarding the prevalence of different types of crime.

To assess the potential impact of missing data on the generalizability of our results, we compared those having missing data for any crime measure to those with responses for each crime measure across a number of demographic characteristics. Those missing data for any crime measures were not significantly different from those with complete data for crime measures with regard to homeownership, income, marriage, and age; however, those missing data for any crime measure were more likely to have reported a higher level of education, had lived in their neighborhoods longer and were more likely to be White.

Results

Descriptive statistics for measures of perceived incivilities, perceived crime and reported victimization are presented in Table 2. For measures of perceived social incivilities mean values were highest for noisy neighbors and loitering. Across indicators of perceived physical incivilities unkempt property and vandalism/graffiti had the highest mean values. Mean values for perceived crime show that perceptions of property crime were elevated relative to perceptions of violent crime. With regard to victimization, means for these measures show that reports of indirect victimization were more common than reports of personal victimization.

Table 2 Descriptive statistics

Variable	M	SD	Minimum	Maximum	Observations
Perceived incivilities
Loitering	1.33	.60	1	3	792
Drunks/tramps	1.27	.58	1	3	788
Harassment	1.16	.47	1	3	787
Fighting/arguing	1.22	.52	1	3	789
Noisy neighbors	1.34	.62	1	3	794
Drug sales	1.31	.65	1	3	768
Broken windows	1.14	.42	1	3	792
Unkempt property	1.47	.64	1	3	792
Vandalism/graffiti	1.35	.60	1	3	793
Vacant lots with trash	1.21	.51	1	3	795
Vacant houses or buildings	1.17	.43	1	3	794
Abandoned cars	1.19	.48	1	3	790
Rubbish/litter	1.28	.55	1	3	793
Perceived crime
Burglary	1.55	.69	1	3	757
Car theft	1.49	.72	1	3	742
Robbery	1.47	.68	1	3	749
Assault	1.21	.54	1	3	755
Reported victimization
Personal victimization	.25	.43	0	1	795
Indirect victimization	.30	.46	0	1	760

Exploratory Factor Analysis

The discriminant validity of measures of perceived incivilities was assessed with an exploratory factor analysis, where measures of physical incivilities and measures of perceived crime were initially constrained to load on a single factor. Subsequently, these measures were allowed to load on two factors. Results for these factor analyses are presented on the left half of Table 3. The first column of coefficients are factor loadings for both physical incivilities and perceived crime on a single factor, the second and third columns of coefficients are factor loadings for physical incivilities and perceived crime for two distinct factors.

Table 3 Exploratory factor analysis results for perceived incivilities and perceived crime

	Physical incivilities	Social incivilities
Indicator	1 factor	2 factors	1 factor	2 factors
Physical incivilities
Broken windows	.67	.65	.18
Unkempt property	.65	.67	.25
Vandalism/graffiti	.77	.49	.55
Vacant lots	.70	.66	.29
Vacant houses	.58	.69	.05
Abandoned cars	.73	.75	.25
Trash	.70	.71	.26
Social incivilities
Loitering				.56	.57	.29
Drunks/tramps				.77	.70	.41
Harassment				.63	.86	.13
Fighting/arguing				.72	.78	.29
Noisy neighbors				.65	.63	.33
Drug sales				.73	.68	.38
Perceived crime
Burglary	.73	.27	.76	.76	.39	.71
Car theft	.63	.14	.75	.71	.38	.66
Robbery	.69	.16	.83	.75	.17	.96
Assault	.68	.26	.71	.82	.62	.59
Eigenvalue	5.18	3.28	2.95	5.09	3.73	2.82
Explained variance		.53	.47		.57	.43

In an exploratory factor analysis of this type discriminant validity is indicated when measures of physical incivilities and measures of perceived crime load on two distinct factors. Results show that this was largely the case for our measures of perceived physical incivilities and perceived crime. Six of seven, measures of perceived physical incivilities had elevated values for the first factor in the two‐factor model (second column of coefficients) and had low values for the second factor (third column of coefficients) in the two‐factor model. In a single noteworthy exception to this general pattern, the indicator of respondent perceptions of vandalism/graffiti had a lower loading on the first factor.

Measures of perceived crime had low values for the first factor in the two‐factor model (second column of coefficients) and high values for the second factor in the two‐factor model (third column of coefficients). Overall, these results indicate that with the exception of perceptions of vandalism/graffiti the indicators of physical incivilities included in our factor analysis had discriminant validity relative to measures of crime perceptions.

The procedure outlined above was also applied to measures of social incivilities and measures of crime perceptions. Results are presented in the right half of Table 3. In the two‐factor model, factor loadings for indicators of social incivilities were all elevated for the first factor and lower for the second factor. Factor loadings for three measures of perceived crime—burglary, car theft, robbery—are all low for the first factor and high for the second factor. In contrast, factor loadings for respondent perceptions of assault are similar across the first (.62) and second factor (.59). These results show that measures of social incivilities are discriminable from measures of burglary, car theft and robbery.

Exploratory factor analysis was also used to test the discriminant validity of indicators of perceived incivilities relative to reports of personal and indirect victimization. ⁴ Results are presented in Table 4. In the two‐factor model for perceived physical incivilities, six of seven measures of perceived physical incivilities have large factor loadings on the first factor and relatively small factor loadings for the second factor. A single indicator of perceived physical incivilities, vandalism/graffiti, has a factor loading that is slightly lower on the first factor than on the second factor. Victimization reports have small factor loadings on the first factor and large factor loadings on the second.

Table 4 Exploratory factor analysis results for perceived incivilities and reported victimization

	Physical incivilities	Social incivilities
Indicator	1 factor	2 factors	1 factor	2 factors
Physical incivilities
Broken windows	.73	.72	.14
Unkempt property	.67	.67	.17
Vandalism/graffiti	.71	.49	.59
Vacant lots	.75	.63	.38
Vacant houses	.66	.70	.06
Abandoned cars	.77	.80	.15
Trash	.74	.69	.26
Social incivilities
Loitering				.59	.50	.33
Drunks/tramps				.77	.73	.36
Harassment				.62	.73	.17
Fighting/arguing				.80	.80	.25
Noisy neighbors				.65	.69	.19
Drug sales				.76	.69	.36
Perceived crime
Personal victimization	.43	.08	.76	.53	.35	.50
Indirect victimization	.44	.13	.68	.53	.20	.98
Eigenvalue	4.00	3.23	1.67	3.53	3.07	1.71
Explained variance		.66	.34		.64	.36

For the two‐factor model with perceived social incivilities and victimization reports, all indicators of perceived social incivilities have high factor loadings on the first factor and lower factor loadings on the second factor. This pattern is reversed for victimization reports. This pattern suggests that measures of social incivilities have discriminant validity relative to reports of victimization.

Confirmatory Factor Analysis

Structural equation models used in our confirmatory factor analysis are presented in Figures 1 and 2. When we explore the factor structure of measures of perceived physical incivilities relative to measures of perceived crime, there are 11 observed in indicators (from X₁ to X₁₁). These observed indicators are our seven indicators of perceived physical incivilities and four indicators of perceived crime. In the first model, depicted in Figure 1, each of the observed indicators is jointly determined by a single underlying latent factor (ξ), in combination with a unique error term (from δ₁ to δ₁₁). The magnitude of expected change in each of the observed indicators attendant to change in the latent factor is indicated by lambda (γ).

Figure 1 Confirmatory factory analysis: One factor model.

Figure 2 Confirmatory factory analysis: Two factor model.

Model fit and identification

In confirmatory factor analysis, discriminant validity is indicated when two‐factor models provide a fit to the data that is better than the fit of the one factor model and above the threshold for adequate fit. There are a number of indices of fit for structural equation models. In assessing model fit no single statistic necessarily indicates the fit of a structural equation model. Rather model fit is by assessing values on a number of distinct fit statistics (Hu & Bentler, 1999; Jaccard & Wan, 1996; Kline, 1998). Here we utilize the same four measures of fit used by Worrall (2006). These measures are the ratio of Chi‐square to the degrees of freedom (CMIN/DF), the adjusted goodness‐of‐fit index (AGFI), Hoelter’s (1983) Critical N, and the root mean square error of approximation (RMSEA). These measures represent a reasonable cross‐section of available fit statistics and will jointly allow a comprehensive assessment of model fit. ⁵

Though there is some variability in the literature, it is generally accepted that CMIN/DF ratios smaller than three indicate that the model provides a good fit to the data (Carmines & McIver, 1981; Kline, 1998). The AFGI ranges from 0 to 1.0. Typically values over .9 are regarded as having good fit (Bentler, 1990). Hoelter’s (1983) Critical N represents the largest sample size at which you would fail to reject the hypothesis that the model provides adequate fit. Critical N values over 200 are indicative of good fit. The RMSEA is described by Browne and Cudeck (1989) as asking the question “How well would the model, with unknown but optimally chosen parameter values, fit the population covariance matrix if it were available?” (Browne & Cudeck, 1989, pp. 137–138). Generally it is thought that RMSEA values less than .05 are indicative of good fit (MacCallum, Browne, & Sugawara, 1996), but recently Hu and Bentler (1999) have argued that this criteria should be relaxed with values below .06 indicating good fit.

In confirmatory factor analysis, the unmeasured latent variables, perceived physical incivilities for example, have no predetermined scale. As a consequence, it is necessary to specify structural equation models in such a way as to determine the scale of the latent factor(s). The most straightforward and common method for determining the scale of a latent factor is to constrain the value of one‐factor loading to one for each latent factor included in the structural equation model. In our confirmatory factor analyses there is a single latent factor in the first model, and therefore one‐factor loading constrained to be one, and two latent factors in the second model and therefore two‐factor loadings constrained to be one.

Confirmatory factor analysis results

Results for the confirmatory factor analyses based on indicators of perceived physical incivilities and indicators of perceived crime are presented on the left hand side of Table 5. Fit statistics show that the one‐factor model provided a mediocre fit to the data. Some fit indices indicated adequate model fit (CMIN/DF and Critical N), while others indicated model fit is unsatisfactory (AGFI, RMSEA). When a two‐factor model was used, model fit improved somewhat but the AGFI was still relatively low. A similar pattern of results was found in structural equation models testing the factor structure of perceived social incivilities indicators relative to perceived crime indicators. Again some fit indices indicated adequate model fit, while others indicated model fit is unsatisfactory. Model fit improved with the two‐factor model but the AGFI remained low.

Table 5 Confirmatory factor analysis results for perceived incivilities and perceived crime

	Physical incivilities	Social incivilities
	1 factor	2 factors	1 factor	2 factors
Indicator	Est.	SE	Est.	SE	Est.	SE	Est.	SE
Physical incivilities
Broken windows	1.0		1.0
Unkempt property	1.53	.34	.86	.11
Vandalism/graffiti	1.15	.28	.91	.09
Vacant lots	.58	.25	.63	.12
Vacant houses	1.06	.18	.86	.08
Abandoned cars	1.19	.24	1.02	.09
Trash	1.42	.28	1.11	.10
Social incivilities
Loitering					1.0		1.0
Drunks/tramps					.58	.17	1.32	.29
Harassment					.56	.14	.82	.20
Fighting/arguing					.42	.13	.67	.18
Noisy neighbors					.84	.20	1.40	.31
Drug sales					.94	.21	1.25	.28
Perceived crime
Burglary	3.81	.72	1.18	.08	2.40	.45	1.16	.11
Car theft	3.22	.59	1.0		1.96	.35	1.0
Robbery	3.58	.67	1.12	.07	1.99	.36	1.00	.09
Assault	2.08	.44	.65	.08	1.02	.22	.51	.09
Fit statistics
CMIN/DF	1.94	1.65	2.02	1.56
AGFI	.82	.85	.78	.83
RMSEA	.06	.05	.06	.04
Critical N	213	252	202	263

Next, we explored the extent to which model fit for the two‐factor models presented in Table 5 could be improved by allowing errors to correlate. The two‐factor models specified in Table 5 assumed error terms were uncorrelated. However, it seems reasonable to suggest that the unmeasured characteristics that determine respondents’ perceptions of the prevalence of vacant lots may also influence respondent perceptions of vacant houses, or in structural equation modeling terms, the errors terms for the indicators of perceptions of vacant lost and perceptions of vacant houses are correlated. For readers unfamiliar with the use of structural equation models in confirmatory factor analysis, it may be necessary to note that this is an entirely common procedure that is frequently used in confirmatory factory analysis (see e.g. Jöreskog & Sörbom, 2001).

Results for two‐factor models allowing correlated errors are presented in Table 6. In each of these models, errors were allowed to be correlated if modification indices were four or greater. Modification indices are the improvement in model fit, as measured by decrease in model Chi‐square, that would be realized by allowing two error terms to correlate. Errors were only allowed to correlate within factor, in other words errors for perceived physical incivilities indicators were only allowed to correlate with other errors of perceived incivilities indicators. At no time was an error term for an incivilities indicator allowed to correlate with an error term of an indicator of crime perceptions.

Table 6 Confirmatory factor analysis results for perceived incivilities and perceived crime: Two factor models revisited

	Physical incivilities2 factors	Social incivilities2 factors
Indicator	Est.	SE	Est.	SE
Physical incivilities
Broken windows	1.0
Unkempt property	.97	.15
Vandalism/graffiti	.88	.10
Vacant lots	.60	.13
Vacant houses	.82	.09
Abandoned cars	1.02	.10
Trash	1.13	.11
Social incivilities
Loitering			1.0
Drunks/tramps			1.39	.25
Harassment			.85	.18
Fighting/arguing			.69	.16
Noisy neighbors			1.26	.25
Drug sales			1.08	.22
Perceived crime
Burglary	1.20	.09	1.02	.08
Car theft	1.0		1.0
Robbery	1.11	.07	1.03	.07
Assault	.63	.08	.56	.09
Fit statistics
CMIN/DF	1.37	1.20
AGFI	.87	.87
RMSEA	.04	.03
Critical N	308	343

In the two‐factor model testing the factor structure of perceived physical incivilities indicators relative to perceived crime indicators, there were two correlations among error terms for perceived incivilities and two correlations among error terms for crime measures. ⁶ Model fit statistics show that these models provided an adequate but less than spectacular fit to the data, as AGFI’s are still a bit low, as values above .9 are generally held to be unambiguously indicative a of good fit.

The two‐factor model testing the factor structure of perceived social incivilities indicator relative to perceived crime indicators allowed one correlation among error terms for physical incivilities indicators and one correlation among error terms for crime perceptions. ⁷ Again model fit statistics indicated an adequate fit but were still less than perfect, as the AGFI was improved but still low.

Table 7 presents the results for confirmatory factory analysis based on indicators of perceived social incivilities and reports of victimization. Across both sets of incivilities measures we find that three of four fit indices showed that a one‐factor model provided a good fit to the data with the single exception being the RMSEA. When two‐factor models were estimated, model fit improved and fit statistics indicated that these models provided a good fit to the data. The RMSEA for the two‐factor model incorporating indicators of perceived physical incivilities reached the cut off point advocated by Hu and Bentler (1999) and the RMSEA for the two‐factor model incorporating indicators of perceived social incivilities meet the more stringent criteria advocated by others (MacCallum, Browne, & Sugawara, 1996).

Table 7 Confirmatory factor analysis results for perceived incivilities and reported victimization

	Physical incivilities	Social incivilities
	1 factor	2 factors	1 factor	2 factors
Indicator	Est.	SE	Est.	SE	Est.	SE	Est.	SE
Physical incivilities
Broken windows	1.00		1.00
Unkempt property	2.17	.45	1.86	.36
Vandalism/graffiti	2.79	.58	2.33	.45
Vacant lots	1.99	.46	1.76	.38
Vacant houses	1.01	.23	1.08	.20
Abandoned cars	1.60	.33	1.41	.26
Trash	3.03	.62	2.50	.47
Social incivilities
Loitering					1.00		1.00
Drunks/tramps					1.46	.21	1.52	.20
Harassment					.99	.15	1.07	.16
Fighting/arguing					1.08	.19	1.19	.19
Noisy neighbors					1.25	.20	1.27	.19
Drug sales					1.19	.20	1.22	.19
Reported victimization
Personal victimization	1.92	.47	1.00		.53	.16	1.00
Indirect victimization	2.48	.56	1.35	.29	.95	.20	2.17	.89
Fit statistics
CMIN/DF	2.90	2.45	2.88	1.51
AGFI	.98	.98	.90	.95
RMSEA	.07	.06	.07	.04
Critical N	189	226	191	369

Discussion

The results of exploratory and confirmatory factor analysis provide mixed evidence regarding the discriminant validity of measures of perceived incivilities relative to measures of perceived crime and victimization reports. Overall, we find fairly consistent evidence that measures of incivilities enjoy discriminant validity relative to victimization reports. Our results for the discriminant validity of perceptual incivilities measures relative to respondents’ perceptions of crime were less consistent. Below we first discuss our results for analyses testing the discriminant validity of measures of perceived incivilities relative to victimization reports. We then discuss results for analyses testing the discriminant validity of measures of perceived incivilities relative to measures of perceived crime.

Perceptual Incivilities Measures and Victimization Reports

The majority of results for exploratory and confirmatory factor analysis showed that perceptual incivilities measures and victimization reports loaded on distinct factors. The single exception to this general pattern came in exploratory factor analysis where an indicator of physical incivilities—vandalism/graffiti—had a factor loading opposite of other indicators of physical incivilities and consistent with victimization reports. Regarding the discriminant validity of social incivilities measures relative to victimization reports exploratory and confirmatory factor analyses both indicated incivilities measures and victimization reports loaded on distinct factors.

Perceptual Incivilities Measures and Perceptions of Crime

Exploratory and confirmatory analysis of perceived incivilities measures and measures of perceptions of crime provided equivocal results. Exploratory factor analysis showed that perceived incivilities indicators and crime perceptions indicators loaded on distinct factors with two important exceptions. A single indicator of physical incivilities—vandalism/graffiti—had a higher factor loading on the factor favored by each of the crime perceptions indicators. As noted earlier, factor loadings for this same indicator of physical incivilities were also reversed when indicators of perceived incivilities were included in exploratory factor analyses with victimization reports.

In the second exception the pattern outlined above, exploratory factor analysis showed that one of our crime measures, perceptions of assault, had a factor loading consistent with loadings from social incivilities measures. This result shows that respondents were unable to discriminate perceptions of assault from perceptions of social incivilities. Respondents may be unable to discriminate perceptions of assault from perceptions of social incivilities due to the content of some of the social incivilities measures. One measure in particular, fighting/arguing seems to index a range of behaviors that overlaps considerably with the behaviors that would qualify as assault in the eyes of those responding to the survey.

In confirmatory factor analysis of perceived incivilities measures and measures of perceived crime, we were able to estimate two‐factor models that provided what can be regarded as an adequate fit to the data by allowing a limited number of correlations among errors for perceived incivilities measures and among errors for perceived crime measures.

Comparing Our Results with those of Worrall (2006)

In our analyses we extended Worrall’s (2006) work by using perceived incivilities measures and perceived crime measures taken at the ordinal level. As a whole, our results are more supportive of the discriminant validity of perceptual incivilities measures. At this time it seems reasonable to attribute this difference, at least in part, to differences across the two studies in the level of measurement for incivilities measures. We use ordinal level data, while Worrall’s (2006) analysis was based on binary data. Therefore when contrasted with the work of Worrall (2006), our results suggest that the measures of perceived incivilities taken at higher levels of measurement will provide stronger evidence of discriminant validity. We recognize, however, that other differences between our study and that of Worrall (2006) may also play a part in explaining differences in results across the two studies. Other important differences include differences in measures of incivilities and differences in measures of crime perceptions.

In both studies, the measures of perceived physical incivilities included indicators of abandoned cars/buildings, unkempt property, trash and vacant lots. Unique indicators of perceived physical incivilities included in our study were broken windows, vandalism, and vacant houses. Unique indicators of perceived physical incivilities in the work of Worrall (2006) were poor lighting and overgrown shrubs/trees.

Indicators of perceived social incivilities held in common by the respective studies included drug sales, loitering, drunks/tramps, and harassment. Items used to measure perceived social incivilities in the current work but not employed in the measure of perceived social incivilities used by Worrall (2006) were fighting and arguing and noisy neighbors. Items used by Worrall (2006) to measure perceived social incivilities, but not included in the measure of perceived social incivilities employed in the current work were vandalism/graffiti, prostitution, and truancy/skip school. It is important to note, differences across the results presented here and those of Worrall (2006) may be influenced by the differences in measurement outlined above. Similarly, differences in results across the two studies may also be influenced by differences in the types of crime assessed by perceptual crime measures.

Though still best described as equivocal, the results of our exploratory and confirmatory factor analysis of perceived incivilities measures and perceived crime measures suggest that these two sets of measures load on distinct factors. In contrast, Worrall (2006) concluded that the poor fit of two‐factor models indicated “respondents were unable to distinguish between perceived crime and either physical or social incivilities” (p. 376).

While some of the differences between the results of our factor analysis and those of Worrall’s (2006) may be attributable to differences in level of measurement and differences in incivilities measures, differences may also be attributable to the type of crime perceptions measured in the two respective studies. Worrall (2006) included measures of respondents’ perceptions of auto theft, theft of property, breaking and entering, violent attacks, gun crime, and sexual assault/rape. In our study we included measures of respondent perceptions of burglary, car theft, robbery and assault.

Worrall’s (2006) exploratory factor analysis showed that respondents were able to clearly distinguish perceptions of theft of property and breaking and entering from both physical and social incivilities. Perceptions of violent attacks and gun crime were also distinct from both but to a lesser degree. In our exploratory factor analysis, respondents were able to distinguish perceptions of burglary, car theft and robbery from perceptions of both physical and social incivilities.

Differences across the two sets of results under consideration here can also be found for exploratory and confirmatory factor analysis of perceptions of incivilities relative to victimization reports. Our results showed that measures of both physical and social incivilities and reports of victimization tended to load on distinct factors. Worrall (2006) reported that “exploratory and confirmatory factor analysis revealed that perceptions of incivilities (of the physical variety) loaded on factors separate from personal victimization” (p. 360), but found two‐factor confirmatory factor analysis models for measures of social incivilities and victimization reports provided a poor fit to the data.

To summarize, our results advance the literature by testing the discriminant validity of perceptual incivilities measures with ordinal data, while previous work relied on binary data. Contrasting the results of our work with that of Worrall (2006) suggests that ordinal measures of perceptual incivilities provide stronger evidence of discriminant validity than binary measures. We also advance the literature by testing the discriminant validity of perceptual incivilities measures relative to a different set of crime measures than that used in earlier research. This proves to be an important extension of the earlier literature when exploratory factor analysis shows that our measure of perceptions of robbery, a measure not included in earlier work, is distinct from measures of perceptions of both physical and social incivilities. Finally, we find that evidence for the discriminant validity of perceptions of incivilities is strongest when these measures are contrasted with victimization reports rather than measures of crime perceptions. We should note that this finding is consistent with the work of Worrall (2006), suggesting that future work incorporating perceptual incivilities measures should also incorporate measures of crime that include victimization reports.

Conclusions

Overall, the pattern of results discussed above suggests that the answer to the question, “Do measures of perceived incivilities have discriminant validity relative to measures of perceived crime,” depends on just what we mean by perceived incivilities and perceived crime. To the extent to which these measures tend to overlap conceptually, we may anticipate that they will also lack discriminant validity. This is illustrated in our data by the fighting/arguing indicator of social incivilities and of respondents’ perceptions of the prevalence of assault. Conversely, we may anticipate that conceptually distinct measures may provide stronger evidence for discriminant validity.

What is used as an indicator of incivilities and what is used as an indicator of crime is a particularly nuanced issue among theories specifying a relationship between incivilities and crime. In what is perhaps the foremost of these theories, Wilson and Kelling (1982, 2006) argue that incivilities include pubic order crimes such as drug dealing and prostitution can lead to more serious crime. Wilson and Kelling go so far as to specify that robbery is the crime that should be most influenced by variation in incivilities. If this is the case we should anticipate that incivilities and robbery, though related, should enjoy discriminant validity relative to each other. Indeed our analysis provides some evidence that this is indeed the case (see factor loadings for robbery Table 3).

Wilson and Kelling (1982, 2006) also allow that perceived incivilities may influence other forms of serious crime including burglary/breaking and entering. Results from exploratory factor analysis and those of Worrall (2006) suggest that respondents are able to distinguish between perceptions of burglary and breaking entering and perceptions of incivilities. These results, in contrast with results showing respondents have a difficult time distinguishing other measures of crime from perceptions of incivilities, underscore the extent to which the discriminant validity of perceptions of incivilities relative to perceptions of crime depends heavily on what are included as indicators of crime and what are included as indicators of incivilities. This variability in turn underscores the need to further assess the extent to which we can use exploratory factor analysis to inform confirmatory factor analyses.

When we use exploratory factor analyses to inform confirmatory factor analysis, perceived crime measures with factor loadings inconsistent with discriminant validity are dropped when we estimate confirmatory factor analysis models. Such an analysis moves beyond testing the discriminant validity of existing sets of indictors of crime perceptions and perceptions of incivilities, in an attempt to develop guidance regarding the creation of measures that actually have discriminant validity.

Recently scholars have called into question the discriminant validity of measure of perceived incivilities (Sampson & Raudenbush, 1999; Taylor, 1999). While far from closing the question, results to date suggest respondents’ perceptions of physical incivilities are distinct from their reports of victimization. Results regarding the extent to which respondents are able to distinguish between social incivilities and victimization reports and between incivilities in general and crime are far less conclusive.

Exploratory factor analyses presented here and in the work of Worrall (2006) show that the relationship between perceptions of incivilities and perceptions of crime varies across different crime types. Further, analyses presented here provide modest evidence for the discriminant validity of incivilities measures relative to particular types of crime. Future research should employ additional confirmatory factor analyses outlined above in an attempt to determine whether or not more conclusive evidence for the discriminant validity of perceptions of incivilities relative to perceptions of crime may be developed.

Notes

1. Data collected with the Bureau of Justice Statistics Survey entitled “Criminal Victimization and Perceptions of Community Safety in 12 United States Cities, 1998” (ICPSR Study No. 2743).

2. About 75% of those contacted agreed to participate in the survey. Just fewer than 2% were unable to complete it due to language barriers; all others who refused stated that they were not available to complete the survey at the time, or asked that the interviewer call back at another time. Our sample contained slightly more Whites and included a greater number of older persons and women when compared with the city’s general population.

3. For a discussion of the derivation of polychoric and tetrachoric correlations, see Drasgow (1988) and Harris (1988). Schumacker and Beyerlein (2000) review the use of polychoric and tetrachoric correlations in confirmatory factor analysis, and Bollen (1989) details the use of polychoric and tetrachoric correlations in structural equation modeling.

4. For the exploratory factor analysis of perceived incivilities and reported victimization measures, indicators of perceived incivilities were dichotomized to match the level of measurement of victimization reports.

5. With four fit statistics we meet the recommendation of Jaccard and Wan (1996) who recommend the use of at least three fit statistics and the more stringent recommendation of Kline (1998, p. 130) who suggests the use of four or more.

6. Correlations among error terms for physical incivilities measures were between error terms for broken windows and unkempt properties and between error terms for unkempt property and vacant lots. Correlations among error terms for perceived crime were between error terms for burglary and assault and between error terms for car theft and robbery.

7. The correlation among error terms for social incivilities measures was between error terms for loitering and drunks/homeless. The correlation among error terms for perceived crime indicators was between the error term for car theft and the effort term for burglary.