Contextual land-cover classification: incorporating spatial dependence in land-cover classification models using random forests and the Getis statistic

Abstract

Land-cover characterization of large heterogeneous landscapes is challenging because of the confusion caused by high intra-class variability and heterogeneous landscape artefacts. Neighbourhood context can be used to supplement spectral information, and a novel way of incorporating spatial dependence in a heterogeneous region is tested here using an ensemble learning technique called random forests and a measure of local spatial dependence called the Getis statistic. The overall Kappa accuracy of the random forest classifier that used a combination of spectral and local spatial (Getis) variables at three different neighbourhood sizes (3 × 3, 7 × 7, and 11 × 11) ranged from 0.85 to 0.92. This accuracy was higher than that of a non-spatial random forest classifier having an overall Kappa accuracy of 0.78, which was run using the spectral variables only. This study demonstrated that the use of the Getis statistic with different neighbourhood sizes leads to substantial increase in per class classification accuracy of heterogeneous land-cover categories.

1. Introduction

Methods of land-cover characterization using medium spatial resolution data (15–30 m) are well established and near operational (Franklin and Wulder 2002). Unfortunately, at fine spatial scales, the size of the pixel is still often larger than the land-cover element causing class mixing within pixels (Blaschke et al. 2004), and over large areas map products lack spatial precision owing to heterogeneous landscape artefacts such as topographic displacement, moisture gradients, and antecedent disturbances (i.e. natural and anthropogenic) (Rogan and Miller 2006). Random forest classifiers provide a new way to produce thematic maps that are potentially robust to variations in class reflectance caused by gradients or disturbances in regional scale mapping and high intra-class variability influenced by landscape heterogeneity, which is evident in the class labelling errors in the calibration data used in land-cover characterization (see Ham et al. 2005, Gislason et al. 2006, Lippitt et al. 2008, Rogan et al. 2008).

Random forest classifiers are ensemble algorithms developed in the field of machine learning and use bootstrap samples with replacement to grow a large set of classification trees (Breiman 2001). In a land-cover classification context, pixels are assigned to categories that receive the maximum number of votes from the collection of multiple trees. Random forests do not overfit the data distributions because the large number of trees grown reduces generalization error (Breiman 2001, Pal 2005) and has been shown to increase land-cover classification accuracy (Ham et al. 2005, Pal 2005, Gislason et al. 2006), because the classification error of one permutation can be overcome by the ensemble of permutations (Kotsiantis and Pintelas 2004). Furthermore, random forests provide measures of variable importance (e.g. mean decrease in Gini coefficient) that can be used to understand the contribution of specific variables in a classification (Gislason et al. 2006).

Random forests represent the state of the art in land-cover classification but, as with most classification methods, do not include spatial dependence among neighbouring pixels even though spatial dependence or context can be used to increase land-cover classification accuracies (Stuckens et al. 2000). The incorporation of spatial dependence into random forests classification has the potential to increase the precision of class designation by minimizing intra-class variation (Wulder and Boots 1998). Thus, the incorporation of spatial dependence in random forests has the potential to complement its inability to utilize spatial relationships among neighbouring pixels. The incorporation of spatial dependence or context has been approached in the literature using (i) texture-based transformed spectral bands as variables in land-cover classification in order to capture the local variability of the digital numbers within a neighbourhood using geostatistics (Atkinson and Lewis 2000) and spatial autocorrelation statistics (Myint et al. 2007) and (ii) majority smoothing filter applied to the classified image (Mather 2004). In this context, this study presents a novel contextual texture-based classification method to assess the capability of a random forest algorithm to classify land cover by integrating spectral and Getis variables in a heterogeneous landscape using Landsat-7 Enhanced Thematic Mapper-plus (ETM+) imagery.

An improved approach to incorporate spatial dependence in land-cover classification is through the Getis statistic, Gi*. This statistic behaves similarly to a moving filter in a remote sensing context by considering pixel values within a local neighbourhood of the focus pixel (which helps to remove labelling errors caused by noisy data or complex spectral measurement space) while simultaneously accommodating the values in the entire image reflecting global landscape heterogeneity characteristics (Wulder and Boots 1998, Richards and Jia 2006).

2. Methodology

2.1 Random forests

A random forest classifier (i.e. random forests) consists of a group of tree-based classifiers {h ( x , Θ _k ), k= 1,…} where x is the input vector and Θ _k are the independent identically distributed random vectors (Breiman 2001). Random forests use bootstrap samples with replacement to grow a large collection of classification trees, which assign each pixel to a class based on the maximum number of votes that a class receives from the collection of trees. Each tree is grown from a randomly and independently selected subspace (certain proportion of pixels) of the measurement space (training pixels) that is used to train the random forest classifier and the remaining samples called out of bag cases are used to assess the accuracy of the classification. Variable importance is calculated as the sum of the decrease in Gini for each variable calculated from the group of tree-based classifiers. This algorithm is easy to implement because the user has to adjust for only two parameters: (i) number of trees to grow and (ii) number of randomly selected split variables at each node. The strong law of large numbers ensures that the solution always converges with no over fitting.

2.2 Getis statistic

The Getis statistic is a local indicator of spatial dependence (Getis and Ord 1992, Ord and Getis 1995), which describes local variability in spatial dependence (Wulder and Boots 1998). The application of the Getis statistic results in the creation of a new image representing the spatial structure of a given spectral image (Wulder and Boots 1998). The Getis statistic, , can be used to identify clusters of high values called hot spots or clusters of low values called cold spots. It is computed as

(1)

where is the Getis statistic value, w_ij are the contiguity-based weights specified as w_ij = 1 if the pixel j falls in the filter window of the central pixel i and w_ij = 0 if the pixel j falls outside, and x_j is the value of pixel j (including the value at the central pixel i). Conceptually behaves similarly to a moving filter window and can be computed for each pixel of a given spectral band as the ratio of the sum of radiometric values in the neighbourhood of a central pixel to the sum of radiometric values of all pixels in the study area.

2.3 Data and methods

Satellite sensor images of Cape Cod, Massachusetts, captured by Landsat-7 ETM+  on 7 September 2001 (path/row 11/31) were used in this analysis. True-colour 1:5000 orthorectified aerial photographs from April 2001 (http://www.mass.gov/mgis/) were used to identify land-cover categories in the area and serve as a source for calibration and validation datasets. The eight land-cover classes considered in this study were cranberry bogs, pasture or row crops, forest, grassland, urban, water, wetland, and sand quarry. The ground reference dataset of land-cover categories was split randomly into a 75% subset of calibration samples and 25% subset of validation samples. Table 1 presents the different land-cover categories examined in this analysis and the number of calibration and validation pixels assigned to each category.

Table 1. Number of calibration and validation pixels for different land-cover classes

Class name	Calibration pixels	Validation pixels
Cranberry bogs	23	9
Pasture or row crops	22	10
Forest	157	67
Grassland	91	33
Urban	163	69
Water	81	39
Wetland	133	65
Sand quarry	48	15

A random forest classification model, hereafter called non-spatial random forest classifier, was run on the six spectral bands using the random forest package in the R statistical software (R Development Core Team 2008). This model was calibrated by varying two input parameters: (i) the number of trees from 100 to 1000 in increments of 100 and (ii) the number of randomly selected split variables from 1 to 6 (where 6 refers to the number of bands) in increments of 1 in order to select a single model built by the combination of these input parameters having the highest accuracy. Post-classification smoothing majority filters of 3 × 3, 7 × 7, and 11 × 11 pixel sizes using a mode decision rule were applied to this non-spatial random forest classified image.

In contrast, another classification model, hereafter called random forest Getis classifier, utilized the random forest classification algorithm run on the six Getis variables computed on each spectral band in addition to the six spectral bands. Three different random forest Getis classifiers were tested for Getis statistic neighbourhood sizes of 3 × 3, 7 × 7, and 11 × 11 pixels with the same input parameters as those used for the non-spatial random forest classifier, in order to provide consistent results for comparison with the non-spatial random forest classifier. These three neighbourhood sizes were considered to investigate the influence of different spatial scales on the classification accuracy. Comparison of the accuracy of the different variants of the random forest model was performed using the Kappa statistic that takes into account the actual agreement specified by the major diagonal of the confusion matrix and chance agreement indicated by the row and column totals of the confusion matrix (Congalton 1991). Furthermore, accuracies of individual classes were assessed with per class Kappa measures (Stehman 1997).

3. Results

The non-spatial random forest classifier parameterized with 600 trees and 3 randomly selected split variables at each node had the highest overall validation accuracy (overall Kappa = 0.78 with Z= 29.15; average per class Kappa = 0.72 with standard deviation = 0.24) amongst the non-spatial random forest classifiers parameterized by varying the number of trees and number of split variables (table 2). Figure 1 shows the classified map produced by the non-spatial random forest classifier. Table 3 shows the confusion matrix of the non-spatial random forest classifier. The post-classification smoothing majority filters applied to the non-spatial random forest classified image showed a decreasing trend in accuracy with increasing neighbourhood size (table 2), reflecting the trade-off between retaining the signal versus the unwarranted noisy pixels. A majority filter neighbourhood size of 3 × 3 pixels produced the highest accuracy (overall Kappa = 0.89 with Z= 43.81; average per class Kappa = 0.82 with standard deviation = 0.30), whereas 11 × 11 pixels had the lowest accuracy (overall Kappa = 0.72 with Z= 24.79; average per class Kappa = 0.61 with standard deviation = 0.32).

Table 2. Accuracy measures of classifiers with and without spatial dependence

Classification method	Overall accuracy (%)	Overall Kappa	Mean per class Kappa ± standard deviation
Non-spatial random forest	81.76	0.7782	0.7245 ± 0.2417
Non-spatial random forest – 3 × 3 pixels majority filter	90.55	0.8850	0.8215 ± 0.2968
Non-spatial random forest – 7 × 7 pixels majority filter	86.64	0.8363	0.7600 ± 0.3153
Non-spatial random forest – 11 × 11 pixels majority filter	77.52	0.7214	0.6112 ± 0.3205
Random forest Getis – 3 × 3 pixels	87.62	0.8496	0.8123 ± 0.1708
Random forest Getis – 7 × 7 pixels	93.49	0.9209	0.8961 ± 0.1199
Random forest Getis – 11 × 11 pixels	93.49	0.9212	0.9303 ± 0.0629

Table 3. Confusion matrix of non-spatial random forest classifier using validation samples

Class name	Cranberry bogs	Pasture or row crops	Forest	Grassland	Urban	Water	Wetland	Sand quarry	Total	Commission error
Cranberry bogs	6	0	0	0	1	0	0	0	7	0.1429
Pasture or row crops	1	2	0	0	1	0	0	0	4	0.5
Forest	0	0	60	5	2	0	7	0	74	0.1892
Grassland	2	4	0	27	6	0	1	1	41	0.3415
Urban	0	4	2	1	53	0	6	1	67	0.209
Water	0	0	0	0	0	39	0	0	39	0
Wetland	0	0	5	0	5	0	51	0	61	0.1639
Sand quarry	0	0	0	0	1	0	0	13	14	0.0714
Total	9	10	67	33	69	39	65	15	307
Omission error	0.3333	0.8	0.1045	0.1818	0.2319	0	0.2154	0.1333

Figure 1. Land-cover maps produced by (a) non-spatial random forest classifier (top left), (b) random forest Getis classifier using 11 × 11 pixels moving window (top right) and (c) difference map of non-spatial random forest and random forest Getis (11 × 11 pixels) land-cover maps (bottom left).

The random forest Getis classifier that integrates the spectral and Getis variables led to a significant increase in land-cover map accuracy (table 2). In general, there was an increase in overall Kappa value with increasing neighbourhood sizes from 3 × 3 to 11 × 11 pixels for the random forest Getis classifiers. The largest neighbourhood size of 11 × 11 pixels resulted in highest overall accuracy (overall Kappa of 0.92 with Z= 53.87; average per class Kappa = 0.93 with standard deviation = 0.06) for the random forest Getis classifier that was significantly different at 0.05 level (Z= 4.51) than that created by the non-spatial random forest classifier and was also higher than that produced by the post-classification majority filters of sizes ranging from 3 × 3 pixels to 11 × 11 pixels applied on the non-spatial random forest classified image. The Getis variables were selected by the random forest classifier because the Getis variables were able to explain a substantial amount of variance indicated by the Gini index variable importance measure (figure 2). The predicted categories of the random forest Getis classifier having a window or neighbourhood size of 11 × 11 pixels are shown in figure 1. The difference map showing areas of difference and no difference between the land-cover maps produced by the non-spatial random forest classifier and random forest Getis classifier is also revealed in figure 1. Table 4 shows the confusion matrix along with the class-specific accuracy measures for the random forest Getis classifier.

Table 4. Confusion matrix of random forest Getis classifier with 11 × 11 pixels moving window using validation samples

Class name	Cranberry bogs	Pasture or row crops	Forest	Grassland	Urban	Water	Wetland	Sand quarry	Total	Commission error
Cranberry bogs	8	0	0	0	0	0	0	0	8	0
Pasture or row crops	0	9	0	0	2	0	0	0	11	0.1818
Forest	0	0	62	0	2	0	1	0	65	0.0462
Grassland	0	0	0	33	2	0	0	0	35	0.0571
Urban	0	1	2	0	60	0	3	0	66	0.0909
Water	0	0	0	0	0	39	0	0	39	0
Wetland	1	0	3	0	2	0	61	0	67	0.0896
Sand quarry	0	0	0	0	1	0	0	15	16	0.0625
Total	9	10	67	33	69	39	65	15	307
Omission error	0.1111	0.1	0.0746	0	0.1304	0	0.0615	0

Figure 2. Variable importance contributions of different bands in terms of percent mean reduction in Gini index of the random forest Getis classifier with 11 × 11 pixels moving window.

Table 5 shows the increase in per class Kappa value of each land-cover category for the random forest Getis classifier over the non-spatial classifier. The highest increase in per class Kappa values was found for heterogeneous map categories, such as pasture or row crops (increase in per class Kappa = 373.23%) and cranberry bogs (increase in per class Kappa = 34.45%). The pasture or row crops category was initially confused with grassland and urban categories and the canberry bog category was confused with the pasture or row crops and grassland categories when the non-spatial classifier was applied, because the pasture or row crops and cranberry bog categories were characterized by low inter-class separability and high intra-class variability. However, the use of the random forest Getis classifier provided additional dimensions in the feature space that increased inter-class separability and reduced intra-class variability. In contrast, small increases in Kappa values were observed for homogeneous classes, such as water (increase in per class Kappa = 0%) and forest (increase in per class Kappa = 4.99%), because the homogeneous categories were highly separable using the spectral variables only and additional Getis variables did not contribute to increasing inter-class separability.

Table 5. Per class Kappa values of non-spatial and random forest Getis classifiers

Class name	Kappa value (non-spatial random forest)	Kappa value (random forest Getis-11 × 11 pixels)	Increase in Kappa of random forest Getis over non-spatial random forest
Cranberry bogs	0.6589	0.8859	34.45
Pasture or row crops	0.1894	0.8963	373.23
Forest	0.8623	0.9053	4.99
Grassland	0.7902	1	26.55
Urban	0.7034	0.8338	18.54
Water	1	1	0.00
Wetland	0.7312	0.9213	26.00
Sand quarry	0.8603	1	16.24

4. Conclusion

Spatial dependence can describe the local spatial structure and variability of land-cover categories and can increase land-cover classification accuracies in heterogeneous landscapes that are difficult to classify because of high intra-class variability and low inter-class separability. The increase in accuracy of the random forest Getis classifier over the non-spatial random forest classifier can be attributed to the large increase in average per class Kappa of heterogeneous map categories in comparison to homogeneous classes. The variation in Kappa accuracy of the random forest Getis classifier for different neighbourhood sizes is related to the differences in the size of pixel neighbourhood and the shape and size of the land-cover patches or objects represented by the pixel neighbourhood. It is hypothesized that the Getis statistic used to describe contextual or spatial information has distinct advantages over conventional contextual texture-based classification approaches because, unlike standard contextual methods that consider only values at a given neighbourhood of each pixel, the Getis statistic is a ratio of values of the neighbourhood for each pixel versus values of the entire image.