An improved Fuzzy Kappa statistic that accounts for spatial autocorrelation

Abstract

The Fuzzy Kappa statistic expresses the agreement between two categorical raster maps. The statistic goes beyond cell‐by‐cell comparison and gives partial credit to cells based on the categories found in the neighborhood. When matching categories are found at shorter distances the agreement is higher. Like the well‐established Kappa statistic the Fuzzy Kappa statistic expresses the mean agreement relative to the expected agreement. The model underlying the expected agreement assumes absence of spatial autocorrelation in both compared maps. In reality however, spatial autocorrelation does lower the expected agreement as matching categories become less likely to be found close‐by. Since most maps have some degree of spatial autocorrelation, the calculated expected agreement is generally higher than the true expected agreement. This leads to counterintuitive results when maps that appear to have considerable agreement obtain negative Fuzzy Kappa values. Furthermore, the Fuzzy Kappa may be biased, as it systematically attributes lower agreement to maps with stronger spatial autocorrelation. This paper proposes an improved Fuzzy Kappa statistic that is based on the same local agreement and has the same attractive properties as the original Fuzzy Kappa. The novelty is that the new statistic accounts for spatial autocorrelation, such that the expected Fuzzy Kappa for maps that are not cross‐correlated is equal to zero. The improved statistic is applied on two cases to demonstrate its properties.

Keywords:

• Fuzzy Kappa
• Map comparison
• Accuracy
• Validation

Acknowledgements

This paper addresses the most frequently asked question of Map Comparison Kit users: ‘Why do I get negative Fuzzy Kappa values for maps that appear quite similar?’ I would like to thank the users for sending their feedback and enabling RIKS to improve on the methods and software. The elaborate and detailed feedback of three anonymous reviewers has been very helpful and is greatly appreciated.