Prediction accuracy in mass appraisal: a comparison of modern approaches

Abstract

The advancement of computational software within the last decade has facilitated enhanced uptake of mass appraisal methodologies by the valuation and prediction accuracy in computer-assisted mass appraisal community for price modelling, estimation and tribunal defence. Applying a sample of 2694 residential properties, this paper assesses and analyses a number of geostatistical approaches relative to an artificial neural network (ANN) model and the traditional linear hedonic pricing model for mass appraisal valuation accuracy and price estimation purposes. The findings demonstrate that the geostatistical localised regression approach is superior in terms of model explanation, reliability and accuracy. ANNs can be shown to perform very well in terms of predictive power, and therefore valuation accuracy, outperforming the traditional multiple regression analysis (MRA) and approaching the performance of spatially weighted regression approaches. However, ANNs retain a ‘black box’ architecture that limits their usefulness to practitioners in the field. In relation to cost-effectiveness and user-friendly applicability for the valuation community, the MRA approach outperforms the ‘black box’ nature of the ANN technique, with the geographically weighted regression approach providing the best balance of outright performance and transparency of methodology. It is this spatially weighted approach utilising absolute location which appears to represent the way forward in developing the practice of mass appraisal.

Keywords:

• valuation accuracy
• artificial neural networks
• hedonic regression
• geographically weighted regression

Notes

1. Different model architectures input predictor variables differently, for example ANN software cannot handle discriminating binary variables but rather accepts classification variables. Therefore different model architectures whilst utilising the same variables will input them into the models differently. Regression techniques are powerful in terms of providing statistical validation such as R² and adj. R², F-stat, t-stats etc, but AI modelling architectures such as ANNs do not provide this level of statistical transparency, therefore performance measurement is applied.

2. As noted by Miron (1984), amongst others, the SAR approach illustrates non-zero covariances between the error terms in the model demonstrating that these non-zero covariances result from misspecification, hinting towards model bias within the parameter estimates.

3. Increasing alpha fashions a higher weighting and a more local scale. Nonetheless, care must be taken to ensure that model error term is not over or under specified. Alpha can be adjusted to remove autocorrelation in the error term, however, a higher term can result in overcorrection.

4. For a full discussion on the effects of changing spatial structures on errors see: Bini et al. (2009) Coefficient shifts in geographical ecology: an empirical evaluation of spatial and non-spatial regression, Ecography, Vol. 32, pp. 193–204.

5. For a full discussion and specification of SAR modelling approaches see Kissling and Carl (2008), Anselin (1998), Haining (2003) and Fortin and Dale (2005).

6. The GWR approach produces an overall R² value for the model which is a ‘pseudo’ R² as it is the squared correlation coefficient between the observed and predicted values for all 2,694 regressions (observations).

7. For a full discussion surrounding the kernel function and bandwidth selection criteria applied within this research see de Smith, Goodchild, and Longley (2007, pp. 46–48).

8. To run the SAR_err this paper applies an open access software product SAM, having developed the data and distance calculations in ArcGIS.

9. Land and Property Services are a division of Department for Personnel and Finance.

10. Each of the properties included in the dataset used has a discrete spatial reference, being an X and Y co-ordinate. This is utilised in the modelling for the GWR and SAR analysis. When undertaking the MRA analysis the absolute spatial reference (X,Y co-ordinates) are not utilised and therefore the greatest level of spatial delineation is indeed the electoral ward.

11. The travel-to-work parameter is sourced from the Northern Ireland Neighbourhood Information Service and is defined as the average travel-to-work time from a specific super output area to employment centres.

12. The SAR estimated coefficients are calculated taking the partial derivative of the equation with regards to a given independent variable. The resultant matrix produces direct, indirect and total effects which can be determined. This research uses the total effects estimated coefficients as this is the most similar in interpretation to the OLS coefficients. For a full discussion see LeSage and Pace (2009).

13. The ANN model does not provide an F-statistic.