Automated Valuation Services: A case study for Aberdeen in Scotland

ABSTRACT

Automated valuation services (AVSs) offered by listings platforms predict market values based on property characteristics supplied by users. We investigate the implementation of such a service for the City of Aberdeen. We fit different market value models with machine learning methods and assess them in a rolling windows procedure that mimics an AVS setting. We also investigate the ease and robustness with which the models can be implemented. We discuss how prediction uncertainty can be measured and reported to users. If implemented in the future, such a service has the potential to improve the transparency of the local housing market.

KEYWORDS:

• Housing market
• machine learning

Acknowledgments

We are grateful to the three anonymous referees and the editor of the journal for suggestions and comments that helped to improve the paper. We thank seminar participants at the Technische Universität Berlin, Fiona Stoddard, and Verity Watson for helpful comments. The usual disclaimer applies.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. Platforms such as ImmoScout24 and Immowelt-Immonet in Germany, Hometrack and Zoopla in the UK, and Redfin and Zillow in the US, offer such automated valuation services.

2. Keeping details about the implementation private prevents that competitors can copy it. Matysiak (2017) finds that US providers are more open than European providers with respect to information on accuracy.

3. As residential markets differ, it seems likely that models that work well for one market might not do so for another market. The no free lunch theorem from machine learning applies here, see Murphy (2012, p. 24): no single statistical model will be best for each and every application.

4. Examples include: Anglin and Gençay (1996), Parmeter et al. (2007), and Haupt et al. (2010), which examine the same data set but use different models (semi-, non-, or fully parametric). Martins-Filho and Bin (2005) is another example of a semiparametric model. Bourassa et al. (2010) fit different spatial models for a given data set and compare predictions. McCluskey et al. (2013) fit linear, neural net, and geospatial models to a given data set and compare predictions.

5. A small number of characteristics will also make it easy to check whether AVS users request plausible characteristic combinations. Naturally, a larger number of characteristics describe properties better and should result in better predictions.

6. The remaining 26,817 properties were social housing and thus in the public sector, which is not relevant here. The above numbers are collated from Aberdeen City Council (2018), National Records of Scotland (2018), and Registers of Scotland (2018).

7. The ASPC distributes also a printed register of the listings in the Aberdeen area.

8. Erroneous values are obvious mistakes such as a property with zero rooms or location coordinates that are outside Aberdeen City.

9. We conducted the analysis also with separate room categories. The results of this analysis are qualitatively identical to the ones reported here, see the web-appendix for details.

10. The mean squared error on the right-hand side of Eq. 2(2) $\mathrm{E}[p|\mathbf{x}]=\underset{g\in \mathcal{F}}{min}\mathrm{E}\left[{\left(p-g(\mathbf{x})\right)}^2\right]$(2) becomes smaller the more (price determining) characteristics $\mathbf{x}$ contains.

11. For instance, with a cubic polynomial basis, $d=3$, we obtain $f(x,\theta )=[x,x^2,x^3]\theta$.

12. The cubic splines basis is $f(x,\theta )=[x,x^2,x^3,|x-{\kappa }_1{|}^3,,|x-{\kappa }_K{|}^3]\theta$. It extends a polynomial with truncated terms $|x-{\kappa }_k|$, which join at knots, ${\kappa }_K>>{\kappa }_1$. The larger the number of knots $K$, the more flexible will $f(x,\theta )$ be. $\theta$ is constrained further to ensure that the function is linear beyond the boundary knots. The thin plate splines basis is a two-dimensional extension of the cubic splines basis (Wood, 2017, Ch. 5).

13. If properties have up to three bathrooms, split sets are $\{1,(2,3)\}$, $\{2,(1,3)\}$ $\{3,(1,2)\}$.

14. The random forest is also robust against outliers, as unusual observations will appear in only few re-sampled training samples. We deal with unusual observations by setting bounds, but additional robustness can be useful.

15. We also examined the predictive performance when the windows are rolled forward by one month. The results are comparable to those reported here, see the web-appendix. We did not examine the effect the length of the training sample. The samples covering four quarters (twelve months) produce good in-sample fits for all five models and we believe that improvements, if any at all, would be only marginal.

16. The fractions can be obtained by computing the observation-weighted averages of $\mathrm{R}\mathrm{E}\mathrm{R}(0.1)$ and $\mathrm{R}\mathrm{E}\mathrm{R}(0.2)$ from Table 3 (boosting machine) and Table 4 (penalised spline).

17. Over the period of our case study, the quality-controlled house price index for Aberdeen declined, on average, by 1.5% per quarter. A statistical model could be used to forecast changes of the price trend and these could then be used to adjust market value predictions. However, this task is separate from finding the best market value model. We do not approach it here.

18. Since we fit the models to location coordinates, a web service also needs to integrate digital maps to match a street address – as provided by the user – to coordinates.

19. The large MSRE is caused by a single property in the test sample that has a floor area just outside the range observed in the training sample.

20. The bootstrap is a third approach for the construction of prediction intervals, which often produces better finite sample results than standard method. We refer to Krause et al. (2020) for a comparison of the bootstrap and standard approaches. For an application of conformal prediction intervals see Bellotti (2017).

Additional information

Notes on contributors

Rainer Schulz

Rainer Schulz studied Economics at Freie Universität Berlin and received his doctoral degree in Economics and Econometrics from the Humboldt-Universität zu Berlin. He is a senior lecturer at the University of Aberdeen Business School. His research has been published, amongst others, in the Journal of Real Estate Finance and Economics, the Journal of Housing Economics, and the Journal of Urban Economics.

Martin Wersing studied Business Economics at Freie Universität Berlin and received his doctoral degree in Economics and Econometrics from Technische Universität Berlin. He is a lecturer at the University of Aberdeen Business School. His research has been published, amongst others, in Empirical Economics, the Journal of Housing Economics, and the Journal of Real Estate Finance and Economics.