Mitigating housing market shocks: an agent-based reinforcement learning approach with implications for real-time decision support

References

• Agarwal, A., Kakade, S. M., Lee, J., & Mahajan, G. (2020). Optimality and approximation with policy gradient methods in markov decision processes. Conference on learning theory, Graz, Austria.
   Google Scholar
• Almahamid, F., & Grolinger, K. (2021, 9). Reinforcement learning algorithms: An overview and classification. Canadian Conference on Electrical and Computer Engineering , 2021- September, Toronto, Canada.
   Google Scholar
• Axtell, R. L. (2014). An agent-based model of the housing market bubble in metropolitan. SSRN.
   Google Scholar
• Bae, J. W., Paik, E., Dongoh, K., Jung, J., & Lee, C. H. (2019, 1). Simulation framework for self-evolving agent-based models: A case study of housing market model. Proceedings - Winter Simulation Conference, Gothenburg, Sweden, 2018–December, (pp. 1120–1131).
   Google Scholar
• Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., & Mordatch, I. (2019). Emergent tool use from multi-agent autocurricula. arXiv. https://doi.org/10.48550/arXiv.1909.07528
   Google Scholar
• Baptista, R., Farmer, J. D., Hinterschweiger, M., Low, K., Tang, D., & Uluc, A. (2016, 10). Macroprudential policy in an agent-based model of the UK Housing Market. SSRN Electronic Journal. https://papers.ssrn.com/abstract=2850414
   Google Scholar
• Barhate, N. (2021). Minimal PyTorch implementation of proximal policy optimization. GitHub. https://github.com/nikhilbarhate99/PPO-PyTorch.
   Google Scholar
• Been, V., Ellen, I., Figlio, D. N., Nelson, A., Ross, S., Schwartz, A. E., & Ellen, A. (2021). The effects of negative equity on children’s educational outcomes. NBER. http://www.nber.org/papers/w28428
   Google Scholar
• Bogdoll, J., Hartmanns, A., & Hermanns, H. (2012). Simulation and statistical model checking for modestly nondeterministic models. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 7201, pp. 249–252). https://link.springer.com/chapter/10.1007/978-3-642-28540-0₂
   Google Scholar
• Cai, W., & Lu, X. (2015). Housing affordability: Beyond the income and price terms, using China as a case study. Habitat International, 47, 169–175. https://doi.org/10.1016/j.habitatint.2015.01.021
   Web of Science ®Google Scholar
• Carstensen, C. L. (2015). An agent-based model of the housing market: Steps toward a computational tool for policy analysis. Analysis & Policy Observation.
   Google Scholar
• Chen, J. H., Ong, C. F., Zheng, L., & Hsu, S. C. (2017, 7). Forecasting spatial dynamics of the housing market using support vector machine. International Journal of Strategic Property Management, 21(3), 273–283. https://doi.org/10.3846/1648715X.2016.1259190
   Web of Science ®Google Scholar
• Chiri¸t˘a, M. (2011). Usefulness of artificial neural networks for predicting financial and economic crisis. Economics and Applied Informatics 2, 61–66. https://ideas.repec.org/a/ddj/fseeai/y2012i2p61-66.html
   Google Scholar
• Cincotti, S., Guerci, E., & Raberto, M. (2005, 5). Price dynamics and market power in an agent-based power exchange. Noise and Fluctuations in Econophysics and Finance, 5848, 233. https://www.researchgate.net/publication/228675480_P rice_dynamics_and_market_power_in_an_agent−based_power_exchange
   Google Scholar
• Dhar, S. S., Chakraborty, B., & Chaudhuri, P. (2014, 8). Comparison of multivariate distributions using quantile–quantile plots and related tests. Bernoulli, 20(3), 1484–1506. https://doi.org/10.3150/13-BEJ530
   Web of Science ®Google Scholar
• Donkin, E., Dennis, P., Ustalakov, A., Warren, J., & Clare, A. (2017, 6). Replicating complex agent based models, a formidable task. Environmental Modelling & Software, 92, 142–151. https://doi.org/10.1016/j.envsoft.2017.01.020
   Web of Science ®Google Scholar
• Dou, X., & Wang, J. (2014, 2). Asset securitization and bubbles: An illustration of subprime mortgage default crisis. Advances in Economics and Business, 2(2), 112–119. http://www.hrpub.org
   Google Scholar
• Epstein, J. M., & Axtell, R. (1997, 3). Artificial societies and generative social science. Artificial Life and Robotics, 1(1), 33–34. https://doi.org/10.1007/BF02471109
   Google Scholar
• Erlingsson, E. J., Teglio, A., Cincotti, S., Stefansson, H., Sturluson, J. T., & Raberto, M. (2014, 12). Housing market bubbles and business cycles in an agent-based credit economy. Economics, 8(1), 2014–2022. https://www.degruyter.com/document/doi/10.5018/economics-ejournal.ja.2014-8/html
   Google Scholar
• Filatova, T. (2015, 11). Empirical agent-based land market: Integrating adaptive economic behavior in urban land-use models. Computers, Environment and Urban Systems, 54, 397–413. https://doi.org/10.1016/j.compenvurbsys.2014.06.007
   Web of Science ®Google Scholar
• Fischer, T., & Riedler, J. (2014, 11). Prices, debt and market structure in an agent-based model of the financial market. Journal of Economic Dynamics and Control, 48, 95–120. https://doi.org/10.1016/j.jedc.2014.08.013
   Web of Science ®Google Scholar
• Ge, J. (2014). Who creates housing bubbles? An agent-based study. In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Vol. 8235, pp. 143–150). https://link.springer.com/chapter/10.1007/978-3-642-54783-6₁
   Google Scholar
• Ge, J. (2017, 3). Endogenous rise and collapse of housing price: An agent-based model of the housing market. Computers, Environment and Urban Systems, 62, 182–198. https://doi.org/10.1016/j.compenvurbsys.2016.11.005
   Web of Science ®Google Scholar
• Geanakoplos, J., Axtell, R., Farmer, J. D., Howitt, P., Conlee, B., Goldstein, J. & Yang, C. Y. (2012). Getting at systemic risk via an agent-based model of the housing market. American Economic Review, 102(3), 53–58. https://doi.org/10.1257/aer.102.3.53
   Web of Science ®Google Scholar
• Gilbert, N., Hawksworth, J. C., & Swinney, P. A. (2009). An agent-based model of the english housing market. In Technosocial predictive analytics, papers from the 2009 aaai springsymposium, technical report ss\-09\-09 (pp. 30–35). AAAI. http://www.aaai.org/Library/Symposia/Spring/2009/ss09-09-007.php
   Google Scholar
• Groff Elizabeth, R., Johnson, S. D., & Amy, T. (2018, 2). State of the art in agent-based modeling of urban crime: An overview. Journal of Quantitative Criminology, 35(1), 155–193. https://doi.org/10.1007/s10940-018-9376-y
   Web of Science ®Google Scholar
• Hamill, L., & Gilbert, N. (2015). Agent‐Based Modelling in Economics (1st ed.). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781118945520
   Google Scholar
• Heppenstall, A., Evans, A., & Birkin, M. (2006). Using hybrid agent-based systems to model spatially-influenced retail markets. JASSS, 9(3). https://www.jasss.org/9/3/2.html
   Google Scholar
• Ho, N. (2020 8). How AI can help build resiliency for small businesses in a global economic crisis. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 3606–3606).
   Google Scholar
• Housing Team. (2022). House price to workplace-based earnings ratio office for national statistics. https://www.ons.gov.uk/peoplepopulationandcommunity/housing/datasets/ratioofhousepricetoworkp
   Google Scholar
• Imam, I., & Kerschberg, L. (1997). Adaptive intelligent agents. Journal of Intelligent Information Systems, 9(3), 211–213. https://link.springer.com/article/10.1023/A:1008672326807
   Google Scholar
• Inman, P. (2022). What is the Bank of England doing in bid to stabilise UK economy? Bank of England. The Guardian. https://www.theguardian.com/business/2022/sep/28/what-bank-of-england-doing-pound-dollar-uk-e
   Google Scholar
• Jalalimanesh, A., Shahabi Haghighi, H., Ahmadi, A., & Soltani, M. (2017). Simulation-based optimization of radiotherapy: Agent-based modeling and reinforcement learning. Mathematics and Computers in Simulation, 133, 235–248. https://doi.org/10.1016/j.matcom.2016.05.008
   Web of Science ®Google Scholar
• Jordan, R., Birkin, M., & Evans, A. (2011). Agent-based simulation modelling of housing choice and urban regeneration policy. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 6532. pp. 152–166). https://link.springer.com/chapter/10.1007/978-3-642-18345-4₁
   Google Scholar
• Jordan, R., Birkin, M., & Evans, A. (2012, 1). Agent-based modelling of residential mobility, housing choice and regeneration. Agent-Based Models of Geographical Systems, 511–524. https://doi.org/10.1007/978-90-481-8927-42
   Google Scholar
• Juliani, A., Berges, V.-P., Vckay, E., Gao, Y., Henry, H., Mattar, M., & Lange, D. (2018). Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627.
   Google Scholar
• Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237–285. https://doi.org/10.1613/jair.301
   Web of Science ®Google Scholar
• Kang, D. O., Bae, J. W., Lee, C., Jung, J. Y., & Paik, E. (2019, 1). Data assimilation technique for social agent-based simulation by using reinforcement learning. Proceedings of the 2018 IEEE/ACM 22nd International Symposium on Distributed Simulation and Real Time Applications, DS-RT 2018, Madrid, Spain (pp. 220–221).
   Google Scholar
• Kothari, V., Blythe, J., Smith, S., & Koppel, R. (2014). Agent-based modeling of user circumvention of security. ACM international conference proceeding series, Sanibel Island Florida USA.
   Google Scholar
• Lelei, D. E. K., & McCalla, G. (2019). How many times should a pedagogical agent simulation model be run? In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Vol. 11625, pp. 182–193). Springer. https://link.springer.com/chapter/10.1007/978-3-030-23204-71
   Google Scholar
• Littman, M. L. (2015). Reinforcement learning improves behaviour from evaluative feedback. Nature, 521(7553), 445–451. https://www.nature.com/articles/nature14540
   PubMed Web of Science ®Google Scholar
• Liu, S., See, K. C., Ngiam, K. Y., Celi, L. A., Sun, X., & Feng, M. (2020, 7). Reinforcement learning for clinical decision support in critical care: comprehensive review. Journal of Medical Internet Research, 22(7), e18477. https://www.jmir.org/2020/7/e18477
   PubMed Web of Science ®Google Scholar
• Loukis, E., Kyriakou, N., & Maragoudakis, M. (2020). Using government data and machine learning for predicting firms’ vulnerability to economic crisis. In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Vol. 12219, pp. 345–358). Springer. https://link.springer.com/chapter/10.1007/978-3-030-57599-12
   Google Scholar
• Luhnow, D., & Colchester, M. (2022). U.K.’s central banker struggles with inflation, a financial crisis and his own government - WSJ. https://www.wsj.com/articles/u-k-central-banker-andrew-bailey-inflation-financial-crisis-1166
   Google Scholar
• Lynch, J., & Adam, K. (2022). Bank of England intervenes to stabilize UK finances after Liz Truss budget. The Washington Post. https://www.washingtonpost.com/world/2022/09/28/boe-uk-pound-intervention/
   Google Scholar
• Maghdid, H. S., & Ghafoor, K. Z. (2020, 9). A smartphone enabled approach to manage COVID-19 lockdown and economic crisis. SN Computer Science, 1(5), 1–9. https://link.springer.com/article/10.1007/s42979-020-00290-0
   Google Scholar
• Maliene, V., Howe, J., & Malys, N. (2008). Sustainable communities: Affordable housing and socio-economic relations. Local Economy: The Journal of the Local Economy Policy Unit, 23(4), 267–276. https://doi.org/10.1080/02690940802407989
   Google Scholar
• Malyshev, N. A. (2015, 12). The importance of regulatory policy. SSRN Electronic Journal. https://papers.ssrn.com/abstract=3323598
   Google Scholar
• Martin, C., Schmitt, N., & Westerhoff, F. (2022, 3). Housing markets, expectation formation and interest rates. Macroeconomic Dynamics, 26(2), 491–532. https://www.cambridge.org/core/journals/macroeconomic-dynamics/article/abs/housing-markets-ex
   Web of Science ®Google Scholar
• Mehta, D. (2020). State-of-the-art reinforcement learning algorithms. International Journal of Engineering Research Technology, 8, 717–722. https://doi.org/10.17577/IJERTV8IS120332
   Google Scholar
• Melzer, B. T. (2010). Mortgage debt overhang: Reduced investment by homeowners with negative equity. Journal of Finance, 72(2), 575–612. https://doi.org/10.1111/jofi.12482
   Google Scholar
• Miller, H. J. (2004, 6). Tobler’s first law and spatial analysis. Annals of the Association of American Geographers, 94(2), 284–289. https://doi.org/10.1111/j.1467-8306.2004.09402005.x
   Web of Science ®Google Scholar
• Mnih, V., Badia, A. P., Mirza, L., Graves, A., Harley, T., Lillicrap, T. P., & Kavukcuoglu, K. (2016). Asynchronous methods for deep reinforcement learning. 33rd international conference on machine learning, ICML 2016, New York, USA (pp. 1–19).
   Google Scholar
• Morescalchi, A., van Veldhuizen, S., Voogt, B., & Vogt, B. (2018). Negative home equity and job mobility. Data-driven policy impact evaluation: how access to microdata is transforming policy design, 183–202. https://link.springer.com/chapter/10.1007/978-3-319-78461-8₁
   Google Scholar
• Munro, M. (2018, 10). House price inflation in the news: A critical discourse analysis of newspaper coverage in the UK. Housing Studies, 33(7), 1085–1105. https://doi.org/10.1080/02673037.2017.1421911
   Web of Science ®Google Scholar
• Narayan, P. K., & Narayan, S. (2011, 5). The importance of real and nominal shocks on the UK Housing Market. SSRN Electronic Journal. https://papers.ssrn.com/abstract=2052122
   Google Scholar
• Nik, P. A., Jusoh, M. A., Shaari, A. H., & Sarmdi, T. (2016). Predicting the probability of financial crisis in emerging countries using an early warning system: Artificial neural network. Journal of Economic Cooperation and Development, 37, 25–40. https://api.semanticscholar.org/CorpusID:157881366
   Google Scholar
• Norwegian, T. (2007). A heterodox economic analysis of the housing market structure in budapest using neural network classification. Journal of Real Estate Literature, 15(1). https://doi.org/10.1080/10835547.2006.12090194
   Google Scholar
• Oguibenine, B. (2011). Economic recession and mental health: An overview. Neuropsychiatrie: Klinik, Diagnostik, Therapie und Rehabilitation: Organ der Gesellschaft Osterreichischer Nervenarzte und Psychiater, 25(3), 113–117.
   PubMedGoogle Scholar
• Oldford, R. W. (2016, 1). Self-calibrating quantile–quantile plots. American Statistician, 70(1), 74–90. https://doi.org/10.1080/00031305.2015.1090338
   Web of Science ®Google Scholar
• Olmez, S. (2022, 3). SedarOlmez94/pythonic UK housing market ABM 2022: UK Housing Market Model 2022. Zenodo. https://zenodo.org/record/6362146
   Google Scholar
• Olmez, S., Birks, D., & Heppenstall, A. (2022). Learning complex spatial behaviours in ABM: an experimental observational study. arXiv. https://arxiv.org/abs/2201.01099v1
   Google Scholar
• Olmez, S., Thompson, J., Marfleet, E., Suchak, K., Heppenstall, A., Manley, E., & Vidanaarachchi, R. (2022). An agent-based model of heterogeneous driver behaviour and its impact on energy consumption and costs in urban space. Energies, 15(11), 4031. https://doi.org/10.3390/en15114031
   Web of Science ®Google Scholar
• Park, D., & Ryu, D. (2021). A machine learning-based early warning system for the housing and stock markets. Institute of Electrical and Electronics Engineers Access, 9, 85566–85572. https://doi.org/10.1109/ACCESS.2021.3077962
   Google Scholar
• Picascia, S., Camarca, A., Picascia, S., DiLuccia, A., & Gianfrani, C. (2014). Cereal-based gluten-free food: How to reconcile nutritional and technological properties of wheat proteins with safety for celiac disease patients. Nutrients, 6(2), 575–590. https://doi.org/10.3390/nu6020575
   PubMed Web of Science ®Google Scholar
• Popescu, I. V. (2014, 1). Analysis of the behavior of central banks in setting interest rates. The case of central and eastern european countries. Procedia Economics and Finance, 15, 1113–1121. https://doi.org/10.1016/S2212-5671(14)00565-6
   Google Scholar
• Ramey, V. A. (2016). Macroeconomic Shocks and Their Propagation. Handbook of Macroeconomics, 2, 71–162. https://doi.org/10.1016/bs.hesmac.2016.03.003
   Google Scholar
• Rosenfield, A., Chingcuanco, F., & Miller, E. J. (2013, 1). Agent-based housing market microsimulation for integrated land use, transportation, environment model system. Procedia Computer Science, 19, 841–846. https://doi.org/10.1016/j.procs.2013.06.112
   Google Scholar
• Sali, R., Adewole, S., & Akakpo, A. (2021). Feature selection using reinforcement learning. arXiv. https://doi.org/10.48550/arXiv.2101.09460
   Google Scholar
• Schulze, J., Müller, B., Groeneveld, J., & Grimm, V. (2017). Agent-based modelling of social-ecological systems: Achievements, challenges, and a way forward. Journal of Artificial Societies and Social Simulation, 20(2). https://doi.org/10.18564/jasss.3423
   Web of Science ®Google Scholar
• Secchi, D. (2015, 3). A case for agent-based models in organizational behavior and team research. Team Performance Management, 21(1/2), 37–50. https://doi.org/10.1108/TPM-12-2014-0063
   Web of Science ®Google Scholar
• Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014 6). Deterministic policy gradient algorithms. Proceedings of the 31st international conference on machine learning (Vol. 32. pp. 387–395). PMLR, Bejing, China. https://proceedings.mlr.press/v32/silver14.html. In E. P. Xing & T. Jebara (Eds.).
   Google Scholar
• Sledge, I. J., & Principe, J. C. (2017 6). Balancing exploration and exploitation in reinforcement learning using a value of information criterion. ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing -Proceedings, New Orleans, USA (pp. 2816–2820).
   Google Scholar
• Spatharis, C., Blekas, K., Bastas, A., Kravaris, T., & Vouros, G. A. (2019). Collaborative multiagent reinforcement learning schemes for air traffic management. 10th international conference on information, intelligence, systems and applications, IISA 2019, Patras, Greece.
   Google Scholar
• Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press.
   Google Scholar
• Suzuki, M., Nakatani, R., & Nishikawa, I. (2014). A mechanism design of solar power trading by autonomous agent based on reinforcement learning. Frontiers in Artificial Intelligence and Applications, 262, 392–401. https://ebooks.iospress.nl/doi/10.3233/978-1-61499-405-3-392
   Google Scholar
• Tang, W., & Bennett, D. A. (2010). Agent-based modeling of animal movement: A review. Geography Compass, 4(7), 682–700. https://doi.org/10.1111/j.1749-8198.2010.00337.x
   Google Scholar
• Todd A., Beling P., Scherer W., & Yang S. Y. (2017, 2). Agent-based financial markets: A review of the methodology and domain. 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016.
   Google Scholar
• Tse, C. B., Rodgers, T., & Niklewski, J. (2014, 2). The 2007 financial crisis and the UK residential housing market: Did the relationship between interest rates and house prices change? Economic Modelling, 37, 518–530. https://doi.org/10.1016/j.econmod.2013.08.013
   Web of Science ®Google Scholar
• Valadkhani, A., Nguyen, J., & O’Brien, M. (2019). Asymmetric responses of house prices to changes in the mortgage interest rate: Evidence from the Australian capital cities. Applied Economics, 51(53), 5781–5792. https://www.tandfonline.com/doi/abs/10.1080/00036846.2019.1619026
   Web of Science ®Google Scholar
• Westerhoff, F. (2010). A simple agent-based financial market model: Direct interactions and comparisons of trading profits. Nonlinear Dynamics in Economics, Finance and Social Sciences: Essays in Honour of John Barkley Rosser Jr, 313–332. https://link.springer.com/chapter/10.1007/978-3-642-04023-8₁
   Google Scholar
• White, M. (2015, 5). Cyclical and structural change in the UK housing market. Journal of European Real Estate Research, 8(1), 85–103. https://doi.org/10.1108/JERER-02-2014-0011
   Google Scholar
• Whitehead, C., & Williams, P. (2011, 10). Causes and consequences? Exploring the shape and direction of the housing system in the UK post the financial crisis. Housing Studies, 26(7–8), 1157–1169. https://doi.org/10.1080/02673037.2011.618974
   Web of Science ®Google Scholar
• Yamaguchi, S., Naoki, H., Ikeda, M., Tsukada, Y., Nakano, S., Mori, I., Ishii, S., & Brown, A. (2018, 5). Identification of animal behavioral strategies by inverse reinforcement learning. PLOS Computational Biology, 14(5), e1006122. https://doi.org/10.1371/journal.pcbi.1006122
   PubMed Web of Science ®Google Scholar
• Yingfei, X. (2009). A language-based approach to model synchronization in software engineering.
   Google Scholar
• Yun, T. S., & Moon, I. C. (2020). Housing market agent-based simulation with loan-to-value and debt-to-income. Journal of Artificial Societies and Social Simulation, 23(4), 1–19. https://www.jasss.org/23/4/5.html
   PubMed Web of Science ®Google Scholar
• Zhan, C., Wu, Z., Liu, Y., Xie, Z., & Chen, W. (2020, 7). Housing prices prediction with deep learning: An application for the real estate market in Taiwan. IEEE International Conference on Industrial Informatics (INDIN), Beijing, China, 2020–July, (pp.719–724).
   Google Scholar
• Zhou, X., Tong, W., & Li, D. (2019). Modeling housing rent in the atlanta metropolitan area using textual information and deep learning. ISPRS International Journal of Geo-Information, 8(8), 349. https://www.mdpi.com/2220-9964/8/8/349
   Web of Science ®Google Scholar
• Zhou, Y., Wu, J., Long, C., Cheng, M., & Zhang, C. (2017, 12). Performance evaluation of peer-to-peer energy sharing models. Energy Procedia, 143, 817–822. https://doi.org/10.1016/j.egypro.2017.12.768
   Google Scholar
• Zhu, B., Betzinger, M., & Sebastian, S. (2017). Housing market stability, mortgage market structure, and monetary policy: Evidence from the euro area. Journal of Housing Economics, 37, 1–21. https://doi.org/10.1016/j.jhe.2017.04.001
   Web of Science ®Google Scholar