Forecasting emergent risks in advanced AI systems: an analysis of a future road transport management system

References

• Abduljabbar, R., H. Dia, S. Liyanage, and S. A. Bagloee. 2019. “Applications of Artificial Intelligence in Transport: An Overview.” Sustainability 11 (1): 189. doi:10.3390/su11010189.
   Web of Science ®Google Scholar
• Banks, V.A., N.A. Stanton, G. Burnett, and S. Hermawati. 2018. “Distributed Cognition on the Road: Using EAST to Explore Future Road Transportation Systems.” Applied Ergonomics 68: 258–266. doi:10.1016/j.apergo.2017.11.013.
   PubMed Web of Science ®Google Scholar
• Barrett, A. M., and S. D. Baum. 2017. “A Model of Pathways to Artificial Superintelligence Catastrophe for Risk and Decision Analysis.” Journal of Experimental & Theoretical Artificial Intelligence 29 (2): 397–414. doi:10.1080/0952813X.2016.1186228.
   Web of Science ®Google Scholar
• Baum, S. 2017. “A Survey of Artificial General Intelligence Projects for Ethics, Risk, and Policy.”. Global Catastrophic Risk Institute Working Paper, 17–11.
   Google Scholar
• Baum, S. D., B. Goertzel, and T. Goertzel. 2011. “How Long Until Human-Level AI? Results from an Expert Assessment.” Technological Forecasting and Social Change 78 (1): 185–195. doi:10.1016/j.techfore.2010.09.006.
   Web of Science ®Google Scholar
• Bostrom, N. 2003. Superintelligence: Paths, Dangers, Strategies. UK: Oxford University Press.
   Google Scholar
• Bostrom, N. 2012. “The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents.” Minds and Machines 22 (2): 71–85. doi:10.1007/s11023-012-9281-3.
   Web of Science ®Google Scholar
• Bostrom, N. 2014. Superintelligence: Paths, Dangers, Strategies: UK: Oxford University Press Inc.
   Google Scholar
• Bostrom, N., and E. Yudkowsky. 2018. “The Ethics of Artificial Intelligence.” In Artificial Intelligence Safety and Security, 57–69. Chapman and Hall: CRC.
   Google Scholar
• Bubeck, S., V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, … Y. Zhang. 2023. “Sparks of Artificial General Intelligence: Early Experiments with Gpt-4.” arXiv preprint arXiv:2303.12712.
   Google Scholar
• Bura, H., N. Lin, N. Kumar, S. Malekar, S. Nagaraj, and K. Liu. 2018. “An Edge Based Smart Parking Solution Using Camera Networks and Deep Learning.” In 2018 IEEE International Conference on Cognitive Computing (ICCC), San Francisco. doi:10.1109/ICCC.2018.00010.
   Google Scholar
• Dallat, C., P. M. Salmon, and N. Goode. 2018. “Identifying Risks and Emergent Risks across Sociotechnical Systems: The NETworked Hazard Analysis and Risk Management System (NET-HARMS).” Theoretical Issues in Ergonomics Science 19 (4): 456–482. doi:10.1080/1463922X.2017.1381197.
   Web of Science ®Google Scholar
• Davis, E., and G. Marcus. 2015. “Commonsense Reasoning and Commonsense Knowledge in Artificial Intelligence.” Communications of the ACM 58 (9): 92–103. doi:10.1145/2701413.
   Web of Science ®Google Scholar
• de Zwart, F. 2015. “Unintended but Not Unanticipated Consequences.” Theory and Society 44 (3): 283–297. doi:10.1007/s11186-015-9247-6.
   Web of Science ®Google Scholar
• Dekker, S. W., and D. D. Woods. 1999. “To Intervene or Not to Intervene: The Dilemma of Management by Exception.” Cognition, Technology & Work 1 (2): 86–96. doi:10.1007/s101110050035.
   Google Scholar
• Duffy, V. G., S. J. Landry, J. D. Lee, and N. A. Stanton. 2023. Human-Automation Interaction: Transportation. Berlin: Springer Verlag.
   Google Scholar
• Firt, E. 2020. “The Missing G.” AI & SOCIETY 35 (4): 995–1007. doi:10.1007/s00146-020-00942-y.
   Web of Science ®Google Scholar
• Furlan, Andrea D., Tara Kajaks, Margaret Tiong, Martin Lavallière, Jennifer L. Campos, Jessica Babineau, Shabnam Haghzare, Tracey Ma, and Brenda Vrkljan. 2020. “Advanced Vehicle Technologies and Road Safety: A Scoping Review of the Evidence.” Accident; Analysis and Prevention 147: 105741. doi:10.1016/j.aap.2020.105741.
   PubMed Web of Science ®Google Scholar
• Glikson, E., and A. W. Woolley. 2020. “Human Trust in Artificial Intelligence: Review of Empirical Research.” Academy of Management Annals 14 (2): 627–660. doi:10.5465/annals.2018.0057.
   Web of Science ®Google Scholar
• Goertzel, B. 2014. “Artificial General Intelligence: concept, State of the Art, and Future Prospects.” Journal of Artificial General Intelligence 5 (1): 1–48. doi:10.2478/jagi-2014-0001.
   Google Scholar
• Goertzel, B., and C. Pennachin. 2007. Artificial General Intelligence. Springer.
   Google Scholar
• Goertzel, B., and J. Pitt. 2014. “Nine Ways to Bias Open‐Source Artificial General Intelligence toward Friendliness.” In Intelligence Unbound: Future of Uploaded Machine Minds, 61–89. John Wiley & Sons.
   Google Scholar
• Goertzel, B., C. Pennachin, and N. Geisweiller. 2014. “Advanced Self-Modification: A Possible Path to Superhuman AGI.” In Engineering General Intelligence, Part 1: A Path to Advanced AGI via Embodied Learning Cognitive Synergy, 365–373. Springer.
   Google Scholar
• Hamidi, H., and A. Kamankesh. 2018. “An Approach to Intelligent Traffic Management System Using a Multi-Agent System.” International Journal of Intelligent Transportation Systems Research 16 (2): 112–124. doi:10.1007/s13177-017-0142-6.
   Web of Science ®Google Scholar
• Hancock, P. A. 2019. “Some Pitfalls in the Promises of Automated and Autonomous Vehicles.” Ergonomics 62 (4): 479–495. doi:10.1080/00140139.2018.1498136.
   PubMed Web of Science ®Google Scholar
• Hancock, P. A. 2022. “Avoiding Adverse Autonomous Agent Actions.” Human–Computer Interaction 37 (3): 211–236. doi:10.1080/07370024.2021.1970556.
   Web of Science ®Google Scholar
• Lake, B. M., T. D. Ullman, J. B. Tenenbaum, and S. J. Gershman. 2017. “Building Machines That Learn and Think like People.” The Behavioral and Brain Sciences 40: e253. doi:10.1017/S0140525X16001837.
   PubMed Web of Science ®Google Scholar
• Lane, B.R., P.M. Salmon, A. Cherney, D. Lacey, and N.A. Stanton. 2019. “Using the Event Analysis of Systemic Teamwork (EAST) Broken-Links Approach to Understand Vulnerabilities to Disruption in a Darknet Market.” Ergonomics 62 (9): 1134–1149. doi:10.1080/00140139.2019.1621392.
   PubMed Web of Science ®Google Scholar
• Lazer, David M J., Matthew A. Baum, Yochai Benkler, Adam J. Berinsky, Kelly M. Greenhill, Filippo Menczer, Miriam J. Metzger, Brendan Nyhan, Gordon Pennycook, David Rothschild, Michael Schudson, Steven A. Sloman, Cass R. Sunstein, Emily A. Thorson, Duncan J. Watts, and Jonathan L. Zittrain. 2018. “The Science of Fake News.” Science 359 (6380): 1094–1096. doi:10.1126/science.aao2998.
   PubMed Web of Science ®Google Scholar
• Legg, S., and M. Hutter. 2006. “A Formal Measure of Machine Intelligence.” In Proceedings of 15th Annual Machine Learning Conference of Belgium and The Netherlands (Benelearn’06).
   Google Scholar
• Leveson, N. 2004. “A New Accident Model for Engineering Safer Systems.” Safety Science 42 (4): 237–270. doi:10.1016/S0925-7535(03)00047-X.
   Web of Science ®Google Scholar
• Liu, N., A. Nikitas, and S. Parkinson. 2020. “Exploring Expert Perceptions about the Cyber Security and Privacy of Connected and Autonomous Vehicles: A Thematic Analysis Approach.” Transportation Research Part F: Traffic Psychology and Behaviour 75, 66–86.
   Web of Science ®Google Scholar
• Liu, Z., Y. Bi, and P. Liu. 2022. “An Evidence Theory-Based Large Group FMEA Framework Incorporating Bounded Confidence and Its Application in Supercritical Water Gasification System.” Applied Soft Computing 129: 109580. doi:10.1016/j.asoc.2022.109580.
   Web of Science ®Google Scholar
• Martinho, A., N. Herber, M. Kroesen, and C. Chorus. 2021. “Ethical Issues in Focus by the Autonomous Vehicles Industry.” Transport Reviews 41 (5): 556–577. doi:10.1080/01441647.2020.1862355.
   Web of Science ®Google Scholar
• McLean, S., G. J. Read, J. Thompson, P. A. Hancock, and P. M. Salmon. 2022. “Who is in Control? Managerial Artificial General Intelligence (MAGI) for Football.” Soccer & Society 23 (1): 104–109. doi:10.1080/14660970.2021.1956477.
   Web of Science ®Google Scholar
• McLean, S., G. J. Read, K. Ramsay, L. Hogarth, and B. Kean. 2021. “Designing Success: Applying Cognitive Work Analysis to Optimise a Para Sport System.” Applied Ergonomics 93: 103369. doi:10.1016/j.apergo.2021.103369.
   PubMed Web of Science ®Google Scholar
• Merton, R. K. 1936. “The Unanticipated Consequences of Purposive Social Action.” American Sociological Review 1 (6): 894–904. doi:10.2307/2084615.
   Google Scholar
• Miller, M. J., and K. M. Feigh. 2019. “Addressing the Envisioned World Problem: A Case Study in Human Spaceflight Operations.” Design Science 5: e3. doi:10.1017/dsj.2019.2.
   Web of Science ®Google Scholar
• Müller, V. C., and N. Bostrom. 2016. “Future Progress in Artificial Intelligence: A Survey of Expert Opinion.” Fundamental Issues of Artificial Intelligence, 555–572. Springer.
   Google Scholar
• Murino, T., M. D. Nardo, D. Pollastro, N. Berx, A. D. Francia, W. Decré, J. Philips, and L. Pintelon. 2023. “Exploring a Cobot Risk Assessment Approach Combining FMEA and PRAT.” Quality and Reliability Engineering International 39 (3): 706–731. doi:10.1002/qre.3252.
   Web of Science ®Google Scholar
• OpenAI 2023. GTP-4 System Card. OpenAI. https://openai.com/
   Google Scholar
• Pöllänen, E., G. J. Read, B. R. Lane, J. Thompson, and P. M. Salmon. 2020. “Who is to Blame for Crashes Involving Autonomous Vehicles? Exploring Blame Attribution across the Road Transport System.” Ergonomics 63 (5): 525–537. doi:10.1080/00140139.2020.1744064.
   PubMed Web of Science ®Google Scholar
• Read, G. J., A. O'Brien, N. A. Stanton, and P. M. Salmon. 2022. “Learning Lessons for Automated Vehicle Design: Using Systems Thinking to Analyse and Compare Automation-Related Accidents across Transport Domains.” Safety Science 153: 105822. doi:10.1016/j.ssci.2022.105822.
   Web of Science ®Google Scholar
• Read, G. J. M., S. McLean, J. Thompson, N. A. Stanton, C. Baber, T. Carden, and P. M. Salmon. 2023. “Managing the Risks Associated with Technological Disruption in the Road Transport System: A Control Structure Modelling Approach.” Ergonomics 1–17. doi:10.1080/00140139.2023.2226850.
   Web of Science ®Google Scholar
• Salmon, P. M., and G. J. Read. 2019. “Many Model Thinking in Systems Ergonomics: A Case Study in Road Safety.” Ergonomics 62 (5): 612–628. doi:10.1080/00140139.2018.1550214.
   PubMed Web of Science ®Google Scholar
• Salmon, P. M., and K. L. Plant. 2022. “Distributed Situation Awareness: From Awareness in Individuals and Teams to the Awareness of Technologies, Sociotechnical Systems, and Societies.” Applied Ergonomics 98: 103599. doi:10.1016/j.apergo.2021.103599.
   PubMed Web of Science ®Google Scholar
• Salmon, P. M., G. J. M. Read, G. H. Walker, M. G. Lenné, and N. A. Stanton. 2018. Distributed Situation Awareness in Road Transport: theory, Measurement, and Application to Intersection Design. Routledge.
   Google Scholar
• Salmon, P. M., N. A. Stanton, and D. P. Jenkins. 2009. Distributed Situation Awareness: Theory, Measurement and Application to Teamwork. Ashgate Publishing, Ltd.
   Google Scholar
• Salmon, P. M., N. A. Stanton, G. H. Walker, A. Hulme, N. Goode, J. Thompson, and G. J. Read. 2022. Handbook of Systems Thinking Methods: CRC Press.
   Google Scholar
• Salmon, P. M., T. Carden, and P. A. Hancock. 2021. “Putting the Humanity into Inhuman Systems: How Human Factors and Ergonomics Can Be Used to Manage the Risks Associated with Artificial General Intelligence.” Human Factors and Ergonomics in Manufacturing & Service Industries 31 (2): 223–236. doi:10.1002/hfm.20883.
   Web of Science ®Google Scholar
• Salmon, Paul M., Chris Baber, Catherine Burns, Tony Carden, Nancy Cooke, Missy Cummings, Peter Hancock, Scott McLean, Gemma J. M. Read, and Neville A. Stanton. 2023. “Managing the Risks of Artificial General Intelligence: A Human Factors and Ergonomics Perspective.” Human Factors and Ergonomics in Manufacturing & Service Industries 33 (5): 366–378. doi:10.1002/hfm.20996.
   Web of Science ®Google Scholar
• Salmon, Paul M., Gemma J M. Read, Vanessa Beanland, Jason Thompson, Ashleigh J. Filtness, Adam Hulme, Rod McClure, and Ian Johnston. 2019. “Bad Behaviour or Societal Failure? Perceptions of the Factors Contributing to Drivers’ Engagement in the Fatal Five Driving Behaviours.” Applied Ergonomics 74: 162–171. doi:10.1016/j.apergo.2018.08.008.
   PubMed Web of Science ®Google Scholar
• Salmon, P. M., M. G. Lenne, G. H. Walker, N. A. Stanton, and A. Filtness. 2014. “Using the Event Analysis of Systemic Teamwork (EAST) to Explore Conflicts between Different Road User Groups When Making Right Hand Turns at Urban Intersections.” Ergonomics 57 (11): 1628–1642. doi:10.1080/00140139.2014.945491.
   PubMed Web of Science ®Google Scholar
• Simsekler, M. C. E., G. K. Kaya, J. R. Ward, and P. J. Clarkson. 2019. “Evaluating Inputs of Failure Modes and Effects Analysis in Identifying Patient Safety Risks.” International Journal of Health Care Quality Assurance 32 (1): 191–207. doi:10.1108/IJHCQA-12-2017-0233.
   PubMed Web of Science ®Google Scholar
• Stanton, N. A. 2014. “Representing Distributed Cognition in Complex Systems: How a Submarine Returns to Periscope Depth.” Ergonomics 57 (3): 403–418. doi:10.1080/00140139.2013.772244.
   PubMed Web of Science ®Google Scholar
• Stanton, N. A. 2021. “Advances in Human Aspects of Transportation.” In Proceedings of the AHFE 2021 Virtual Conference on Human Aspects of Transportation, Vol. 270, July 25-29, Springer Verlag, Berlin.
   Google Scholar
• Stanton, N. A., and C. Harvey. 2017. “Beyond Human Error Taxonomies in Assessment of Risk in Sociotechnical Systems: A New Paradigm with the EAST ‘Broken-Links’ Approach.” Ergonomics 60 (2): 221–233. doi:10.1080/00140139.2016.1232841.
   PubMed Web of Science ®Google Scholar
• Stanton, N. A., K. M. A. Revell, and P. Langdon. 2021. Designing Interaction and Interfaces for Automated Vehicles: User-Centred Ecological Interface Design and Testing. Boca Raton: CRC Press.
   Google Scholar
• Stanton, N. A., R. Stewart, D. Harris, R. J. Houghton, C. Baber, R. McMaster, P. Salmon, G. Hoyle, G. Walker, M. S. Young, M. Linsell, R. Dymott, and D. Green. 2006. “Distributed Situation Awareness in Dynamic Systems: theoretical Development and Application of an Ergonomics Methodology.” Ergonomics 49 (12-13): 1288–1311. doi:10.1080/00140130600612762.
   PubMed Web of Science ®Google Scholar
• Stanton, N., P. Salmon, and G. Walker. 2018. Systems Thinking in Practice: applications of the Event Analysis of Systemic Teamwork Method: CRC Press.
   Google Scholar
• Tegmark, M. 2018. Life 3.0: Being Human in the Age of Artificial Intelligence. Penguin Books Limited.
   Google Scholar
• Thompson, J., G. J. Read, J. S. Wijnands, and P. M. Salmon. 2020. “The Perils of Perfect Performance; considering the Effects of Introducing Autonomous Vehicles on Rates of Car vs Cyclist Conflict.” Ergonomics 63 (8): 981–996. doi:10.1080/00140139.2020.1739326.
   PubMed Web of Science ®Google Scholar
• Torbaghan, M. E., M. Sasidharan, L. Reardon, and L. C. Muchanga-Hvelplund. 2022. “Understanding the Potential of Emerging Digital Technologies for Improving Road Safety.” Accident; Analysis and Prevention 166: 106543. doi:10.1016/j.aap.2021.106543.
   PubMed Web of Science ®Google Scholar
• Wooldridge, A. R., J. Morgan, W. A. Ramadhani, K. Hanson, E. Vazquez-Melendez, H. Kendhari, N. Shaikh, T. Riech, M. Mischler, S. Krzyzaniak, G. Barton, K. T. Formella, Z. R. Abbott, J. N. Farmer, R. Ebert-Allen, and T. Croland. 2022. “Interactions in Sociotechnical Systems: Achieving Balance in the Use of an Augmented Reality Mobile Application.” Human Factors 0 (0): 187208221093830. doi:10.1177/00187208221093830.
   Web of Science ®Google Scholar
• World Health Organisation. 2018. https://www.who.int/publications/i/item/9789241565684
   Google Scholar
• Young, M. S., and N. A. Stanton. 2023. Driving Automation: A Human Factors Perspective. Boca Raton: CRC Press.
   Google Scholar