Navigating the governance challenges of disruptive technologies: insights from regulation of autonomous systems in Singapore

Figures & data

Table 1. Dimensions and explanations of governance challenges of autonomous systems.

Dimensions of governance challenges	Explanation	Evidence/Adapted from
Operational challenges	Technological risks with the potential for negative consequences in the operation of the systems. These include the challenge of safety, unpredictability, and opacity in operation; problems in monitoring, control, and task allocation.	(Grimm, Smart, and Hartzog 2018; Surden and Williams 2016; Mekki-Mokhtar et al. 2012; Sheridan 2016)
Legal challenges	Lack of legal instruments to protect privacy, assigning liability and responsibility in case of harm or damage, and managing cybersecurity issues.	(Eugensson et al. 2013; Tan, Taeihagh, and Pande 2023; Calo 2017; Such 2017)
Economic challenges	Disruptions in employment through job losses and change in workforce structure in the economy due to re-skilling and potential replacement of existing workers with autonomous systems like drivers for cabs, buses, and private car drivers.	(Tan and Taeihagh 2021; Pettigrew, Fritschi, and Norman 2018; Beede, Powers, and Ingram 2017; Nikitas, Vitel, and Cotet 2021)
Social challenges	Challenges concerning systems interactions with human beings with a lack of trust and natural social relationships or human touch undermine the user’s dignity.	(Tan, Taeihagh, and Tripathi 2021; Lahijanian and Kwiatkowska 2016; Sharkey and Sharkey 2012; Etzioni and Etzioni 2017b)
Ethical challenges	Issues related to moral principles guiding judgements of right or wrong result in ethical dilemmas for operating autonomous systems and impacting individuals or groups in society.	(Tan and Taeihagh 2021; Manning et al. 2023; Arkin 2009; Coeckelbergh 2011)

Table 2. Key regulations for specific autonomous systems according to the governance challenges.

	Type	Dimensions	Key regulations in Singapore according to the challenges
1	Autonomous vehicles	Operational	Unpredictability in operation Requirement of integrity and technical competence of AV developer or operator. Record of contact persons with accountability for the life cycle of AVs Safety Quality management system with full quality assurance system, change management. Hazard analysis and risk assessment clause to ensure a robust process for identifying hazards and compliance with rules. Explanation of risk mitigation in hazardous events by the AV operator. Validation methods to ensure the safe operation of AVs through virtual methods, closed road testing, and public road testing are outlined in the standards. Monitoring, control, and task allocation A human operator may be on board to take over dynamic driving tasks. The AV human machine interface (HMI) allows occupants to operate AV-related functions and services. HMI can happen when the AV system is 1) in autonomous driving mode or unavailable to do autonomous driving, 2) is requesting a transition in control from system to operator, 3) in exceptional driving situations, or 4) needs safe response or action requirements in case of an emergency.
		Legal	Data management and privacy Minimum requirements for the consistency of components with communication protocols, maps, and on data management. Data types and formats in AVs are specified for uses such as data recording, use of data, and information management. Structured encoding of messages is defined in the standards. Cybersecurity standards The TR lists cybersecurity principles and requirements for wireless networks. Third-party cybersecurity assessment is required involving the AV operator/developer and assessor using a framework mentioned in the standards. The TR proposes a cybersecurity interface agreement for the involved parties to clarify the roles and responsibilities of actors. TR 64 has listed categories of cybersecurity requirements to avoid threats.
2	Unmanned aerial vehicles	Operational	Monitoring, control, and task allocation The unmanned aircraft must be in the line of sight of the operator at all times. The controller must follow instructions in the user manual. Safety Permits are required for operation – operator permit/activity permit. Safety measures need to be employed when operating.
		Legal	Data management, privacy, and cybersecurity Ethical practices ensuring personal data protection and establishing clear liability and risk assessment protocols.
3	Industrial robots	Operational	Unpredictability and opacity in design and operation HARA methods have been specified before selecting and designing appropriate safeguard measures. Safety General design requirements and protective measures have been outlined. Verification and validation of safety requirements and protective measures. Robots require an instruction handbook with details and instructional guidelines. Monitoring, task allocation, and human-system interaction Collaborative robot operation measures in a common workspace can be used for predetermined tasks when all protective measures are active and for robots with features for collaborative operation.
		Legal	Data management, privacy, and cybersecurity For mobile robots used for delivery, data interchange occurs between robots, lifts, and doors to execute operations. DDS and location and sensing environment modules have been developed to interchange communication during operations. Safeguards and protective measures need to be applied by improving the capability/reliability of sensors, identification algorithms to be designed in a way that the probability of being correct is increased, can be calculated, and monitored. Validity checks have to be made on decisions that can lead to hazardous situations, with decisions to be verified by principles.
	Personal care robots	Operational	Unpredictability in operation A hazard identification strategy has been listed in the standards to identify hazards for cases. Risk estimation methods along with risk reduction methods have been outlined and are supposed to be designed on a case-by-case basis for events. Limits to operational spaces for personal care robots specifying safety-related speed controls, environmental sensing, object sensing, contact sensing, travel surface sensing, stability control, and safety-related force control. Safety Hazards with robot’s operation are listed in the standards to be countered through safety requirements and an inherently safe design. Verification methods of safety requirements are listed in the standards. Proper markings required with details of the manufacturer and manufacturing of the robot. Required performance levels for selected robot types and robot stopping functions are also listed in the standards. User manual to be provided with the robot. Additional measures might be imposed on robots, for instance, motor vehicle regulations if they operate on public roads.

Grimm, C. M., W. D. Smart, and W. Hartzog. 2018. “An Education Model of Reasonable and Good-Faith Effort for Autonomous Systems.“ In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, 117–121. New Orleans LA USA: ACM. 10.1145/3278721.3278732.

 Google Scholar

Surden, H., and M.A. Williams. 2016. “Technological Opacity, Predictability, and Self-Driving Cars.” Cardozo Law Review 38: 121. doi:10.2139/ssrn.2747491.

 Google Scholar

Mekki-Mokhtar, A., J.P. Blanquart, J. Guiochet, D. Powell, and M. Roy.“Elici- tation of Executable Safety Rules for Critical Autonomous Systems.“ Embedded Real Time Software and Systems (ERTS2012), Feb 2012. Toulouse, France.

 Google Scholar

Sheridan, T. B. 2016. “Human–Robot Interaction: Status and Challenges.” Human Factors 58 (4): 525–532. doi:10.1177/0018720816644364.

 PubMed Web of Science ®Google Scholar

Eugensson, A., M. Brännström, D. Frasher, M. Rothoff, S. Solyom, and A. Robertsson. 2013. “Environmental, Safety Legal and Societal Implications of Autonomous Driving Systems.“ In International Technical Conference on the Enhanced Safety of Vehicles (ESV). Seoul, South Korea. Vol. 334.

 Google Scholar

Tan, S. Y., A. Taeihagh, and D. Pande. 2023. “Data Sharing in Disruptive Technologies: Lessons from Adoption of Autonomous Systems in Singapore.” Policy Design and Practice 1–22. doi:10.1080/25741292.2022.2162247.

 Google Scholar

Calo, R. 2017. “Artificial Intelligence Policy: A Primer and Roadmap.” UCDL Rev 51: 399. doi:10.2139/ssrn.3015350.

 Google Scholar

Such, J. M. 2017“ Privacy and Autonomous Systems.“ Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, 4761–4767.

 Google Scholar

Tan, S. Y., and A. Taeihagh. 2021. “Governing the Adoption of Robotics and Autonomous Systems in Long-Term Care in Singapore.” Policy and Society 40 (2): 211–231.

 Web of Science ®Google Scholar

Pettigrew, S., L. Fritschi, and R. Norman. 2018. “The Potential Implications of Autonomous Vehicles in and Around the Workplace.” International Journal of Environmental Research and Public Health 15 (9): 1876. doi:10.3390/ijerph15091876.

 PubMed Web of Science ®Google Scholar

Beede, D. N., R. Powers, and C. Ingram. 2017. “The Employment Impact of Autonomous Vehicles.“ SSRN. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3022818.

 Google Scholar

Nikitas, A., A.E. Vitel, and C. Cotet. 2021. “Autonomous Vehicles and Employment: An Urban Futures Revolution or Catastrophe?” Cities 114: 103203. doi:10.1016/j.cities.2021.103203.

 Web of Science ®Google Scholar

Tan, S. Y., A. Taeihagh, and A. Tripathi. 2021. “Tensions and Antagonistic Interactions of Risks and Ethics of Using Robotics and Autonomous Systems in Long-Term Care.” Technological Forecasting and Social Change 167: 120686.

 Web of Science ®Google Scholar

Lahijanian, M., and M. Kwiatkowska. 2016. “Social Trust: A Major Challenge for the Future of Autonomous Systems.” The 2016 AAAI Fall Symposium Series: Cross-Disciplinary Challenges for Autonomous Systems Technical Report FS-16-03.

 Google Scholar

Sharkey, A., and N. Sharkey. 2012. “Granny and the Robots: Ethical Issues in Robot Care for the Elderly.” Ethics and Information Technology 14 (1): 27–40. doi:10.1007/s10676-010-9234-6.

 Web of Science ®Google Scholar

Etzioni, A., and O. Etzioni. 2017b. “Incorporating Ethics into Artificial Intelligence.” The Journal of Ethics 21 (4): 403–418. doi:10.1007/s10892-017-9252-2.

 Google Scholar

Manning, L., S. Brewer, P. J. Craigon, J. Frey, A. Gutierrez, N. Jacobs, S. Kanza, S. Munday, J. Sacks, and S. Pearson. 2023. “Reflexive Governance Architectures: Considering the Ethical Implications of Autonomous Technology Adoption in Food Supply Chains.” Trends in Food Science & Technology 133: 114–126. doi:10.1016/j.tifs.2023.01.015.

 Web of Science ®Google Scholar

Arkin, R. 2009. Governing Lethal Behavior in Autonomous Robots. New York: Chapman and Hall/CRC.

 Google Scholar

Coeckelbergh, M. 2011. “Is Ethics of Robotics About Robots? Philosophy of Robotics Beyond Realism and Individualism.” Law, Innovation and Technology 3 (2): 241–250. doi:10.5235/175799611798204950.

 Google Scholar