Navigating the governance challenges of disruptive technologies: insights from regulation of autonomous systems in Singapore

ABSTRACT

The proliferation of autonomous systems like unmanned aerial vehicles, autonomous vehicles and AI-powered industrial and social robots can benefit society significantly, but these systems also present significant governance challenges in operational, legal, economic, social, and ethical dimensions. Singapore’s role as a front-runner in the trial of autonomous systems presents an insightful case to study whether the current provisional regulations address the challenges. With multiple stakeholder involvement in setting provisional regulations, government stewardship is essential for coordinating robust regulation and helping to address complex issues such as ethical dilemmas and social connectedness in governing autonomous systems.

KEYWORDS:

• Autonomous systems
• regulation
• standards
• governance
• Singapore

JEL CLASSIFICATION:

• O00
• O32
• O33
• O38
• I18
• K20
• L50
• L88
• L98

1. Introduction

The applications of artificial intelligence (AI) have seen a proliferation in the fourth industrial revolution. A key utilisation of AI has been for autonomous systems like robots that impersonate human cognition and activities (Calo 2017; Russel and Norvig 1994), and operate without human intervention, such as autonomous vehicles (AVs), unmanned aerial vehicles, robots, unmanned ships, autonomous weapons, and exoskeletons. Autonomous systems are defined as powered physical systems that have the cognitive abilities through the computational capacity to self-direct themselves; they are aware of their surrounding known or unknown environment, context and tasks allocated and operate without human intervention to generate outcomes by achieving goals (Pande and Taeihagh Forthcoming). Industrial robots are a means to save resources and develop greener technologies to reach the sustainable development goal of building resilient infrastructure, promoting inclusive and sustainable industrialization, and fostering innovation (IFR 2022). Such technologies also provide benefits of quick decision making, resulting in lower costs, improved performance, and higher productivity (Leikas, Koivisto, and Gotcheva 2019). For instance, service robots can reduce labour costs, work during unpopular hours, and execute repetitive tasks (Seyitoğlu and Ivanov 2021). Another advantage of autonomous systems is their applicability in hostile environments (not feasible for human presence) and where quick actions are required, such as in undersea mining or collapsible buildings (Lloyd’s Register Foundation 2016; Spichkova and Simic 2015). The deployment of AVs has been touted to significantly reduce car accidents (Hancock, Nourbakhsh, and Stewart 2019). AVs have significant social, economic, and environmental benefits via better traffic flow, road safety, and sustainability through reduced emissions and lower energy consumption (Lim and Taeihagh 2018; Williams, Das, and Fisher 2020). Healthcare robots deployed for long-term healthcare are autonomy-enhancing tools for the elderly and burden-relieving for caregivers (Tan and Taeihagh 2021). Autonomous weapons have significant military advantages, with the need for fewer war fighters and robots being better suited for dangerous missions (Etzioni and Etzioni 2017a).

However, operating autonomous systems has ethical, social, and legal risks (McCarthy 2009). These risks lead to governing challenges in regulating AI, which also pass over to the regulation of autonomous systems (Taeihagh 2021; Taeihagh, Ramesh, and Howlett 2021). While scholarly literature groups AI-enabled systems together and national-level AI strategies have been analysed to examine strategic actions and regulation (Fatima, Desouza, and Dawson 2020; Roberts et al. 2021), there is a gap in having a nuanced understanding of the regulations for varied types of autonomous systems. With this backdrop, the research questions for this study are 1) What are the challenges of governing autonomous systems? 2) Do the standards for specific autonomous systems in Singapore address these challenges?

The “pacing problem” is defined as a situation when developments in the sphere of technology surpass the laws and regulations (Thierer 2018). Creating a balance between regulations that become ineffective due to the speed of innovation and clauses that are too general is a major challenge for governments (Guihot, Matthew, and Suzor 2017). Setting performance standards for the safe and reliable operation of a technology focuses on three dimensions: 1) purpose of the technology, 2) frequency and degree of achieving the function, and 3) appropriate contexts of operation defined as the environment in which the technology functions to operate within the limits of tolerance (Danks and John London 2017). The complexity of autonomous systems leads to difficulties in regulating them in two key ways (Danks and John London 2017). The standard methods of metric-based regulation that presuppose contexts will not be feasible for autonomous systems due to the inherent uncertainty in their operation. However, in the initial stages of introducing emerging technologies, standards serve as regulations when neither the best technological options are available (since they are still evolving) nor performance goals have been fixed (Borrás and Edler 2014). Due to the risks arising from their operation and the implications for the economy, environment, and social stability, how to govern autonomous systems remains a dilemma for policymakers.

The legislations for autonomous systems are evolving worldwide, and countries have been setting provisional standards for the testing and deploying of autonomous systems. Using the case of Singapore for studying a context-specific phenomenon in real-life (Yin 2018), we have examined the regulations for autonomous systems through a literature review and thematic analysis. The insights will help understand whether the standards address the challenges in governing autonomous systems and the steps to be taken for their regulation. This study fills the gap in the literature by highlighting the challenges of governing autonomous systems within operational, economic, legal, social, and ethical dimensions and using a novel case to identify the regulations to manage those challenges. This study contributes to the existing literature by providing empirical evidence of standards, assessing whether standards focus on mitigating the risks of operation of autonomous systems and providing recommendations on the regulation of autonomous systems based on the case study in Singapore. Singapore’s experience with regulating autonomous systems can help other jurisdictions in policy learning while deciding on guidelines for testing or adopting autonomous systems.

The paper is structured into six sections. Section 2 presents the methodology, and section 3 provides an overview of the challenges of governing them. We map the challenges to the provisional standards for various autonomous systems in Singapore in Section 4 and discuss the findings in Section 5 before concluding in Section 6.

2. Methodology

2.1 Case selection

Singapore has been a front runner in trials of autonomous systems such as autonomous vehicles, drones, and service robots, keeping in line with the vision of becoming a “Smart nation”. It has ranked first for three years in a row during 2019–21 in the IMD-SUTD Smart City index that ranks cities on the technological provisions for health and safety, mobility, activities, opportunities, and governance (Smart Nation and Digital Government Office 2019). It ranked first in the AI Readiness index (2019) (Oxford Insights 2019) and the KPMG 2020 Autonomous Vehicles Readiness Index (AVRI) (KPMG International 2020). Singapore’s National AI Strategy focuses on developing, testing, deploying and scaling AI applications (Kit 2019). Moreover, Singapore has been considered to have one of the most conducive regulatory regimes for AV testing (The Economist 2019). Considering these developments, Singapore is an appropriate and novel case to study the risks from trials and deployment of autonomous systems along with the standards for managing their risks from operation.

2.2 Method

We searched for different autonomous systems and their regulations in two stages. First, we used keywords for autonomous systems and their different types to identify them with AND/OR operators in SCOPUS. This helped identify the risks of such systems and classify the associated challenges of governing them within operational, economic, legal, social, and ethical dimensions. Second, to search for the regulations/standards/legislations, we used keywords (in Supplemental material A) with an “AND” operator and “Singapore” in the novel database “Singapore Standards” by the Singapore Standards Council, Singapore Enterprise. Using keywords related to autonomous systems, we searched the database to identify the relevant standards and technical reference documents for the following specific autonomous systems: AVs, unmanned aerial vehicles, industrial robots, and personal care robots. We identified 25 documents spanning 2591 pages for examination. When guidelines or regulations for a certain type of autonomous system were unavailable or more information was sought, the search engine Google was used.

The government’s published guidelines, standards, and legislation for different types of autonomous systems were analysed through thematic analysis. Thematic analysis refers to a descriptive method for systematically inferring meanings from data sources through emerging themes (Schreier 2014). A line-by-line reading of the documents was undertaken to identify the rules for the relevant autonomous systems and group them under the appropriate challenges of governing them. In the data analysis process, we examined the regulations to manage the risks arising from the deployment of autonomous systems by anchoring them to themes identified in Stage 1.

3. Challenges of governing autonomous systems

Autonomous systems are supposed to adapt and learn in the process of receiving input, and with changing norms and circumstances in settings with human-robot interaction, the emerging modes of robot behaviour can be unpredictable (Tan, Taeihagh, and Tripathi 2021). Syntactic failure (sensors are unable to detect or identify objects), semantic failure (systems cannot translate human intent into algorithms), testing failure (systems cannot be tested in all scenarios), and warning failure (systems cannot identify and communicate their limitations) can be sources of unpredictability in autonomous systems (Grimm, Smart, and Hartzog 2018). The human-system interaction is significant in autonomous systems to overcome certain challenges, and the responsibility of supervision cannot be undermined, especially in the case of semi-autonomous systems (Sheridan 2016). However, in such interactions between humans and the system, there are questions about when should this be done? How? What are the rules for supervision or take-over? Data management and privacy are the challenges arising from data collection and the use of algorithms for generating insights that might lead to the identification of the characteristics of an individual. Legal challenges in governing autonomous systems involve cybersecurity concerns, as hacking incidents can result in the leakage of private information (Lera et al. 2017). Attribution of moral or legal responsibility on autonomous systems due to the lack of accountability and opacity in allocating liability across the stakeholders is another key challenge. For specific autonomous systems like personal care robots, the challenges in governing lie in the lack of social connectedness and ethical dilemmas.

The advent of autonomous systems such as driverless cars and robots constitutes a disruptive technological change that will impact jobs and constitute economic challenges. While a range of repetitive tasks have been automated, cognitive tasks are also being performed by AI systems (Wong 2020). Based on a survey in Australia, Pettigrew, Fritschi, and Norman (2018) emphasise the detrimental effects of AVs on employment alongside the benefits of new jobs that will emerge. Adopting autonomous systems will also compromise human interactions – they might increase social isolation if humans spend more time with robots, and the lack of human touch is particularly concerning for healthcare and long-term care (Vandemeulebroucke, de Casterlé, and Gastmans 2018; Tan, Taeihagh, and Tripathi 2021).

Challenges in the operation of autonomous systems derived from their risks for robots in long-term care, smart toys for children, and AVs have been studied from the technological, social, individual, and ethical perspectives (Tan and Taeihagh 2021; Taeihagh and Si Min Lim 2019; Fosch-Villaronga et al. 2021). Following the studies focusing on the ramifications of using such autonomous systems from different perspectives, we classify the challenges of governing autonomous systems in the following dimensions: operational, legal, economic, social, and ethical (Table 1). Classifying these governing challenges in silos is difficult since legal challenges related to data management would overlap with ethical dilemmas due to the usage of algorithms that impact fairness in operation.

Table 1. Dimensions and explanations of governance challenges of autonomous systems.

Dimensions of governance challenges	Explanation	Evidence/Adapted from
Operational challenges	Technological risks with the potential for negative consequences in the operation of the systems. These include the challenge of safety, unpredictability, and opacity in operation; problems in monitoring, control, and task allocation.	(Grimm, Smart, and Hartzog 2018; Surden and Williams 2016; Mekki-Mokhtar et al. 2012; Sheridan 2016)
Legal challenges	Lack of legal instruments to protect privacy, assigning liability and responsibility in case of harm or damage, and managing cybersecurity issues.	(Eugensson et al. 2013; Tan, Taeihagh, and Pande 2023; Calo 2017; Such 2017)
Economic challenges	Disruptions in employment through job losses and change in workforce structure in the economy due to re-skilling and potential replacement of existing workers with autonomous systems like drivers for cabs, buses, and private car drivers.	(Tan and Taeihagh 2021; Pettigrew, Fritschi, and Norman 2018; Beede, Powers, and Ingram 2017; Nikitas, Vitel, and Cotet 2021)
Social challenges	Challenges concerning systems interactions with human beings with a lack of trust and natural social relationships or human touch undermine the user’s dignity.	(Tan, Taeihagh, and Tripathi 2021; Lahijanian and Kwiatkowska 2016; Sharkey and Sharkey 2012; Etzioni and Etzioni 2017b)
Ethical challenges	Issues related to moral principles guiding judgements of right or wrong result in ethical dilemmas for operating autonomous systems and impacting individuals or groups in society.	(Tan and Taeihagh 2021; Manning et al. 2023; Arkin 2009; Coeckelbergh 2011)

4. Regulations and provisional standards for autonomous systems in Singapore

The standards and regulations for autonomous systems have developed over time and are still evolving. We discuss the regulations for specific autonomous systems based on the challenges discussed in the previous section. Details about relevant legislation are provided in Supplemental material B, and an overview of the regulations related to specific autonomous systems (AVs, unmanned aerial vehicles, industrial robots, and personal care robots) is provided in Table 2.

Table 2. Key regulations for specific autonomous systems according to the governance challenges.

	Type	Dimensions	Key regulations in Singapore according to the challenges
1	Autonomous vehicles	Operational	Unpredictability in operation Requirement of integrity and technical competence of AV developer or operator. Record of contact persons with accountability for the life cycle of AVs Safety Quality management system with full quality assurance system, change management. Hazard analysis and risk assessment clause to ensure a robust process for identifying hazards and compliance with rules. Explanation of risk mitigation in hazardous events by the AV operator. Validation methods to ensure the safe operation of AVs through virtual methods, closed road testing, and public road testing are outlined in the standards. Monitoring, control, and task allocation A human operator may be on board to take over dynamic driving tasks. The AV human machine interface (HMI) allows occupants to operate AV-related functions and services. HMI can happen when the AV system is 1) in autonomous driving mode or unavailable to do autonomous driving, 2) is requesting a transition in control from system to operator, 3) in exceptional driving situations, or 4) needs safe response or action requirements in case of an emergency.
		Legal	Data management and privacy Minimum requirements for the consistency of components with communication protocols, maps, and on data management. Data types and formats in AVs are specified for uses such as data recording, use of data, and information management. Structured encoding of messages is defined in the standards. Cybersecurity standards The TR lists cybersecurity principles and requirements for wireless networks. Third-party cybersecurity assessment is required involving the AV operator/developer and assessor using a framework mentioned in the standards. The TR proposes a cybersecurity interface agreement for the involved parties to clarify the roles and responsibilities of actors. TR 64 has listed categories of cybersecurity requirements to avoid threats.
2	Unmanned aerial vehicles	Operational	Monitoring, control, and task allocation The unmanned aircraft must be in the line of sight of the operator at all times. The controller must follow instructions in the user manual. Safety Permits are required for operation – operator permit/activity permit. Safety measures need to be employed when operating.
		Legal	Data management, privacy, and cybersecurity Ethical practices ensuring personal data protection and establishing clear liability and risk assessment protocols.
3	Industrial robots	Operational	Unpredictability and opacity in design and operation HARA methods have been specified before selecting and designing appropriate safeguard measures. Safety General design requirements and protective measures have been outlined. Verification and validation of safety requirements and protective measures. Robots require an instruction handbook with details and instructional guidelines. Monitoring, task allocation, and human-system interaction Collaborative robot operation measures in a common workspace can be used for predetermined tasks when all protective measures are active and for robots with features for collaborative operation.
		Legal	Data management, privacy, and cybersecurity For mobile robots used for delivery, data interchange occurs between robots, lifts, and doors to execute operations. DDS and location and sensing environment modules have been developed to interchange communication during operations. Safeguards and protective measures need to be applied by improving the capability/reliability of sensors, identification algorithms to be designed in a way that the probability of being correct is increased, can be calculated, and monitored. Validity checks have to be made on decisions that can lead to hazardous situations, with decisions to be verified by principles.
	Personal care robots	Operational	Unpredictability in operation A hazard identification strategy has been listed in the standards to identify hazards for cases. Risk estimation methods along with risk reduction methods have been outlined and are supposed to be designed on a case-by-case basis for events. Limits to operational spaces for personal care robots specifying safety-related speed controls, environmental sensing, object sensing, contact sensing, travel surface sensing, stability control, and safety-related force control. Safety Hazards with robot’s operation are listed in the standards to be countered through safety requirements and an inherently safe design. Verification methods of safety requirements are listed in the standards. Proper markings required with details of the manufacturer and manufacturing of the robot. Required performance levels for selected robot types and robot stopping functions are also listed in the standards. User manual to be provided with the robot. Additional measures might be imposed on robots, for instance, motor vehicle regulations if they operate on public roads.

4.1 Operational challenges

4.1.1 Unpredictability in the operation of autonomous systems

The AV developer/operator must have integrity and technical competence for testing according to Technical Reference-68 (TR-68). The factors for investigation and handling compliance issues would include a proper record and definition of contact persons with accountability for the life cycle¹ of AVs and description of discovery, reporting, classifying and escalating non-compliance issues, and deciding whether an AV is fit to be on the road (TR 2021).

In the operation of personal care robots, hazard identification is significant when uncertain autonomous decisions can lead to hazards impacting the user. The standards recognize that the different varieties of personal care robots will have different hazards, and hence risk assessments would have to be conducted to identify the protective measures for a particular application of the robot. They need to be designed in a manner such that anticipated environmental conditions do not lead to dangers with inherently safe design measures and safeguards. A hazard identification strategy and risk assessment (HARA)² have been defined for the same (details in Supplemental material C).

The standards also provide examples of operational spaces for the different types of personal care robots – person carrier robots, mobile servant robots, and exoskeletons. There are defined spaces for the operation of the robots depending on their type – maximum space, restricted space, safeguarded space, and protective stop space. This is similar to collaborative industrial robots systems where the collaborative workspace is specified to ensure established limits and restricted areas (clearances around obstacles and fixed equipment, access routes, and hazards which could cause trips and falls) factors that affect human interaction with the equipment, and the timing of collaborative operation (TR ISO/TS 2017). Research studies on human-robot interaction and pain tolerance have been conducted to test the limits of adults and collisions on parts of the human body and are being conducted on different categories of people (children, elderly, pregnant women) for a future edition of the International Standards.

4.1.2 Monitoring, control, and task allocation

A human operator may be on-board for testing and risk management to take over the dynamic driving task in AVs (2021). However, the guidelines in the TR do not account for any human-system interaction and assume a level 5 automated driving system for the AV.

For collaborative industrial robots, when robots work with humans, robots require a monitored stop button and maintenance of a constant speed and distance from the human operator (TR ISO/TS 2017). The design of the user interface is specified by outlining details about status, connection and disconnection, and command devices. The take-over conditions are not specified. In the standards for industrial robots, while conditions for collaborative workspace are defined, there are no guidelines for take-over during collaborative operations. Collaborative robot operation measures in a common workspace can be used for predetermined tasks when all protective measures are active and for robots with features for collaborative operation. For such situations, the standards also specify labelling, requirement for collaborative spaces with a safety-rated monitored stop, hand guiding, and speed and separation monitoring. Similarly, designing risk estimation methods for personal care robots would be done on a case-by-case basis when there is interaction between the robot and the human or safety objects.

The responsibility of the UA lies with the controller according to the safety guidelines of the Civil Aviation Authority of Singapore (CAAS). The controller must have the UA in their line of sight and follow the instructions in the user manual for operation (CAAS 2022).

4.1.3 Provisional standards for safety

4.1.3.1 Autonomous vehicles

The provisional standards for Singapore’s AVs focus on design, production quality, and safe operation. A Quality Management system for AVs has been proposed to include full quality assurance, configuration management, change management, testing environment, and actions for correcting non-compliance (2021). The LTA and CETRAN (Centre of Excellence for Testing and Research of AVs-NTU), with suggestions from Traffic Police, prescribe strict safety assessment measures before the AVs can operate on roads (LTA 2022). AV standards outline the full quality assurance process (details in Supplemental material C).

During COVID-19, ground autonomous robots were deployed for disinfection and using irradiation (UV) for cleaning. In a Technical Reference for the use of technologies and processes for safe management measures, robots or remote operations were preferred for UV-C (100 nm-280 nm wavelength) and cleaning due to the advantages of not being affected by radiation, high precision and performance as compared to human operators, and the ease in deploying them at short notice or unusual hours (TR 88 2020). For their safe operation, it was required to have robot-accessible routes and link them with building systems (TR 88 2020).

4.1.3.2 Unmanned aerial vehicles

In Singapore, unmanned aircraft operations are governed by the Air Navigation Act introduced in 2019. General rules for UA disallow operating it in prohibited areas, resulting in penalties or imprisonment (Air Navigation Act 2019). All unmanned aircraft weighing more than 250 grams must be registered with CAAS before operating for accountability and traceability. For UAs that are more than 7 kilograms, a training certificate or a pilot license is required. The registration labels are available to those above 16 years of age and require a unique UA registration number with complete online registration and an affixed label. After the registration, relevant permits are required to operate the UA – these can be operator permit (granted to individuals or organisations that demonstrate the ability to safely operate the UA), activity permit³ (granted to an organisation or individual for repeated activities provided the location, type of operation and date or time, altitude of use, and contingency plans in case of an accident are decided), and other permits from the Singapore Police Force and Info-communications Media Development Authority of Singapore (IMDA). To address safety concerns, UAs can be flown only outside no-fly zones prescribed by CAAS. Unmanned aircraft systems are deployed for visual inspection of building facades and must adhere to safety measures and ethical practices to ensuring the protection of personal data and establishing clear insurance, liability and risk assessment protocols (TR 78 2020).

4.1.3.3 Industrial robots

To fulfil safety requirements, general design requirements and protective measures have been outlined for industrial robots. The general requirements provide guidelines for the safety of transmission components, power loss or change, component malfunction, isolating any hazardous source of energy, provision for controlled release of energy, appropriate actuating controls, safety-control related measures for hardware and software, functions to stop robots in required situations, controlling speed, buttons for start and emergency stop, and control of multiple robots (SS ISO 2016).

For collaborative robots, there are collaborative operation requirements, including safety-rated monitored stop, maintaining speed and distance from the operator, specifications related to axis limit, and designing electric connectors to prevent unintended separation. The robot should have information on the name and address of the manufacturer/supplier, month and year of manufacture, details on structure and capacity, data for hydraulic systems, and points to move the robot. An instruction manual with details about the robot and instructions for activating safeguards (if they stop) with recommendations for training personnel is to be provided for each robot. Proper verification and validation of the safety and protective measures through inspection, tests, observation during operation, assessment of tasks, and specification of reviews must be undertaken.

4.1.3.4 Personal care robots

Personal care robots can have hazards related to battery overcharge, energy storage in the robot, danger due to robot start-up or shape or noise/lack of awareness, vibration, substances and fluids, environmental conditions, extreme temperatures, radiation, the motion of the robot, collision of the robot with obstacles, risk arising from human-robot interaction, and contact with moving parts or navigation errors (SS ISO 2017). The design principles of personal care robots require a safe design to reduce mental stress, protective measures, and information for use to avoid risks. To counter the dangers of using these robots, the standards propose a safe design, avoiding or not having heat sources and using materials with thermal conductivity. The robot also needs to have proper markings with information about the manufacturer or supplier, type of robot, details such as serial number and date of manufacture, and technical information regarding rated voltage, supply required, power input, mass, and symbols as per IEC standards. The standards list the following ways of verification: inspection, practical tests, measurement, observing the robot while it is being operated, examining the circuit figures and the software, reviewing risk assessment through the assignment of tasks, and inspecting the drawings and documents specifying the layout of robots. While testing and validation are necessary, plant and environmental simulation to test functions such as obstacle avoidance, tracking, and navigation in a real-environment scenario are challenging due to the simulation of reality (Pannocchi et al. 2019). Additional safety measures while deploying robots for cleaning include clarifying sensory requirements and programming logic in case of human presence in a room, safety protocols, and information dissemination in the deployment building (TR 88 2020). A user manual has to be provided with the robot for its intended use, detailed descriptions, applications, and instructions for operation.

Hygiene and infection control are the foremost safety considerations for robots in healthcare environments. The TR provides a checklist for safety assessments to meet disinfection needs, built-in safety mechanisms, cleanliness mechanisms to prevent infection spread, and tasks that must be fully or partially automated (TR 108 2022).

4.2 Legal challenges

4.2.1 Regulations for lability

In the case of accidents that result in injury or harm involving manned vehicles, the issues comprise of a) identifying the liable party, b) establishing the liability of the party, and examining any defence (Singapore Academy of Law 2020). However, ascribing liability in accidents involving autonomous systems attains is difficult due to the difficulty in proving fault due to the harm caused when there is no human operator (Beck 2016; Singapore Academy of Law 2020).

In Singapore, the issues of liability are indirectly addressed by the regulatory sandbox to promote the testing of autonomous vehicles by focusing on safety and attaining approvals for testing (Taeihagh and Si Min Lim 2019). The TR-68 attributes the responsibility for the safe deployment of AVs with appropriate validation and certification to the AV developer/operator. However, the Road Traffic (Autonomous Motor Vehicles) Rules 2017 specify liability regulations. Applicants for conducting trials of AVs need to obtain authorization from LTA. The Rules have specified that the applicant for conducting AV trials must have liability insurance for the period of authorization to conduct the trials or must make a deposit of $1.5 million (Singapore Statutes Online 2017). This is in addition to the appropriate documentation about the testing of AVs, notes describing quality, action items and their handling by the developer/operator. The nature of UAs considered in the current regulations for safety comprises of automated drones that a human operates. Hence, the liability for accidents involving drones can hold human operators responsible.

4.2.2 Data management in the operation of autonomous systems

The standards require consistency of each autonomous system with V2× (vehicle to everything) communication protocols, encrypted security principles, maps, and provisions for storing, tracking, and updating information on a system. In the operation of AVs, (TR 68: Part 4 2021) enlists data types and formats for the following uses: 1) data recording for automated driving, 2) proper use of AV data to improve safety, 3) management of dynamic content like information on road traffic, 4) data for use in investigation in case of accidents. A structure is outlined for encoding of messages defining the smallest divisions of information, data framing, and message that encapsulates header data frame, data elements, and multiple data frames. However, this excludes software updates, fleet management, AV-human interaction, and data ownership and privacy standards. The data must be stored with a series of time-stamped entries to record a signal or event at a specific time.

Communication links need to be clear between the autonomous robot and the building systems, for instance, when a human being needs to be sensed in the area being cleaned so that CCTVs or motion sensors can alert the robot of the same (TR 88 2020). For robots operating in buildings using lifts for transportation and deliveries, a system for communications between the lift and robot needs to be standardised through a sequence of operating commands that can be sent from the robot fleet manager to the communication node and finally to the lift control system for executing the tasks (2021). Security measures for data encryption are required for this to prevent attacks that could cause harm to the robot-lift system.

4.2.3 Cybersecurity standards

The Cybersecurity Act governs cybersecurity in Singapore, passed in 2018, authorising the Cybersecurity Agency of Singapore to deal with cybersecurity threats, set cybersecurity regulations and license service providers. The Singapore Cybersecurity Strategy 2021 was introduced to build robust infrastructure, safer cyberspace, international cyber cooperation, and develop a vibrant ecosystem and cyber talent (CSA 2021). The strategy classifies AVs under the internet of things (IoT), under which promotion and development of objective technical cybersecurity standards are outlined.

The TR lists cybersecurity principles for AVs: 1) To have cybersecurity in the design of AVs, 2) ensure security in the system architecture for protection under threat analysis and risk assessment framework for AVs, 3) robust processes for smooth and safe operations, and 4) resilient safeguard measures that are reviewed regularly. A cybersecurity assessment framework and methods are outlined for the third-party cybersecurity assessment against threats involving the AV operator/developer and operator. A cybersecurity interface agreement for a clear understanding of involved parties is specified through the RASIC (responsible, approve, support, inform, consult) framework. The AV Cybersecurity Assessment Framework includes the steps for system review, threat analysis and risk assessment, cybersecurity testing, and assessment report. For autonomous driving, the TR proposes cybersecurity principles for developers for the full life cycle of the AV (regarding design, development, operations, maintenance, and decommissioning), a frame for independently assessing the cybersecurity of AVs for identifying any cybersecurity susceptibilities and testing AVs against cyber threats (TR 68: Part 3 2021).

4.2.4 Regulations for data management and privacy

The data protection regime in Singapore is governed by the Personal Data Protection Act (PDPA). It came into effect in 2014 and aimed to address the collection, use, and disclosure of personal data by organisations in Singapore (Personal Data Protection Regulations n.d.). The PDPA serves two key legislative purposes of building the trust of individuals in the government at the local level and following international standards for data protection (Wong Yong Quan 2017). Data protection principles have been borrowed from the privacy principles of OECD guidelines. These principles cover the collection, use, disclosure, purpose, access, correct retention, accuracy, and data protection, along with penalties and enforcement (Chik 2013). The PDPA has been amended with new measures having taken effect from February 2021. Additional stricter measures have been undertaken to ensure organisations protect their data by ensuring security arrangements to prevent unsanctioned collection, use, disclosure, or modification of data or any loss of the data storage medium.

To deal with unpredictability, all foreseeable steps have been considered in the provisional standards; for example, in the case of automated ground vehicles (AGVs) and AMRs deployed in multi-stories buildings for unmanned deliveries of material, a minimum set of communication exchanges between digital and discrete lift control systems is specified (TR 93 2021). Data distribution service (DDS) has been developed for communication between different domains and data routing, and specific modules for location data, detecting surfaces and environments (Autonomous delivery workgroup 2021).

4.3 Economic challenges

The job losses and obsolete jobs associated with the adoption of AI have been discussed by the Singapore government to help people find other employment opportunities or update their skills to retain their employment opportunities (Ministry of Manpower 2018). The government introduced the Adapt and Grow initiative, Workforce Singapore (WSG) and an Employment and Employability Institute (e2i) for job facilitation and coaching in 2018 (ibid.). The Skills Future program is another initiative to provide opportunities to Singaporeans for training and career development and to bridge the skills gap with the changing needs of the digital workforce (Skills Future 2022).

4.4 Social challenges

Assisting in social connectedness is a crucial factor for personal care robots. The standards specify ongoing research on risk estimation being conducted for different age groups (SS ISO 2017); however, enabling the social connectedness of robots during human interaction has not been discussed in the standards for personal care robots and robot systems in healthcare environments.

4.5 Ethical challenges

The ethical dilemmas issues are not explicitly discussed in the provisional standards. Ethical issues in the operation of AVs emerge when there is statistical bias, manufacturers/programmers tweak the algorithms to favour AV users, algorithms are opaque, difficulty in proving intent in algorithms, and inconsistencies in the ethical reasoning of the stakeholders (Lim and Taeihagh 2019). The provisional standards in the TR-68 clarify provisions on getting around unpredictability by requiring increased transparency for software and hardware for traceability and requiring integrity of the AV developer/operator. However, there are no direct and clear guidelines for overcoming challenges of ethical dilemmas for AVs, UAs, and industrial robots.

5. Discussion

This study has examined the challenges of governing autonomous systems within the operational, legal, economic, social, and ethical dimensions by corresponding them with the existing provisional standards in Singapore. We examine the standards for AVs, unmanned aerial vehicles, industrial and collaborative robots, and personal care robots. Based on the findings, we can see similarities and differences in the standards proposed across autonomous systems. The standards address most operational and legal challenges of governing autonomous systems (Table 2). There is scope for addressing the social and ethical challenges, given that the government is making efforts to re-skilling the existing workforce to manage the economic challenges. Even though economic challenges have been addressed through employment-creating and re-skilling initiatives, efforts for dealing with the introduction of specific autonomous systems like AVs and their impact on drivers and personal care robots with effects on the nurses and healthcare workers need attention.

A bulk of standards amongst autonomous systems are for AVs considering the car-lite vision of Singapore, the goals of introducing driverless buses in three areas from 2022 onwards, and making the western section of Singapore a test-bed for AVs (Abdullah 2019; Lim 2017). Singapore has received the maximum score in the sub-pillars of AV regulations, pilots for AVs funded by the government, agency for AVs, future orientation of the government, and government readiness for change in the 2020 AVRI (KPMG International 2020). Singapore has used a control-oriented strategy towards trials and testing of AVs (Taeihagh and Si Min Lim 2019), and the current regulations on mandatory liability insurance have further strengthened it.

Ensuring safety in the operation of systems has been the foremost goal of the government within the operational challenges. Most of the standards address physical safety concerns and seek to minimise harm to humans – users, in the case of personal care robots or operators in the case of industrial robots. Physical safety is ensured by causing no discomfort or injury to the human operator, and psychological safety refers to avoiding constant stress to the human operator over extended periods (Lasota, Rossano, and Shah 2014). Thus, maintaining safety in autonomous systems acquires a multidisciplinary approach. While the standards list guidelines to deal with physical safety, insights on psychological safety to reduce mental stress to the user are specified for personal care robots.

To deal with unpredictability, all possible and foreseeable situations that can be hazardous have been listed in the standards for personal care robots. To address the safety concerns, the standards cover quality management in design and hazard management in operation for personal care robots and collaborative industrial robots. To reduce the risk of harm, the standards specify technical measures for tweaking the design or reducing the risk by substitution and ensuring a safe state for the operators of robots. Emphasis is also on having safeguarded environments for their operation to ensure no harm to the operators or users. The standards acknowledge the dynamic characteristics of robots and the environment of operation, along with the difference in applications impacting the design and options of safeguarding measures.

Data management and cybersecurity-related challenges have been specified for AVs and industrial robots, with the overarching legislation of PDPA for autonomous systems in safeguarding the privacy of individuals. Issues of creating safe environments and enabling infrastructure for trials, identifying hazards, and developing frameworks for data collection and privacy would be significant points for developing economies to leapfrog and adopt autonomous systems so that industry 4.0 does not become a source of a gap within and between countries (Primi and Toselli 2020).

5.1 Regulation strategy in Singapore

This study brings also brings forth the complexities in regulating emerging technologies. While national AI strategies and plans have been analysed across countries (Fatima, Desouza, and Dawson 2020; Radu 2021), a case study like this provides in-depth insights into existing regulatory frameworks in a country and the improvements required. The pacing problem can be solved in two ways: 1) by slowing the pace of innovations and 2) by boosting the capacity of the legal system to propose timely regulations (Marchant 2011). Sunset clauses can help adapt to rules and determine the expiry of irrelevant regulations, and experimental regulations can form part of temporary regulations (Ranchordas 2014).

The current provisional standards in Singapore for selected autonomous systems are voluntary; in this case, experimental regulations and sunset clauses can be imposed to formalise the regulations. Experimental clauses permit deviations from a regulation for alternative approaches to examine how the regulation can be adapted in the future (WEF 2020). However, there needs to be a balance between the time limit and duration of experimental clauses to ensure competition among businesses is not undermined, and yet, they still can test and learn so that regulations can be changed (WEF 2020). The analysis of standards in this study demonstrates the need for government to focus on social and ethical governance challenges while testing the autonomous systems in different environments and review the provisional standards between shorter durations as opposed to the current schedule of three years.

We find that clearer guidelines need to be developed for addressing ethical and social challenges of ethical dilemmas and lack of social connectedness. These are especially relevant for personal care robots, and the challenge of technological opacity related to obscure algorithms in autonomous systems, along with guidelines on cybersecurity, need to be better addressed in the standards. While standards for collaborative industrial robots acknowledge the collaboration between the systems, the details about take-overs and operation on different levels of autonomy need to be specified.

Regulation is the continuous act of monitoring, assessing, and refining rules rather than a one-shot or ad hoc operation that involves bureaucratic and administrative rule-making (Levi-Faur 2011). The characteristics of technology affect the regulatory intervention (Roca et al. 2017). The initial stages of emerging technology are engulfed with uncertainty, and development from “infancy to maturity” depends on production outputs (Vincenti 1990). The test-beds provided for AVs instead of introducing them in the mainstream in Singapore is indicative of the practice that for uncertainty in immature technologies, the outputs improve as knowledge is garnered, and it takes time to explain the behaviour of the technology with replicable results to be called “science” (Bohn 2005).

5.2 Existence of multiple actors

The standards focus on the responsibilities of different stakeholders – designers, manufacturers/operators, and the government indicating multiple actors in regulating the operation of autonomous systems. The TR describing the eco-system for AVs lists a variety of actors: insurance companies, testing, inspection, and certification bodies, AV owner, AV operator, AV passengers, charging networks, infrastructure (V2I), vulnerable road users, vehicle gateway, regulator, government agencies, private service and data provider, and system providers. A key catalyst for developing autonomous systems in Singapore has been the collaboration between statutory bodies, technical committees, and research laboratories to conceptualise and test them. All standards have been proposed by working groups comprising of stakeholders from relevant organisations. For instance, an unmanned surface vehicle in Singapore that has been designed indigenously has undergone trials for collision-free activities along with the Collision Detection and Collision Avoidance system for anti-collision (MINDEF 2021). This initiative was undertaken with the collaboration of the Defence Ministry of Defence and DSO Laboratories. Similarly, the working groups for the TRs and standards comprised members from relevant organisations for the autonomous systems. For instance, the organisations of experts in the working groups for standards on AVs were A*STAR, Centre of Excellence for Testing and Research of AVs, Singapore University of Technology and Design, DSO National Laboratories, Land Transport Authority, consultants like Ernst & Young Singapore, and companies involved in manufacturing AVs like NuTonomy.

The multiplicity of actors and their views involved in the ecosystem of autonomous systems is an important consideration, as differing views would impact the nature of regulation. There are conflicting views on factors impacting robot autonomy between operators and observers of robots (Burri 2016). Operators have a technical view of robots emphasizing that they operate on programmed code, and robot autonomy depends on the human-system interaction due to this. On the other hand, observers view robot autonomy from as unsupervised autonomy without thinking about the programming that determines the control and direction of the robot. While operators view robot autonomy in the present, observers usually have a futuristic outlook by thinking about what lies ahead. The difference in the conceptions will impact how the actors interact with different systems. Such divergence underscores the importance for policymakers to consider diverse perspectives while developing regulations (Burri 2016).

5.3 Government’s role in coordination for regulation

A novel challenge for the governance of autonomous systems is the coordination problem whereby regulators might not be able to coordinate with other agencies since they lie outside their sectoral or geographical jurisdiction (WEF 2020). While the governments might not have been in the commanding position of regulating autonomous systems, their presence is significant for AVs (as an example) due to their role in existing driving mobility systems (Borrás and Edler 2020). Singapore has followed the strategy of reviewing guidelines periodically, considering the views of stakeholders from diverse sectors for different autonomous systems. There is an overlap in some of the features of autonomous systems, such as AVs and automated ground vehicles (that are being used for deliveries). In such scenarios, some standards for AVs will apply to automated ground vehicles as well. Coordination between agencies that are involved in standard setting can be carried out by the government, which is being done by Singapore Enterprise. The responsibility problem arises when regulators struggle to attribute responsibility for managing risks to varied actors in a complex environment (WEF 2020). To overcome this, the government can act as a steward in coordinating the actions of stakeholders and fixing responsibilities on individuals or sets of actors.

6. Conclusion

This research highlights the provisional standards for autonomous systems in Singapore to understand how governments can manage the operational, legal, economic, social, and ethical governance challenges. Singapore has adopted provisional standards from ISO by preparing technical references for autonomous vehicles, personal care robots, and industrial and collaborative robots. By drawing a parallel between the governance challenges across operational, legal, economic, social, and ethical dimensions to the provisional standards, we find that while there is an intention to safeguard against cybersecurity threats, clear regulations are yet to be proposed for personal care robots and industrial robots. Moreover, the challenges relating to ethical dilemmas and lack of social connectedness are yet to be addressed through the standards. These challenges can be understood and addressed through consultations with experts in relevant fields.

As Singapore is advancing towards being a Smart City, deploying autonomous systems would require more trials and pilots. To manage the risks from their operations, there is a need for a robust legal system and policy capacity that can provide timely regulations. A regular review of standards should be undertaken to enhance the existing standards and include those that will address hazards recognised during testing or omit redundant standards. Currently, the guidelines in the TRs and standards are voluntary, and through consultations and reviews, they must be made mandatory for the uniform implementation of rules for testing.

Future work could focus on examining more case studies so that cross-country comparisons of the different autonomous systems can provide more insights on their regulation. A focus on actor networks and the influence of individuals or groups on standard setting would throw light on the nature of regulation required and the responsibilities to be assigned.

CRediT authorship contribution statement

Conceptualisation, A.T.; methodology, A.T. and D.P; validation, A.T.; formal analysis, D.P, and A.T.; investigation, D.P. and A.T.; resources, A.T.; data curation, D.P.; writing and editing D.P. and A.T.; supervision, A.T.; project administration, A.T.; funding acquisition, A.T. All authors read and agreed to the published version of the manuscript.

Acknowledgments

Araz Taeihagh is grateful for the support provided by the Lee Kuan Yew School of Public Policy, National University of Singapore.

Disclosure statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/17487870.2023.2197599.

Additional information

Funding

This research is supported by Ministry of Education Singapore AcRF Tier 1 funding support and the Lee Kuan Yew School of Public Policy, National University of Singapore.

Notes

1. The system life cycle refers to the time period of activities that start when a system is conceived and end when it is decommissioned and disposed (like stages starting from commissioning, set-up, production, maintenance, repair, and ending at decommissioning).

2. Due to the uncertainty in the operation and difference in knowledge of users, hazard identification strategy is outlined by listing foreseeable hazards in the standards, and an elimination of hazards is carried out for risk assessment of the autonomous system.

3. Activity permits are of two kinds: Class 1 Activity Permit is required with an operator permit for non-recreational and non-educational purposes, or for UAs above 25 kilograms to be used for recreational purposes, or UAs above 7 kilograms to be used for educational purposes. Class 2 Activity Permit is required if the UA to be used for recreational purposes is 25 kilograms or below, or if it is used for educational purposes, it is 7 kilograms or below, and the conditions for usage are 1) operation altitude higher than 200 feet, 2) within 5 kilometres of a civil airport or military establishment, 3) within any restricted or protected area.