The potential mental health effects of remote control in an autonomous maritime world

ABSTRACT

Many maritime activities, such as loading, unloading and transporting cargoes, consist primarily of long periods of low-stress, with some moments of high stress during complex manoeuvres or unanticipated, dangerous, incidences. The increase in autonomy provided by machines and AI is beginning to take over certain tasks in the maritime sector, to reduce costs and mitigate human error. However, with the current levels of autonomous technology available, legislation, and public trust in the technology, such solutions are only able to remove majority of tasks associated with low-stress periods. In fact, many current remote control solutions still suggest relying on human operators to deal with the complex situations AI struggle with. Such a human–automation relationship could endanger the human element. The concern is that, if the human user is spending a disproportionate part of their time dealing with multiple, unconnected, high-stress tasks, without periods to de-stress, this could increasingly put workers at risk. This paper seeks to highlight potential technical, social, and mental, issues that may arise as the sector begins implementing semi-autonomous and fully autonomous maritime operations.

KEYWORDS:

• Autonomous shipping
• maritime
• mental health

Introduction

In most occupational contexts, even with the rise of automated tools and services, it is well established that human error will always be relevant as long there is a social-technical element. This is true with the maritime sector, as no matter how prevalent technology becomes, the human element will always be a part of the picture through training, legal, and ethics issues (Kim et al. 2020). Moreover, the role of humans and human error with autonomous systems are often grossly misunderstood, as human–autonomous relationships do exist and are often highly complex (Porathe, Prison, and Man 2014).

In the maritime industry, which forms a major part of the modern world’s wider transportation sector, it is estimated that between 50% and 90% of all accidents (that result in injury and death) are due to human error (Raby et al. 2001; Michael Regan and Stevens 2014). The role and rapid evolution of technology in this sector continues to change this risk landscape, increasing the types and volume of human error. It is worth noting that it is not uncommon for cyber-attacks in this sector to be mislabelled as human, or mechanical, error instead of an attack (Tam and Jones 2018). That said, at the root of many cyber-attacks, like data breaches, is human vulnerabilities and error (Safa, Solms, and Futcher 2016).

The International Maritime Organization (IMO), the UN agency charged with ensuring maritime safety and security, as early as 1997 identified the relationship between crew performance and safety (International Maritime Organisation 1997). Later revised in 2003 (International Maritime Organisation 2003), the IMO define crew endurance as the ability for crew to maintain performance within safety limits. This performance is a function of many complex, interacting variables, including individual capabilities, management policies, cultural factors, experience, training, job skill, work environment and countless other factors. Thereby, signifying the important relationship between the human element and safety, regardless of the number of technical solutions.

Furthermore, the IMO and others have argued that central to human error in the maritime sector, both with traditional physical and newer cyber/cyber-physical risks, is stress brought on by overall fatigue, workloads, and work environment (International Maritime Organisation 1993b; Tam and Jones 2019b). Therefore, as the sector adopts new technologies that drastically alters both working environment and load, understanding the changes in stress is critical to mitigate human error. However, while there is some research into human-autonomy relationships in other sectors, there has been a lack of similar works for the future of autonomous shipping and smart ports.

While it is important to consider operations at port and on ships, it is estimated that approximately 90% of human-caused accidents occur in confined waters (Cockroft 1984). It would be interesting to see if remote control will balance these statistics in the future by reducing the difference in work environment, if both port and ship operations are controlled from similar office-like control centres or base control station (Maritime UK 2018).

Physiological studies conducted in the past, before many now commonplace technologies such as ECDIS (i.e., Electronic Chart Display and Information System) became mandatory, have suggested that complex navigation tasks can elicit an acute stress response (Cook and Shipley 1980). Technology advancements, such as ECDIS, have since then aspired to assist humans and are often designed to reduce user stress. However, there is concern that the new wave of technology may be designed to reduce human involvement, enable machines, and consider the human element less than systems specifically designed for human users.

As stated in other studies exploring stress experienced by mariners, there is very little evidence quantifying stress in contemporary maritime pilotage, let alone how the future may affect human mental health (Main, Wolkow, and Chambers 2017; Main and Chambers 2015). The pilotage phase of the voyage, in particular, is typically a critical, high-stress, stage of the passage with maximum crewing often in restricted channels. That said, during the wider voyage of the complete journey there can still be similarly complex or high-stress situations, ranging from known (e.g., straits) to less know, (e.g., accidents, weather). Given that autonomy and remote control have become reality in shipping, albeit on a small scale, the potential for new incidents and accidents may also increase. Therefore, understanding the new stressors in maritime work environments shaped by autonomy is critical for protecting mental health and promoting a healthy human-autonomy relationship.

It is important to note that a healthy human-autonomy relationship does not necessarily mean stress-free. More generally, moderate stress levels in shipping have been found to positively affect not only human productivity but also health and wellness (Hetherington, Flin, and Mearns 2006). However, today’s seafarers are constantly exposed to stressors on-board for several months, and, on average, it seems that stress is higher than a normalized onshore population (Oldenburg and Jensen 2019; Tony et al. 1997). Therefore, if technology evolves while seriously considering the human-in-the-loop, it is possible that this stress could be reduced. However, this would require research and awareness of the potential ways stress could be increased or amplified since, as mentioned, excessive stressors and fatigue often lead to more human errors (Arenius, Athanassiou, and Strter 2010; Kim and Jang 2018; Liu et al. 2020). For example, significant crew workloads could increase fatigue and decrease situational awareness, something that has been reported to be the main contributor to 23% of 98 critical vessel and personnel injuries in one year (McCallum, Raby, and Rothblum 1996). Other research efforts have emphasized the importance of analysing cognitive workloads of maritime crews and how increased a mental workload is associated with higher levels of ship collision risks (Robert et al. 2003; Lochner et al. 2018). Therefore, the effects of workloads need to be given greater consideration as more crew could become remote drivers and supervisors is important.

The remainder of this article is as follows. Section 2 will discuss the wider context of that autonomy brings to the maritime sector. This begins with discussion on the levels of autonomy maritime ships may see in the near future, and the drivers behind its implementation. Section 3 will explore the human-autonomy relationship and discuss the impacts that autonomy can have on stress. As there has been similar research in other sectors, such as air, comparisons will be made to highlight some of the stressors crews of autonomous ships may face in the future. Last but not least, relevant social impacts of autonomy will be considered. Section 4 then examines the potential effects moving forward, focusing primarily on the change in stress felt by human workers in the maritime sector. Lastly, throughout Section 4 and 5 a discussion will be had on what can be done now, and in the future to ensure changes in technology do not further stress the human-in-the-loop and instead enhance work conditions.

Autonomous shipping

There are several drivers for maritime autonomy, aside from the belief that reducing human decision-making at sea will facilitate a reduction in human-error triggered incidents. It has been estimated that in the near future the potential reductions of annual operation costs using autonomous ships could be up to 90%, by removing human crew and life support for those crews (Morris 2017). These changes could have a positive impact on carbon emissions, as slower ships on longer voyages would reduce environmental impact. If implemented correctly, autonomous systems could mitigate traditional stressors (e.g., long hours) by providing a better work environment or meet the global demands for qualified seafarers (ICS 2019).

However, there are some drawbacks to autonomy as it could lead to a situation where a manned ship would mean both increased wages, and less attractive voyages available to seafarers (Porathe, Prison, and Man 2014). Furthermore, fully autonomous ships have also been theorized as a way to remove human error; however, this may not be as straightforward an assertion as it seems. Many solutions across social, technical and social-technical domains have been developed to minimize human-caused accidents in the maritime industry. Despite this, accident rates have rarely deviated significantly over a considerably long history.

Comparing other types of autonomous vehicles, cars and unmanned air vehicles (UAVs), yields a similar trade-off between cost and benefit. In the public sector, the number of autonomous passenger flights is very low, as there are negligible cost savings to be had in removing the pilot cockpit while a significant increase in risk. There is, however, a greater uptake of military UAVs as these save costs and lower risk to the physical safety of personnel by removing the human pilot. Similarly, most of the pushback on autonomous cars comes from the presence of human passengers and pedestrians, with more incentives to pursue autonomous trucks on long hauls in less populated areas. Despite this, both UAVs and trials of self-driving trucks have focused on remote controlled solutions, relying on remote pilots and drivers for complex manoeuvres (Hancock et al. 2007; Clevenger 2019). Similarly, while the benefits of an autonomous passenger ferry or cruise ship could be considered low, other specializations in shipping could benefit more from higher levels of autonomy (e.g., LNG tankers).

Therefore, while the sector moves towards autonomous ships and ports, humans will continue to play an important role, although possibly altered ones, as will be discussed in a later section. For example, as long as there are manned ships sharing the same physical environments, situations will require cross-communication and interactions (Ahvenjrvi 2016). Moreover, no matter the level of autonomy, maintenance and ship-to-shore operations are still likely to include people. This could contribute to both stress and human error, meaning even highly autonomous systems could be negatively affected by these issues.

It is also important to note that physically servicing and interacting with autonomous systems may in itself be stressful, as the environment may be hazardous and not fit for humans’ long term. However, that is not the focus of this article. Instead, the authors are interested in: how remote monitoring and control will affect the human driver or manager; their role in a “fully” autonomous world; and the stress they may experience.

Degrees of automation

Lloyd’s Register was amongst the first maritime stakeholders to consider autonomy, and initially introduced six theoretical autonomous levels for shipping. These levels mirrored the SAE definitions for different levels of autonomous cars. The IMO has had maritime automation on their work programme since 2017 (International Maritime Organisation 2017). As such, the IMO have gone through several rounds of discussion to determine the correct number of degrees of autonomy which provided enough clarity, while not overcomplicating the regulatory process (International Maritime Organisation 2018b). To this end, the IMO decided upon four different degrees of maritime autonomy (DA) (International Maritime Organisation 2018a) (see Table 1).

Table 1. Tiers of ship autonomy including exterior human involvement based on (International Maritime Organisation 2018a)

Degreeof Automation	Description	Level of human involvement
Degree One	Ship with automated processes and decision support	Seafarers are on board to operate and control shipboard systems and functions. Some operations may be automated and at times be unsupervised but with seafarers on board ready to take control.
Degree Two	Remotely controlled ship with seafarers on board	The ship is controlled and operated from another location. Seafarers are available on board to take control and to operate the shipboard systems and functions.
Degree Three	Remotely controlled ship without seafarers on board	The ship is controlled and operated from another location. There are no seafarers on board.
Degree Four	Fully autonomous ship	The operating system of the ship is able to make decisions and determine actions by itself.

These fine-grained definitions have asserted how “autonomy” does not necessary mean “unmanned”, and that ship or port systems can operate at different levels depending on situation and capabilities. However, it is important to note that definitions for both autonomous cars and maritime systems primarily address a human driver/crew and how their involvement decreases as automation increases. Human workers, and therefore human error, are not limited to the “driver” or “decision making” roles. Tasks like maintenance and interactions with other manned entities (e.g., other ships, offshore structures, scheduling, passengers, port services), are likely to include people for the foreseeable future. One reason for this is that systems are not only limited from reaching DA4 by technological challenges but distrust in the system may prevent full autonomy. This has been seen in other sectors, where the main challenges to autonomy are often legal, social, political, or ethical (Ryan 2019).

More generally, DA1-2 (and to some extend DA3) is already seen today, where tools can provide suggestions and users can delegate some simpler tasks to software (e.g., car cruise control). Most solution developments today are aiming for DA2-3, where humans have indirect control of the system and instead primarily play the role of supervisor (Wärtsilä 2017; DNV-GL 2020). This could be through notifying the system on-route of critical events such as weather changes or newly declared “no go” zones like an oil spill in action. This way people can indirectly, locally or remotely, affect the system’s execution of voyage decision-making. Therefore, to understand the stressors faced by crews it is important to explore this human-autonomy relationship further.

The human-autonomy relationship

One of the major components of both traditional and autonomous shipping is situational awareness. In this paper, the authors determine that situation awareness can take on two differing definitions. Firstly, the term is defined specifically as an operator’s adaptive understanding of their surroundings and context when operating. Secondly, a much broader definition can be adopted, encompassing an understanding of the sector today and in the near future.

It has been shown that significant technological advancements are becoming a critical aspect of situational awareness. For example, in recent years, industrial robots in manufacturing have evolved to the point where they have changed the relationship between man and machine to become more reciprocal (Man, Lundh, and MacKinnon 2019). In the maritime industry, over the years, efficiency, safety, and environmental factors have similarly been addressed and reinforced by state-of-the-art technologies. Therefore, there is an increasing interface between humans and autonomy, which leads to the human-autonomy problem.

Automation tends to focus on less complex and menial tasks, which aims to lighten human workloads. However, despite well intentions, reductions in workload could actually cause more issues based on the implementation. For example, in experiments for autonomous cars, when driver workloads go below certain thresholds during automatized driving segments, drivers can experience what is called passive fatigue. This phenomenon is often the result of low cognitive loads when the driver does not have direct control, decreasing performance (Neubauer et al. 2012; Frederik Naujoks and Neukum 2014). When experiencing passive fatigue, road drivers are more likely to become distracted during high levels of automation (Natasha Merat et al. 2012).

In situations of this making, stress is even more likely to spike whenever a driver is surprised. This is most common when the driver’s situational awareness is suddenly proven incorrect, particularity when re-engaging with the vehicle or autonomous systems. The resulting confusion, trust or distrust in the automated systems often leads to a poor human decision-making in an increasingly stressful, and potentially dangerous, situation.

This dichotomy is also known as mode confusion, where the discrepancy between drivers understanding, situational awareness, and beliefs regarding their vehicle’s operations differs from how the vehicle is actually operating. This mode confusion is highly prevalent in autonomous cars up to level 3 (Drexler et al. 2018) and UAVs (Werner et al. 2020), meaning the future of autonomous ships will be equally affected. Not only is there confusion whether autonomy is “off” or “on”, but autonomous ships may be designed to operate at different levels of autonomy depending situations, and these may be triggered automatically or manually, as seen in Figures 1,¹ possibly complicating matters.

Figure 1. Possible autonomous situations that one, or several remote crews may be expected to handle when switching between modes of autonomy

This is relevant to the maritime sector as, prior to autonomy, accidents due to stress and fatigue occured when handovers were between just people (Andrew Smith et al. 2003). More specifically, it can be surmised from accident data (1989–1999) that fatigue-related incidences primarily occurred during the first week of the voyage, while situation awareness of vessel and context were lowest, and during the first four hours of a watch after a person-to-person hand-over, even in calm conditions for long sea routes. Challenges and likely stressors when introducing higher levels of autonomy, and therefore remote control, are multi-fold.

Firstly, automating the majority of a voyage could greatly decrease situational awareness and increase fatigue and hand-over confusion. To mitigate this, as well as justify salaries and with the nature of remote control, the authors predict that, in a future autonomous world, one remote crew may be expected to oversee multiple autonomous ships. Therefore, a remote control environment could increase the issue of situational awareness particularly if crew need to jump in to control ships possibly in different locations and contexts.

As seen in Figures 1–2, this could add further issues if incidences happen to occur simultaneously in different ships, locations, and situations, adding the stress of prioritizing actions and assets. Furthermore, it is possible that at DA3, when autonomy takes care of most tasks, a crew monitoring multiple ships could, on average, be under stress more often. For shorter sea and coastal routes, (Andrew Smith et al. 2003) found that seafarers had higher levels of fatigue, with 52.6% of the respondents asserting that the stressful conditions encountered were potentially dangerous. While this is not a direct comparison, these shorter routes, with a higher ratio of high to low stress moments, demonstrates how automating away low-stress periods out of longer voyages can be detrimental to seafarer mental health. Moreover, in past research, experiment duration may be an issue, as most which referenced air and road used 60–90 min trials, and it is not clear how working under these conditions long term affect stress cumulatively on the order of weeks, months or over a career.

Figure 2. Possible workload stress ratio comparison between manned and autonomous ships

Lastly, although rare, it has been reported that people may feel motion sickness in simulations, autonomous vehicles, and visualizations used during remote control. For example, 10% of participating adults often experienced motion sickness in autonomous vehicles (Byrne and Parasuraman 1996). It has been theorized that this due to a dissonance between reality and internal beliefs of the car driver (Michael Regan and Stevens 2014). This has also been seen in drone pilots (Koch et al. 2018) in addition to drivers, or passengers, of autonomous cars. Motion sickness could therefore be a potential issue for seafarers working remotely. Depending on the visuals employed for remote control, experienced seafarers may feel motion sickness without the physical movements they normally associate with their surroundings, while both experienced and inexperienced may feel motion sickness due to the dissonance between visual and vestibular inputs (Koch et al. 2018). Daily discomfort, even with 10% of users, if unmitigated could increase stress, decrease the appeal, and result in more incidences. Therefore, it is important to consider how realistic situational awareness can be achieved by human operators, without adding new stressors.

The reality of remote control

To examine a part of the human-autonomy problem one project explored a solution where ships were manned while departing and re-entering ports (DA2), but unmanned and controlled autonomously with remote monitoring and control during ocean passages (DA4) (Porathe, Prison, and Man 2014). In this particular project, the challenge was to maintain adequate situational awareness through remote sensors for a shore-based crew, as inadequate or confusing information would increase stress, and likely human error as well. This in itself may be difficult, as there are many different elements (e.g., wind, waves) a mariner takes into consideration when at sea (Prison, Dahlman, and Lundh 2013). It is important to note that the trans-oceanic phase, often the least stressful with the exception of incidences, is the least difficult part of the voyage. However, in more stressful situations such as rough weather the crew is then expected to increase control over the ship to deal with the situation.

If fully implemented a remote control crew may be expected to oversee the operations of several ships, which could significantly increase difficulties and stress. An example of how remote crews may be expected to assist operations at DA3-4 for multiple assets can be seen in Figure 2. The manned part of this image shows one crew per ship. The red sections demonstrate moments of high stress on board two DA1 ships, particularly during pilotage in and out of ports at the start and end of each journey. The high stress moments during the middle part of the voyage are contributed to difficult manoeuvres, rough weather, incidences with other ships, attacks, break downs etc.

The second part of the image shows the high stress moments that a single crew in charge of multiple assets face. This is one area the authors would highlight for further research in Section 5. Single crews operating multiple assets is a real possibility in the future as many digital products and services have been introduced in the shipping industry to access system information and maintain control, such as energy-efficient fuel monitors (Zhang et al. 2019), unmanned cargo handling systems automated terminals (Luo, Wu, and Mendes 2016), and autonomous ships (Ahvenjrvi 2016; Tam and Jones 2018; Levander 2017), etc. The growth of smart systems continues to shift the human operator’s role from a traditional controller to a collaborator, manager, or supervisor of the system (Dekker and Woods 2002). With more equal roles, it is important to maintain user-friendly approaches but also see if stress can be decreased for the human element, and efficiency increased with the technical component. However, it is important to also consider that at low stress times a single operator could be in charge of monitoring multiple vessels simultaneously.

If fully implemented, a remote control crew may be expected to oversee the operations of several ships, which could significantly increase difficulties and stress. An example of how remote crews may be expected to assist operations at DA3-4 for multiple assets can be seen in Figure 2. The manned part of this image shows one crew per ship. The red sections demonstrate moments of high stress on board two DA1 ships, particularly during pilotage in and out of ports at the start and end of each journey. Grey represents periods of low-stress. The high stress moments during the middle part of the voyage are contributed to difficult manoeuvres, rough weather, incidences with other ships, attacks, break downs etc.

The second part of the image shows the high stress moments that a single crew in charge of multiple assets face. This is one area the authors would highlight for further research in Section 5. Single crews operating multiple assets is a real possibility in the future as many digital products and services have been introduced in the shipping industry to access system information and maintain control, such as energy-efficient fuel monitors (Zhang et al. 2019), unmanned cargo handling systems automated terminals (Luo, Wu, and Mendes 2016), and autonomous ships (Ahvenjrvi 2016; Tam and Jones 2018; Levander 2017), etc. The growth of smart systems continues to shift the human operator’s role from a traditional controller to a collaborator, manager, or supervisor of the system (Dekker and Woods 2002). With more equal roles, it is important to maintain user-friendly approaches but also see if stress can be decreased for the human element, and efficiency increased with the technical component. However, it is important to also consider that at low stress times a single operator could be in charge of monitoring multiple vessels simultaneously.

In terms of technology, at the various degrees of autonomy, there is still a question of whether situational awareness can be maintained by sensors and data, no matter the detail or amount, without high fidelity. While this aspect has not been thoroughly explored for remote monitoring or control, there has been similar research in simulations that are worth discussing.

A critical element in current simulation technologies, in maritime and aviation, is high-fidelity so trainers receive accurate mariner and pilot training (Liu et al. 2020; Myers III, Starr, and Mullins 2018; Tam, Moara-Nkwe, and Jones 2020). Although costly, the benefits seem clear when preparing trainees for real environment, as highlighted in (Winn et al. 2006) students learning oceanography using field experience, even with little prior understanding could learn quickly.

Simulation, on the other hand, was more effective to help students connected with what they have learned in other classes. A study involving medical students has also demonstrated how high-fidelity simulation was not usurious to low-fidelity ones, but did also lead students to be overconfident in their learned skills (Massoth et al. 2019). It is not clear how this may translate to crews moving from local control to remote, or the mental effects of being disconnected from the environment they are normally immersed in. If mariners no longer physically enter environments like a ship, and instead employ remote control from a centre, fidelity in simulations, cyber-ranges, and visualizations may be decreased without the cost justifications.

Major stressors in remote control

Within transport sectors there are two areas of remote control, that are most likely to increase stress, these are (1) the handover between manual and automation and, as previously insinuated, (2) inadequate situation awareness, feedback, or response times from vessel/vehicle to control (Porathe, Prison, and Man 2014; Larsson 2010). It is important to note that there is a precedent for errors within handovers in the maritime sector. (Andrew Smith et al. 2003) asserts that errors occur during human-to-human crew handovers especially during the first week of a voyage as crew get used to their surroundings. Thus, also illustrating the important relationship between the handover of control and situational awareness.

The handover between manual and automation

Around 75% of all car accidents are consider human error as a casual factor, however, it has been shown that autonomy brings with it a variety of other challenges that could actually increase risks regarding road safety (Neville and Marsden 1996; Dennerlein, Ronk, and Perry 2009). Studies show that prolonged autonomous driving periods can contribute to an increase in fatigue, with drivers in automated conditions displaying a 2% increase in drowsiness when compared to the same drivers in manual mode (Hamish Jamson, Merat, and Oliver 2013). Other studies have also highlighted the link between, driver fatigue, the sudden need to perform operations designed to be performed by AI and algorithms, and the additional or shifted stress that is placed on the remote driver (Shaikh and Krishnan 2012).

DA2 is the point at which similar challenges will be faced by maritime operators and the handover between human and machine. However, in most cases the change in DA will not be routine, but instead be triggered by an unpredictable situation. For example, (Natasha Merat et al. 2014) stressed that most real-world handover situations are likely to be unpredictable. Under these assumptions, it is important that research and studies can determine how best to alert drivers of an imminent handover event. This would allow an appropriate amount of time for the human supervisor to gain an appropriate level of situation awareness and switch to human driver, regaining manual driving more efficiently. However, alarm fatigue is a real issue in vessel operations and something that should be monitored particularly when considering adding more, as this could lead to alarms being ignored or lost in the confusion (Fan et al. 2017; Lees and Lee 2007).

As discussed earlier, there are benefits, especially within military UAVs, to employing remote control capabilities. However, it is still important to give consideration to the different stressors these controllers now face. UAV work is still considered risky and can lead to significant psychological stress. All of which can be contributed to various factors including mission workloads, working hours which may be affected by the UAV’s time zone instead of the remote pilot’s, or mission-related situations where lives are at stake.

To different degrees all of these may be applicable to the remote control of maritime activity, from military operations to the shipment of important goods (e.g., fuel, medicine, food). While the physical safety of UAV operators is more secure, studies have shown that pilots’ mental well-being can be severely affected by specific stressful tasks (Wayne et al. 2014). UAVs can be used in the civilian sector they are frequently for inspection and monitoring, and in the military they can also be used for reconnaissance and rescue (Chappelle et al. 2014; Guznov et al. 2011).

These mental demands have been shown to bear consequences such as higher workloads, stress, and diminished situational awareness (Chappelle et al. 2012, 2014; Hancock et al. 2007). More specifically, it has been found that 14 to 16% of the UAV operators in the US military reported high levels of mental fatigue during missions (Wayne et al. 2014; Werner et al. 2020). Further existing research shows very little empirical experiments in this field, despite demonstrations of increased emotional distress among drone remote UAV pilots (Valenzano et al. 2018). While there may be some overlaps of interest, more generally the context in which remote crew and UAV pilots operate differ in operation, outcome, and duration. However, if navies develop equivalent drone capabilities, differences will lessen.

Inadequate situation awareness, feedback, or response times from vessel/vehicle to control

Considering that while workers on site or on-board may have more traditional stressors, like long hours (Raby et al. 2001), their situational awareness may be impaired, making critical decisions harder and more prone to error (Hancock et al. 2007). This then means that as the degree of automation increases, and the autonomous systems have more direct control, it becomes more important to build human trust in these systems. It is not unfeasible to assume that there are cases where the system does not know how to react to new situations, even with user guidance, meaning autonomous systems many need to be bypassed. This could be achieved by disabling the autonomous systems or, as mentioned above, decreasing the automation level if designed to allow different autonomous levels.

However, consider the situation were a local crew or are not available to receive and give commands, a remote situation room would need to be used (Porathe, Prison, and Man 2014). Thus, a suite of technology is required to support all these human-autonomous actions, at all levels of autonomy. This suite includes large sensor or IoT networks, which are designed to improve remote monitoring and control for both machine learning AI algorithms and human supervisors.

However, while an increase in devices and data could help to lessen operational stress, they also lead to an increase in cyber risks leading to misinformation, loss of control, delays, damage, and theft (Tam and Jones 2019b, 2018) could have the opposite effect and increase stress. How to balance the use of technology, the new maintenance and cyber issues it causes, and the effect on human workers are therefore all critical questions that need to be researched in depth.

Social factors

Apart from the technical advancement of the Internet-of-Things (IoT), information technology, operational technology, and more across all sectors to support autonomous systems, there are also social factors involved in the human part of the human-autonomous relationship. As mentioned previously, seafarers face a considerable amount of stress, often more than those on shore (Raby et al. 2001; Oldenburg and Jensen 2019; Tony et al. 1997). Work hours vary greatly depending on job title (e.g., engineering, wheelhouse), but all will either have long shifts and/or will always be on call after shift. Seafaring contracts typically last between three and nine months, with an average duration of voyages increasing due to environmental regulations. This can be dangerous to physiochemical and physiological functions (Lu et al. 2010), which is why international maritime labour laws say seafarers have a right to return home at the end of their contracts, with no cost to themselves (ILO 2006).

However, longer voyages and current situations such as Covid-19 lock-downs and demands continually add stress to these situations (Apuzzo and Gebrekidan 2020). Communication and entertainment technology have been introduced to ships to try and mitigate isolation and work conditions. In one survey for seafarers, 92% said that Internet access strongly affects their decision on where they choose to work. This was ranked just below necessities such as accurate and timely salaries, benefits (e.g., healthcare), training, and quality food (Navarino 2018).

While 69% of mariners considered Internet connectivity as a positive effect in 2018, 17% saw new technologies as a threat. This could be a reference to cyber-attacks, which can take advantage of this connectivity to socially engineer people or attack cyber vulnerabilities in both IT and OT systems (Tam and Jones 2019b). It varies between surveys, but (Navarino 2018) results show 47% of seafarers believe they have sailed on a vessel that was the target of a cyber-attack. While the initial cyber-attacks are dangerous, growing distrust in systems and information can become a troublesome secondary effect. What is more, there is evidence that staff are not provided adequate cyber risk awareness training, (Markit 2019) asserts that in 2019 only 64% of staff received training, with 75% rating it as good.

However, distrust in technology could also be a reference to social factors, as 53% of seafarers in 2018 believed that Internet and other crew communication tools has led to a decline in social interaction on board. This factors into both the job aspect, when communicating remotely, but also during personal time as people may be more likely to reach out via the Internet for entertainment and social interactions. Due to significant workloads, automating a set of smaller tasks has been well received, however it is not clear whether further automating tasks and having humans work in potentially siloed centres to support autonomous systems could exasperate these issues.

In this survey (Navarino 2018), more than half seafarers have also seen at least one element of their role automated since 2016, with 98% saying this had a positive impact on their role. However, there is a significant difference between using the Internet to access the outside world from the ship as opposed to accessing a ship to execute a task, and there is a difference between automating menial tasks versus the all or most of a job’s tasks. Given this, in comparison, only 38% of seafarers viewed unmanned ships as an opportunity. To meet the demands for seafarers (ICS 2019), it is important to consider the possible social issues and benefits they may face in an autonomous world as isolation, workload, work hours, could be negatively affected if the focus is on the technology. That said, as discussed more in the following section, this could also improve these issues and more, such as worker diversity (e.g., disabilities, gender balance).

Potential effects

As discussed, the limitations in situational awareness and its bearing on human error could still result in remote operators misinterpreting information or being overly reliant or even distrustful of the autonomous systems (Saffarian, Joost, and Happee 2012). This leads to poor human–autonomous relationship and could cause human-error due to stress and poor handover of information or controls. Within literature regarding fatigue and stress, there is a significant amount of studies using controlled laboratory environment for pilots, mariners, nurses, and more (Allen et al. 2014; Chinda and Hamer 2008; Hays et al. 1992; Foronda, Liu, and Bauman 2013). However, these laboratory-based tests do not accurately translate to real, specific, contexts. The ratio of high-stress to low-stress operations is much higher in simulators, as a long flight or voyage can be drastically reduced to a shorter, often 90 minutes or less, exercise. Therefore more studies on stress outside a controlled environment, while more difficult, may be needed to fully understand this issue.

However, it is important to note that these studies did see a clear correlation between stress and the complexity or difficulty of a task (Main, Wolkow, and Chambers 2017). Therefore, achieving complex tasks that technology either cannot do, or humans to not trust machines to do, will continue to have significant effect on worker stress levels. Examples of guidance for human-automation relationships across sectors include Unmanned Aerial Vehicle (UAV) guidance, health care, and computer security (Chen 2010; Holzinger et al. 2016). Given Section 3’s awareness of the sector and current context, this section attempts to determine the possible changes in stress, workforce diversity, and training moving forward into an autonomous world where human element will still be present. Moreover, remote control and monitoring will be needed for DA2-3, as full autonomy (DA4) is unlikely to be fully realized in the near future, if ever, as seen with both autonomous cars and UAVs.

Workforce diversity

If implemented correctly autonomy and remote control can help enable more workers, increasing workforce diversity. Remote-controlled robots, or avatars, have been created for a number of different situations that can be piloted by disabled people who would otherwise be unable to perform those actions and tasks (Takeuchi, Yamazaki, and Yoshifuji 2020). That said, while there is an extensive amount of research on how autonomous vehicles can improve the lives of passengers, it is less clear on what benefits it could have on new, possibly disabled, drivers (e.g., remote controlled trucks) (Bradshaw-Martin and Easton 2014). As indicated earlier, the reason for this could simply be that experiments surrounding autonomous control normally use people that have several years of experience driving instead of participants with no experience.

Therefore, it seems current studies on introducing autonomy to air, land, and sea tend to focus on the transition of the existing workforce into remote control autonomy, with little to no focus on the impact of new/minority demographics entering or growing into the workforce. This is another research area worth exploring, particularly if high, but not fully, autonomous vehicles become more common as the workforce will have a decreasing amount of people with real-world experience, and an increase in workers with remote access or managing experience. Lastly, much like military pilots and truck drivers, seafarers traditionally have had low percentages of women. It has been estimated that only 2% of the world’s 1.2 million seafarers are women, and with 94% of those working solely in the cruise industry (International Maritime Organization 2020; Thomas 2004). There is a possibility that the change in work environment could be both more appealing to prospective female applicants, but also lessen social constructs that have previously limited opportunities to women in traditional seafarer jobs and careers.

Furthermore, remote control, especially in the higher DAs (3–4) could bring about a change in the traditional watches adhered to by on board crew. Utilizing office spaces for remote control stations frees up the time constraints of operators, allowing them to share the burden of operations with others. This could potentially lead to operators following a more traditional 9–5 working pattern, with others taking over in shifts. Following a similar work pattern to traditional office based jobs could help to improve the diversity of the maritime workforce, by reducing the negative appeals of the work to those with families or travel limitations.

Training and regulations

It has long been ingrained in IMO practices to consider the importance of training and safety. The IMO stipulate that the Company should establish and maintain procedures for identifying any training which may be required in support of the SMS and ensure that such training is provided for all personnel concerned (International Maritime Organisation 1993a). What is more, the IMO have drawn a clear distinction between using training as a way to manage the impacts of stress and fatigue on crew (International Maritime Organisation 1993b). So, while increasing levels of automation does not necessarily reduce the number of human error-related accidents, it does mean training should be provided to ensure a greater level of maritime safety.

However, standards such as ISO 26,262 only address functional safety and not the role of human users and interventions (i.e., remote access). Moreover, in most difficult, high-stress and high-risk situations, it is not technically and/or ethically possible to fully automate systems, meaning training and guidance is needed on top of stress mitigating strategies. Other studies have stated that current managerial procedures do not provide sufficient support for learning to evolving systems within the organisations (Johnson and Andersson 2014; Tam and Jones 2019c). Social-culture constraints also factor into this situation, as many offshore and onshore organizational structures mostly promote top-down management, with practitioners often too far from key decision-making processes that will affect the difficulties and stressors in their duties (Johnson and Andersson 2014).

When asked directly, 85% of seafarers either agreed or strongly agreed that seafarers will remain an essential component in the long-term future of shipping, even with autonomous ships (Meadow, Ridgwell, and Kelly 2018). However, 54% thought many on-board roles could be relocated to shore, with many voicing doubts about deck and engineering officers. In terms of education, 63% agreed or strongly agreed that the current workforce at sea could be trained to handle remote operations. It is unclear if the participants would believe less common worker demographics, as discussed in Section 3.2, would feel similarly about new generations of seafarers. Within the same survey, 50.7% of participants strongly agreed that current sea and shore-based training will need to be changed to prepare the workforce for remote operations of autonomous, with 38% in choosing “agree” (i.e., 89% in total), with 4.5% neutral and under 4% disagreeing or strongly agreeing.

Although this aspect needs to be addressed globally by the IMO, the International Convention on Standard of Training, Certification and Watchkeeping for Seafarers (STCW) has however, as of 2020, has not addressed this aspect of seafarers training for the future of shipping (Emad, Khabir, and Shahbakhsh 2020). The MASS regulatory scoping exercise (RSE) is primary reason for a delay including autonomy within the STCW Convention. The RSE was a systematic review of all the major IMO instruments to determine the impacts that autonomy (at all levels) would have. Following the completion of the review in late 2019 the next phase of the RSE is to suggest amendments to, or develop new, instruments which address autonomy (International Maritime Organisation 2020). However, (MARITIME UK 2018) provides a brief overview of some of the training requirements that crews of DA3-4 ships will require.

As mentioned previously, basing any training off of experiments may be dangerous as they may biased to the current workforce, despite the fact that autonomy could significantly diversify and add to existing workforces. Similarly, most land and air-based studies in the past have hired experienced drivers and pilots. This can introduce a new challenge, as moving pilots, drivers, and mariners into a 100% remote control environment could degrade their sector-specific skills long term. For example, in one simulated driving experiment, it was found that brief periods of highly-automated driving was enough to impair driver performance in a subsequent, real-world, manual driving tasks (Skottke et al. 2014). This highlights the possibility that experienced drivers may perform well in automated settings, but that their skill to perform those tasks may degrade over time when removed from real-world settings. It is not clear whether learned skills only relevant to remote control were developed. Furthermore, it is uncertain what long term effects remote control will have on newer workers with minimal experience the real system in the first place. It is likely that seafarers with little to no real-world experience will struggle with some situational contexts they have no reference for, but it is also possible that remote monitoring and control will develop into a separate skill set from previous manned-ship skills.

When discussing training for remote access, there are several areas of concern. Both machine and human should learn to predict the need of a handover in order to mitigate the shock of a transition from machine to human, or vice versa, to reduce error. This can be done with heightened situational awareness and would decrease stress by providing time and information to ease the transition as far in advance from a potential incident as possible. Training in this area would be to teach personnel how to effectively, and passively, monitor systems remotely and practice handovers and practice the timing. A technical contribution that can help with this is well-designed human-machine interactions (HMI) so remote workers can have quick and easy access to processable situational information when a handover is imminent. This must also take things like alarm fatigue into consideration, as information at the wrong time or a flood of data could lead to inaction or a harmful action (Fan et al. 2017). It is also very important to education the limitations and strengths of the autonomous systems they are controlling, as the right amount of trust is imperative (Drexler et al. 2018; Saffarian, Joost, and Happee 2012). This again highlights how the human-autonomy relationship is more collaborative than previous human and maritime technology dynamics, and knowing the weakness and strengths of both sides is critical for effective partnership (Dekker and Woods 2002).

Future research and discussion

With high levels of autonomy (i.e., DA3-4) solutions are becoming more accepted in the maritime sector, there are several areas of concern relating to the human component of the developing human-autonomous picture. This paper has reviewed works showing that, because of trust and technical challenges, it is unlikely full autonomy will be reached in any sector in the near future, if at all. Because of this, it is likely that remote access, monitor, and control technology will evolve at the same rate as autonomy in order to satisfy high levels of autonomy, but with the option to switch back to manual. However, existing projects have shown that, because of all of these factors, autonomy is mostly likely to automate away “easier” tasks, i.e., tasks that were low-stress for people to execute. Given the review of existing work above, on stressors in the human-autonomy relationship across sea, air, and land, this paper discussed the following areas for future research which the authors feel like have not been fully investigated by other works:

• Approximately 90% of human-caused accidents occur in confined waters (Cockroft 1984), hence the focus of this paper was on autonomous ships. It would be interesting to see if remote control will balance these statistics more in the future by reducing the difference in work environment.

• What would crew workloads be like if majority of low-stress tasks were automated and only primarily high-stress tasks were left to be dealt with manually (Porathe, Prison, and Man 2014)?

• With more tasks automated, would crews only be assigned one ship and risk fatigues associated with long periods of automated inaction (Neubauer et al. 2012)? What affect would that have to situational awareness during handovers, fatigue and stress?

• With more tasks automated, will crews be expected to monitor and control more than one asset? How would this effect situational awareness during handovers, fatigue and stress?

• Will it be easier for future workforces to be more diverse for gender equality and disabled individuals? Will they handle or react to stress differently?

• What are the long-term effects of remote-controlling autonomous systems, must these skills be built on real-world experience or can it be built entirely on remote access training? Do both manned and remote skills degrade without continuous real-world training (Skottke et al. 2014)?

• What existing skills can be transitioned to remote control, and what new skills need to be created? How can be training be improved to teach these skills?

The reasoning for these questions can be found in previous sections, with the remainder of this sector discussing additional areas of future research and topics for discussion. Despite the focus on autonomous ships here, it is important to examine stress for humans in autonomous ports, not only regarding the handover between man and machine but also the transition of goods from different modes of transport, for example, sea to rail. The last three topics to be reviewed are how to measure stress, in order to better quantify, study, and mitigate stressors, discuss good HCIs, which have shown to be critical in optimal human-machine interactions in previous studies, and lastly cyber security. While each of these are significant topics alone, this paper will primarily be looking at the aspects that would most affect stress in workers in an autonomous world. Another possible area of influence, although probably with impact, is ongoing research on remote working stressors during the COVID-19 pandemic (Hayes and Curran 2020). With a large proportion of people working remotely for parts of 2020, data on stress related to remote working is likely to increase and lessons learned from this can also be implemented in conjunction with autonomous solutions. However research in that topic has not yet begun being published. Similar, previous, studies have highlighted weaker ties with the organization when working remotely (Perry, Rubino, and Hunter 2018), and while a member might on average have more ties working in a control centre, they may also lose stronger ties with those they may have worked with physically closer. This issue may also be compounded if individuals are a part of multiple “remote crews”, as postulated earlier.

Limitations in current research

While these studies are useful in ascertaining some of consequences of remote control vehicles, they are limited as many of these psychological studies have mainly centred around perception and experience, not measuring stress (Ryan et al. 2010). Furthermore, the authors were unable to find any studies in transportation that measured stress of remote monitoring and control, especially when preformed as a full-time job instead of a controlled experiment that rarely lasted more than 90 minutes. What is more, ships that may be designed to operate at different DAs depending on situation may complicate this issue, and the fact that ships often have longer trips than air and truck, on the order of weeks, can create unique issues, which are not accounted for in previous studies (Tam and Jones 2019b).

Other findings from similar transportation sectors are limited due to the employment of their test subjects and simulated driving methodologies. Many of the controlled experiments included experienced drivers, but in the long term as more pilots, drivers, or seafarers develop their careers in an autonomous world, they may not have enough real experience to help them navigate simulations and remote control actions. Furthermore, to be discussed further in a later section, there is a concern that real-world skills may degrade when working remotely and that may negatively affect both remote and real-world skills. Therefore, further research is needed to ensure a harmonious handover between human and machine.

Measuring stress

Being able to measure stress is an important part of assessing acute stress effects in both experiments but also during daily activities. As many past studies have focused on short intervals for study, data taken over longer periods could give the sector insight into areas that have been less addressed. While there are issues surrounding this, as measuring stress in real-life situations could actually add to the stress, modern day devices and the IoT may be able to mitigate that. For example, in a survey mariners say they have increased the number of personal devices they bring on board to an average of three per person (Meadow, Ridgwell, and Kelly 2018). This includes wearables for measuring and tracking fitness, and some of the following stress measurements could use that information to better determine levels of stress.

Reading mariner biology readings, i.e., bio-signals, has two unique advantages over other methods such as post exercise surveys, interviews, and simulated experiments. Advantages include continuous psychophysiological monitoring during real-tasks tasks, and the option of acquiring high temporal resolution of the measurements, although the method for acquiring quality of data should also have minimal impact on the participant. Different bio-signals are used to detect the psychophysiological states. Examples what can be studied to measure stress are salivary cortisol and heart rate (Main, Wolkow, and Chambers 2017; Main and Chambers 2015). Measuring electroencephalography (i.e., EEG) is also another method, which observes the electrical activity within the human brain for stress (Liu et al. 2017). This, however, is an example of high resolutions achieved, but may not be as applicable to real-world experiments as, for example heart rate. For further shortened lab exercises or simulations however, this method could yield interesting research (Liu et al. 2020).

There are limitations to bio-signals from wearables, as these results need to be put into operational context. For example, readings that just show heart rate over a voyage may be a good indicator of the periods of time that crew spent with elevated heart rate. However, this data does not ascertain why it was high, i.e., what situation or event was occurring to initiate that rise. Therefore, measuring stress in this way would need to include copious logs and records of particular events so that some form of correlation could be drawn.

Human-control interaction

To better facilitate handovers and other interactions, it is important to determine that the best HCI practices and solutions are to enable smooth handovers and maintain situational awareness without surprising mariners or inducing alarm fatigue. Optimal human-machine interfaces are critical as the main point of information and command exchanges between system and human supervisor or controller. Examples of data needed from remote monitoring and controlling of maritime autonomous surface ship (MASS) from a BCS can be found here (Maritime UK 2018). The right HCI solution might vary between different mariner jobs, not just port and ship, but within those contexts. Signals for manual take-overs must carefully balance urgency, driver workload, and alarm fatigue. In a recent simulated driving study (Zeeb, Buchner, and Schrauf 2015), findings showed that tracking eye movement during automated driving can predict if a person is ready to take over a vehicle. In contrast to other studies this is, in a way, a method for the machine to gain better awareness of their human counterpart. It is also still unclear how multiple unfolding conflicts should be communicated to drivers. In a situation where a person is in charge of monitoring and aiding multiple assets (e.g., ships) this is an issue surrounding unforeseen issues as seen in Figure 2. Moreover, different alerts that signalling multiple conflicts in a small period of time may confuse the driver, potentially delaying reaction time or stressing them enough to make an error (Fitch, Bowman, and Llaneras 2014). More research surrounding these topics in particular is likely needed if industry moves to one crew monitoring multiple sets of autonomous systems. It is important to also find what information is best displayed for the remote monitor (e.g., current AL level), and with how much urgency.

Cyber security

Lastly, cyber security is critical for protecting ship assets, goods, infrastructure, and workers in the maritime sector, onshore or offshore. Understanding the security of a remote access environment from is critical for preventing both attacks and high-stress stress during an attack, whether or not it is successful. Maritime-cyber is a relatively new field and may need new facilities for researching (Tam and Jones 2019a) and training (Tam, Moara-Nkwe, and Jones 2020). Similar to the issue of training for an autonomous age, a survey in (Tam and Jones 2019c) found that 70% participants have not had any training cyber security, but 75% thought that addition cyber safety training would be useful for their tasks. Making sure systems are trustworthy despite security risks is also a significant, as a hacked system could not only stop working, but it could also provide false or manipulative information to intentionally change human or machine behaviours (Tam and Jones 2019b). A cyber-attack could therefore significantly damage trust in autonomous systems, which as stated in previous studies can cause as many human errors as over reliance. It is also unclear how quickly a person can regain trust in a system if an incident were to happen. That said, it is also unclear on how long a person would trust false information coming from a system that is normally functioning correctly, and how too much trust could help hackers in achieving their goals or extending the duration of an attack.

Conclusions

Despite some beliefs, the arrival and growth of autonomous systems and drivers in the maritime sector does not mean the human element will be phased out. With the limitations of technology and an inherit distrust in autonomous systems dealing with difficult situations, full autonomy is likely to never be reached at ports or at sea. The sector is facing increasing diversification of shipping technology and practices, with some old norms disappearing. With this, the range of skill requirements and work environment also diversifying. Besides maintenance and other tasks that will need to be done in person, it is likely that many existing jobs will be remote monitor or control enabled to promote high levels of autonomy, but always with the option of switching to manned control.

The potential issue is, due to trust and complex tasks being more difficult to programme, automated systems are likely to be given the simpler, more menial tasks, potentially leaving the human component of the human-autonomy equation with majority of the more complex, and therefore high-stress, tasks. Not only could autonomy not reduce human error, but this could increase the amount of stress felt by mariners. There are a variety of questions that need to be asked when implementing autonomous solution, including, what skills are required on board if an autonomous ship changes DA, what is the minimum staffing level required to maintain safe and secure operations and all levels of autonomy, where are the personnel responsible for maintaining these systems going to be based?

It is therefore clear that managing stress will be key to future shipping, as it has been, but with a different emphasis to provide a supportive learning and working environments. In a review of other sectors and the way autonomous systems are growing those contexts, this article attempts to highlight real-world concerns when moving ahead with autonomous ships in particular. This article concludes with several areas of future research that may be important in creating a healthy work environment, and non-harmful workloads, for mariners as autonomy continues to develop.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This paper is a partly funded by the research efforts under Cyber-MAR. Cyber-MAR project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 833389. Content reflects only the authors’ view and European Commission is not responsible for any use that may be made of the information it contains.

Notes

¹ Background image taken from https://www.shipmap.org/