A Systematic Literature Review of Augmented Reality for Maritime Collaboration

Abstract

Augmented Reality (AR) could improve maritime collaboration by facilitating human factors like situation awareness and decision-making. However, it is not clear in which ways AR can be beneficial and for which maritime operations. This paper addresses the need for a systematic literature review of state-of-the-art ways AR can facilitate maritime collaboration. Following the PRISMA statement, we searched for the constructs “AR,” “maritime operation,” and “collaboration” in the Web of Science and IEEE Xplore databases. Out of 691 search results, we retained 32 publications for analysis. Ten publications from other sources were included. Our results provide a review of AR applications for the maritime operations of ship navigation, personal navigation, maritime construction, maritime maintenance and inspection, and other. We suggest that research focuses on bringing promising AR applications to higher technology readiness levels, learning from other industries, applying robust evaluation methods, and grounding more on human factors like decision-making, situation awareness, and communication.

Keywords:

• Augmented reality
• augmented reality
• mixed reality
• maritime operation
• safety-critical
• collaboration
• situation awareness
• decision-making
• communication

1. Introduction

The role of human collaboration has been raised as a challenge for maritime safety (see Batalden & Sydnes, 2014; Chauvin et al., 2013; Lee et al., 2020; Sánchez-Beaskoetxea et al., 2021; Zhang et al., 2016). In a study of 39 maritime vessel collisions, (Chauvin et al., 2013) pointed out crucial causal factors such as decision-making, inter-ship communication, and Bridge Resource Management. In a study of 94 cases of maritime vessel accidents, Batalden and Sydnes (2014) identified several causes related to collaboration such as shipboard management and the planning of shipboard operations. Lee et al. (2020) discusses the role of collaboration for safe operations in vessel traffic systems and on offshore platforms. According to Sampson et al. (2022) collaboration also influences safety in maritime evacuations.

Maritime safety becomes more relevant as the maritime industry grows (Nilsson, van Overloop, Ali Mehdi, & Pålsson, 2018). Further, as climate change leads to more intense and frequent extreme weather events, maritime safety should be further emphasized (United Nations Economic Commission for Europe (UNECE), 2020). Improved collaboration has been shown to increase energy efficiency (Waterborne Technology Platform, 2021). Collaboration plays a role in a wide spectrum of maritime operations, such as ship operations (Sánchez-Beaskoetxea et al., 2021), offshore platform operations (Lee et al., 2020), offshore windmill maintenance (Preisser et al., 2019), and ship construction (Blanco-Novoa et al., 2018). In this publication, we focus on maritime operations in which humans play a role; we use the definition of collaboration proposed by (Bedwell et al., 2012, p. 3): “An evolving process whereby two or more social entities actively and reciprocally engage in joint activities aimed at achieving at least one shared goal.” Given the importance of safe and efficient collaboration in the evolving field of maritime operations, there would be great value in identifying ways to improve this collaboration.

Various factors have been argued to be important for high-quality collaboration. For example, Warner et al. (2005) emphasized team decision-making and related cognitive factors, such as shared mental models and communication. Similar factors were identified by Cooke et al. (2013), namely: team planning, team decision-making, team problem-solving, team performance, shared mental models, team mental models, shared situation awareness (SSA) and team situation awareness, shared understanding, team coordination, and team communication. These collaborative factors are underpinned by individual cognitive factors such as decision-making and situation awareness (SA). Collaboration is also influenced by contextual factors, such as time constraints, risk, and task characteristics (Reader, 2017).

Augmented reality (AR) is a relatively new technology in the field of maritime operations, but it is seen to have the potential for facilitating collaboration (see Blanco-Novoa et al., 2018; Davidson & Sanderson, 2022; Merino et al., 2020; Rowen, Grabowski, Rancy, et al., 2019). In this publication, we use the definition of AR proposed by Azuma (1997) and refined by Azuma et al. (2001): “An AR system supplements the real world with virtual (computer-generated) objects that appear to coexist in the same space as the real world.” AR can be displayed on different types of hardware, such as immersive headsets (Frydenberg et al., 2021) and fixed displays (Oh et al., 2016). Often, the virtual objects are visual; in some cases, other senses are also involved, such as hearing (Lyons et al., 2000).

Some research has been done on AR for maritime collaboration. For example, AR can display a ship’s heading and speed (Frydenberg et al., 2021) or camera images on fixed displays showing traffic information (Oh et al., 2016). Alternatively, AR can be provided through hand-held tablets showing objects obscured by other objects (Stylianidis et al., 2020). While there are some studies of AR applications for maritime operations (e.g., Blanco-Novoa et al., 2018; Frydenberg et al., 2021; Rowen et al., 2019), their prototypes generally were developed to a low technology readiness level (TRL). TRLs are a method for estimating the maturity of a technology during its development by grading them on a scale (Héder, 2017, p. 11). Another reason we think AR may be useful for collaboration in maritime operations is that it is being studied and implemented for collaboration in other fields, such as product design, factory planning, and maintenance (Lukosch et al., 2015), and manufacturing (Marques et al., 2022). Similar operations can be carried out in maritime contexts. While AR has been implemented in several industries, its effects on operator performance are understudied (Grabowski et al., 2018), particularly in the maritime industry (De la Peña Zarzuelo et al., 2020; Gernez et al., 2020). Furthermore, there is a need to understand which categories of maritime operations may benefit from AR (Gernez et al., 2020). Finally, we see a need for understanding the potential of AR for collaboration (Billinghurst, 2021) including in maritime operations (Davidson & Sanderson, 2022; Merino et al., 2020; Pidel & Ackermann, 2020).

While AR may have the potential to facilitate maritime collaboration, it is not clear in which ways AR can be beneficial and for which maritime operations. A review of the existing research and development of AR for collaboration in maritime operations is needed to create an overview of the state-of-the-art ways in which AR can facilitate collaboration in maritime operations. Therefore, a systematic literature review could advance the knowledge base of ways AR can facilitate collaboration in maritime operations. Thus, we present a systematic literature review with the aim of guiding researchers and developers in understanding and developing AR for collaboration in maritime operations and other safety-critical operations. To achieve this aim, this study has three objectives: (1) creating an overview of the state-of-the-art ways in which AR can facilitate collaboration in maritime operations; (2) creating a categorization of these maritime operations; (3) giving insight into cognitive and collaborative challenges in these maritime operations.

Previous literature reviews have identified valuable knowledge on AR applications in maritime operations. For example, Gernez et al. (2020) and Laera et al. (2021) reviewed AR applications for ship navigation, and Vasiljević, Borović, and Vukić (2011) and Von Lukas, Vahl, and Mesing (2014) reviewed AR applications for several maritime operations, including ship construction. Our current review contributes to these existing reviews in several ways. Firstly, it includes the most recent developments. Secondly, compared to Gernez et al. (2020) and Laera et al. (2021), it includes maritime operations other than ship navigation, such as inspection and maintenance. Thirdly, compared to Vasiljević et al. (2011) and Von Lukas et al. (2014), it presents a more systematic approach and includes more recent developments. Lastly, it focuses on collaborative and cognitive challenges related to collaboration in maritime operations. This review is timely because AR has passed the top of the Gartner hype curve in 2010 (?). This is shown in Figure 1, which depicts where AR was on the Gartner hype curve in the years 2005–2018, indicating that, around 2010, proposed and tested use cases became realistic and industry adoption of AR started (Gartner, n.d.a). For examples of AR implementation for collaboration see Blanco-Novoa et al. (2018); Lukosch et al. (2015); Marques et al. (2022).

Figure 1. Augmented reality plotted on the Gartner hype curve from 2005 until 2018. Data from (Gartner, n.d.b), figure made by (Wikitude, n.d.)

2. Method

We made a systematic literature review of scientific publications on the topic of AR applications relevant to collaboration in maritime operations. Literature reviews are vulnerable to researcher bias. Therefore, it is important to follow guidelines to reduce the impact of bias (Drucker, Fleming, & Chan, 2016). Following such guidelines also makes it easier to assess the trustworthiness of a literature review and allows others to replicate and update the literature review. Our review was done in accordance with the guidelines of the PRISMA statement (Page, McKenzie, et al., 2021). We choose the PRISMA guidelines because it is endorsed by many journals and organizations from various disciplines (Page, McKenzie, et al., 2021) and its focus on interventions translates well to our focus on prototypes. Other guidelines we considered were the Synthesis Without Meta-analysis (SWiM) and the Cochrane guidelines. PRISMA was preferred over other guidelines such as Synthesis Without Meta-analysis (SWiM) which is intended to examine quantitative effects of interventions (Campbell et al., 2020), as we were mainly interested in prototype development and evaluation studies, which often do not report quantitative effects. PRISMA was also preferred over the Cochrane guidelines, which are focused on randomized clinical trials of medical interventions, and are less comprehensive for the synthesis of methods and study characteristics (Arteaga et al., 2022). This chapter describes the process of selecting inclusion and exclusion criteria, the search for publications, the screening and reviewing process, the data collection process, and our attempts to avoid and mitigate biases.

2.1. Inclusion and exclusion criteria

Our aim was to identify literature related to AR, maritime operations, and collaboration. We only included publications that related to all three of these topics. The selected publications used various methods, such as empirical studies, reviews, prototype designs, and theoretical discussions. We think research on AR for collaboration in other industries may be relevant to collaboration in maritime operations. Therefore, when AR for collaboration was studied in maritime operations, a similar AR application studied in another industry could be included. As such, Stylianidis et al. (2020, 2016) and Oskiper et al. (2013, 2015) are included despite their lack of discussion of maritime operations because others developed similar AR applications for maritime operations (e.g., Lee et al., 2016; Morgere et al., 2014b). We minimized the bias introduced by including these publications as described in the section 2.4. Publications used for our review are restricted to those written in English. The reasons for exclusion of each publication found is given in our data sheet available on the Open Science Framework (OSF).

2.2. Search, screening, and full-text review

We followed the guidelines of the PRISMA statement for systematic reviews (Page, McKenzie, et al., 2021) in an approach similar to that of Laera et al. (2021) and to some of our previous work (Van den Oever, 2019). We searched literature in the Web of Science and IEEE Xplore databases and added ten publications from other sources. We believe our review satisfactorily covers its topic with these databases and by including these ten publications from other sources. We choose these databases because the Web of Science is one of the most comprehensive academic literature databases (Chadegani et al., 2013) and IEEE Xplore specializes in computer science (About IEEE Xplore, 2023). Based on the Gartner hype curve AR passed the peak of inflated expectations in 2010, as depicted in Figure 1, which depicts where AR was on the Gartner hype curve in the years 2005 to 2018, indicating that around 2010, proposed and tested use cases became realistic and industry adoption started (Gartner, n.d.a). Thus, we limited our search to literature published after 2010. The last inclusion date was September 22, 2022. The search string was based on the constructs “augmented reality,” “maritime operations” and “collaboration,” and synonyms thereof. We used negative keywords to exclude search results that were researched in a medical context. For example, we excluded “blood” and “vascular” because these terms led to results about blood vessels. We searched the same search terms in both databases. To further filter out irrelevant publications, search results from subject areas that did not have any clear relevance to maritime operations involving humans were excluded, such as radiology, physics, and biology. The exact search strings, full list of filters of the respective databases, and instructions for replicating the literature search are available on OSF. Here, we present a readable version of the search string:

Title, abstract, and keywords: “augmented reality” OR AR OR HMD OR “immersive augmented reality” OR “head mounted display” OR HUD OR “head up display” OR MR OR “mixed reality” OR HWD OR “head-worn display”

AND Title, abstract, and keywords: collaboration OR cooperation OR communication OR operation OR “team communication” OR “situation awareness” OR “situational awareness” OR decision* OR decision-making OR “decision making” OR joint OR “shared situation awareness” OR “team situation awareness” OR “shared decision-making” OR “shared decision making” OR “team decision-making” OR “team decision making” OR “joint decision making” OR “joint decision-making” OR orientation

AND Title, abstract, and keywords: “Maritime operation” OR “offshore operation” OR offshore OR maritime OR ship OR “ship bridge” OR vessel OR marine OR navigation OR “Safety-critical system” OR” safety critical” OR “safety-critical”

NOT Title, abstract, and keywords: blood OR vascular OR surgery OR surgical OR medical

Search outputs were filtered by the first author in two major phases: a screening and a full-text review. The screening of the search results was based on title, subtitle, summary, and keywords. Full-text reviews were performed on search results that passed the screening. After the full-text review, ten additional publications from other sources than the database search were included. In section 3, we discuss the process of screening, full-text review, and categorization in more detail. The full search output and exclusion reasons in the screening and full-text review are visible in our data sheet.

2.3. Data collection process

Publications included after the full-text review were structurally summarized in an Excel spreadsheet, similar to Schraagen and Verhoeven (2013) and Van den Oever (2019), available in our data sheet. Each publication was classified according to the following categories: publication type, study method, AR hardware type, visualized information, validation level, TRL, maritime operation type, and methodological limitations. Table 1 explains the categories by showing their classification format and the rationale for them. Column one lists the categories we summarized in our data sheet; column two explains how we filled in cells for each publication for the category listed in column one, often including standardized answer options; column three explains why and how we summarized publications in the category listed in column one and why and how we applied the classification format described in column two.

Table 1. Categories of the structured summaries of individual publications.

Category	Classification format	Rationalization
Publication type	Gives one of these options with a short description in brackets: Prototype development publication Prototype development and evaluation publication Theoretical discussion Review article	Gives insight in the types of publications treating AR for maritime collaboration. Based on the example of Van den Oever (2019), adapted to fit the field of design research
Study methods	Discusses study methods, the prototype development method, the evaluation method (e.g., conditions, measured variables), and participants.	Gives insight in the methods applied in researching AR for maritime collaboration. Reflects the method section of publications.
Hardware type	Gives one of these options with a short description in brackets: Head-mounted display (HMD) HMD optical see-through HMD video see-through Unspecified HMD Handheld optical see-through Handheld video see-through Fixed display Not specified	Gives an overview of types of hardware used for AR for maritime collaboration. Based on the types defined by Azuma et al. (2001). Fixed display was added because of its use in several publications (e.g., Laera et al., 2021; Oh et al., 2016).
Visualized information	Lists the information visualized in the prototypes	Gives insight in the information that can be visualized with AR for maritime collaboration. Based on the descriptions in the publications.
Validation level	Gives one of these options with a short description in brackets including the number of participants and tests, as well as the method of validation Tested. Prototype was empirically evaluated. Not tested. Prototype was not empirically evaluated. Not specified. Unclear if the prototype was empirically evaluated.	Makes the level of empirical evaluation visible. Based on the method section in the publication
TRL	Gives one of these options: TRL 1—basic principles observed TRL 2—technology concept formulated TRL 3—experimental proof of concept TRL 4—technology validated in lab TRL 5—technology validated in relevant environment TRL 6—technology demonstrated in relevant environment TRL 7—system prototype demonstrated in operational environment TRL 8—system complete and qualified TRL 9—actual system proven in operational environment	Gives an overview of the TRL to which prototypes are developed. Derived from annex G of the Horizon 2020 Work Programme 2014–2015 (Directorate-General for Research and Innovation, 2014) using descriptions of Héder (2017).
Maritime operation type	Gives one of these options with a short description in brackets: Ship navigation   ○ Ship navigation: normal   ○ Ship navigation: traffic control   ○ Ship navigation: ROV surface   ○ Ship navigation: ROV underwater Personal navigation   ○ Personal navigation: normal   ○ Personal navigation: crisis   ○ Personal navigation: diving Maritime construction Maritime maintenance and inspection Other   ○ Other: piloting helicopter   ○ Other: drone operation   ○ Other: cargo handling	Provides a categorization of AR prototypes for maritime collaboration. Inspired by the categories of Laera et al. (2021), but expanded to fit the broader range of maritime operations that emerged in the reviewed publications.
Methodological limitations	Describes limitations mentioned in the publication and limitations observed by the current authors	Gives insight in methodological limitations in research of AR for maritime collaboration. Based on the focus topics discussed in Page, Moher, et al. (2021).

The first author estimated the TRL of the AR applications by comparing the full-text description in the manuscript with the TRL criteria shown in row five of Table 1. To verify this assessment, the inter-rater reliability of the TRL judgments were assessed for a subset of 15 of the publications. This was done by a research assistant who separately estimated the TRLs for a random selection of the publications. A linear weighted Cohen’s Kappa showed an inter-rater reliability of 0.81, which is conventionally considered to be acceptable (McHugh, 2012). TRL estimations that differed between the first author and the research assistant were resolved by discussion of the criteria by which we estimated TRLs. The estimation criteria that came forth from this discussion were used for the final TRL estimations in our review.

Data from individual publications selected for review were synthesized and numerical summaries were made when relevant (see our data sheet on OSF). Distributions were summarized in tables for several data categories: reasons for exclusion in the screening and full-text review, publication types, chronological and geographical distribution of publications, maritime operations, TRLs, and hardware used. Summary statistics, such as means and modes, were calculated when possible. Chronological distribution and TRL data are summarized in graphs. Structured summaries were grouped to synthesize their characteristics per maritime operation category.

2.4. Bias avoidance and mitigation

Literature reviews carry a risk of promoting a biased view of knowledge on a topic Page, Moher, et al. (2021). In the current study, we attempted to minimize this risk of bias in several ways. In terms of bias in individual publications, we evaluated and discussed elements that add to the risks of bias within each publication (i.e., research methods, and reporting).

In terms of bias across publications, the perspectives of the authors may lead to bias. The ten publications found through other sources than the database search may have introduced some bias in favor of research known to the authors. In addition, the authors’ judgments on the relevance of publications to collaboration, maritime operations, and AR may have been biased in favor of topics that are better known to the authors. Namely, Stylianidis et al. (2020, 2016) and Oskiper et al. (2013, 2015) are included despite their lack of discussion of maritime operations because other researchers developed similar AR applications for maritime operations (e.g., Lee et al., 2016; Morgere et al., 2014b). We attempted to minimize the risk of this bias by limiting the number of publications from other sources and by ensuring their thematic relatedness to publications found through the structured search.

Another risk of bias comes from using scientific databases as our main source, which creates a blind spot for developments that are not published in scientific literature. Some of the publications we found (e.g., Şakar & Sürücü, 2018) discuss non-academic sources, which implies there are more AR applications under development than is reflected in the scientific literature. Besides that, research institutes like the Maritime Research Institute Netherlands (MARIN), the German Aerospace Center (DLR), and companies such as Kongsberg develop and study maritime AR applications beyond those discussed in publicly available publications. This bias is partly resolved by including three publications from other sources which were published by DLR. Nevertheless, there may remain some bias in favor of scientific works published in scientific journals. We accept this bias because our target readers are in the scientific community. As such, readers should assume our literature review falls short concerning the representation of non-scientific and non-public developments of AR for maritime collaboration.

3. Results

We performed a systematic literature review of scientific publications on the topic of AR applications relevant for collaboration in maritime operations, following the guidelines of the PRISMA guidelines (Page, McKenzie, et al., 2021). The next section describes our search results and the characteristics of the publications we found. After that, we discuss AR applications for each of the maritime operation categories we delineated.

3.1. Search results and publication characteristics

We searched in the Web of Science (WOS) and IEEE XPlore (IEEEX) databases. We filtered publications following the PRISMA statement (Page, McKenzie, et al., 2021), as depicted in Figure 2. The top row shows how many publications were identified in the databases and how many publications we excluded for being duplicates; the second row shows how many publications we screened based on their titles and abstracts; the third row shows how many publications we full-text reviewed; the bottom row shows how many publications we included in our review. In the screening and full-text review, publications were excluded for being insufficiently related to maritime operations (e.g., Mouri et al., 2012; Tran et al., 2021), to AR (e.g., Tsivgoulis et al., 2018), to human collaborations (e.g., Bang et al., 2016; Rogers et al., 2015), or combinations thereof. Exact numbers per reason of exclusion in the screening and full-text review are listed in Table 2. Column one gives a list of reasons for excluding publications; column two gives the number of publications excluded in the screening for each reason in column one; column three gives the number of publications excluded in the full-text review for each reason in column one; the first row gives the total of publications considered in each selection phase; the last row gives the total publications included in each selection phase;” NA” means not applicable. Aside from the structured database search, Ten additional publications from other sources than the database search were included (i.e., Falk et al., 2020; Laera et al., 2021; Şakar & Sürücü, 2018; Stylianidis et al., 2020; Von Lukas et al., 2014) because they were known to the authors, were found via citations in publications in the database search, or suggested by the reviewers. Thus, we retained 42 publications for analysis. Figure 3 displays the chronological distribution of publications, with the x-axis giving the years 2010 to 2022 and the y-axis showing the number of publications in each year as a bar with a label. We see a mild trend towards increased publication on the topic. We think the dip in publications in 2021 and 2022 may be explained by the Corona pandemic and our latest inclusion date being September 9 2022, rather than a waning of interest in the subject. Included publications come from Europe (21) North America (9), Asia (8), Oceania (3), and South America (1).

Figure 2. PRISMA flow diagram.

Figure 3. The included publications distributed by year of publication.

Table 2. Reason for exclusion of publications in screening and full-text review.

Reason for exclusion	Screening	Full-text review
Publication considered	691	164
Excluded	527	122
Insufficiently related to collaboration (human)	1	14
Insufficiently related to AR	12	14
Insufficiently related to maritime operations	116	47
Insufficiently related to collaboration and maritime operations	91	36
Insufficiently related to AR and collaboration	12	3
Insufficiently related to AR and maritime operations	10	1
Insufficiently related to AR, collaboration and maritime operations	258	1
Insufficiently scientific	0	2
Only abstract available	NA	4
Duplicate	27	NA
Publication from other source	NA	10
Included	164	42

Examining the publications along the categories by which we summarized them gives insight into publication characteristics. Firstly, we found four different publication types. There were 26 publications that developed a prototype and evaluated it. 11 publications developed a prototype without evaluating it. One publication was a theoretical discussion. Four publications were review articles. Secondly, these 42 publications discussed 57 unique AR applications. Thirdly, researchers used different hardware for their prototypes. As shown in Figure 4, in which the x-axis gives the different hardware types we identified, and the y-axis shows how many of the prototypes were made using each hardware type with a bar and a label. the most popular devices were optical see-through head-mounted displays (HMD), followed by fixed displays. Eight studies did not clearly specify the hardware they used. Fourthly, TRLs ranged from 1 to 7, with a mode of 2 as visible in Figure 5, which shows on the x-axis the TRLs and on the y-axis how many of the prototypes were on each TRL according to our estimation. Structured summaries of all selected publications and calculations for the figures and tables in this paper are available in our data sheet on OSF.

Figure 4. Hardware types used in the prototypes of AR for maritime collaboration.

Figure 5. Technology Readiness Levels of the AR application prototypes presented in the included literature.

3.2. AR applications for maritime operations

Our literature review showed that AR application prototypes have been developed for various maritime operations. Table 3 shows how many times AR applications for the different maritime operations were discussed in the literature we found. Column one lists the maritime operation categories we made; column two gives the number of publications in each category. To see which publication was in each category, we refer to the column” Maritime operation type” in our data sheet available on OSF. In each of the following subsections, we discuss a group of maritime operations. For each operation, we will list the following: how many AR applications are relevant for this type of operation, the cognitive and collaborative challenges that the AR applications attempt to address, what kind of information is visualized in the AR applications, the hardware that was used, and the TRLs of the AR applications.

Table 3. Prevalence of maritime operations discussed in included publications.

Maritime operation	Prevalence
Ship navigation: normal	22
Ship navigation: traffic control	3
Ship navigation: ROV underwater	4
Ship navigation: ROV surface	3
Personal navigation: normal	2
Personal navigation: crisis	4
Personal navigation: diving	3
Maritime construction	5
Maritime maintenance and inspection	6
Other: piloting helicopter	2
Other: drone operation	1
Other: cargo handling	2
Total	57

3.2.1. AR for ship navigation

In terms of implementing AR, ship navigation is the most discussed operation. We found AR applications intended for four types of ship operation: (1) normal operations such as sailing, (2) traffic control, (3) control of remotely operated vehicles (ROV) on the surface, and (4) control of ROV underwater.

Ship navigation in normal operations using AR was the most discussed; 22 prototypes were made for this operation, as shown in Table 3. The AR application sometimes addresses more specific tasks, like navigating ships in narrow waterways (Rowen et al., 2021), in coastal waters (Morgere et al., 2014b), or while docking (Okazaki, Takaseki, et al., 2017). The cognitive and collaborative challenges that these operations pose include maintaining SA (Grabowski et al., 2018; Okazaki, Takaseki, et al., 2017; Rowen et al., 2019; Şakar & Sürücü, 2018), SSA (Rowen, Grabowski, Rancy, et al., 2019), decision-making (Grabowski et al., 2018; Morgere et al., 2014a, 2014b; Rowen et al., 2019; Wisernig et al., 2015), collaborative decision-making (Grabowski et al., 2018; Rowen, Grabowski, Rancy, et al., 2019), communication (Grabowski et al., 2018; Rowen, Grabowski, Rancy, et al., 2019; Satriadi et al., 2019), operator mobility, self-efficacy, and trust (Rowen et al., 2021), and mental workload (Morgere et al., 2014a, 2014b; Oh et al., 2016; Rowen et al., 2019). The AR applications visualize information such as the following: collision warnings, position, rudder information (Oh et al., 2016), roll and pitch, engine data (Laera et al., 2021), speed over ground (SOG), course over ground (COG), heading (Oh et al., 2016; Rowen et al., 2019; Rowen et al., 2021; Rowen, Grabowski, Rancy, et al., 2019), Automatic Identification System (AIS) information, bearing, closest point of approach (CPA) and time to closest point of approach (TCPA) of other ships or buoys (Leite et al., 2022; Oh et al., 2016; Rowen et al., 2019, 2021; Rowen, Grabowski, Rancy, et al., 2019), information about objects and areas of interest (Laera et al., 2021, 2020; Leite et al., 2022; Morgere et al., 2014a, 2014b; Oh et al., 2016; Okazaki, Takaseki, et al., 2017; Şakar & Sürücü, 2018), tide and sea current information (Laera et al., 2021), track lines, sea route lines, course lines (Leite et al., 2022; Oh et al., 2016; Okazaki, Takaseki, et al., 2017), cross track errors (Rowen et al., 2019, 2021; Rowen, Grabowski, Rancy, et al., 2019), maps (Satriadi et al., 2019), markers in geological locations (Leite et al., 2022; Okazaki, Takaseki, et al., 2017; Oskiper et al., 2013, 2015), speeds of bow and stern, distance of bow and stern to quay, wind information (Falk et al., 2020), and no-go zones (Laera et al., 2020). AR hardware used included: HMD optical see-through (e.g., Rowen et al., 2021), HMD video see-through (Satriadi et al., 2019), unspecified HMD (e.g., Şakar & Sürücü, 2018), handheld optical see-through in the form of binoculars (Oskiper et al., 2013, 2015), fixed display video see-through (e.g., Okazaki et al., 2017). The AR applications had TRLs that ranged from 1 to 7, but the majority of these applications were developed to TRL 2 (18 out of 25 prototypes).

Maritime traffic control using AR was discussed three times (Nad et al., 2019; Şakar & Sürücü, 2018; Von Lukas et al., 2014). All three publications focused on surveilling ships in ports. Such surveillance poses challenges such as the following: mental workload (Nad et al., 2019), communicating, remotely collaborating with ship operators (Şakar & Sürücü, 2018), and SA (Şakar & Sürücü, 2018; Nad et al., 2019). AR applications have also been proposed for remotely observing situations (Şakar & Sürücü, 2018), for overlaying AIS information on the world (Nad et al., 2019; Von Lukas et al., 2014), and for providing communication tools such as virtual pointers, markers, and pose indicators (Nad et al., 2019; Von Lukas et al., 2014). The hardware used were: HMD video see-through (Nad et al., 2019), fixed displays (Von Lukas et al., 2014), and HMD optical see-through (Şakar & Sürücü, 2018). We judged these prototypes to be at a TRL of 2 (Nad et al., 2019; Şakar & Sürücü, 2018) and 3 (Von Lukas et al., 2014).

We found three AR applications for navigating ROV on the water surface (Burmeister et al., 2020; Byeon, Grundmann, & Burmeister, 2021; Laera et al., 2021). Burmeister et al. (2020) developed a prototype for remotely operating a simulated tugboat, which Byeon et al. (2021) improved to remotely operate a physical model tugboat. In a literature review, Laera et al. (2021) described Hong et al.’s (2015) prototype for remote vessel navigation. Remote vehicle operations pose challenges of maintaining sufficient SA with reduced sensory perception and remotely collaborating with other vessels (Byeon et al., 2021; Laera et al., 2021). Visualized information in the prototype of Byeon et al. (2021) included: live 360° video, an Electronic Chart Display and Information System (ECDIS), virtual representations of control elements, such as joysticks, measured forces on the tug lines, distance to other vessels, AIS information about other vessels, and a laser pointer. The AR application described by (Laera et al., 2021) visualized the following information: forward and rear 180° video with AIS information of other ships, a range ring, compass, and heading. The hardware used was: HMD video see-through. We judged these prototypes to be at TRL 4 (Burmeister et al., 2020; Laera et al., 2021) and TRL 5 (Byeon et al., 2021).

AR for underwater navigation of ROV was discussed four times (Candeloro et al., 2015; He et al., 2019; Shih et al., 2018; Vasilijevic et al., 2013). Such operations pose challenges related to: maintaining SA (Candeloro et al., 2015; Vasilijevic et al., 2013), reduced sensory perception (He et al., 2019; Shih et al., 2018), mental workload (Vasilijevic et al., 2013), and collaboration between ROV operators (Candeloro et al., 2015). Visualized information included: dehazed live video streaming (He et al., 2019; Shih et al., 2018), depth, distances to targets, heading (Candeloro et al., 2015), and virtual targets presented with an auditory display (Vasilijevic et al., 2013). The hardware types used were: HMD video see-through (Candeloro et al., 2015), HMD optical see-through (Shih et al., 2018), fixed display video see-through with headphones (Vasilijevic et al., 2013). The hardware used by He et al. (2019) was not specified. We judged these prototypes to be at TRL 2 (Vasilijevic et al., 2013), TRL 3 (He et al., 2019; Shih et al., 2018), and TRL 6 (Candeloro et al., 2015).

3.2.2. AR for personal navigation

During maritime operations, people have to find their way around ships, windmills, and shipyards, and while diving, for example, to find ship parts or evacuation routes. To improve safety and efficiency, AR tools are under development to support personal navigation in maritime operations. We found AR applications for three types of personal navigation: normal operations such as finding ship parts in a shipyard, crisis operations such as evacuations, and diving operations such as salvaging.

We found two instances of AR for personal navigation in normal operations (Blanco-Novoa et al., 2018; Şakar & Sürücü, 2018). When people have to navigate in large and unfamiliar areas such as shipyards and ports, they encounter challenges to SA and remote collaboration (Blanco-Novoa et al., 2018). The prototypes aimed to help operators by means of geotags (Blanco-Novoa et al., 2018; Şakar & Sürücü, 2018), a map-like overview of the area (Blanco-Novoa et al., 2018), and live video of the point of view of local operators to help remote experts orientate (Blanco-Novoa et al., 2018). The hardware types proposed were: handheld video see-through and HMD optical see-through (Blanco-Novoa et al., 2018). Şakar and Sürücü (2018) did not specify hardware. We judged these prototypes to be at TRL 2 (Şakar & Sürücü, 2018) and TRL 3 (Blanco-Novoa et al., 2018).

We found four AR applications for personal navigation in crisis operations (Bhattarai et al., 2020; Chusetthagarn et al., 2018; Şakar & Sürücü, 2018; Sampson et al., 2022). Bhattarai et al. (2020) and Şakar and Sürücü (2018) proposed AR to improve the sensory perception of emergency personnel in critical situations. For example, during a fire, people have to cope with limited visibility and high time pressure, which poses challenges to SA, decision-making, and remote collaboration. The information that was visualized in these AR applications highlighted important points of interest, such as other emergency personnel, civilians and doors (Bhattarai et al., 2020), and an overlay to improve vision(Şakar & Sürücü, 2018). It also shared sensor data with commanding officers and other emergency personnel (Bhattarai et al., 2020). Sampson et al. (2022) developed an AR application for evacuating people from large passenger ships. Such operations pose challenges to SA, complex problem-solving, and the coordination of emergency personnel and passengers. The AR application visualized a map, directional features, a route, and information about passengers and emergency personnel in need of assistance. Chusetthagarn et al. (2018) developed an AR application prototype to manage crowds during evacuations by visualizing how many people are located in particular crisis areas. This prototype is useful for supporting SSA, communication, and collaborative decision-making in evacuations of, for example, offshore structures, ports, and ships. The visualized information was a map of crisis areas and sensor data per area. The hardware used were: HMD optical see-through (Bhattarai et al., 2020; Chusetthagarn et al., 2018) and handheld video see-through (Sampson et al., 2022). Şakar and Sürücü (2018) did not specify the hardware for the AR application they discussed. We judged these prototypes to be at TRL 2 (Bhattarai et al., 2020; Şakar & Sürücü, 2018), TRL 3 (Chusetthagarn et al., 2018), and TRL 6 (Sampson et al., 2022).

We found three AR applications for personal navigation while diving (Gallagher et al., 2017; Oppermann et al., 2013; Şakar & Sürücü, 2018). These applications can be used in diving operations such as search and rescue, construction, maintenance, and salvage. Such operations pose challenges to communication (Gallagher et al., 2017; Oppermann et al., 2013) and SA (Gallagher et al., 2017). The visualized information included images and outlines of objects (Şakar & Sürücü, 2018), high-resolution sonar video, sector sonar images, navigation aids, underwater night vision, text messages (Gallagher et al., 2017), a crosshair in the center of the display, a mission description, and virtual search targets (Oppermann et al., 2013). All three prototypes used HMD optical see-through. We judged these prototypes to be at TRL 3 (Şakar & Sürücü, 2018; Gallagher et al., 2017) and TRL 2 (Oppermann et al., 2013).

3.2.3. AR for maritime construction

AR has been proposed for use in maritime construction of, for example, ships, ports, and offshore structures. We found five AR applications for construction in the maritime industry (Lee et al., 2016; Şakar & Sürücü, 2018; Stylianidis et al., 2020, 2016; Von Lukas et al., 2014). Maritime construction operations pose challenges to SA, decision-making (Stylianidis et al., 2020, 2016), communication, collaborative decision-making, and complex problem-solving in highly parallel and time-sensitive construction processes (Von Lukas et al., 2014). Both Lee et al. (2016) and Von Lukas et al. (2014) developed TRL 2 prototypes to design piping in ships or offshore structures. Lee et al.’s (2016) prototype converted pipe drawing data to 3D models and connected sensor data of the physical piping to the 3D models to be viewed on-site. Von Lukas et al.’s (2014) prototype also visualized 3D models of pipes on-site, focusing on the design and placement of new pipes in the context of a bundle of previously placed pipes. Both these prototypes used handheld video see-through AR devices (tablets). Stylianidis et al. (2020, 2016) developed a similar application using CAD models to visualize hidden objects, such as pipes and electricity cables, and their characteristics to increase SA for decision-making in maritime construction projects. The AR hardware used was handheld video see-through. We judged this prototype to be at TRL 7. Von Lukas et al. (2014) developed another prototype to support collaborative decision-making in retrofitting a rescue boat and davit crane on a ship, which we judged to be at TRL 5. The prototype visualized a davit crane and rescue boat in different positions with characteristics such as size and maximum load. The hardware used for this prototype was a handheld video see-through device. Şakar and Sürücü (2018) discussed a prototype for planning terminals and port expansions. Such operations pose challenges to collaboration, efficiency, and decision-making. While the used hardware was not specified, the visualized information included virtual models of objects, such as new buildings projected in their planned location within the physical port or terminal. We judged this prototype to be at TRL 2.

3.2.4. AR for maritime maintenance and inspection

AR has been proposed for use in maritime maintenance and inspection of, for example, ships, offshore windmills, and shipyards. AR applications for maintenance and inspection in maritime contexts were discussed six times (Blanco-Novoa et al., 2018; Hou et al., 2014; Von Lukas et al., 2014), focused on shipyards, ships, and offshore structures. AR applications like these can support maintenance personnel through remote human support and computer support. Such operations pose challenges related to: SSA, collaborative decision-making (Von Lukas et al., 2014), SA, remote collaboration (Blanco-Novoa et al., 2018), mental workload, efficiency, and complex problem-solving (Hou et al., 2014; Von Lukas et al., 2014). Visualized information included point-of-view sharing, descriptive textual information about objects under inspection, historical data about maritime maintenance and inspection (Blanco-Novoa et al., 2018), object locations, visual maintenance guides, diagrams and schematics, a mini-map (Hou et al., 2014), and on-site visualized CAD data with information about objects (Blanco-Novoa et al., 2018; Hou et al., 2014; Von Lukas et al., 2014). The AR hardware devices used were: handheld video see-through (Blanco-Novoa et al., 2018; Hou et al., 2014; Von Lukas et al., 2014) and fixed display video see-through (Hou et al., 2014). We judged two of Blanco-Novoa et al.’s (2018) prototypes to be at TRL 1, Hou et al.’s (2014) prototype to be on TRL 2. Another prototype of Blanco-Novoa et al. (2018) and Von Lukas et al.’s (2014) prototype we judged to be on TRL 3.

3.2.5. AR for other maritime operations

We found AR applications for the maritime operations of helicopter piloting in an offshore wind park (Ernst et al., 2019; Maibach, Jones, et al., 2022; Maibach, Walko, et al., 2022; Walko & Schuchardt, 2021), drone operation for large-scale maritime construction, such as offshore windmills and port terminals (Piumsomboon et al., 2018), and handling cargo in ports (Şakar & Sürücü, 2018).

Flying during maritime operations makes it possible to reach locations hard to reach by ship. Helicopters are used to transfer equipment and technicians to windmills in offshore wind parks (Maibach, Jones, et al., 2022). Due to increasing maritime activity, the number of offshore helicopter operations is growing (Walko & Schuchardt, 2021). For helicopter pilots, operations pose high demands on mental workload, SA, orientation, attention, collaboration with co-pilots, coordination with ship crew (Maibach, Jones, et al., 2022; Maibach, Walko, et al., 2022; Walko & Schuchardt, 2021), and communication with the hoist operators and persons to be hoisted (Ernst et al., 2019). To support pilots, Ernst et al. (2019) developed an application prototype for HMD video see-through (Oculus Rift), which we judged to be at TRL 3. The visualized information included virtual flight displays, a chase-view perspective (behind and above the helicopter), an unobstructed view of the surroundings by making the cockpit see-through, and a synthetic ocean surface that gave more valuable information than the real ocean surface by identifying wave types. Walko and Schuchardt (2021) developed an HMD video see-through (HoloLens) prototype for helicopter pilots that was further developed by Maibach, Jones, et al. (2022). Maibach, Walko, et al. (2022) continued the development on HoloLens 2. This prototype visualized information in three categories: world fixed (heading indicator, artificial horizon, highlighted obstacles), helicopter fixed (attitude indicator, drift indicator), and head fixed (airspeed, First Limit Indicator, altitude, vertical speed, flight path marker). We judged this prototype to be at TRL 4.

In large-scale maritime construction, drones can be used for remote collaboration and multi-scale collaboration. Such operations pose challenges to remote collaboration, communication, and SSA. Partly, these challenges are posed because collaborators have different visual perspectives on a structure under construction. The visualized information included augmented interpupillary distance and movement gain in the virtual presence to give the user the impression of either flying or being a giant. The hardware used was HMD video see-through (Piumsomboon et al., 2018). We judged this prototype to be at TRL 3.

Much of the maritime industry is directed at handling cargo and warehousing cargo. These operations pose challenges related to SA and efficiency. In addition, small increments in time spent and errors avoided can create large improvements in efficiency (Şakar & Sürücü, 2018). Şakar and Sürücü (2018) discuss an AR application to support cargo handling by visualizing picking lists, best routes to cargo, loading instructions, and characteristics of the cargo. The hardware was head-mounted optical see-through. We judged this prototype to be at TRL 2.

4. Discussion

To advance research and development into AR for collaboration in maritime operations, we have provided an overview of recent and related scientific work. Following the PRISMA statement for structured literature reviews (Page, McKenzie, et al., 2021), we searched the Web of Science and IEEE Xplore databases. Through a filtering process, we selected 34 publications for in-depth analysis and comparison and added ten publications from other sources. With this review, we address a knowledge gap of an overview of the state-of-the-art knowledge base of ways AR can be used to facilitate collaboration in maritime operations.

In section 3 we have presented a structured overview of the publications we examined. Our overview showed that AR applications for collaboration in maritime operations have been studied for various maritime operations. We categorized these operations into ship navigation, personal navigation, maritime construction, maritime maintenance and inspection, and other. Our work addresses Gernez et al. (2020) call for categorizing maritime operations that could be supported by AR. For each operation, we have discussed what tasks it consists of, the AR applications that have been developed for it, the cognitive and collaborative challenges the AR applications have addressed, the hardware used, and the information visualized in AR. Most of the publications we examined were based on work from Europe, North America, and Asia. Most prototypes were at TRL 2. In this section, we will discuss (1) important recurring themes across publications we analyzed, with an emphasis on what research and development should focus on in the future, (2) limitations of our study, and (3) recommendations for further research.

4.1. Recurring themes

We will now discuss some of the common themes we discovered. Recognizing and understanding these themes may help focus and structure further research and development. The themes we discuss are: TRLs of AR application prototypes were rather low; AR applications were developed more for some maritime operations than for others; certain cognitive and collaborative challenges recur often; most studies lack explicit discussion of and grounding in cognitive and collaborative challenges.

In general, the AR application prototypes that were identified had been developed and described at rather low TRLs. Most prototypes were at TRL 2 (technology concept formulated), with some being on slightly higher TRLs (see Figure 4). The prototypes described in the publications we examined may have been further developed to higher levels of TRL, but we were not able to identify such instances. That prototypes were developed to low TRLs could indicate that the field appears to be more interested in proof-of-concept than in continued development. However, we think the field would benefit from higher TRL development of AR applications that were already tested on lower TRLs with positive results, i.e., ship navigation. Therefore, we agree with Gernez et al. (2020) and Laera et al. (2021) who suggested that developing and testing AR applications for ship navigation at higher TRLs may accelerate the growth of our understanding of the topic. AR development could also be extended to the other categories of maritime operations we have discussed. Besides the low TRLs, the methods sections in most of the publications we reviewed were inadequate for reproduction or validation because they focused on the design and development of prototypes rather than on evaluation. Although this has been previously addressed by, among others, Martini et al. (2022), we believe this shortcoming should be mentioned. We would therefore also suggest more focus on scientific standards for methods and reporting when testing prototypes, including discussing the limitations of the evaluation.

There were more AR applications developed for some maritime operations than for others. Most AR applications were developed for ship navigation operations; several other categories of maritime operations did not receive much attention (see Table 3), even though AR could improve collaboration. We found no publications discussing AR for other maritime operations, such as maritime crane operations and aquaculture. This is an interesting finding since AR is being researched for crane operations (Sitompul & Wallmyr, 2019) and agriculture (de Oliveira & Correa, 2020). Based on our findings, it appears that AR has significant potential for ship navigation, as also mentioned by Gernez et al. (2020) and Laera et al. (2021). Maritime crane operations may have challenges relating, for example, to SSA, communication, and decision-making. At the same time, there are research opportunities for AR for other maritime operations, including operations not listed in our review.

We discussed several cognitive and collaborative challenges in the publications we reviewed. Cognitive challenges that were repeatedly mentioned were: SA, reduced sensory perception, decision-making, mental workload, complex problem-solving, and efficiency. SA especially was frequently named as a cognitive challenge to be addressed by AR. Recurring collaboration challenges were SSA, communication, collaborative decision-making, and remote collaboration. These cognitive and collaborative challenges could be interesting to explore in further research as well. We suggest focusing on these collaborative challenges when answering Brandao and Pinho (2017) call to do more academic research on AR for collaboration.

While cognitive and collaborative challenges were discussed in most publications, many publications lacked discussion of how these challenges had been addressed by their prototypes. For example, Şakar and Sürücü (2018) suggested implementing AR for traffic control to support communication and SA without discussing SSA. Other researchers discussed SA, but not decision-making (e.g., Blanco-Novoa et al., 2018), even though these challenges are usually interconnected (Endsley & Garland, 2000). This may reflect the fact that SA has become a popular concept across both academic and practical fields, almost on the verge of passing into layman’s language. At the same time, the concept of SA has also been criticized for lacking specificity and having an unclear theoretical basis (Hoffman, 2015). Thus, simply stating that an AR application could” enhance SA” for a given maritime operation may have limited value.

Several publications focused on cognitive and collaborative challenges without a discussion of their relationship to the relevant theory. For example, Okazaki, Takaseki, et al. (2017) developed a prototype without explicitly discussing any cognitive or collaborative challenge, while others developed similar prototypes clearly aimed at addressing cognitive and collaborative challenge (e.g., Rowen et al., 2021). In future AR research, we suggest putting more emphasis on relating prototype development and evaluation to relevant cognition and collaboration theories. This should improve the prototype’s applied value for maritime operations, facilitate academic research, and increase human-centeredness in design. Following this advice may also help in improving scientific methods and reporting when testing prototypes, as exemplified by Rowen et al. (2021). This advice is in concurrence with Aylward’s (2022) suggestion for more human-centered design.

4.2. Limitations

Some limitations should be considered when interpreting our current findings. As we mentioned in section 2.4, our filtering judgment may have introduced bias in favor of publications reporting on topics better known to us. As also mentioned in section 2.4, our literature review focuses on academic developments of AR for maritime collaboration. Thus, readers should assume our literature review falls short in its representation of non-scientific, non-public developments. Another limitation to consider is that, due to resource limitations, the TRL inter-rater reliability has been checked for only 15 out of 42 publications. However, since the inter-rater reliability score was high for that sample, we may assume that the rest of the TRL judgments are also fairly reliable. This impression was supported by the way the few cases of inter-rater disagreement were resolved. A third limitation is that we searched literature in only two databases: the Web of Science and IEEE Xplore. For example, we also could have repeated the search in the SCOPUS database. However, we believe that performing the same search in the broad Web of Science database as well as the IEEE Xplore which specializes in computer science will satisfactorily cover our topic of interest. A fourth limitation is that the wording in our search terms may have favored visual AR. For example, we searched for head-mounted display and head-up display and did not explicitly search for tactile AR and audio AR. This may have resulted in the omission of AR with tactile and audio displays. However, tactile and audio displays did appear in our search results, but these prototypes were insufficiently related to maritime operations to include in our review. Only two of the publications in our review discussed non-visual displays (e.g., Burmeister et al., 2020; Vasilijevic et al., 2013). This indicates a possible research gap in audio and tactile displays for maritime AR.

4.3. Implications for further AR research

To our knowledge, our study is the first to perform a structured literature review of AR applications in maritime operations, while dealing with cognitive and collaborative challenges and categorizing AR applications according to types of maritime operations. AR promises to be a useful tool for improving collaboration and therefore safety and efficiency in maritime operations. Our findings can guide such development and evaluation of AR for collaboration in maritime operations. Firstly, like Gernez et al. (2020) and Laera et al. (2021), we think AR applications for maritime collaboration are ready to be developed and evaluated at higher TRLs, especially for operations for which many prototypes have been developed at lower TRLs, i.e., ship navigation. Secondly, we agree with Aylward (2022) that AR development should apply a human-centered design approach to a larger extent. Thirdly, in concurrence with Brandao and Pinho (2017), we have noticed a shortage of academic research on AR for collaborative challenges, to which we add a shortage of attention to cognitive challenges. Therefore, we suggest research and development of AR applications attend to cognitive and collaborative challenges. Fourthly, future studies would benefit from task analyses of the maritime operations we have discussed in order to identify further cognitive and collaborative challenges that AR can address. Fifthly, our overview of maritime operations can help researchers choose operations to research for which there is a lack of AR development. This includes operations for which we found no academic publications, such as maritime crane operations and aquaculture operations. Studying the applicability of AR in these operations may help increase safety and efficiency.

Our results and findings may also be useful for wider AR research and development. Some aspects of maritime operations have similarities to operations in other industries. For example, AR to support the handover of control from automation to human operator (Langlois & Soualmi, 2016) and between human operators (Sampaio et al., 2020); AR for maintenance and inspection (e.g., Le Chenechal et al., 2016; Park et al., 2020). AR is also used in the design and construction of other things than ships and harbors (see Lukosch et al., 2015; Marques et al., 2022). Insights may even be relevant to AR applications that appear dissimilar to the maritime AR applications we treated in this paper, as cognitive and collaborative challenges may be similar (Billinghurst, 2021). For example, AR may support surgeons in decision-making and collaboration (Vávra et al., 2017). If the cognitive and collaborative challenges are the same, similar AR features may be useful, such as highlighting areas of interest. Transferring knowledge from the maritime industry to other sectors and vice versa can stimulate innovation. This may especially be the case for safety-critical operations in which cognitive and collaborative challenges are similar.

Further research into AR for maritime operations could attend to other challenges besides cognitive and collaborative challenges. Billinghurst (2021) discusses other” grand challenges for AR” such as technological challenges (e.g., developing displays and tracking), interaction challenges (e.g., natural user interfaces and multi-modality), and social and ethical challenges (e.g., acceptance and privacy). Organizational, environmental, and ergonomic challenges are also important to understand when implementing AR (Masood & Egger, 2019). Further research into AR for maritime operations could attend to these challenges. More foundational research into the grand challenges of implementing AR may also benefit the maritime and other sectors.

Further research can use our data sheet available on OSF to review topics we summarized in the data sheet but did not cover in this review. We chose to focus our review on identifying maritime operations in which AR can support collaboration, estimating TRLs of developed prototypes, identifying cognitive and collaborative challenges AR can address, and relating prototype development and evaluation to relevant theory. We did not extensively treat information visualization, and prototype evaluation methods, which are summarized in our data sheet. Our summary of information visualized in AR can be used to suggest design guidelines for developing AR applications for collaboration in safety-critical operations (maritime and in other industries) similar to existing design guidelines for ship bridges (Nordby et al., 2019). Our summary of prototype evaluation methods may be used for making an overview and assessment of prototype evaluation methods in the maritime and other industries. As mentioned in section 4.1, we think evaluation methods could be more scientific, which has been addressed by, among others, Martini et al. (2022). A study reflecting on common issues in prototype evaluation methods may inform the research and development of prototypes.

4.4. Conclusion

Our systematic review found a thriving literature on AR prototypes for maritime operations. Our evaluation of the literature has led us to conclude that it would be more beneficial to spend more resources on developing the prototypes to higher TRLs, rather than developing new prototypes that overlap with previous functionality. At the same time, we conclude that it would be beneficial to develop prototypes for a wider array of maritime operations. For this, design guidelines would be useful. Further, grounding AR development in cognitive, collaborative, and other” grand challenges” (Billinghurst, 2021) may facilitate the transfer of AR applications to new contexts with similar challenges. De la Peña Zarzuelo et al. (2020) envisioned that AR can become a pillar of the maritime industry. We believe that, if our recommendations are followed, their vision can be made true.

Acknowledgements

We wish to thank Anna Kartnes Andersson for helping with the inter-rater reliability checking of the TRLs.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

We have made our data available in a data sheet on the Open Science Framework: https://osf.io/5eur3/?viewonly=7eadfa2f2e85446ea8c4dcb8d6f917de for evaluating the validity and reliability of our work and to facilitate follow-up research.

Additional information

Notes on contributors

Floris van den Oever

Floris van den Oever is a Ph.D. researcher in Psychology and Human-Computer Interaction at the University of Bergen, Norway. His work focuses on AR and collaboration. He holds an MSc in Human Factors and Engineering Psychology with honors in design from the University of Twente, The Netherlands.

Morten Fjeld

Morten Fjeld is a professor of Human-Computer Interaction at the University of Bergen, Norway, and Chalmers, Sweden. His work focuses on tangible user interfaces and founded the t2i Interaction Lab. He holds a dual MSc in applied mathematics from NTNU, Norway, and ENSIMAG, France, and a Ph.D. from ETH, Switzerland.

Bjørn Sætrevik

Bjørn Sætrevik is a professor of general psychology at the University of Bergen, Norway. His work focuses on decision-making, perceived risk, team-processes, and human factors. He is a clinical psychologist (2003) with a Ph.D. in cognitive neuroscience (2007), both from the University of Bergen, Norway.