Abstract
Acute bleeding during surgery or after trauma harms patients, and challenges involved physicians. Protocols and cognitive aids can help in such situations. This dual-centre study investigated physicians' opinions regarding two ways to present cognitive aids, graphic ‘Haemostasis Traffic Light’ and text-based, using the example of a coagulation management algorithm to identify the strengths and limitations of both presentation modalities. Using qualitative research methods, we identified recurring answer patterns and derived major topics and subthemes through inductive coding. Eighty-four physicians participated. We assigned each half randomly to one of the cognitive aids (graphic/text-based) and determined 447 usable statements. We qualitatively deduced the importance of having a cognitive aid for physicians. Furthermore, it is noticeable that the graphic group made more positive comments (154 of 242 (64%) statements), while the text-based participants made more negative annotations (126 of 205 (61%) statements), suggesting a generally stronger approval of this cognitive aid.
Practitioner summary: This qualitative study provides an overview of physicians` positive and negative perceptions regarding two presentation ways for a coagulation management algorithm. Participants perceived the graphic method created according to user-centred design principles more positively. The analysis reveals components that an ideal algorithm should have to help streamline the decision-making process.
Introduction
Acute bleeding and resulting coagulopathy are challenging and stressful situations for the involved physicians. Early identification and effective treatment are required to avoid patient harm in the form of increased morbidity and mortality. Modern medicine uses diverse laboratory and viscoelastic coagulation tests, which physicians must analyse in the context of the overall clinical situation like pre-existing anticoagulation medication and circulatory depression of the patient. Many authors define targeted haemostasis management, especially in trauma patients, guided by point-of-care (POC) diagnostics as the gold standard. However, these processes generate a large number of complex results, which are difficult to interpret correctly and quickly. This decision-making process forms the basis for the subsequent adequate and often life-saving therapy (Gruen et al. Citation2006; Spahn et al. Citation2019; Stein et al. Citation2017; Winearls et al. Citation2016; Endsley Citation1995). Especially for inexperienced physicians, the correct interpretation of diagnostic methods and the subsequent decision-making in acute situations under time pressure is difficult and easily leads to errors (Dambacher and Hübner Citation2015; Larsson, Rosenqvist, and Holmström Citation2006). Therefore, various professional societies regularly publish guidelines for diverse medical situations, and studies found evidence that checklists and protocols improve patient outcomes (Spahn et al. Citation2019; Haynes et al. Citation2009; Konstantinides et al. Citation2019; Starmer et al. Citation2014). Furthermore, a presentation layout following user-centred design principles may positively influence the anaesthesiologist's situation awareness and clinical performance (Drews and Westenskow Citation2006; Jones et al. Citation2018; Keebler et al. Citation2018; Wung Citation2018). These results demonstrate the demand for simple instruments facilitating diagnosis and decision-making in the care of bleeding patients.
Hence, our research group developed the ‘Haemostasis Traffic Light’, an algorithm concept aimed to simplify coagulation management through situation awareness-oriented presentation of essential diagnostic and therapeutic information. Instead of only text, it provides the participants with a graphic scheme itemised by a triple-colour code representing the clinical situation (Mileo et al. Citation2019). According to Mica Endsley`s model, an adequate situation awareness for required data is necessary for good decision-making. Environmental factors such as cognitive aids, support this process of perception, comprehension, and projection (Endsley Citation2016). The clarity of our graphic scheme may relieve the user from additional mental stress, helping to reduce the information load for better situation awareness and consecutive decision-making.
In this study, we qualitatively assessed the opinions of anaesthesiologists regarding two presentation methods (graphic/text-based) in two different parts of the world, representing diverse socio-cultural and economic circumstances. These results will help define the advantages and disadvantages of graphic and text-based algorithms, which may support developing better-designed cognitive aids to assist physicians in challenging clinical situations and thus improve the quality of patient care.
Methods
The Cantonal Ethics Committee of Zurich, Switzerland, reviewed the study protocol and issued a declaration of no objection (Business Management System for Ethics Committees Number 2019–01090). The ethics committee responsible for the centre in Buenos Aires approved the study protocol in a separate statement (N° 5357, dated November 14th, 2019). Before participation, we obtained written consent from each physician to use the collected data for research purposes.
Study and participant characteristics
This was an investigator-initiated, prospective, international dual-centre survey study. Using qualitative research methods, we explored the opinions of anaesthesiologists regarding two different ways to present an algorithm for the management of acute bleeding. We included participants at the University Hospital Zurich in Switzerland and the Italian Hospital of Buenos Aires in Argentina. These tertiary referral hospitals are regularly confronted with bleeding situations, so that we assume at least a basic competence regarding coagulation management. We conducted this study between January and May 2020 simultaneously in both centres and included 21 residents and 21 staff physicians in each centre. All staff physicians held an anaesthesia board certification in their respective country.
Designing the ‘Haemostasis Traffic Light’
We created the ‘Haemostasis Traffic Light’ according to principles of user-centred design fostering situation awareness. Such a layout aims to transmit the necessary information as quickly as possible and with the lowest possible cognitive effort. Models that are intuitively understandable and therefore promote accessibility and interpretation are particularly suitable (Endsley Citation2016). Due to its universal use across various domains (traffic, everyday language use, e.g., ‘to green-light something’), the traffic light system is suitable as an intuitive way for distinguishing between severities of bleeding situations. After the user has assigned the severity, they work through the points from left to right. We indicate the direction by arrows, which also intuitively specify the path. When rotational thromboelastometry values (ROTEM) need to be interpreted, we display them in the same colour as they appear on the ROTEM machine display. In addition, we work with different fonts, shadows, and text colours for further highlighting.
Previous ‘Haemostasis Traffic Light’-study and participant interviews
The previously published study investigated the coagulation management algorithm called ‘Haemostasis Traffic Light’ (graphic). In this computer-based, between-subjects comparison study, the graphic algorithm enabled more correctly solved cases and faster therapeutic decisions. Furthermore, when using the ‘Haemostasis Traffic Light’, participants indicated improved therapeutic confidence, conjoined with a lower perceived workload (Kataife et al. Citation2021).
We allocated the participants randomly to either a black and white text-based coagulation algorithm or the graphic ‘Haemostasis Traffic Light’ version. Both cognitive aids were adapted to the locally pre-existing coagulation protocols. The medical content differed slightly between the study centres, however, not within one site. We provided standardised instructional videos shown on an iPad (Apple Inc, Cupertino, California, United States) explaining the respective algorithm. Using the randomly assigned cognitive aid as a printout, the participants managed six bleeding scenarios. gives an overview of the two cognitive aids. For further information, all used versions (English and Spanish) are available in the Supplemental Appendix 1. We designed the simulated cases to have as few ambiguities as possible (Kataife et al. Citation2021). After solving all scenarios, we conducted the interviews for this study. This procedure allowed us to question the participants right after they had used the algorithm. In an undisturbed environment, we asked the physicians about advantages and disadvantages of their used type of cognitive aid. The original questions were: ‘What did you LIKE about the used cognitive aid (text-based algorithm or “Haemostasis Traffic Light”)? “and” What did you DISLIKE about the used cognitive aid (text-based algorithm or “Haemostasis Traffic Light”)?’. We instructed the participants to type their opinions as field notes into two separate text boxes on the iPad and left it open to answer in Spanish, German, or English without a time constraint.
Figure 1. The used cognitive aids for coagulation management (Zurich variant = English language). (A) Text-based algorithm. (B) ‘Haemostasis Traffic Light’ (graphic) algorithm.
Qualitative analysis
Beginning our qualitative analysis, we translated the original answers from German and Spanish to English using Deepl.com (DeepL GmbH, Cologne, Germany). In Supplemental Appendix 2, we provide all translated field notes.
We followed the Consolidated Criteria for Reporting Qualitative Research (COREQ) and the Standards for Reporting Qualitative Research (SRQR) for reporting our analysis (Dossett, Kaji, and Cochran Citation2021; O'Brien et al. Citation2014; Tong, Sainsbury, and Craig Citation2007). The choice of appropriate sample size in qualitative research is a matter of controversy (Vasileiou et al. Citation2018). We applied a pragmatic approach and included the 84 anaesthesiologists of the ‘Haemostasis Traffic Light’-study.
First, we counted the most frequently mentioned keywords and grouped word families like ‘clear/clearly/clearness’ to process the answers. Common English words like ‘the, you, to’ were ignored. For further data analysis, we used the template approach. It allows predefining themes that are likely to be relevant to our research question. In our field notes obtained, we identified recurring answer patterns and developed a coding template. We assigned the participant answers to that coding template. During the analysis process, we revised major topics with subthemes several times until we reached our final template. This ultimately resulted in four coding trees, each presenting either the advantages or disadvantages of one of the two algorithms ( and ). Complying with COREQ, the study authors LB and DJW independently applied the final coding templates to our translated data set (Dossett, Kaji, and Cochran Citation2021; O'Brien et al. Citation2014). LB and DJW, both absolving anaesthesia residency, were full-time research fellows at the Institute of Anaesthesiology of the University Hospital Zurich, Switzerland. Both coders were not involved in the interview process. In case of disagreement, they jointly determined a code for the respective answer in a consensus review.
Figure 2. The coding trees concerning the positive and negative perceptions of the ‘Haemostasis Traffic Light’ algorithm. We created the coding trees by word count and inductive free coding. This figure was generated using draw.io (Seibert Media GmbH, Wiesbaden, Germany), N = 42 participants.
Figure 3. The coding trees concerning the positive and negative perceptions of the ‘text-based’ algorithm. We created the coding trees by word count and inductive free coding. This figure was generated using draw.io (Seibert Media GmbH, Wiesbaden, Germany), N = 42 participants.
Statistical analysis
We report the number of field notes and their respective percentages that went into each major topic and subtheme. For data management we used Microsoft Excel and Microsoft Word (both: Microsoft Corporation, Redmond, Washington, United States). Inter-rater reliability is the extent to which the two raters (LB and DJW) agree. It addresses consistency in implementing a rating system, in this case, the generated coding trees to the field notes (Lange Citation2011). To calculate the statistical dimension ‘Cohen's kappa’, we used R version 4.0.5. (R Foundation for Statistical Computing, Vienna, Austria). Furthermore, we assessed percent agreement using Microsoft Excel. To compare the proportions of positive and negative statements for both algorithms, we used Chi-square test calculated in Graph Pad 9.1.2 (Graphpad Software LCC, San Diego, CA, USA).
Results
Study and participant characteristics
We performed this study between 9 January 2020 and 12 May 2020 including the same participants as in the ‘Haemostasis Traffic Light’-study. We interrogated 21 residents and 21 staff anaesthesiologists in each study centre, giving us field notes from 84 physicians. The analysis of the participant characteristics showed a balanced relationship in terms of gender and qualification status. Supplemental Appendix 3 outlines the participant and study characteristics in detail. In total, we analysed 462 statements from the given answers. We declared 15 of the 462 (3%) statements as ‘not codable’ because they were incongruent in content or proposed modifications, so that we continued to work with 447 statements. and outline all answers' distribution in more detail, including the ‘not codable’ statements.
Table 1. Haemostasis Traffic Light algorithm (N = 252 statements).
Table 2. Text-based algorithm (N = 210 statements).
The evaluation of the consensus rating showed that the participants who worked with the ‘Haemostasis Traffic light’ algorithm made more positive comments (154 of 242 (64%) statements) in percentage terms than the text-based group (79 of 205 (39%) statements). Using quantitative methods, the difference is statistically significant (Odds ratio: 2.79; 95% confidence interval: 1.90–4.10; p < 0.001). displays this allocation of statements.
Table 3. Overview of the distribution of the statements codable (N = 447).
Qualitative analyses of participant field notes
Within the investigation of the word counts, we selected the three most frequently mentioned positive and negative perceptions for each algorithm. shows the analysis of the word counts.
Table 4. Table with the analysis of the word count. We display the three most mentioned words per group.
Using the final coding template, after the first round of coding, inter-rater reliability between the two study authors LB and DJW, was almost perfect according to Landis and Koch (Landis and Koch Citation1977) with a percentage agreement of 82% and a Cohen`s kappa of 0.81 related to the subtheme levels. After consensus finding, the interrater agreement between the coders was 100%. We describe all topics with percentages and examples in the subsequent sections.
Positive comments about the graphic algorithm (154 responses [100%])
Design (76 statements/49%)
Seventy-six of all 154 (49%) statements concerning the graphic algorithm related positive design features. We were able to divide this major topic into the subthemes Step-by-step structure and Colour code. Seventy-two of 154 (44%) comments were assigned to the two subgroups. In addition, four statements (5%) like ‘Little text’ (participant # 56) or ‘Appealing’ (participant # 60) were matched with the major topic Design.
Easy use (42 statements/27%)
This major topic was the content of 42 of 154 (27%) statements. The participants made 25 answers like ‘Simple in stressful situations’ (participant # 9). Furthermore, we specified Intuitive (10 statements) and Beginner friendly (7 statements) as subthemes.
Consideration of the clinical status (17 statements/11%)
In 17 of 154 (11%) comments, the participants mentioned that the patient's clinical status is considered. Especially, they noted the haemodynamic ‘Classification from stable to unstable’ (participant # 1) and the ‘Intermediate step whether the patient is anticoagulated or not’ (participant # 22) positively.
Concrete recommendations (11 statements/7%)
Many participants expressed that they liked the concrete recommendations (11 of 154 (7%) statements) given by the graphic algorithm. They gave statements like ‘Uncertainties are resolved’ (participant # 7), ‘Clearly defined’ (participant # 33) or ‘Clear cutoff values’ (participant # 35) in total eleven times.
Completeness (4 statements/3%)
Four of 154 (3%) answers considered the algorithm to be complete because it contains ROTEM interpretation (participant # 71) and therefore ‘There is hardly any danger of forgetting something when using it’ (participant# 18).
Processing time (4 statements/3%)
Four of 154 (3%) statements showed that the participants found that the graphic algorithm accelerates treatment, because of ‘Allowing a quick entrance’ (participant# 27) and a ‘Rapid interpretation of analytic data’ (participant # 53).
Negative comments about the graphic algorithm (88 responses [100%])
Incompleteness (23 statements/26%)
In 23 of 88 (26%) statements, the participants accused the algorithm of Incompleteness, which we defined as a major topic. As subthemes Unclear definition of the circulatory situation, Lack of dosage, Lack of recommendations for vasopressors and Pre-existing medication appeared. We allocated annotations like ‘There are only qualitative descriptions (normal/prolonged)’ (participant # 20) to this major topic. Ten statements concerned the unclear definition of stable/unstable in the algorithm. For example, participant # 18 mentioned: ‘Deciding in stable/unstable is difficult. If you do this wrong, you are on the wrong path’.
Design (21 statements/24%)
The content of 21 of 88 (24%) statements let us define this major topic in the negative section. We broke down this topic into two subthemes: Small font size (5%) and Cluttered (14%). It seems that, like participant # 51 (‘Font is small. Looks restless’), also others had problems reading the ‘Visually overloaded’ (participant# 27) cognitive aid. Additionally, we assigned statements as the one of Participant # 71 who criticised the reading flow to the major topic Design (5%).
Treatment steps (21 statements/24%)
Twenty-one of 88 (24%) statements considered the negative major topic Treatment steps. After inductive coding, we identified the subthemes Combined criteria and Redundant. Many participants considered the AND/OR criteria to be confusing. Others criticised the sequence of the steps, like participant # 26: ‘The anticoagulation column creates a delay’.
Scientific validity (12 statements/14%)
Twelve of 88 (14%) negative comments on the graphic algorithm criticised its Scientific validity. For example, the given mean arterial blood pressure and haematocrit limits were not always accepted. As participant # 7 noted: ‘For fresh frozen plasma (FFP) and prothrombin complex concentrate (PCC) it is better to either/or stand, because you never give both’.
Nothing (6 statements/7%)
As a further major topic, we defined Nothing. A total of six participants stated that they had nothing to criticise about the graphic algorithm.
Training required (5 statements/6%)
Five of 88 (6%) negative comments showed that participants considered prior training to be necessary.
Positive comments about the text-based algorithm (79 responses [100%])
Facilitates decision-making (37 statements/47%)
In 37 of 79 (47%) statements, the participants noticed how the text-based algorithm facilitated their Decision-making in fulfilling the given task. Additional inductive coding revealed Clear statements and Priorities as subthemes. Statements like ‘limits clearly defined’ (participant # 19) or ‘clear instructions’ (participant # 10) were made often (29 of 79 (37%) statements). The Priorities provided by the algorithm were considered helpful (8 of 79 (10%) statements), because the ‘Pathophysiological hierarchy’ (participant # 39) of bleeding is respected and ‘You can just go from top to bottom’ (participant # 19).
Completeness (17 statements/22%)
Like participant # 80, who stated, ‘It is complete’, 17 of 79 (9%) positive statements made about the text-based algorithm related to this major topic.
Structured design (15 statements/19%)
The inductive coding resulted in Structured design as another major topic. Fifteen of 79 (19%) comments directly addressed the structure as a positive aspect. The linear respectively organised structure appealed to participants # 50 and # 14.
User friendly (10 statements/13%)
As a further major topic, we identified User-friendliness in 10 of 79 (13%) statements. Under this topic, we summarised statements like ‘Simple for reading’ (participant # 81), ‘Flow of reading was intuitive’ (participant # 29) and ‘Good mix between suitability for everyday use and complexity’ (participant # 50).
Negative comments about the text-based algorithm (126 responses [100%])
Design (57 statements/45%)
Most of the negative comments (57 of 126 times [45%]) related to the Design of the text-based algorithm. As subthemes, we identified the aspects Extensive text and information, Unremarkable and Non-intuitive. Participants complained about the amount of information and that there was ‘Too much text, way too much to read’ (participant # 6). Moreover, the participants saw the colourless design without any highlights as problematic. They reported ‘Little overview at first contact’ (participant # 16) and ‘Not easy to find the information’ (participant # 40).
Unpractical (26 statements/21%)
Twenty-six of 126 (21%) statements considered that the text-based algorithm was Unpractical.
Through inductive coding, we subdivided this major topic into the subthemes Time consuming (7%), Challenging (5%) and Training required (3%). More general comments like ‘Not practical to be used in a bleeding situation’ (participant # 57) were gathered under this major topic (6%).
Unclear concept (24 statements/19%)
Twenty-four of 126 (19%) statements fit the major topic Unclear concept. For participant # 30 the text-based algorithm ‘Didn`t give a clue where to start from’. Other participants found the algorithm confusing and undifferentiated. Similar comments like ‘Not very clear’ (participant # 49) were given several times.
Missing information (19 statements/15%)
In nineteen of 126 (15%) statements, the participants expressed concern about Missing information. They mentioned ‘Unclear abbreviations’ (participant# 16) and that ‘There are no standard values’ (participant # 14). Further analysing derived that, for example, the management in case of pre-existing anticoagulation (participant # 48: ‘Anticoagulants are hardly mentioned’) and the circulatory situation (participant # 49: ‘Clinical situations not included’) were not taken into account accurately.
Discussion
This international dual-centre study analysed the opinions of 84 physicians regarding two different ways to present a coagulation management protocol. Our aim was to identify the strengths and limitations of both presentation modalities to foster the development of even better cognitive aids. We mainly concentrated on the statements relating to the structure and visual layout because this is the point in which the two studied cognitive aids differ fundamentally.
Critical events like haemorrhage call for rapid and accurate manoeuvres, despite stress and increased task load. Stress can cause cognitive and behavioural changes, which can lead to deficits in recalling information or missing treatment steps (van Pelt and Weinger Citation2017; Wheelock et al. Citation2015). Furthermore, returning to systemic thinking is more difficult in case of an emergency (Lupien et al. Citation2007; McEwen and Sapolsky Citation1995). Fittingly, the participants in our study expressed that they appreciated the step-by-step structure, ease of use, and concrete recommendations (graphic group), respectively, the user-friendliness of the structured design, and clear statements (text-based group) provided by their cognitive aid. Displaying cognitive aids during emergencies reduces omissions, time to perform tasks and improves team skills in most instances (Marshall Citation2017; Gawande and Arriaga Citation2013). In addition, the authors of the current European guideline on management of major bleeding suggest that adhering to coagulation protocols improves patients’ outcomes (Spahn et al. Citation2019). Nevertheless, like Marshall`s work suggested (Marshall Citation2013), our participants noted that poorly designed cognitive aids might even act as a distractor. The graphic group described the small font and the overcrowding as a disturbance. The participants of the text-based group criticised the high amount of text and the black and white format without any highlights as impractical and non-intuitive. User-centred protocols should be designed to relieve mental strain and not to overburden by cluttered design (Hales and Pronovost Citation2006). When using a purely text-based algorithm, physicians must create their mental model of the treatment from the given text in their minds by considering the content and prioritisation. We found that colour coding and step-by-step structure was perceived as helpful by the participants in the graphic algorithm.
The graphic representation has proven to be advantageous in the literature. For example, graphic display improved anaesthetists` situation awareness and clinical performance (Drews and Westenskow Citation2006; Jones et al. Citation2018; Marshall Citation2013; Harrison et al. Citation2006). This is also reflected in our survey, in which the graphic group made a significantly higher proportion of positive comments. Six participants also stated that they could not criticise anything at all about the graphic algorithm. On the other hand, all participants who worked with the text-based algorithm made at least one negative comment. We assume that the negative perceptions regarding the design of the graphic algorithm, such as the small font, could be rectified easily. According to the statements of the text-based group and the cited literature, a complete redesign of the layout seems necessary to achieve a user-centred design. The inventors of the ‘Haemostasis Traffic Light’ algorithm respected well-studied principles of user-centred design, aiming to transfer the required information to the decision-maker as quickly as possible and with the lowest cognitive demand (Endsley Citation2016). This graphic concept has already been tested and found as superior to the text-based algorithm in the theoretical management of six bleeding scenarios (Mileo et al. Citation2019).
Frequently, the participants mentioned the colour code as a positive feature. Using the ‘red-yellow-green’ coding as a traffic light has also been used in other medical protocols, e.g., emergency caesarean sections. It showed positive effects such as fewer referrals to neonatology of the newborn (Benazza et al. Citation2019). In the non-medical sector, traffic light labelling of food is a researched strategy to promote healthy eating (Osman and Thornton Citation2019; Thorndike et al. Citation2014), which underlines how universally known and highly intuitive this triple colour code is. In our case, it represents the severity of haemorrhage and the start in a particular arm of the algorithm. Timely recognition of bleeding severity is essential for initiating correct treatment (Schöchl et al. Citation2010).
Nevertheless, there was also criticism of the content, such as the treatment steps or scientific validity mentioned by the graphic group, or the lack of information – which both groups noted. This must be taken seriously. Existing algorithms should always be kept up to date with constant scientific re-evaluation to achieve the gold standard of evidence-based medicine and to foster adherence and acceptance of the algorithm. The above criticisms reflect the complexity of medical cases and the large number of diagnostic values with which physicians are confronted. Thus, it is important to have algorithms as a reliable basis (Jones et al. Citation2018). For example, Mehta et al. found that low guideline adherence in treating patients with acute coronary syndrome was associated with an increased risk of major bleeding and mortality (Mehta et al. Citation2015). Conversely, the introduction of a surgical safety checklist by the World Health Organisation was associated with a significant reduction in morbidity and mortality in routine practice (World Health Organization Citation2009).
This qualitative study highlighted physicians` positive and negative perceptions of two presentation methods for a coagulation management algorithm. It reveals the components that an ideal algorithm should have. On the one hand, the participants demand a clear structure with unambiguous statements without much text, which are reinforced in their hierarchy by visual highlighting. At the same time, the information given should correspond to the current state of research and guidelines. Furthermore, our analysis supports the superiority of a graphic layout. The traffic light layout seems to be a good choice, as it is known worldwide and therefore intuitive for a large user group.
The study has the inherent limitations of qualitative research. The participants` opinions cannot be transferred to broader populations with the same certainty as quantitative research, because qualitative results are developed inductively. Because participants indicated the quality of their statement- as positive or negative, we decided to exert quantitative tests which revealed statistical significance favouring the graphical algorithm. A generalisable validity and reliability for this result can only be postulated with caution. However, qualitative analysis gives a detailed description of the widest possible range of subjective opinions and each observation receives attention (Dossett, Kaji, and Cochran Citation2021; Carr Citation1994; Ochieng Citation2009). Using a qualitative approach enabled us to identify everyday problems with the presentations of checklists and standard operating procedures, which are nowadays required by the quality management of hospitals and lawmakers (Miljak and Zaar Citation2017; Qaseem et al. Citation2012; Nachtigall et al. Citation2008).
Furthermore, a potential response bias cannot be excluded as the interviewers knew most of the participants personally from daily work. Anyhow, this study focussed on analysing both design variants. Ultimately, the text-based algorithm`s layout is very similar to the existing standard operating procedure for coagulation management in massive haemorrhage of the University Hospital of Zurich. Indeed, we saw that more statements from Buenos Aires commented on the text-based approach`s design negatively. This finding may be explained by the familiarity of the other participant group with the text-based modality, which probably compensates for deficits in design aspects. Further within-subject comparisons could clarify this point. However, we postulate that subjective opinions can be evaluated even with a pre-existing design.
A strength of the study is the international dual-centre design, which helped thereby to analyse a broader range of opinions. Both study centres are tertiary care hospitals with great diagnostic possibilities. The algorithms mapped using cognitive aids may not be helpful in many, especially smaller health care facilities where ROTEM diagnostic is not available. However, this kind of graphic algorithm is supposed to have the advantage of providing a standard structure that can be easily adapted to local resources. Nevertheless, studies show that point of care diagnostics provide a benefit in patient care and should be expanded (Spahn et al. Citation2019; Theusinger, Stein, and Levy Citation2015).
Conclusions
In conclusion, both groups found it helpful to gain structured information through clear statements. We found a higher proportion of positive comments on the graphic algorithm, suggesting a generally stronger approval of this cognitive aid. Our analysis shows how important visual presentation of medical information is and many variables to consider for designing. The triple colour code ‘red-yellow-green’ of the graphic algorithm is familiar to most people, and the colours intuitively represent the severity of bleeding. In contrast, when using a text-based algorithm, physicians must create their own mental models, assembling them from text fragments, which means an additional cognitive workload. The user perceptions identified by this study may inform the development of future algorithms. The importance of user-centred design in algorithms to simplify and accelerate decision-making cannot be overstated. Care providers can only be as good as the information presented to them.
Authors’ contributions
LB, EDK, FGM, TRR, SS, DRS, DWT and DJW helped to design the study. EDK, FGM, TRR, SS and DWT helped to collect the data. LB, DRS, DWT and DJW helped to analyse the data. LB, EDK, FGM, TRR, SS, DRS, DWT and DJW helped to write the article and approve the final version.
Ethics approval and consent to participate
The Cantonal Ethics Committee of Zurich in Switzerland reviewed the study protocol and issued a declaration of no objection (Business Management System for Ethics Committees Number 2019-01090). The ethics committee responsible for the centre in Buenos Aires approved the study protocol in a separate statement (No 5357, dated 14th November 2019). Before participation, we obtained written informed consent from all participants to the anonymous use of their data.
| Abbreviations | ||
| IQR | = | Interquartile range |
| HIBA | = | Hospital Italiano de Buenos Aires |
| POC | = | point of care |
| ROTEM | = | Rotational thromboelastometry |
| graphic | = | ‘Haemostasis Traffic Light’ algorithm |
| USZ | = | University Hospital Zurich |
TERG-2021-0284-File003.docx
Download MS Word (33.2 KB)TERG-2021-0284-File002.pdf
Download PDF (2.8 MB)Acknowledgements
The authors are thankful to the study participants for their time and effort. The authors thank biostatistician Dr. Julia Braun- Gruebel Ph.D. for her statistical advice regarding this manuscript. This study was carried out within the framework of a collaborative research and education agreement between USZ and HIBA.
Disclosure statement
EDK and FGM received honoraria for their participation in ROTEM frontier council from Werfen, Instrumentation Laboratory, Bedford, MA.
DRS's academic department is receiving grant support from the Swiss National Science Foundation, Berne, Switzerland, the Swiss Society of Anaesthesiology and Reanimation (SGAR), Berne, Switzerland, the Swiss Foundation for Anaesthesia Research, Zurich, Switzerland, Vifor SA, Villars-sur-Glâne, Switzerland and Vifor (International) AG, St. Gallen, Switzerland.
DRS is co-chair of the ABC-Trauma Faculty, sponsored by unrestricted educational grants from Novo Nordisk Health Care AG, Zurich, Switzerland, CSL Behring GmbH, Marburg, Germany, LFB Biomédicaments, Courtaboeuf Cedex, France and Octapharma AG, Lachen, Switzerland.
DRS received honoraria/travel support for consulting or lecturing from:
Danube University of Krems, Austria, US Department of Defense, Washington, USA, European Society of Anaesthesiology, Brussels, BE, Korean Society for Patient Blood Management, Seoul, Korea, Korean Society of Anaesthesiologists, Seoul, Korea, Network for the Advancement of Patient Blood Management, Haemostasis and Thrombosis, Paris, France, Baxalta Switzerland AG, Volketswil, Switzerland, Bayer AG, Zürich, Switzerland, B. Braun Melsungen AG, Melsungen, Germany, Boehringer Ingelheim GmbH, Basel, Switzerland, Bristol-Myers-Squibb, Rueil-Malmaison Cedex, France and Baar, Switzerland, CSL Behring GmbH, Hattersheim am Main, Germany and Berne, Switzerland, Celgene International II Sàrl, Couvet, Switzerland, Daiichi Sankyo AG, Thalwil, Switzerland, Haemonetics, Braintree, MA, USA, Instrumentation Laboratory (Werfen), Bedford, MA, USA, LFB Biomédicaments, Courtaboeuf Cedex, France, Merck Sharp & Dohme, Kenilworth, New Jersey, USA, Novo Nordisk Health Care AG, Zurich, Switzerland, PAION Deutschland GmbH, Aachen, Germany, Pharmacosmos A/S, Holbaek, Denmark, Pfizer AG, Zürich, Switzerland, Pierre Fabre Pharma, Alschwil, Switzerland, Portola Schweiz GmbH, Aarau, Switzerland, Roche Diagnostics International Ltd, Reinach, Switzerland, Sarstedt AG & Co., Sevelen, Switzerland and Nümbrecht, Germany, Shire Switzerland GmbH, Zug, Switzerland, Tem International GmbH, Munich, Germany, Vifor Pharma, Munich, Germany, Neuilly sur Seine, France and Villars-sur-Glâne, Switzerland, Vifor (International) AG, St. Gallen, Switzerland, Zuellig Pharma Holdings, Singapore, Singapore.
DWT is a designated inventor of Visual Patient technology, for which the University of Zurich holds various patents and trademarks. There are cooperation and licencing agreements with Philips Medizin Systeme Böblingen GmbH, Böblingen, Germany; Koninklijke Philips NV, Amsterdam, The Netherlands; and Philips Research/Philips Electronics Nederland BV, Eindhoven, The Netherlands. Under these agreements, DWT may receive royalties. DWT and DRS are designated inventors of Visual Clot technology, for which the University of Zurich holds various patents and trademarks. The University of Zurich signed a letter of intent for a cooperation and licencing agreement with Instrumentation Laboratory Company/Werfen Corporation, Bedford, MA, USA and Barcelona, Spain. Under this and future agreements, DWT and DRS may receive royalties. DWT and DRS received travel support for consulting Instrumentation Laboratory, Bedford, MA, USA.
The other authors do not have any competing interests.
Data availability statement
The datasets supporting the conclusions of this article are available in this article and its supplementary information files.
Related Research Data
References
- Benazza, N., L. Touzart, C. Muszynski, and J. Gondry. 2019. “Impact of Establishment of a Color Code in Emergency Caesareans in Secondary Health Care Maternity.” Journal of Gynecology Obstetrics and Human Reproduction 48 (4): 261–264. doi:10.1016/j.jogoh.2018.11.010.
- Carr, L. T. 1994. “The Strengths and Weaknesses of Quantitative and Qualitative Research: What Method for Nursing?” Journal of Advanced Nursing 20 (4): 716–721. doi:10.1046/j.1365-2648.1994.20040716.x.
- Dambacher, M., and R. Hübner. 2015. “Time Pressure Affects the Efficiency of Perceptual Processing in Decisions under Conflict.” Psychological Research 79 (1): 83–94. doi:10.1007/s00426-014-0542-z.
- Dossett, L. A., A. H. Kaji, and A. Cochran. 2021. “SRQR and COREQ Reporting Guidelines for Qualitative Studies.” JAMA Surgery 156 (9): 875. doi:10.1001/jamasurg.2021.0525.
- Drews, F. A., and D. R. Westenskow. 2006. “The Right Picture is Worth a Thousand Numbers: Data Displays in Anesthesia.” Human Factors 48 (1): 59–71. doi:10.1518/001872006776412270.
- Endsley, M. R. 1995. “Toward a Theory of Situation Awareness in Dynamic Systems.” Human Factors: The Journal of the Human Factors and Ergonomics Society 37 (1): 32–64. doi:10.1518/001872095779049543.
- Endsley, M. R. 2016. Designing for Situation Awareness: An Approach to User-Centered Design. 2nd ed. Boca Raton, FL: CRC Press.
- Gawande, A. A., and A. F. Arriaga. 2013. “A Simulation-Based Trial of Surgical-Crisis Checklists.” The New England Journal of Medicine 368 (15): 1460. doi:10.1056/NEJMc1301994.
- Gruen, R. L., G. J. Jurkovich, L. K. McIntyre, H. M. Foy, and R. V. Maier. 2006. “Patterns of Errors Contributing to Trauma Mortality: lessons Learned from 2,594 Deaths.” Annals of Surgery 244 (3): 371–380. doi:10.1097/01.sla.0000234655.83517.56.
- Hales, B. M., and P. J. Pronovost. 2006. “The Checklist-A Tool for Error Management and Performance Improvement.” Journal of Critical Care 21 (3): 231–235. doi:10.1016/j.jcrc.2006.06.002.
- Harrison, T. K., T. Manser, S. K. Howard, and D. M. Gaba. 2006. “Use of Cognitive Aids in a Simulated Anesthetic Crisis.” Anesthesia and Analgesia 103 (3): 551–556. doi:10.1213/01.ane.0000229718.02478.c4.
- Haynes, Alex B., Thomas G. Weiser, William R. Berry, Stuart R. Lipsitz, Abdel-Hadi S. Breizat, E. Patchen Dellinger, Teodoro Herbosa, Sudhir Joseph, Pascience L. Kibatala, Marie Carmela M. Lapitan, Alan F. Merry, Krishna Moorthy, Richard K. Reznick, Bryce Taylor, and Atul A. Gawande. 2009. “A Surgical Safety Checklist to Reduce Morbidity and Mortality in a Global Population.” New England Journal of Medicine 360 (5): 491–499. doi:10.1056/NEJMsa0810119.
- Jones, C. P. L., J. Fawker-Corbett, P. Groom, B. Morton, C. Lister, and S. J. Mercer. 2018. “Human Factors in Preventing Complications in Anaesthesia: A Systematic Review.” Anaesthesia 73 (Suppl 1): 12–24. doi:10.1111/anae.14136.
- Kataife, E. D., S. Said, J. Braun, T. R. Roche, J. Rössler, A. Kaserer, D. R. Spahn, F. G. Mileo, and D. W. Tscholl. 2021. “The Haemostasis Traffic Light, a User-Centred Coagulation Management Tool for Acute Bleeding Situations: A Simulation-Based Randomised Dual-Centre Trial.” Anaesthesia 76 (7): 902–910. doi:10.1111/anae.15314.
- Keebler, J. R., E. H. Lazzara, E. Blickensderfer, and T. D. Looke. 2018. “Human Factors Applied to Perioperative Process Improvement.” Anesthesiology Clinics 36 (1): 17–29. doi:10.1016/j.anclin.2017.10.005.
- Konstantinides, Stavros V., Guy Meyer, Cecilia Becattini, Héctor Bueno, Geert-Jan Geersing, Veli-Pekka Harjola, Menno V. Huisman, Marc Humbert, Catriona Sian Jennings, David Jiménez, Nils Kucher, Irene Marthe Lang, Mareike Lankeit, Roberto Lorusso, Lucia Mazzolai, Nicolas Meneveau, Fionnuala Ní Áinle, Paolo Prandoni, Piotr Pruszczyk, Marc Righini, Adam Torbicki, Eric Van Belle, and José Luis Zamorano. 2019. “2019 ESC Guidelines for the Diagnosis and Management of Acute Pulmonary Embolism Developed in Collaboration with the European Respiratory Society (ERS): the Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC).” European Respiratory Journal 54 (3): 1901647. doi:10.1183/13993003.01647-2019.
- Landis, J. R., and G. G. Koch. 1977. “The Measurement of Observer Agreement for Categorical Data.” Biometrics 33 (1): 159–174. doi:10.2307/2529310.
- Lange, R. T. 2011. “Inter-Rater Reliability.” In Encyclopedia of Clinical Neuropsychology, edited by J. S. Kreutzer, J. DeLuca, and B. Caplan, 1348–1348. New York, NY: Springer New York.
- Larsson, J., U. Rosenqvist, and I. Holmström. 2006. “Being a Young and Inexperienced Trainee Anesthetist: A Phenomenological Study on Tough Working Conditions.” Acta Anaesthesiologica Scandinavica 50 (6): 653–658. doi:10.1111/j.1399-6576.2006.01035.x.
- Lupien, S. J., F. Maheu, M. Tu, A. Fiocco, and T. E. Schramek. 2007. “The Effects of Stress and Stress Hormones on Human Cognition: Implications for the Field of Brain and Cognition.” Brain and Cognition 65 (3): 209–237. doi:10.1016/j.bandc.2007.02.007.
- Marshall, S. 2013. “The Use of Cognitive Aids during Emergencies in Anesthesia: A Review of the literature.” Anesthesia & Analgesia 117 (5): 1162–1171. doi:10.1213/ANE.0b013e31829c397b.
- Marshall, S. D. 2017. “Helping Experts and Expert Teams Perform under Duress: An Agenda for Cognitive Aid Research.” Anaesthesia 72 (3): 289–295. doi:10.1111/anae.13707.
- McEwen, B. S., and R. M. Sapolsky. 1995. “Stress and Cognitive Function.” Current Opinion in Neurobiology 5 (2): 205–216. doi:10.1016/0959-4388(95)80028-X.
- Mehta, R. H., A. Y. Chen, K. P. Alexander, E. M. Ohman, M. T. Roe, and E. D. Peterson. 2015. “Doing the Right Things and Doing Them the Right Way: Association between Hospital Guideline Adherence, Dosing Safety, and Outcomes among Patients with Acute Coronary Syndrome.” Circulation 131 (11): 980–987. doi:10.1161/circulationaha.114.013451.
- Mileo, F. G., E. D. Kataife, A. Adrover, G. Garcia Fornari, M. S. Lopez, and M. E. Martinuzzo. 2019. “The Haemostasis Traffic Light: A Pragmatic Tool for Bleeding Management.” Anaesthesia 74 (12): 1623–1624. doi:10.1111/anae.14874.
- Miljak, T., and P. Zaar. 2017. “Standard Operating Procedures in Clinical Medicine.” Deutsche Medizinische Wochenschrift 142 (18): 1390–1395. doi:10.1055/s-0043-108436. Standards in der klinischen Medizin.
- Nachtigall, I., M. Deja, S. Tafelski, A. Tamarkin, K. Schaller, E. Halle, P. Gastmeier, KD. Wernecke, T. Bauer, M. Kastrup, and C. Spies. 2008. “Adherence to Standard Operating Procedures is Crucial for Intensive Care Unit Survival of Elderly Patients.” Journal of International Medical Research 36 (3): 438–459. doi:10.1177/147323000803600308.
- O'Brien, B. C., I. B. Harris, T. J. Beckman, D. A. Reed, and D. A. Cook. 2014. “Standards for Reporting Qualitative Research: A Synthesis of Recommendations.” Academic Medicine 89 (9): 1245–1251. doi:10.1097/acm.0000000000000388.
- Ochieng, P. 2009. “An Analysis of the Strengths and Limitation of Qualitative and Quantitative Research Paradigms.” Problems of Education in the 21st Century 13: 13–18.
- Osman, M., and K. Thornton. 2019. “Traffic Light Labelling of Meals to Promote Sustainable Consumption and Healthy Eating.” Appetite 138: 60–71. doi:10.1016/j.appet.2019.03.015.
- Qaseem, A., F. Forland, F. Macbeth, G. Ollenschläger, S. Phillips, and P. van der Wees. 2012. “Guidelines International Network: Toward International Standards for Clinical Practice Guidelines.” Annals of Internal Medicine 156 (7): 525–531. doi:10.7326/0003-4819-156-7-201204030-00009.
- Schöchl, H., L. Forster, R. Woidke, C. Solomon, and W. Voelckel. 2010. “Use of Rotation Thromboelastometry (ROTEM) to Achieve Successful Treatment of Polytrauma with Fibrinogen Concentrate and Prothrombin Complex Concentrate.” Anaesthesia 65 (2): 199–203. doi:10.1111/j.1365-2044.2009.06188.x.
- Spahn, Donat R., Bertil Bouillon, Vladimir Cerny, Jacques Duranteau, Daniela Filipescu, Beverley J. Hunt, Radko Komadina, Marc Maegele, Giuseppe Nardi, Louis Riddez, Charles-Marc Samama, Jean-Louis Vincent, and Rolf Rossaint. 2019. “The European Guideline on Management of Major Bleeding and Coagulopathy following Trauma: fifth Edition.” Critical Care 23 (1): 98. doi:10.1186/s13054-019-2347-3.
- Starmer, Amy J., Nancy D. Spector, Rajendu Srivastava, Daniel C. West, Glenn Rosenbluth, April D. Allen, Elizabeth L. Noble, Lisa L. Tse, Anuj K. Dalal, Carol A. Keohane, Stuart R. Lipsitz, Jeffrey M. Rothschild, Matthew F. Wien, Catherine S. Yoon, Katherine R. Zigmont, Karen M. Wilson, Jennifer K. O’Toole, Lauren G. Solan, Megan Aylor, Zia Bismilla, Maitreya Coffey, Sanjay Mahant, Rebecca L. Blankenburg, Lauren A. Destino, Jennifer L. Everhart, Shilpa J. Patel, James F. Bale, Jaime B. Spackman, Adam T. Stevenson, Sharon Calaman, F. Sessions Cole, Dorene F. Balmer, Jennifer H. Hepps, Joseph O. Lopreiato, Clifton E. Yu, Theodore C. Sectish, and Christopher P. Landrigan. 2014. “Changes in Medical Errors after Implementation of a Handoff Program.” New England Journal of Medicine 371 (19): 1803–1812. doi:10.1056/NEJMsa1405556.
- Stein, P., A. Kaserer, G. H. Spahn, and D. R. Spahn. 2017. “Point-of-Care Coagulation Monitoring in Trauma Patients.” Seminars in Thrombosis and Hemostasis 43 (4): 367–374. doi:10.1055/s-0037-1598062.
- Theusinger, O. M., P. Stein, and J. H. Levy. 2015. “Point of Care and Factor Concentrate-Based Coagulation Algorithms.” Transfusion Medicine and Hemotherapy 42 (2): 115–121. doi:10.1159/000381320.
- Thorndike, A. N., J. Riis, L. M. Sonnenberg, and D. E. Levy. 2014. “Traffic-Light Labels and Choice Architecture: promoting Healthy Food Choices.” American Journal of Preventive Medicine 46 (2): 143–149. doi:10.1016/j.amepre.2013.10.002.
- Tong, A., P. Sainsbury, and J. Craig. 2007. “Consolidated Criteria for Reporting Qualitative Research (COREQ): A 32-Item Checklist for Interviews and Focus Groups.” International Journal for Quality in Health Care 19 (6): 349–357. doi:10.1093/intqhc/mzm042.
- van Pelt, M., and M. B. Weinger. 2017. “Distractions in the Anesthesia Work Environment: Impact on Patient Safety? Report of a Meeting Sponsored by the Anesthesia Patient Safety Foundation.” Anesthesia & Analgesia 125 (1): 347–350. doi:10.1213/ane.0000000000002139.
- Vasileiou, K., J. Barnett, S. Thorpe, and T. Young. 2018. “Characterising and Justifying Sample Size Sufficiency in Interview-Based Studies: Systematic Analysis of Qualitative Health Research over a 15-Year Period.” BMC Medical Research Methodology 18 (1): 148. doi:10.1186/s12874-018-0594-7.
- Wheelock, Ana, Amna Suliman, Rupert Wharton, E. D. Babu, Louise Hull, Charles Vincent, Nick Sevdalis, and Sonal Arora. 2015. “The Impact of Operating Room Distractions on Stress, Workload, and Teamwork.” Annals of Surgery 261 (6): 1079–1084. doi:10.1097/SLA.0000000000001051.
- Winearls, James, Michael Reade, Helen Miles, Andrew Bulmer, Don Campbell, Klaus Görlinger, and John F. Fraser. 2016. “Targeted Coagulation Management in Severe Trauma: The Controversies and the Evidence.” Anesthesia and Analgesia 123 (4): 910–924. doi:10.1213/ane.0000000000001516.
- World Health Organization. 2009. “WHO Guidelines Approved by the Guidelines Review Committee.” In WHO Guidelines for Safe Surgery 2009: Safe Surgery Saves Lives. Geneva: World Health Organization.
- Wung, S. F. 2018. “Human Factors and Technology in the ICU.” Critical Care Nursing Clinics of North America 30 (2): xi–xii. doi:10.1016/j.cnc.2018.03.001.