Reducing failures in daily medical practice: Healthcare failure mode and effect analysis combined with computer simulation

Abstract

This study proposes a risk analysis approach for complex healthcare processes that combines qualitative and quantitative methods to improve patient safety. We combine Healthcare Failure Mode and Effect Analysis with Computer Simulation (HFMEA-CS), to overcome widely recognised HFMEA drawbacks regarding the reproducibility and validity of the outcomes due to human interpretation, and show the application of this methodology in a complex healthcare setting. HFMEA-CS is applied to analyse drug adherence performance in the surgical admission to discharge process of pheochromocytoma patients. The multidisciplinary team identified and scored the failure modes, and the simulation model supported in prioritisation of failure modes, uncovered dependencies between failure modes, and predicted the impact of measures on system behaviour. The results show that drug adherence, defined as the percentage of required drugs received at the right time, can be significantly improved with 12%, to reach a drug adherence of 99%. We conclude that HFMEA-CS is both a viable and effective risk analysis approach, combining strengths of expert opinion and quantitative analysis, for analysing human-system interactions in socio-technical systems.

Practitioner summary: We propose combining Healthcare Failure Mode and Effects Analysis with Computer Simulation (HFMEA-CS) for prospective risk analysis of complex and potentially harmful processes, to prevent critical incidents from occurring. HFMEA-CS combines expert opinions with quantitative analyses, such that the results are more reliable, reproducible, and fitting for complex healthcare settings.

Keywords:

• Patient safety
• system failure modelling
• risk assessment and management
• human reliability
• system performance modelling

Introduction

There is a high focus on the reduction of adverse events in complex systems, such as complex healthcare settings. To this end, complex and potentially harmful processes can be prospectively assessed on their risks and preventing them from occurring before (patient) harm occurs (Abrahamsen, Abrahamsen, and Høyland 2016), instead of waiting for a critical incident to retrospectively assess the root causes (Franklin, Shebl, and Barber 2012).

Healthcare Failure Mode and Effects Analysis (HFMEA) is a widely used prospective risk analysis approach in healthcare (Abrahamsen, Abrahamsen, and Høyland 2016). HFMEA is a five-step multidisciplinary approach to map the process flow of a high-risk healthcare process, identify potential failures and their causes and effects, assess these failure modes, and propose risk mitigation measures. It has been applied in various settings, such as drug prescription and pharmacy services (van Tilburg et al. 2006; Potts et al. 2014; Vélez-Díaz-Pallarés et al. 2013), neonatal care transitions (Moyer et al. 2010), and radiotherapy services (Vlayen 2011).

Although HFMEA is frequently applied, a widely recognised major drawback of HFMEA is the reproducibility (and thus reliability) and validity of the outcomes (Abrahamsen, Abrahamsen, and Høyland 2016; Potts et al. 2014; Faiella et al. 2018; Shaqdan et al. 2014). The subjective nature of HFMEA results for the same mapped process-flows in variation of the identified failure modes, and even larger variation of hazard scores (Shebl, Franklin, and Barber 2009). This leads to the question on how to assess the prioritisation of failure modes within the HFMEA framework, to overcome the challenges with reproducibility and validity (Franklin, Shebl, and Barber 2012).

The contribution of this research is to propose and pilot a mixed-methods approach that enhances the multi-stakeholder HFMEA methodology with objective risk assessment through Computer Simulation (CS), to overcome the reproducibility and outcome validity drawbacks of HFMEA. This brings the field of risk management a strong and interrelated combination of a multi-stakeholder (qualitative) perspective for risk identification and mitigation measure identification and selection, with data-driven objective prospective risk assessment and prioritisation. The strong elements of HFMEA (e.g. the focus on mapping and understanding the healthcare process in a structured and multidisciplinary way) are combined with the quantitative prospective assessment of failure modes using CS, to overcome the qualitative risk assessment deficiencies of HFMEA.

CS is a computational modelling approach emerging in the field of ergonomics that is able to model complex problems and inform solution development (Read et al. 2020). CS models provide evidence for potential system changes and assess these systems and/or system changes on costs, unintended consequences, and risks of potential process failures, by dynamically simulating the interactions and relationships between the input and output in a system (Pitt et al. 2016; Pooya et al. 2014). Contrary to simulations in a simulation centre or in situ simulations, as e.g. proposed to combine with HFMEA by (Nielsen et al. 2014), CS allows for prospectively assessing numerous potential strategies without actual intervening in real-life practice (Zhang, 2018), and therefore without large additional time-investments of the HFMEA team. In healthcare, the most widely used CS approach is Discrete Event Simulation (DES), which allows for prospective assessment of a system evolving over time with interacting, possibly heterogeneous, entities (typically patients), for example in care process flow studies (Brailsford 2007; Law, Kelton, and Kelton 2000). If the entities are mutually independent, such as for example in drug adherence studies, static simulation models from the class of Monte Carlo Simulation (MCS) models may be used (Law, Kelton, and Kelton 2000; Zhang, 2018). A third class of CS models, System Dynamics Simulation (SDS) models, are recommended for health policy and population studies (Brailsford 2007; Katsaliaki and Mustafee 2011). CS has been applied in a wide range of healthcare settings to assess the severity of problems, and to assess the impact of possible solutions (Zhang, 2018), such as smoking-cessation behaviour (Igarashi et al. 2016), patient flow in orthopaedic fracture pathways (Anderson et al. 2017), the effects of nursing workload (Farid, Purdy, and Neumann 2020), and the impact of emergency department crowding (Ahalt et al. 2018).

Recently, the extension of HFMEA with DES was first introduced in the literature (Ershadi, Ershadi, and Niaki 2020). In this work, First HFMEA was executed, after which a DES model determined the effects of selected measures before actual implementation in practice. However, an integrated HFMEA and CS (HFMEA-CS) approach, in which CS also supports the impact and prioritisation of failure modes, is not yet performed or described in the medical literature. However, in other safety-critical industries, this integrated combination of CS and FMEA is more common (Liu, Liu, and Liu 2013; Spreafico, Russo, and Rizzi 2017). FMEA is a general safety and (human) reliability analysis tool for products, including medical devices, and processes in a wide range of safety-critical industries, such as manufacturing, aerospace, and maintenance management (Lin et al. 2014). In contrary to the hazard score of HFMEA, FMEA scores failure modes using the risk priority number (RPN), which also incorporates the detectability based on expert opinions, besides occurrence and severity (Rah et al. 2016). Just as HFMEA, FMEA is also subjective in nature as the RPNs are influenced by the stakeholders’ disciplines, perceptions and experiences. Therefore the reproducibility of the approach is a joint deficiency of both approaches (Lin et al. 2014; Sagnak et al. 2020; Steinfeld et al. 2015). Therefore, CS can be beneficial for FMEA as well. For example, Bevilacqua et al. (Bevilacqua, Braglia, and Gabbrielli 2000) extended FMEA with MCS, to test the weights assigned to the RPN elements. The prioritisation of policies did not require a deterministic evaluation anymore, as through the MCS the final priority rank was derived. Besides addressing the RPN, other main reasons for combining FMEA and CS are increased benefits in considering the relation between various failure modes (e.g. Neghab et al. 2011), as well as the effects within the entire process chain, while incorporating more reliable data on uncertainty in these processes (e.g. Damiani et al. 2017; Lillie, Sandborn, and Humphrey 2015).

Rare complex conditions often require a multidisciplinary approach where specialists from different disciplines work together for delivering care tailored to the individual patient’s needs. This care is often delivered in centres of expertise. Because of the complexity of the care process and the need for individualised care, there is a high-risk for failures and adverse events. Therefore, such care processes highly benefit from a thorough risk analysis.

This paper combined HFMEA and CS by using CS to quantitatively assess the impact of failure modes with moderate to high hazard scores from the HFMEA. The impact of the combination of proposed measures was analysed using the CS model, to aid the multi-disciplinary team in their decision making and provide quantified evidence for the suggested measures. In order to verify the feasibility of HFMEA-CS, this approach was applied in a complex clinical environment: the drug adherence in the surgical admission to discharge process of pheochromocytoma patients.

Materials and methods

Hospital setting

The study was conducted in University Medical Center Utrecht, which is a reference centre within the European Reference Network ‘rare endocrine conditions’. The proposed methodology, HFMEA-CS, was verified with the medication processes related to the admission to discharge process of pheochromocytoma patients. A pheochromocytoma is rare tumour of the adrenal gland. Drug adherence for patients with pheochromocytoma that are operated is of utmost importance. When operated, patients with pheochromocytomas are at high risk of hemodynamic instability. Therefore, patients are prepared with a carefully titrated dose of alfa blockers. Furthermore, if pheochromocytomas occur in both adrenals, patients are in per- and post-operative need for hydrocortisone and fludrocortisone. Therefore, multi-disciplinary care paths are in use for standardising the care, which must be adjusted per patient according to the individual patient’s (drug) needs. Due to the care pathway complexity, transfers from and to several wards are involved, divided over five phases: pre-operative screening, hospitalisation at internal medicine ward, operating theatre/recovery, post-operative care at surgical oncological ward, and discharge, as shown in Figure 1. Each of these phases involves specialised healthcare professionals, which requires careful attention to information transfers over the various disciplines, specifically related to medication requirements, to ensure patient safety.

Figure 1. Process flow an drug requirements of pheochromocytoma admission to discharge process.

5 step process flow and drug requirements of a pheochromocytoma patient in the admission to discharge process, starting with pre-operative screening, followed by hospitalisation at the internal medicine ward, followed by surgery and recovery, followed by post-operative care at the surgical oncological ward, ending with discharge to home. Relevant drugs, such as home medication, alpha and beta blockades, pain and nausea medication, hydrocortisone and fludrocortisone are associated with each step.

Data of all admitted pheochromocytoma patients from 2017 to 2018 were used, with a total of 18 patients. For retrieving the anonymized and de-identified patient data we followed the safety monitoring system of the involved department.

HFMEA-CS procedure

In HFMEA, a multidisciplinary team maps the process steps of the entire care process, identifies the potential failure modes and scores their severity and likelihood, using a five-step process: (1) Define the HFMEA topic; (2) Assemble the HFMEA team; (3) Describe the process; (4) Conduct a hazard analysis; and (5) Determine actions and potential improvement measures (Abrahamsen, Abrahamsen, and Høyland 2016; DeRosier et al. 2002).

In HFMEA-CS, the five-step HFMEA methodology is extended with a CS step, and therefore consists of the following six steps:

Step 1 defines the process under evaluation

This study analysed the surgical admission to discharge process of pheochromocytoma patients.

Step 2 assembles the multidisciplinary HFMEA-CS team

The multidisciplinary team existed of one or two nurses of each involved department (four in total), two pharmacy assistants of each involved ward (internal medicine ward for pre-operative alfa blockade titration, and surgical ward for post-operative period), one endocrinologist, one endocrinology nurse practitioner, one resident at each involved ward, one endocrinology fellow, one project manager, and one simulation expert. Since this research was inspired by a patient that pointed us towards an opportunity of improvement with respect to (failed) drug transfers, this patient was also involved in designing the control and elimination measures in the end phase of this research. Starting from February 2019, the HFMEA-CS team was scheduled to meet weekly in 1.0–1.5 hour sessions, with a total of six meetings. Since the time schedules did not allow for more than six meetings to be held with all members present, and some participants were (last-minute) not able to be present in two sessions, additional interviews with individual participants were held to further specify quantitative information as input for the simulation model. All important decisions were made in the multi-disciplinary meetings, to represent the multi-stakeholder collaboration perspective of HFMEA. The HFMEA-CS procedure ended in June 2019 with a final meeting with the entire team, in which the results and recommendations were discussed.

In step 3, the multidisciplinary HFMEA-CS team maps the process under review using flowcharts, and identifies failure modes, their causes, and the effects

During the first 3 meetings the processes were discussed, and mapped in a flowchart. Subsequently, the failure modes were identified, based on the experience of the participating team members. For each failure mode, the potential causes and effects were identified by the team members as well.

Step 4 determines the hazard score of each failure mode (severity multiplied with likelihood)

During the subsequent two meetings of the HFMEA-CS team, the team scored the severity and likelihood of each failure mode on a 4-point numerical scale, as shown in Table 1 (based on DeRosier et al. 2002; Habraken et al. 2009). This consensus team score was solely based on subjective assessment of the team, given their experience.

Table 1. Likelihood and severity scoring model.

Score	Severity	Likelihood
1	Minor	Remote (<1x per 50 patients)
2	Moderate	Uncommon (≥1x per 50 patients)
3	Major	Occasional (≥1x per 20 patients)
4	Catastrophic	Frequent (≥1x per 10 patients)

Step 5 performs the CS study to determine the quantitative effects of failure modes with moderate or high hazard scores

The multi-disciplinary team decided which failure modes are input to the simulation model. Typically, all failure modes with a moderate or high hazard score (i.e. hazard score ≥ 4) are ranked, but exceptions can be made based on team consensus (DeRosier et al. 2002). Therefore, these selected failure modes will be the input for the CS model. In the CS model, the process’ flowchart of Step 3 is used as the conceptual model, and the target Key Performance Indicators (KPIs) are determined. Relevant input data is gathered, and statistical distributions are fitted to this data where relevant. The conceptual model is programmed in a simulation environment, and verified and validated. For each of the failure modes, as well as the ase case scenario, a simulation is run using a one-factor-at-a-time design, to assess the performance in the best case, worst case, and current situation, and results are presented.

As the pheochromocytoma patients under review are part of a rare disease patient group, mutually independency of the patients within the admission to discharge process can be assumed. Therefore, a MCS was performed. The patient care process was modelled based on the flow-chart of Step 3. The model used real data from all pheochromocytoma patients admitted to the hospital in 2017–2018. Based on this limited data, distributions on the length-of-stay and drugs stock per department were determined by beta distributions and empirical distributions respectively. The drug requirements were modelled by partially correlated uniform distributions using the copula method, to achieve a correlation coefficient of τ = 0.85.

With each failure mode, an input parameter or variable (e.g. ‘percentage of patients that forgot their home medication’) was varied to see the effect on the predicted outcomes. The expected occurrence was based on data derived from the hospital information system. If no reliable data was available, interviews with responsible hospital staff were held and a sensitivity analysis with the outcomes was performed for a variety of values.

The simulation model was programmed in Plant Simulation version 13.

In step 6, the multidisciplinary HFMEA-CS team decides if a failure mode should be accepted, controlled, or eliminated, and provides recommendations

Acceptance could mean that the multidisciplinary team accepts the risk of a failure, but could also mean that there are already sufficient measures to ensure that the risk of the failure is minimal. Controlling a failure mode means that the team comes up with a control measure. Eliminating a failure mode means that the team designs a measure such that the failure mode cannot occur anymore.

As a result from Step 5, all failure modes were positioned in one of three categories and ranked on their priority based on their performance. These categories are:

1. Significant improvement with current situation. Proposed decision: eliminate.

2. Significant improvement with worst case situation. Proposed decision: control.

3. No significant improvement. Proposed decision: accept.

Given these proposed decisions, the CS model subsequently determined the predicted performance improvement if these decisions are effectuated.

Based on this proposal and the predicted outcomes, the multi-disciplinary team discussed the ranking and made a final decision. For each eliminated or controlled failure mode, representative measures were determined, and the CS model reran to assess the predicted performance after implementation of these measures. Note that this is an iterative process, as changing the decisions led to changes in the predicted performance. For example, a team may determine to eliminate a category 2 failure mode to create support for other measures, or to implement a uniform way of working across departments.

Note that the final recommendations, are therefore not necessarily identical to the assessment based on the aforementioned priority ranking. The HFMEA-CS team should use these rankings as input to the final decision-making on the outcome measures to implement in practice. In this final iterative decision-making stage, the CS model is rerun multiple times to assess the predicted performance after implementation of multiple measures, which shows the impact of combining multiple measures.

Comparison of HFMEA and HFMEA-CS recommendations

For both the HFMEA as well as the HFMEA-CS a rule of thumb for recommendations regarding acceptance, control and elimination is defined, to guide the discussion on outcome measures. To assess the impact of HFMEA-CS compared to HFMEA on the outcomes, we will compare these recommendations of both procedures. In the hazard score matrix of HFMEA, the scoring for each failure mode is divided in 0–3 (accept), 4–7 (control), and 8+ (eliminate). In HFMEA-CS, the scoring is divided based on the (in)significant difference between worst-, current-, and best-case scenarios for each failure mode. In HFMEA-CS the final decision is made based on team consensus, informed by the HFMEA and CS recommendations.

Results

Results HFMEA-CS

The multidisciplinary team identified 30 process steps, divided over the five phases, as shown in Figure 1.

69 failure modes were identified, all regarding drug administration during hospitalisation. 14 of these failure modes were scored with a hazard score ≥ 4 (see Table 2). Note that all other failure modes were checked by the multi-disciplinary team to see whether they needed to be included in the CS study, but none was selected. Furthermore, recall that only failure modes related to medication were taken into account, as this was the focus of this study.

Table 2. Failure modes and outcome measures.

Process phase	Failure mode	Hazard score (severity, likelihood, proposed recommendation)	CS performance category (proposed recommendation)	Recommendation	Outcome measure(s)
Internal medicine ward	1. The patient leaves medication at home	4 (2,2, control)	3 (eliminate)	Eliminate	The internal medicine admission planner, who calls the patients one day before their hospitalisation, tells the patient explicitly to bring their own (home) medication.
Internal medicine ward	2. The resident registers required drugs too late within the electronic patient record, resulting in drugs that are delivered too late	6 (3,2, control)	3 (eliminate)	Accept	–
Internal medicine ward	3. Medication is stopped completely instead of temporarily	6 (3,2, control)	3 (eliminate)	Accept	–
Internal medicine ward	4. Medication is incorrectly stopped	8 (4,2, eliminate)	2 (control)	Control	If the resident stops medication preoperatively, he/she has to discuss this with the endocrinologist and the patient.
Internal medicine ward	5. A drug is replaced by a similar drug	4 (2,2, control)	2 (control)	Eliminate	If the resident replaces a drug by a similar drug, he/she has to discuss this with the endocrinologist and the patient.
Internal medicine ward and surgical ward	6. There is a supply failure at the pharmacy, resulting in a lack of supply of medicines at both wards	6 (3,2, control)	2 (control)	Control	During pre-operative screening, the pharmacy assistant should go through all medication together with the patient, and check if there are any supply failures. If so, he/she advises that a patient brings a larger amount of this drug type from home.
Internal medicine ward	7. The resident incorrectly processes the medication proposal of the pharmaceutical assistant	8 (4,2, eliminate)	2 (control)	Control	The resident has to discuss the medication proposal with the endocrinologist and the patient.
Internal medicine ward and operating theatre and recovery	8. Drugs are not handed over between the internal medicine ward and the operating theatre	12 (3,4, eliminate)	1 (accept)	Eliminate	The activity plan of the nurses of the internal medicine ward states that the pheochromocytoma patients need to have medication for 24 hours when they go to the operating theatre. To check this, the nurses at the holding should only accept a pheochromocytoma patient with medication. A medicine box should be available for each patient at the recovery.
Operating theatre and recovery and surgical ward	9. Drugs are not handed over between the operating theatre and the surgical ward	12 (3,4, eliminate)	3 (eliminate)	Accept	–
Surgical ward	10. A drug is replaced by a similar drug	4 (2,2, control)	3 (eliminate)	Eliminate	If the resident replaces a drug by a similar drug, he/she has to discuss this with the endocrinologist and the patient.
Surgical ward	11. The resident registers required drugs too late within the electronic patient record, resulting in drugs that are delivered too late	6 (3,2, control)	3 (eliminate)	Accept	–
Surgical ward	12. The resident does not prescribe the required ‘stress medication scheme’	4 (2,2, control)	3 (eliminate)	Accept	–
Discharge	13. Unjustified reactivation of the medication required	4 (2,2, control)	1 (accept)	Eliminate	Before discharge, the nurse practitioner cheques the home medication together with the patient. After discharge, the nurse practitioner and resident make sure that the home medication is correctly reactivated.
Discharge	14. Unjustified reactivation of stopped medication on discharge	6 (3,2, control)	2 (control)	Control	The nurse practitioner has to check the reactivation of medication together with the patient and resident.

Based on the 14 failure modes, the KPI to be assessed in the CS model is drug adherence, and defined as: The percentage of the correct drugs that the patients receive at the correct point in time in the correct dosage out of the total amount of drugs that the patients should receive during and immediately after their hospitalisation.

The MCS of the current situation showed a drug adherence performance of 90% (95%-CI: 87–92%, n = 150). When comparing the best case scenarios of the single failure modes to the current situation, a significant drug adherence improvement is possible for two failure modes (p < 0.05), i.e. the handover of drugs from the clinical ward where the patients stayed to the operating theatre (FM8), and the wrongful prescription of home medication at discharge (FM13). In addition, for some of the failure modes there was a significant improvement of the KPI between the best case scenario and worst case scenario, or there was an improvement between the best case scenario and the current situation of the performance of a part of the care process.

Failure modes and recommendations

Based on the input of the multidisciplinary teams during the HFMEA-CS meetings and the quantitative results of the MCS, the team decided to accept five failure modes, to control four failure modes and to eliminate five failure modes, by means of the outcome measures as shown in Table 2.

It is recommended to implement the control and elimination measures and to prioritise the proper handover of drugs from the internal medicine ward to the operating theatre. Moreover, it is pointed out that the wrongful prescription of home medication at discharge is prevented by proper communication between the doctor, nurse specialist, and the patient. The patient who inspired this research, confirmed that the communication between the doctor and patient about drug administration is necessary.

Jointly simulating all recommendations gives a drug adherence performance of 99% (95%-CI: 98–99%, n = 150) which is a significant improvement of 12% on average compared to the current situation (p ≤ 0.05), as depicted in Figure 2.

Figure 2. Computer simulation based pre and post intervention patient drug adherence performance (in %) for each process step, given no backup mechanisms would be activated, related to eliminated failure modes.

A line graph plotting the drug adherence percentage for the current situation and proposed situation over the six sections of process steps. Differences in performance are related to the failure modes that are eliminated. Large performance differences are shown in the operating theatre and recovery process step, due to failure mode eight.

Comparison of recommendations derived from hazard scores and CS model

The resulting proposed and final recommendations (before and after qualitative assessment) are shown in Figure 3. Although all failure modes were scored by the multidisciplinary team, based on experience and expectations, the CS model was able to formalise the dependencies between departments and therefore the effects of failure modes across the entire care chain, as well as the interaction between failure modes.

Figure 3. Proposed and final recommendations based on hazard scores CS model outcomes and HFMEA-CS procedure.

Three matrices in which failure mode numbers are plotted, with (a) four-by-four matrix of severity and likelihood; (b) one by three matrix of Computer Simulation recommendations; and (c) one by three matrix of final recommendations of HFMEA-CS procedure.

As expected, the final recommendations, as presented in Figure 3(c), are not identical to the assessment based on the priority ranking of Figure 3(a,b). The CS model with multiple measures showed that there was no need for implementing measures for FMs 2, 3, 9, 11 and 12, despite having high hazard scores and CS scores, as these were already covered by the measures for other failure modes.

Notably, FM9, which received a high hazard and CS score, was not assessed similarly by the HFMEA-CS team. This unawareness by the HFMEA-CS team is caused by failure modes and existing working protocols earlier in the process. Based on interviews with nurses on the wards, the CS model showed that the medication is handed over between the internal medicine ward and the surgical ward, which means that despite the presence of FM9, there is no lack of medication at the surgical ward due to the current working protocols. Also, FM1 has a lower hazard score than FM2, but is eliminated, whereas FM2 is accepted. The CS model showed that when eliminating FM1, FM2 has no impact to the drug adherence process anymore. Furthermore, FM3 is accepted by the HFMEA-CS, whereas FM6, which had a similar hazard score, is controlled. The CS model showed that the worst-case scenario of FM6 had a much larger impact on the process compared to FM3, which explains the difference in outcome.

Discussion

This study proposes a combination of HFMEA and CS, as an improved prospective risk analysis approach for complex care processes in healthcare compared to the current standards.

The combination of these methods is strong, as it combines the qualitative expert opinions of a multidisciplinary team, as prevalent in the HFMEA methodology, with quantitative prospective risk assessment, using CS methods. This way, the results are more reliable, reproducible, and fitting within a healthcare setting.

In the case study, the HFMEA-CS method significantly improved the drug adherence performance during hospitalisation of pheochromocytoma patients from 90% to 99%. The HFMEA multi-disciplinary meetings resulted in involved team members who were motivated to change their way of working to improve the drug adherence process. The CS results enabled the multi-disciplinary team to prioritise the failure modes and to decide which failure modes to eliminate, to control and to accept.

HFMEA-CS increases the reliability of the outcomes comparing to the outcomes of a regular HFMEA in three ways:

1. Through CS, the impact of a failure mode on the entire process is considered, such that it distinguishes between two failure modes which might be scored similarly in HFMEA, but that have a different impact on the entire process.

2. The CS model gives insights in the impact of mitigating a certain failure mode, on the probability and impact of the other failure modes, showing the effects of failure modes on each other. This relation between failure modes and measures, is especially relevant in the assessment of complex care paths covering multiple departments. As an example from this pheochromocytoma case study, it was shown that when patients would always bring their medication from home, mitigating FM1, the impact of a delayed prescription (FM2) on the drug adherence was non-existing, which made us accept this failure mode. HFMEA would not directly identify this connection.

3. HFMEA only prioritises failure modes on a highly aggregated level, based on a 1–4 numerical scale. However, the CS model requires the likelihood of failure modes as a percentage, such that the relevant KPIs can be calculated. This requires the research input to be more detailed, either through data analysis or based on in-depth interviews, and results in more detailed outcomes, as well as a priority scaling.

When combining HFMEA and CS, multiple elements of HFMEA are used as input for the CS model. However, some of these elements are non-mandatory elements of the HFMEA methodology, such as the design of a visualised flowchart in Step 3. However, for HFMEA-CS, this element is required. In Step 3, the flowchart should be agreed upon by the entire HFMEA-CS team, such that the CS model is built correctly in Step 5, based on this process flow.

In the proposed HFMEA-CS approach, failure modes with a hazard score ≥ 4 are automatically included in the CS model. However, this does not mean that failure modes with a lower score should be discarded. All failure modes should be checked on their relevance by the team, and in case of doubt, the failure mode should be included in the simulation. This gives the quantitative assurance that is missing based on qualitative assessment.

In this study, based on the input of the medical experts, it is assumed that every failure mode can be entirely solved. Therefore, all failure modes are evaluated in a best-case scenario, using a one-factor-at-a-time simulation experiment design. This allows for benchmarking failure modes against each other. IThe worst-case scenarios are also evaluated, to anticipate on eliminating or controlling failure modes given all precautionary measures are failing. This is especially relevant in rare disease settings due to the low incidence of events. Finally, when assessing the impact of the outcome measures, the relation between failure modes and elimination measures is considered, showing the impact and dependencies of failure modes. In this final decision-making stage, HFMEA-CS is effective in assessing the predicted outcomes of control and eliminating measures, increasing their evidence before actual implementation.

It is important to realise that the CS model outcomes are always meant as decision support, not as decisions itself. The CS model gives a proposed ranking of failure modes, but the HFMEA-CS team should use this ranking as input to the final decision making on the outcome measures to implement in practice. Based on qualitative and subjective conditions that were not included in the CS model, the team might decide to adopt another strategy. As an example, the HFMEA-CS team in the pheochromocytoma case study decided to implement the same measure for FM5 and FM10, to create a uniform way of working across wards, whereas the HFMEA-CS model only recommended to eliminate the FM10 measure. The length of stay at the surgical ward is shorter, which explains the difference in proposed recommendations. However, patients and nursing staff experience the effects similarly on both wards, which argues for implementing the same standards.

A well-known drawback of HFMEA is the high time-intensive schedule for all involved participants (see e.g. Shaqdan et al. 2014; Habraken et al. 2009), which even increased with the CS extension. For these reasons, targeted interviews were held with individual members in the CS phases by the project manager and simulation expert to derive the required quantitative input. An additional advantage is that during individual interviews, participants might open up more easily about possible failures in their working processes. However, it is preferred that all information is gathered in multi-disciplinary meetings with all team members present at all meetings, and therefore does require a joint meeting in which all team members verify the outcomes of these individual interviews.

Next to the time-intensity of the approach, the HFMEA-CS also requires substantial knowledge on computational modelling by at least one of the team members. Although we see an increasing amount of healthcare organisations hiring people with a data-analytics background that would be able to support the HFMEA-CS team with this type of knowledge (and expect this will become the norm in the near future), this knowledge might be not readily available to all organisations.

A second drawback of the HFMEA methodology is the definition of the likelihood parameter, which is typically not scaled to the patient population. The scaling of likelihood, also referred to in the literature as frequency or probability, is often formulated as experienced occurrence (e.g. frequent, occasional, uncommon, remote) (Abrahamsen, Abrahamsen, and Høyland 2016; Habraken et al. 2009), or in terms of a certain number of events per unit of time (e.g. daily, weekly, monthly, yearly, bi-yearly) (Moyer et al. 2010; DeRosier et al. 2002). However, especially when considering complex care paths for rare diseases, even frequent failure modes typically only occur on a rare basis, as only a limited amount of patients are seen per year. In order to overcome this issue, especially in the context of rare diseases with small patient populations, the number of occurrences per patient is proposed as the likelihood scale, in order to be able to differentiate between frequent and remote occurrence of a failure mode.

Besides incorporating the likelihood in the hazard score based on the HFMEA-CS team ratings, the occurrence is also input to the simulation study, if possible based on data derived from the hospital information system. In our study, these numbers were in line with each other. However, the experienced and measured occurrences may vary for specific failure modes. In this case, we recommend to discuss the discrepancy in the multi-disciplinary team, and to perform a sensitivity analysis with the variety of occurrence input data in the CS model to assess the impact of these failure modes’ occurrences.

Leadership support is important for the success of HFMEA-CS, both for enabling resources to take part in the time-consuming process, as well as for implementing the outcome measures. Currently, the first measures are implemented, such as the handover of medication between departments (measure 8), and including medication questions in the discharge process (measure 13). Additional longitudinal research has to show whether the expected performance increase of all proposed measures has indeed occurred.

Next to the (expected) improvements based on the proposed recommendations from the HFMEA-CS results, the HFMEA-CS meetings with the multidisciplinary team also created awareness for this very specific patient group amongst the members of the multidisciplinary team. This is already expected to result in a more careful administration of drugs of this patient group.

In future research, a comparative study should be performed to see the differences in outcomes between traditional HFMEA and HFMEA-CS, to further strengthen the evidence on this approach. In this study, it is important to consider cases with more, and more reliable data.

Healthcare can greatly benefit from Human Factors and Ergonomics evaluations, for example in risk assessment, prioritisation, and mitigation – not only for the use of medical devices, but increasingly also for care processes, as human behaviour and the design of the system play a key role in patient safety. A deeper understanding of this complex environment can not only improve patient safety, but also be generalised to address safety in other contexts. Especially the interplay between qualitative and quantitative research methodologies (mixed-methods approach) is of key importance in such complex environments, as shown in our study. This interplay preferably requires integration of these methods, instead of being successively applied. The qualitative research results are input to quantitative methods, quantitative methods are enhanced with stakeholder and contextual information, and is followed by qualitative research to understand and interpret these results.

In conclusion, this study illustrates the value of combining the HFMEA procedure with CS, which increases the validity and reliability of the risk assessment outcomes. The implementation of this new HFMEA-CS approach in a rare disease setting, led to a reduction in errors related to drug delivery and system errors, and increased drug adherence performance, ultimately enhancing patient safety.

Abbreviations
CI	=	confidence interval
CS	=	computer simulation
DES	=	discrete event simulation
FM	=	failure mode
FMEA	=	failure mode and effects analysis
HFMEA	=	healthcare failure mode and effects analysis
KPI	=	key performance indicator
MCS	=	monte carlo simulation
RPN	=	risk priority number
UMC	=	Utrecht University Medical Center Utrecht

Acknowledgements

The authors thank all members of the HFMEA-CS team, in particular the pheochromocytoma patient that inspired this research project by pointing us towards opportunities for improvement.

Disclosure statement

No potential conflict of interest was reported by the author(s).