A national needs assessment study to determine procedures for simulation-based training in cardiology in Denmark

Abstract

Objectives. New training methods such as simulation have been introduced in cardiology as in other specialties; however, the development of effective simulation-based training programs is challenging. They are often unstructured and based on convenience or coincidence. The objective of this study was to perform a nationwide general needs assessment to identify and prioritize technical procedures that should be included in a simulation-based curriculum for cardiology residency in Denmark. Design. We completed a needs assessment using the Delphi method among key opinion leaders in cardiology. Brainstorming in round 1 identified technical procedures that future cardiologists should learn. Round 2 was a survey to examine frequency of procedure, number of cardiologists performing the procedure, operator-related risk and/or discomfort for patients and feasibility for simulation. Round 3 was final elimination and prioritization of procedures. Results. Ninety-four key opinion leaders were included, and the response rates were 77% (round 1), 62% (Round 2), and 68% (Round 3). Twenty-four technical procedures were identified in Round 1 and pre-prioritized in Round 2. In round 3, 13 procedures were included in the final prioritized list. The five highly prioritized procedures eligible for simulation-based training were advanced life support, pleurocentesis, transesophageal echocardiography, coronary angiography, and pericardiocentesis. Conclusion. The general needs assessment following the Delphi process identified and prioritized 13 technical procedures in cardiology that should be integrated in a simulation-based curriculum. The final list provides educators a guide when developing simulation-based training programmes for cardiology residents.

Keywords:

• simulation-based training
• needs assessment
• curriculum
• Delphi technique
• cardiology
• simulators
• training in residency

Introduction

Cardiology has evolved dramatically in the last three decades with the advent of advanced imaging procedures and highly technical interventional procedures mandating a high level of proficiency. The current training approach to achieve proficiency still follows the traditional apprenticeship model where trainees learn under supervision of an expert practitioner. However, this approach has several challenges and limitations, e.g. limited training time due to work-hour restrictions, scarcity of expert faculty for supervision, and patient safety concerns when patients have to bear the burden of new trainees’ learning curve [1]. Hands-on training on mannequins, animals, cadavers, and high-fidelity virtual-reality simulators offer efficient and patient-safe training opportunities for future cardiologists [2]. Simulation-based training in (SBT) advanced life support is already fully integrated in the European and American guidelines for resuscitation training and many simulation-based methods for training of diagnostic and management strategies in cardiology have been developed [3,4]. It is believed that it is possible to reduce procedural risk associated with novice operators by first demonstrating competency on simulators [5]. Furthermore, there is growing evidence that competency acquired in a simulation-based environment can be directly transferred to better patient outcome [6].

Unfortunately, the development of simulation-based curricula has not followed a systematic approach and many training programs are based on availability of commercial simulators or local interests [7,8]. A systematic approach should be followed to ensure that formal training programs are established and are effectively integrated into the curriculum. Kern et al. suggested that the development of curriculum should start with problem identification and general needs assessment [9]. The objective of this study was to perform a national needs assessment to identify and prioritize technical procedures in cardiology that are relevant and suitable to integrate in a simulation-based curriculum.

Methods

The study was conducted November 2016 - May 2017 and consisted of a three-round iterative Delphi process to establish consensus and prioritize technical procedures for SBT (Figure 1). The Delphi method is widely accepted as a valid tool to obtain expert opinion, elicit consensus, and support decision-making [10]. A research steering group was formed to manage all processes throughout the Delphi rounds including data collection and analyses. This consisted of four members: two consultant cardiologists (PG, DGN), a professor of medical education and expert in SBT (LK), and a nurse and simulation research associate (LJN).

Figure 1. Illustration of the Delphi process.

Figure 2. High correlation between the ranking order in round 2 and the ranking order in the final round.

Participants

The key opinion leaders were chosen for their important roles in the training and education of cardiology trainees across Denmark. They were identified by contacting the main course leader of the Danish residency program in cardiology. The board members of the Danish Society of Cardiology and the Association of Junior Cardiologists were identified through their respective websites. A total of 94 key opinion leaders with a variety of background (Table 1) were invited and introduced to the study design through e-mail. Some of the participants belonged to two or three groups as presented in Table 1. All e-mails in the study were sent individually to each participant to avoid the excessive influence from dominant individuals [11]. Reminders were sent twice in round 1 and round 3, but three times in round 3. Participation was voluntary.

Table 1. Key opinion leaders in cardiology in this study.

Role	N
Heads of clinical education and training	26
Heads of departments	24
Course directors for Residency Programs in Denmark	11
Professors	19
Board members of the Danish Society of Cardiology	10
Board members of the Association of Junior Cardiologists	8
Members of the Educational Council: Danish Society of cardiology	7
Postgraduate Clinical associates	2

Several assume more than one position.

Round 1

Round 1 was brainstorming where the participants were asked to suggest all technical procedures that a new cardiology specialist should be proficient in. Technical procedures were defined as hands-on and/or applied skills. All suggested procedures were summarised by the steering group and non-technical procedures were removed, e.g. Electrocardiogram (ECG) interpretation and managing hemodynamic problems in the intensive care patient. The remaining procedures were grouped into categories that were used in Round 2.

Round 2

We used a previously developed needs assessment formula (NAF) from the Copenhagen Academy for Medical Education and Simulation (CAMES) to explore and pre-prioritize the included procedures from Round 1 [8,12]. The formula includes four different factors: 1) Number of procedures performed annually, 2) Number of doctors that should be able to perform the procedure, 3) Discomfort/risk if the procedure is performed by an inexperienced doctor, and 4) Feasibility of learning the procedure in a simulation-based environment.

The first three factors were answered by all key opinion leaders in a web-based survey. The feasibility SBT was determined by the steering group considering suitability, associated cost, and availability of equipment. A five-point Likert Scale was used by the steering group to categorize their subjective opinions for suitability (1 = safely learned in a clinical environment – 5 = should be practiced in a simulation-based setting), availability (1 = no known simulation equipment for training the proceedure –5 = effective and realistic equipment readily available), and costs (1 = very expensive to practice the procedure in a simulation-based environment – 5 = very cheap to practice the procedure in a simulation-based environment), respectively.

All procedures were prioritized based on the four factors in the NAF. This ensured that commonly performed procedures that all cardiologists should be able to master would receive high priority. Procedures that can be safely taught on patients and/or are difficult to teach in a simulated environment would receive less priority

Round 3

Round 3 was the final step of the Delphi process where the key opinion leaders had the opportunity to suggest removal of procedures that they did not find applicable for SBT and change the prioritization of the remaining procedures.

Data analysis and statistics

Qualitative analysis using a summative approach was used in round 1 to eliminate non-technical procedures and group similar procedures into categories. In round 2, the mean score for each question was calculated and the procedures were ranked from highest to lowest priority based on the CAMES NAF score. In round 3, we calculated the number of occurrences for each response using frequency analysis and applied the principle of 2/3 qualified majority to establish consensus. Procedures were eliminated if less than two thirds of the key opinion leaders found them relevant for SBT. The mean scores of the remaining procedures were calculated to produce a prioritized list. Spearman’s Rho correlation coefficient was calculated to examine any changes between the order of prioritization in rounds 2 and 3. IBM SPSS (Statistical Package for Social Sciences Version 22, Chicago, Illinois) was used for all statistical calculations.

Results

Round 1

The response rate was 77% (72/94 participants) and 62 clinical procedures were identified. These were reduced to 24 technical procedures after a summative analysis.

Round 2

Round 2 had a response rate of 62% (58/94 participants). Peripheral venous catheter and advanced life support came up with the highest priority while tilt table testing, and massage of the carotids had the lowest priority (Table 2).

Table 2. Preliminary prioritization of procedures after round 2 including mean scores.

	Procedure	No. of procedures performed (Frequency)	No. of Doctors	Impact	Feasibility	CAMES NAF Score
1	Peripheral venous cannulation and arterial puncture	4.7	4.7	3.4	4.6	4.4
2	Advanced heart lung resuscitation	3.0	4.9	4.2	4.6	4.2
3	Auscultation (heart and lung)	5.0	5.0	2.5	3.3	3.9
4	Transoesophageal echocardiography	4.2	3.3	4.0	4.0	3.9
5	Direct current (DC) conversion	3.9	4.6	4.4	2.6	3.9
6	Coronary angiography	4.7	1.8	4.8	4.0	3.7
7	Electrocardiogram (ECG) performance	5	4.7	2.6	3.0	3.8
8	Transthoracic echocardiography	5.0	4.3	2.56	3.3	3.8
9	Ultrasound-guided pleurocentesis	2.7	3.3	4.4	4.7	3.8
10	Percutaneous transluminal coronary angioplasty	4.8	1.1	5.0	4.0	3.7
11	Test and troubleshooting of PM and ICD	4.5	2.7	3.9	3.8	3.7
12	Central venous catheter insertion	3.2	1.7	4.5	4.7	3.5
13	Implantation of permanent pacemaker	4.3	1.2	4.8	3.7	3.5
14	Lumbar puncture	1.8	3.2	4.2	4.7	3.5
15	Pericardiocentesis	2.3	2.0	4.8	4.7	3.4
16	Electrophysiology test	3.6	1.0	4.8	4.0	3.4
17	Exercise testing	3.5	3.8	3.3	2.7	3.3
18	Insertion of temporary pacemaker catheter	2.6	1.9	4.7	4.0	3.3
19	Myocardial biopsy performance	2.7	1.0	5.0	4	3.2
20	Transcutaneous pacing	1.9	4.3	4.1	2.3	3.2
21	Right heart catheterisation	2.7	1.1	4.8	4.0	3.1
22	Ascites puncture	1.6	2.3	3.9	4.7	3.1
23	Carotid massage	2.5	4.0	3.3	1.3	2.8
24	Tilt table testing	2.5	2	3.4	2.7	2.6

ECG: Electrocardiogram; DC: Direct Current; PM: Pacemaker; ICD: Implantable Cardioverter defibrillator.

CAMES NAF = Copenhagen Academy for Medical Education and Simulation (CAMES) Needs Assessment Formula (NAF).

No. of procedures = Defined as how frequent the procedure was estimated to be performed in the department per year.

No. of Doctors = Refers to the approximate number of doctors in the department that are able to perform the procedure.

Impact = Defined as the risk or discomfort to patients when performed by an inexperienced doctor.

Feasibility = Refers to the feasibility of simulation-based training of a procedure in a simulation-based environment.

Round 3

The response rate in round 3 was 68% (64/94 participants). Eleven procedures were eliminated, resulting in a final prioritized list of 13 procedures (Table 3). The list included diagnostic procedures (n = 5), interventional/treatment procedures (n = 4), and combined procedures (n = 4). Seven procedures were invasive, one procedure semi-invasive, and five procedures non-invasive. Advanced Life Support, pleurocentesis, transesophageal echocardiogram (TEE), coronary angiography (CA), and pericardiocentesis gained the highest priority for SBT while implantation of permanent pacemaker and right heart catheterization were included and ranked with lowest priority.

Table 3. Final prioritized list of technical procedures to include in a simulation-based curriculum in cardiology (after round 3).

Table 3. Prioritized list of technical procedures to include in a simulation-based curriculum in cardiology.

1	Advanced Heart Lung Resuscitation
2	Transesophageal echocardiography
3	Ultrasound guided pleurocentesis
4	Coronary angiography
5	Pericardiocentesis
6	Test and troubleshooting of PM and ICD
7	Transcutaneous pacing
8	Transthoracic echocardiography
9	Implantation of a temporary pacemaker
10	Lumbar puncture
11	Central venous catheter
12	Permanent pacemaker
13	Right heart catheterization

PM: Pacemaker; ICD: Implantable Cardioverter defibrillator.

A Spearman’s rho of r_est = 0.71 showed that there was a strong correlation between the order of prioritization of procedures after calculation of the CAMES NAF score in round 2 and the order of prioritization according to the participants in round 3 (Figure 2).

Discussion

We used a three-round Delphi process to identify and prioritize the need for SBT by including key opinion leaders in cardiology. The result of this process was a prioritized list of 13 technical procedures which should be included in a SBT curriculum for residents in cardiology.

We had a relatively high response rate in all three rounds, indicating strong support from the key opinion leaders and adding credibility to the results. While the aim of the study was to identify technical procedures for SBT, using the CAMES-NAF helped the key opinion leaders and the steering committee to prioritize the procedures based on predefined factors. This is supported by the high correlation between the preliminary prioritization from round 2 and the prioritization by the participants in round 3. The same strong correlation between the CAMES-NAF and priority ranking was found in needs assessment studies performed in other specialties using the same methodology [8,13].

Advanced life support (ALS) topped the prioritized list showing that resuscitation is regarded as a fundamental competence to be achieved in the cardiology specialty. ALS comprises several hands-on procedures in the form of CPR, airway management, defibrillation and pacing, and arterial and venous puncture. SBT is an integral part of resuscitation advanced level training [3,4]. A systematic review and meta-analysis of 182 studies involving 16,636 participants on SBT for resuscitation showed improvement in knowledge and skill performance compared to training without simulation [14].

TEE came in second on the prioritized list, while transthoracic echocardiography (TTE) was number eight. Echocardiography is highly operator dependent, which implies that the utility of the examination is reliant on the skill of the operator in obtaining adequate views. Advanced simulation-based models for training TEE and TTE are available (i.e. U/S Mentor, 3D Systems, California USA; Vimedex Ultrasound Simulator, CAE Healthcare, Canada)

TEE is a semi-invasive procedure; therefore, a small risk of harm to the patient and some discomfort in the awake patient should be considered. TEE is also a complex task reliant on 3D spatial awareness of cardiac anatomy and dexterity to manipulate the transducer probe. Existing evidence on the effect of SBT is based on studies from either the cardiology specialty or the anesthesia specialty. It is assumed that overall the results apply to trainees in both specialties [15]. Existing evidence is restricted to single-group analysis on pre and post simulation training [16,17] or improvement in simulation procedural ability through SBT [18,19]. Importantly however, two studies provided evidence that SBT can improve ability when carrying out the procedure in real patients. In one study, second-year cardiology residents were divided into three groups; the first group (A) completed the standard TEE training with no SBT training, one group (B) received SBT alongside standard training during the first month of apprenticeship while the other group (C) received SBT in the second month of apprenticeship. Ability was assessed in terms of proficiency in obtaining appropriate TEE views on a real patient after 1 month. At 1-month, fellows in the both SBT groups had significantly higher number of views achieved without assistance, while group B achieved higher total scores and number of views without assistance compared with group C [20]. In a study by Fererro et al. [21] a total of 42 clinical anesthesia residents were randomized into two groups: a control group receiving traditional didactic training, and a simulator group training on a transesophageal echocardiography–mannequin simulator. The simulator group obtained significantly higher-quality images on real patients compared to the control group. Seventy one percent of the acquired images taken by the simulator group were acceptable for clinical use as compared to 48% taken by the control group.

A TTE, as opposed to TEE, is performed without any risk of harm to the patient which might explain why it is lower on the prioritized list. However, prolonged training can be uncomfortable to the patient. Basic training on a simulator is obviously a less stressful learning environment and gives the opportunity to get familiar with standard views and both frequent and rare pathologies before performing TTE on patients. Furthermore, TTE is a basic competence that every resident in cardiology should master which is probably the reason why this procedure is still in the prioritized list though not highly listed. The role of TTE simulation in training residents and sonographers has been explored in several studies. A randomized prospective trial including 61 residents in anesthesiology demonstrated that residents trained with simulation acquired better skills in TTE image acquisition and anatomy identification on volunteer subjects compared to residents trained with traditional teaching methods [22]. Platts et al. [23] conducted a study on the feasibility of utilizing a TTE simulator for training sonographers. All the participants found that the TTE simulator was useful in obtaining the superior vena cava view, whereas 88% of trainees found it useful in acquisition of the apical 2 chamber view. A randomized study of 46 anesthesiology trainees and faculty members demonstrated that a high-fidelity simulator is not inferior to the use of a human model based on practical and written examinations [24]. Bowcock et al. [25] included twenty-seven trainees and eight experts and demonstrated that the use of echocardiography simulation to determine competence in basic critical care echocardiography is feasible and concluded that competence in image acquisition appears to be achieved with less experience compared with correct interpretation and management decisions . All the referred studies were performed in other specialties; however, the results are believed to be applicable to cardiology trainees too.

CA came in fourth on the prioritized list as well as in the preliminary list. This procedure poses a risk of serious complications and is feasible to learn in a simulated environment using existing, however expensive virtual-reality simulators. Bagai et al. [26] randomized 27 cardiology trainees to either a short 2–4 hours simulation-based training programme in addition to 1-week standard apprenticeship training or 1-week apprenticeship training alone. On repeat assessment of ability on real patients, the simulation-based group had significantly improved technical ability compared to the control group. This is the first study that documented the addition of simulation-based training to conventional training, showing an improvement in coronary catheterization skill. Jensen et al. [27] randomized sixteen senior cardiology residents to proficiency-based SBT or control to explore if the skills achieved from proficiency-based training in CA can be transferred to the clinical setting. In the study, two consecutive CAs were performed on patients. The performances of the two groups were compared and the SBT group showed superior performance with shorter fluoroscopy and total procedure times than the control group in real life. The study documents transfer from SBT to real life performance in coronary angiography, which is considered the highest level of evidence in interventional educational research [27].

Pericardiocentesis was the fifth procedure on the final prioritized list. While periocardiocentesis is an invasive procedure with some risk of serious complications, commercially available equipment for SBT in echo-guided pericardiocentesis is sparse especially if demanding apical and parasternal approach and a beating heart, which is necessary for optimal training of cardiology residents. There are a few phantoms that are available for ultrasound-guided emergency periocardiocentesis. However, to our knowledge, there are no studies published on the effect of SBT in pericardiocentesis for cardiologists.

For ultrasound guided pleurocentesis, the order of ranking changed notably from round 2 to round 3. It ranked lower in round 2 because it is primarily performed by other specialties such as radiologists and pulmonologists. However, the key opinion leaders recognized the importance to learn ultrasound-guided pleurocentesis as a basic competence that residents in cardiology should be able to perform. A study has investigated the validity of using a phantom to perform pleural drainage using the Seldinger technique. A scoring system was used to evaluate the performance of the participants and was found to be reliable in the assessment of procedural competence [28].

Several pacemaker related procedures are on the prioritized list both invasive such as insertion of a temporary pacemaker and non-invasive such as test and troubleshooting of pacemakers and intracardiac defibrillators. Simulation-based equipment is commercially available to practice these skills and our key opinion leaders found simulation training relevant in cardiology residency, however this field is still awaiting the first studies.

The results of this study can be compared to the current European Society of Cardiology (ESC) core curriculum for the general cardiologist in Europe (2013) [29]. Of the 13 procedures in our final list, 8 are represented in the competence list in the ESC core curriculum of skills relating to investigations, as well as procedures with competence level II or level III. While the majority of the procedures were represented, two procedures (ultrasound guided pleurocentesis and lumbar puncture) were not included in the ESC core curriculum since these procedures are not specific to cardiology, but part of the basic common trunk of internal medicine. The key opinion leaders in our study found that these common trunk procedures are important competences for a Danish cardiologist to master. The top prioritized procedure in our final list is advanced heart lung resuscitation. This comprises a variety of basic skills competences which are also represented in the ESC curriculum such as arterial and venous puncture. Interestingly, CA is in top five in our final list as well as a recommendation with competence level III in the ESC core curriculum, however it is not included in the new Danish core curriculum (2017) for the general Danish cardiologist [30]. In both the Swedish and Norwegian core curricula [31,32], CA is included as a procedure that a cardiologist resident should be able to perform under supervision. These examples stress the importance of performing a national needs assessment when establishing SBT programs to ensure alignment to current training needs.

The strength of this study is its multicenter design wherein key opinion leaders from all cardiology departments across the five regions of Denmark were included. However, one limitation of the study is the three-round survey design, which is time consuming and consequently can result in low response rates, especially round 2 and 3. The response rates in this study varied in the three rounds, with the lowest response rate in round 2. It is a characteristic of the Delphi method that the number of responses decreases with every round [33]. Reminder emails were sent to encourage the participants to respond and to be active in the process. The assessment of feasibility by the steering committee in round 2 is also a limitation. However, feasibility only accounts for 25% in the round 2, while round 3 give the key opinion leaders the possibility to re-prioritize procedures and eliminate the procedures according to their own judgment. Another inherent limitation is that this study was performed at a national level, therefore reflecting local medical practices and traditions. Some of the identified procedures in the list may not be applicable to other countries since core curriculums differ across countries [29]. It is important that local centers develop SBT programs for the prioritized procedures that conform to local standards and training practices. Finally, it is important to stress that only technical procedures for simulation were identified in our study. However, other factors that are important to train such as communication and other team-based training skills can be incorporated when developing simulation-based programs on the identified procedures.

Conclusion

This needs assessment following the Delphi method has identified and prioritized a list of 13 technical procedures that are relevant for simulation in cardiology. Program directors may refer to the list as a guide in the development of simulation-based training programs for cardiology residents.

Disclosure statement

No potential conflict of interest was reported by the authors.