Cardiac Anesthesia Intraoperative Interpretation Accuracy of Transesophageal Echocardiograms: A Review of the Current Literature and Meta-Analysis

Abstract

Background

In the United States, echocardiography is an essential component of the care of many cardiac patients. Recently, increased attention has been given to the accuracy of interpretation of cardiac-based procedures in different specialties, amongst them the field of cardiac anesthesiology and primary echocardiographers for transesophageal echocardiogram (TEE). The purpose of this study was to assess the TEE skills of cardiac anesthesiologists in comparison to primary echocardiographers, either radiologists or cardiologists. In this systematic review, we evaluated available current literature to identify if cardiac anesthesiologists interpret TEE procedures at an identical level to that of primary echocardiographers.

Methods

A PRISMA systematic review was utilized from PubMed from the years 1952–2022. A broad keyword search of “Cardiology Anesthesiology Echocardiogram” and “Echocardiography Anesthesiology” to identify the literature was used. From reviewing 1798 articles, there were a total of 9 studies included in our systematic review, 3 of which yielded quantitative data and 6 of which yielded qualitative data. The mean accuracy from each of these three qualitative studies was calculated and used to represent the overall accuracy of cardiac anesthesiologists.

Results

Through identified studies, a total of 8197 TEEs were interpreted by cardiac anesthesiologists with a concordance rate of 84% to the interpretations of primary echocardiographers. Cardiac anesthesiologists had a concordance rate of 83% when compared to radiologists. On the other hand, cardiac anesthesiologists and cardiologists had a concordance rate of 87% in one study and 79% in another study.

Conclusion

Based on these studies, cardiac anesthesiologists are shown to interpret TEEs similarly to that of primary echocardiographers. At this time, there is no gold standard to evaluate the accuracy of TEE readings. One way to address this is to individually assess the TEE interpretation of anesthesiologists and primary echocardiographers with a double-blind study.

Keywords:

• echocardiography
• transesophageal echocardiogram
• cardiac anesthesia

Introduction

In the recent literature, cardiology based training in different procedures and techniques has been garnering a lot of attention.^1–3 As of 2019 over 90,000 physicians specialize in cardiac-based procedures and interpretation in the United States. One such procedure is echocardiography, and physicians who specialize in interpreting them are called primary echocardiographers, which include cardiologists and radiologists.⁴ 22,521 active physicians practice in the field of cardiology and 28,025 active physicians practicing in radiology.⁵ In comparison, there are only 1667 anesthesiologists who practice cardiac anesthesia as a subspecialty.⁶ Subspecialization in cardiac anesthesiology requires at least 4 years of training in an anesthesiology residency program and at least 1 year of a cardiac anesthesiology fellowship.^7–16

During their residency and fellowship years, most anesthesiologists will be trained in the use of echocardiography. One such type of echocardiography is transthoracic echocardiogram (TTE), in which a handheld transducer is held outside the heart.^17,18 Although other forms of echocardiograms exist, such as intracardiac echo and stress echo, a transesophageal echocardiogram is often the approach in the perioperative setting. Compared to TTE, a transesophageal echocardiogram can be more sensitive at identifying etiologies of an embolic stroke.^19–21 One study suggests that TEE may be more suitable than TTE for detecting infective endocarditis.²² Transesophageal echocardiograms can assess the heart’s function and detect symptoms of atherosclerosis, cardiomyopathy, heart failure, and more.^23,24 This is because an ultrasound probe is guided into the esophagus, providing a closer view of the heart.^25–30 Interpreting TEEs have a significant impact throughout perioperative care in order to make a proper diagnosis.³¹ Although cardiac anesthesiologists, cardiologists, and radiologists are all trained in interpreting transesophageal echocardiography, an overwhelming majority of perioperative TEEs are performed by cardiac anesthesiologists. A study by Poterack recognized that out of 98 institutions surveyed, 54% of them have anesthesiologists in charge of TEE interpretations.³² Therefore, it is of utmost importance that cardiac anesthesiologists are well-trained in these procedures.

TEE specifically has seen major growth in terms of technology, use, and indications since its introduction to the medical community nearly half a decade ago.³³ These advancements include the increase in TEE use from 29% in 2009 to 45% in 2011, and upgrades in technology such as the 3-D TEE systems.³⁴ 3-D TEE imaging has been shown to improve detecting infective endocarditis in a study by Chahine et al.³⁵ Additional advancements include continuous TEE monitoring, strain imaging, and diastolic function assessment.³⁶ These advancements have also increased the complexity of the procedure itself. For this reason, diagnostic evaluation of the TEE exams may vary disparately depending on who delivers the procedure and the expertise of the examiner.³⁷ Despite the active role that cardiac anesthesiologists have in the perioperative setting, there is limited literature on the assessment of their ability to interpret intraoperative TEE. Our paper conducts a systematic literature review to assess the effectiveness with which cardiac anesthesiologists interpret TEE examinations compared to primary echocardiographers, such as cardiologists and radiologists.

Methods

The PRISMA systematic review model was used to execute this study and identify relevant literature.³⁸ A comprehensive search was used on the MED-LINE database (PubMed) to yield articles used for our study. Step 1 included using a broad keyword search using the phrases “Cardiology Anesthesiology Echocardiogram” and “Echocardiography Anesthesiology” to produce 1114 and 684 articles, respectively, dating from 1952 to 2022. The criteria for inclusion and exclusion are shown below in Figure 1 including but not limited to articles being written in the English language.

Figure 1 Study flow chart.

From the search, a total of 363 articles were included based on the relevance of the title (Figure 1, step 1), and duplicates were then removed (Figure 1, step 2). The remaining articles were then screened based on their abstract (Figure 1, step 3). The last step executed was reading the full article to determine which publications will be used in the study (Figure 1, step 4). This process yielded a combination of quantitative and qualitative information that amounted to a total of 9 relevant articles for our topic of interest. Three researchers carried out the procedures to obtain the final sample. The investigation team agreed on the final selection of the literature (Table 1).

Table 1 Publications Included in the Systematic Review

Authors	Year	Objective	Methods
Bick et al^Citation45	2013	To compare expert and novice performance of simulated TEE exams	TEE exams evaluated by blinded raters using a standardized assessment
Hahn et al^Citation39	2013	To outline current guidelines for TEE examinations	Outline general guidelines, and guidelines for comprehensive TEE exams and imaging specific cardiac structures
Konstadt et al^Citation47	1996	To measure physician competence in TEE	Written exam was administered to both anesthesia faculty and residents
Mathew et al^Citation4	2002	To evaluate agreement of TEE amongst cardiac anesthesiologists and primary echocardiographers. Measurements included left ventricular area change. Cardiac anesthesiologists interpreted TEE intraoperatively.	CQI program evaluated TEE examinations
Miller et al^Citation43	2001	To evaluate intraoperative accuracy of cardiac anesthesiologists after revision of TEE exam. The study examined size of the heart chambers, FAC, and degree of stenosis or insufficiency of heart valves.	Interpretations were compared to evaluation by cardiology expert review
Mishra et al^Citation41	1998	To assess safety and reliability of TEE during cardiac surgery. The study examined left ventricle regional wall motion, valve function, and ventricle function. Cardiac anesthesiologists interpreted TEE intraoperatively.	Comparison of interpretation by anesthesiologist and analysis by cardiologist
Rafferty et al^Citation40	1993	To evaluate 846 TEE procedures	Two software packages used to evaluate TEE
Sawasdiwipachai et al^Citation46	2022	To determine success rate in first 20 intraoperative TEEs in anesthesiologist trainees	TEE studies performed by trainees were reviewed by certified reviewers
Sloth et al^Citation44	2001	To determine whether TEE is beneficial during pediatric cardiac surgery	Intraoperative TEE was performed on 532 consecutive children

After assembly of the 9 articles, they were divided according to whether they contained quantitative or qualitative data. There were three quantitative data containing the accuracy of cardiac anesthesiologists’ TEE readings. Accuracy is defined as the degree to which cardiac anesthesiologists’ TEE interpretation agreed with that of primary echocardiographers. The quantitative studies examined different parameters as part of the TEE procedure and also used different methods to assess accuracy. Cohen’s kappa coefficient and high-fidelity videotape evaluation were the methods of analysis used to evaluate the accuracy of the interpretation of these parameters. The number of correctly interpreted TEEs and the total number of TEEs were obtained from each of the three quantitative studies. These numbers were then used to calculate the mean accuracy in the interpretation of all TEEs to represent the overall accuracy of cardiac anesthesiologists (Table 2).

Table 2 Results of Transesophageal Echocardiogram Results in Systematic Review

Authors	N-Cardiac Anesthesiologists	A-Number of TEEs Interpreted	% Correctly Interpreted	# of Correct TEEs	Total TEEs	Total Average % Correct
Mathew et al^Citation4	10	154	83%	2045	2464
Miller et al^Citation43	8	131	79%	1242	1572
Mishra et al^Citation41	N/A*	3846	87%	3620	4161
				6907	8197	84.26%

Note: *Number of cardiac anesthesiologists was not mentioned.

Results

PRISMA systematic review yielded a total of 3 quantitative studies and 6 qualitative studies for a total of 9 relevant studies. The three quantitative studies contained comparisons between cardiac anesthesiologists and radiologists, cardiac anesthesiologists and cardiologists, and cardiologists and radiologists.

Mathew et al contained the concordance rate of TEE interpretations amongst cardiac anesthesiologists, cardiologists, and radiologists. In the study, radiologists interpreted the same number of TEEs as cardiac anesthesiologists. For this reason, we decided to compare anesthesiologists to radiologists in this study. They found that anesthesiologists with less than 5 years of experience underestimated left ventricular fractional area change (FAC). On the other hand, anesthesiologists with greater experience had higher levels of concordance with radiologists, particularly in the assessment of the aorta, right atrium, pulmonary vein flow, and transmitral flow. Furthermore, cardiac anesthesiologists correctly interpreted 83% of TEEs when compared specifically to radiologists. Out of 2464 TEE exams, this comes out to a total of 2045 correctly interpreted TEEs. Nevertheless, comparisons between anesthesiologists and cardiologists (80% concordance) and cardiologists and radiologists (82% concordance) were all similar.

The study by Mishra et al contained information regarding the concordance between online interpretation by cardiac anesthesiologists and offline analysis by cardiologists. This study specifically examined left ventricle regional wall motion, valve function, and left and right ventricle function. 3620 out of 4161 TEEs were correctly interpreted by the cardiac anesthesiologists, amounting to an accuracy rating of 87%. Although this study did not state the number of anesthesiologists involved, they examined 3217 TEEs in a group of patients who underwent coronary bypass graftings and 629 TEEs in a group of patients who underwent valve procedures, yielding a total of 3846 TEEs that were interpreted.

The final quantitative study by Miller et al compared the performance of anesthesiologists to an expert cardiologist in recording and interpreting TEEs. Parameters measured in this study included size of the heart chambers, FAC, and degree of stenosis or insufficiency of heart valves. They found that their cardiac anesthesiologists correctly interpreted 1242 out of 1572 TEEs, a 79% accuracy rating. As indicated in Table 2, these three studies totaled 8197 interpreted TEEs by cardiac anesthesiologists, 84% of which were correctly interpreted.

Discussion

The American Society of Echocardiography suggests that non-cardiologists such as radiologists and cardiac anesthesiologists who provide optimal TEE services should ideally undergo 6 months of full-time training in an active echocardiography training institution.³⁹ They recommend being involved in 300 total TEE exams and performing at least 150 of those exams, and 15 h of TEE within 3 years per Continuing Medical Education (CME) standards. Thus, all physicians who were not formally trained in TEE should adhere to these standards. It may also be advisable to consider facilitating close interactions between cardiac anesthesiologists and cardiologists or radiologist echocardiographers, at least in the initial training phases.⁴⁰

In our study design, we chose to compare the evaluation of TEE studies between attending anesthesiologists to primary attending echocardiographers, either cardiologists or radiologists. A prospective observational cohort study was performed between 1993 and 1997 meant to evaluate TEE as a safe and reliable technique during cardiac surgery.⁴¹ 3217 TEEs were administered to 944 patients who underwent coronary artery bypass grafting (CABG) procedures, and another 629 TEES to 142 patients who underwent heart valve procedures. The attending anesthesiologists who performed the TEE had a minimum hands-on experience of performing and interpreting 500 TEE studies each. Although the study did not disclose the number of anesthesiologists included in the study, they found that there was a rather high concordance between anesthesiologists and cardiologists (87%). This suggests that anesthesiologists can interpret and perform TEE studies in a manner comparable to that of cardiologists.

Another study was done at Duke University Medical Center that assessed the concordance of TEE interpretation in a continuous quality improvement (CQI) program.⁴ In this study, 10 cardiac anesthesiologists conducted a total of 154 TEE studies that included the estimation of FAC using Bland-Altman methods. Fractional area change is a measure of right ventricular systolic function. It is clinically significant because it can be used to measure any impairments to right ventricle function, such as after a pulmonary valve replacement.⁴² All 154 of the TEE studies were reviewed by radiologists, 50 of which were also reviewed by cardiologists. Cardiac anesthesiologists were found to underestimate the FAC when compared to radiologists, especially if the anesthesiologist had less than 5 years of TEE experience. Anesthesiologists with more experience, however, were found to have higher levels of concordance with the radiologists. Ultimately, the high levels of concordance of anesthesiologists to radiologists (83%) and cardiologists (80%) suggest that anesthesiologists are proficient in TEE interpretation.

A prospective study done at the Madigan Army Medical Center evaluated the ability of anesthesiologists to perform and interpret TEE after revisions were made to their examination protocol.⁴³ Namely, these revisions entailed going from a standard 10 view TEE examination to a 12 view in which 8 were from the original and 4 assessed with color Doppler. Eight cardiac anesthesiologists performed 135 TEE examinations, which were then compared with a final expert evaluation by a cardiologist, yielding an accuracy of 79%. Although this is considerably lower than the other studies we analyzed, this is inclusive of TEE examinations with omitted diagnoses (blanks on evaluation sheets). If these TEE examinations had not been included in the study, the rate of correct interpretation would have been 94%.

It has been shown in a study done at Aarhus University Hospital that anesthesiologists are capable of providing valuable information in interpreting TEE.⁴⁴ A TEE was successfully performed on 525 children undergoing cardiac surgery and according to the results, interpretations of TEE performed by anesthesiologists resulted in a total of 184 alterations to treatment in 143 patients. Additionally, anesthesiologists were able to add 37% of new information and add 8% of decisive information out of all the TEEs interpreted.

Although our study indicates how effective anesthesiologists can be in perioperative care, there have been multiple studies that have shown experience and training is still valuable in both carrying out the TEE procedure and interpreting the results. One study compared the length of time it takes to obtain a TEE exam and how accurate the interpretation was between certified anesthesiologists and anesthesiology residents.⁴⁵ Attending physicians and residents were recruited from both the Vanderbilt School of Medicine and The Icahn School of Medicine at Mount Sinai for a total of 15 residents and 11 attending physicians. Participants were required to obtain 10 standard views using TEE. The certified anesthesiologists were able to interpret 5 out of 10 images better than the residents, whereas the remaining 5 views were comparable to the residents. Results also indicated that certified anesthesiologists were able to acquire TEE images more quickly, suggesting that experience is necessary to become a proficient echocardiographer.

A study done at Mahidol University concurred with this by showing improvement in acquiring TEE images as the procedure was performed more often.⁴⁶ An additional study performed at The Icahn School of Medicine at Mount Sinai suggested that more experienced anesthesiologists were able to score higher on multiple-choice questions that involved TEEs.⁴⁷ Evidently, experience in echocardiography improves both the theoretical knowledge and the practical application of the skills involved in TEE.

Limitations to our study include the circumstances of assessment in our quantitative studies. Specifically, comparisons were made between on-line assessments by anesthesiologists and off-line assessments of the primary echocardiographers. It is plausible that there could have been a higher level of agreement between the two groups if they interpreted TEEs under the same circumstances. For example, there may have been higher concordance if the anesthesiologists evaluated TEE results after operation. Another limitation to our study is that most of our quantitative data were published nearly 20 years ago. If these studies were to be done today, it may be the case that we would see higher concordance between cardiac anesthesiologists and primary echocardiographers, especially because of the guidelines that were established since then.³⁹ Another notable limitation of this study is that there were variations in the gold standard for interpreting TEEs. Some studies used expert echocardiographers as the gold standard, while others relied on the degree to which there was consensus amongst attending echocardiographers.

Quality Improvement

A possible method of improving clinical evaluation is by refining current indications for the use of echocardiography. For example, echocardiography currently plays a major role in the diagnosis and management of infective endocarditis (IE) as part of Duke’s criteria. However, many patients are initially misclassified even though IE is a life-threatening emergency.^48–50 This is partly because a negative echocardiogram does not rule out IE and a false-positive result is not unusual with these tests. The fault here is not so much in the conductor of the test, but the test itself. Therefore, in these cases, it may be worth considering other imaging techniques. An 18F-FDG PET/CT scan has instead shown promising results with these patients.^51–56

Conclusion

Based on the studies presented, it is clear that anesthesiologists have an important role in the perioperative stages of patient care by performing and interpreting transesophageal echocardiograms. With continuous quality improvement, cardiac anesthesiologists are shown to function at a level equivalent to that of primary echocardiographers. The implementation of software programs to routinely test physician TEE skills and the implementation of standardized AI interpretation as a possible gold standard are noteworthy considerations for future investigation.

Disclosure

The authors declare no competing interests in this work.