Validity of a smartwatch for detecting atrial fibrillation in patients after heart valve surgery: a prospective observational study

Abstract

Objectives

Atrial fibrillation (AF) is a common early arrhythmia after heart valve surgery that limits physical activity. We aimed to evaluate the criterion validity of the Apple Watch Series 5 single-lead electrocardiogram (ECG) for detecting AF in patients after heart valve surgery.

Design

We enrolled 105 patients from the University Hospital of North Norway, of whom 93 completed the study. All patients underwent single-lead ECG using the smartwatch three times or more daily on the second to third or third to fourth postoperative day. These results were compared with continuous 2–4 days ECG telemetry monitoring and a 12-lead ECG on the third postoperative day.

Results

On comparing the Apple Watch ECGs with the ECG monitoring, the sensitivity and specificity to detect AF were 91% (75, 100) and 96% (91, 99), respectively. The accuracy was 95% (91, 99). On comparing Apple Watch ECG with a 12-lead ECG, the sensitivity was 71% (62, 100) and the specificity was 92% (92, 100).

Conclusion

The Apple smartwatch single-lead ECG has high sensitivity and specificity, and might be a useful tool for detecting AF in patients after heart valve surgery.

Graphical abstract

Keywords:

• Smartwatch
• single-lead ECG
• atrial fibrillation
• heart valve surgery
• specificity
• sensitivity
• validity

Introduction

Heart valve diseases are on the rise, and its prevalence has reached 2.5% in occident countries [1,2]. The frequency is increasing with age, and due to increased life expectancy, a growing health problem is becoming apparent [2]. Heart valve surgery is well-established with cardiovascular benefits, improved quality of life, and reduced mortality rates [3]. The procedure is complex, and patients often experience postoperative complications, such as low cardiac output, bleeding, infection, and arrhythmias including atrial fibrillation (AF) [4].

AF is the most frequent arrhythmia after cardiac surgery [5–7], and is a heart rhythm in which the atria contract irregularly and rapidly [8,9]. Moreover, AF is increasing in frequency, affects 0.5% of the world population, and is a leading cause of heart failure, stroke, and embolism [10]. The risk of secondary ailments and existing pharmacologic treatments have increased interest in consumer devices with the potential for early detection and treatment to prevent secondary diseases. If proven effective, such devices may play a role in physical activity and prevent serious diseases in patients with AF.

The ability to record an electrocardiogram (ECG) with potential for detecting AF are available in several commercially available smartwatches in many countries [11]. Few have investigated the validity of the smartwatch ECG function after heart valve surgery [11]. In the context of a larger study testing the safety and feasibility of early physiotherapy-led exercise, we aimed to examine the sensitivity and specificity of the Apple Watch Series 5 single-ECG to detect AF within the first 4 days after heart valve surgery.

Methods

Study design and participants

We performed a cross-sectional study to evaluate the validity of detecting AF by using a smartwatch-based ECG function in patients after left-sided open-heart valve surgery. AF was defined according to the 2020 ESC Guidelines for the diagnosis and management of AF: “A standard 12- lead ECG recording or a single-lead ECG tracing ≥ 30 s showing heart rhythm with no discernible repeating P waves and irregular RR intervals is diagnostic of clinical AF” [9].

The study participants (n = 105) were recruited consecutively from the University Hospital of North Norway (UNN) between October 2020 and March 2022 (Figure 1). Due to logistics, acute patients that were- transferred from other hospitals at weekends or holidays were not included. Northern Norway is a large geographical region of 112,951 km² with 483,536 inhabitants according to Statistics Norway (SSB) as of January 2023 [12]. UNN is one of four regional hospitals with cardiac surgery expertise and performs approximately 400 heart operations annually in patients from Northern Norway, of which valve operations make up a third.

Figure 1. Study flow chart.

Graph 1. Bar graphs with 95% confidence intervals as error bars showing the sensitivity, specificity, positive and negative predictive values of smartwatch ECG compared to telemetry and 12-lead ECG.

The inclusion criteria were all patients aged > 18 years who could provide informed consent and had undergone elective left-sided open-heart valve surgery. Non-consenting patients, patients < 18 years, and patients being unable to participate due to serious postoperative complications were not enrolled.

Data collection and equipment

The Apple smartwatch (Apple inc., Cupertino, CA) used in the present study is approved by the US Food and Drug Administration (FDA) [13]. The smartwatch was updated to the latest software between each participant. Measurements were taken with software versions 6.0 to 8.4.1. Before the heart valve surgery, all participants received oral and written information about the Apple Watch 5 single-lead ECG procedure. A trained physiotherapist instructed all participants on how to perform ECGs by placing their index finger on the smartwatch’s digital crown, keeping their arms rest on a table or legs, and practising sample acquisition for few minutes. A detailed description of how to obtain an ECG can be found in the user manual for the Apple Watch Series 5 [14]. By using an Apple Watch Series 5, single-lead ECGs were recorded up to three times a day on the second and third, or third and fourth postoperative days in the hospital. The ECG measurements were taken at specific times (morning, afternoon, and evening) during the day. In accordance with the smartwatch settings to ensure quality and accuracy, the ECG measurements were of 30 s duration. Additionally, patients were encouraged to take an extra ECG in case of clinical symptoms of AF such as palpitations, chest pain, dizziness, or heavy breathing. Participants who were unable to perform the measurements were assisted by a trained nurse or physiotherapist. The participants wore the smartwatch continuously on their left wrist for the whole study period.

The hospital telemetry system (a six-lead ECG, BeneVision TM80, Bio-Medical Electronics Co., Mindray, Shenzhen, China) is used as a standard procedure for the detection and identification of postoperative AF. All participants were continuously monitored with the hospital telemetry system from the first to thee fifth postoperative day in hospital settings. Heart rhythm recordings were forwarded, downloaded, and printed at the monitoring Centre in the intensive care unit at UNN, Tromsø.

A 12-lead ECG was taken with a Mortara XScribe 6, a standard 12-lead ECG machine (Scan-Med a/s, Drammen, Norway) during medical examinations 3 days after heart valve surgery.

The single-lead ECGs from the Apple Watch were extracted to PDF files. Analysable single-lead ECG PDF files were anonymized and compared with telemetry prints at the same time points and with 12-lead ECG. The smartwatch single-lead ECG was taken within 10 min after the 12-lead ECG.

General conditions, surgical methods, diagnoses, conditions related to ECG or AF, and health behaviour data were collected from patients’ electronic medical records.

Evaluation and statistical analysis

To compare the single-lead Apple Watch ECGs with the printed telemetry ECGs and the 12-lead ECGs, a physician with ECG expertise without prior knowledge of the patients reviewed the data, and assessed the presence of sinus rhythm, AF, or another arrhythmia.

Statistical analysis was conducted using the R language for statistical computing (R Core Team 2022 Foundation for Statistical Computing, Vienna, Austria). The smartwatch results were compared with the physician’s evaluation of continuous telemetry and a 12-lead ECG.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each variable, and accuracy for the continuous telemetry monitoring. Moreover, we compared Apple Watch AF detection with continuous telemetry measurements at any given time point and with the 12-lead ECG detection. The Apple watch “Poor Recording” and “Inconclusive” results could not be analysed and were discarded.

The primary outcome in this study is detection of AF with a smartwatch. Power analysis was run with PASS version 16.0.7 (NCSS Statistical Software, Kaysville, UT). The calculations were based on a 95% confidence interval (CI) with a maximum of 15 percentage points. A specificity of 0.80 would require 110 participants to achieve 80% power.

Ethics and consent

The Institutional Data Protection Authority at UNN (PVO) approved this study (Project number: 02509). The study was performed in accordance with the Declaration of Helsinki. All participants provided oral and written consent before surgery and study initiation.

Results

Patients’ characteristics

Between October 2020 and March 2022, we enrolled 105 participants who underwent planned elective heart valve surgery at UNN. Surgery was postponed in two patients; two others suffered serious postoperative complications that made them ineligible (visual disturbances and postoperative bleeding), and one withdrew from the study for personal reasons. Altogether, 100 patients (78 males and 22 females) completed the study, 93 had an analysable dataset, and seven had missing telemetry values owing to technical and administrative failures (Figure 1). All the 93 patients had single-lead ECGs taken by the smartwatch ≥3 times daily on the second to third or third to fourth postoperative day. These results were compared with the continuous 4–5 days ECG telemetry monitoring and a 12-lead ECG on the third postoperative day. The baseline characteristics of the patients are presented in Table 1.

Table 1. Baseline characteristics of the 93 in-hospital participants that had undergone left-sided heart valve surgery.

	Participants with the complete dataset n = 93 (%)
Mean age (SD) in years	68 (9.9)
Sex, n (%)
Male	73 (78.5)
Female	20 (21.5)
Caucasian	93 (100)
Clinical characteristics
Main diagnosis
Aortic Stenosis	85 (91.4)
Mitral regurgitation	9 (9.7)
Endocarditis	–
Secondary diagnosis
Paroxysmal AF	8 (8.6)
Persistent AF	2 (2.2)
Permanent AF	8 (8.6)
Heart Failure	14 (15.1)
Myocardial Infarction	8 (8.6)
AV block	7 (7.5)
Surgical method
Median Sternotomy	93 (100)
TAVI	–
Valve surgery and CABG	22 (23.6)
Valve Types
BAVR	74 (79.6)
MAVR	19 (20.4)
Heart medication
Beta-blockers	33 (35.5)
Antiarrhythmic	4 (4.3)
Smoking
Never smoked	24 (25.8)
Current smoker	9 (9.7)
Former smoker	60 (64.5)
Snuff
Never snuffed	92 (98.9)
Current snuff user	1 (1.1)
Former snuff user	–

AF: atrial fibrillation; TAVI: Transcatheter Aortic Valve Implantation; CABG: Coronary Artery Bypass Graft Surgery; BAVR: Bioprosthetic Aortic Valve Replacement; MAVR: Mechanical Aortic Valve Replacement.

Eighteen patients had AF diagnosis before the surgery (19.3%). Of these, eight had paroxysmal, two persistent, and eight permanent AF. Nineteen patients took extra measurements with the smartwatch due to symptoms. Of these, ten had AF, four had high pulse (> 120/minute), one bradycardia (< 50/minute), and four had inconclusive results. Eight patients took an extra measurement at night between 22.00 and 06.00. Three of these patients had AF on the smartwatch. The telemetry equipment registered episodes of atrial flutter in four patients. The smartwatch app caught two of these and classified them as “not conclusive.” The physician who reviewed the data recognized the episodes as atrial flutter.

Comparison of Apple Watch atrial fibrillation detection with continuous telemetry equipment from the hospital at any given time

Of all patients, 93 (100%) completed the first and second, and 92 (99%), 87 (93%), 82 (88%), and 64 (69%) completed the third, fourth, fifth, and sixth Apple Watch ECG measurements, respectively (Table 2). The main findings of these results were 91% sensitivity, 96% specificity, 83% PPV, and 81% NPV (Table 2). The accuracy to detect AF was 95% (95% CI: 91–99).

Table 2. Comparison of Apple Watch atrial fibrillation detection with continuous ECG measurements at given time points.

Time	n^a	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
1	93	86 (64,100)	97 (93,100)	86 (64,100)	97 (93,100)
2	93	92 (75,100)	95 (90,99)	75 (53,94)	99 (96,100)
3	92	94 (79,100)	95 (89,99)	79 (59,95)	99 (96,100)
4	87	88 (70,100)	99 (96,100)	93 (80,100)	97 (93,100)
5	82	93 (78,100)	97 (93,100)	87 (69,100)	99 (96,100)
6	64	92 (71,100)	96 (92,100)	79 (54,100)	99 (96,100)
All	93	91 (76,100)	96 (92,100)	83 (67,96)	81 (74,89)

^an: number of participants; PPV: Positive Predictive value; NPV: Negative Predictive Value.

Sensitivity, specificity, positive and negative predictive values (PPV and NPV) were generated with Bootstrap and given as percentages with 95% confidence intervals.

Comparison of Apple Watch atrial fibrillation detection with a 12-lead ECG

Table 3 presents the sensitivity, specificity, PPV, and NPV when comparing the 12-lead ECG with the Apple Watch single-lead ECG measurement. The smartwatch single-lead ECG was taken within 10 min after the 12-lead ECG (Graph 1).

Table 3. Comparison of Apple Watch atrial fibrillation detection with a 12-lead ECG.

ECG 12 n	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
93	71 (62,100)	92 (92,100)	72 (54,100)	96 (88,100)

ECG 12: 12-lead electrocardiography; n: number of participants; PPV: Positive Predictive value; NPV: Negative Predictive Value.

Sensitivity, specificity, positive and negative predictive values (PPV and NPV) were generated with Bootstrap and given as percentages with 95% confidence intervals.

Discussion

By comparing the single-lead ECG on the Apple Watch S5 with continuous ECG measurements at given time points and the 12-lead ECG on the third postoperative day, we found that the Apple Watch S5 single lead ECG was able to rule out atrial fibrillation with a high level of specificity. The ability of the Apple Watch single-lead ECG to detect atrial fibrillation was high when compared to the continuous ECG measurements, and a little lower when compared to the 12-lead ECG.

Although the results of our study show high sensitivity and specificity, it is lower than the results in Apple’s study of the single-lead ECG that showed over 98.4% sensitivity and 99.2% specificity for detecting AF when compared to a 12-lead ECG in 588 participants [13]. However, the aforementioned study was with a different setting and from a different population, comparing 301 participants with known AF and 287 without any known cardiac arrhythmias [13]. Because the smartwatch single-lead ECG was taken within 10 min after the 12-lead ECG in our study, the time frame between the two with the paroxysmal nature of AF could have influenced these results. Additionally, small sample size, and the fact that only 19.3% of patients had an AF diagnosis before heart valve surgery in our study could have influenced the detection rate.

A study of 50 patients that underwent cardiac surgery found 96% sensitivity and 100% specificity of the Apple Watch single-lead ECG when compared to telemetry monitoring, which is a little higher than in our study [11]. Notably, 90% of postoperative AF occurs within the first 4 days after cardiac surgery [5]. The length of the first episode averages 7–8 h [5] Additionally, recurrence occurs in 40% of cases and within the first 24 h, and up to 80% of patients remain 24 h in sinus rhythm [15]. Both the time frame, type of surgery, and amount of patients with AF beforehand could possibly explain the small differences in detection rate between our and the aforementioned study results [11]. On the other hand, a recent study that compared Apple Watch 6 single-lead ECG and other single-lead smartwatch algorhytms with a 12-lead ECG in 201 patients showed 85% sensitivity and 75% specificity for the Apple Watch 6, which is lower, but within the confidence intervals of the results in our study [16]. However, this study had intention-to-treat analysis, and opposed to our study, it included all ECGs that could not be evaluated due to “Poor recording” or “Inconclusiv” results.

A strength of our study is that the interpretation of the ECG recordings was made by an experienced medical doctor without previous knowledge of the patients. However, without a level of agreement among different interpreters there is a small risk of inconsistent diagnosis of AF.

The causes of postoperative AF vary and is also influenced by the type of surgery. Additionally, development of AF depend on patient, surgical, anaesthetic, postoperative course and treatment. Heart valve surgery is also associated with many changes in body homeostasis, which could lead to other arrhythmias [17]. According to Apple’s preclinical study of 2500 participants of which 625 (25%) had a diagnosis of AF, both body movements during measurement, sweating, low body mass index (BMI), right axis deviation and other arrhythmia than AF could influence the detection rate of AF [13]. Such conditions could give “Poor Recording” and “Inconclusive” results from the smartwatch. Because the patients repeated the single-lead ECG measurements until a result that was eligible for analysis appeared, we do not think that these issues significantly influenced the smartwatch accuracy in our study. An interesting finding in our study is that the smartwatch classified two out of four episodes of atrial flutter as “not conclusive”, but the physician reviewed the data and classified these two arrhythmias correctly. Although few patients, the finding shows that the smartwatch can often detect, but as of today, is not able to recognize other arrhythmias besides AF. Therefore, close collaboration with health care personnel is warranted to review results manually if there is suspicion of another arrhythmia.

The telemetry equipment registered episodes of atrial flutter in four patients. The smartwatch app caught two of these and classified them as “not conclusive.” The physician who reviewed the data recognized the episodes as atrial flutter.

In contrast, we observed 47 episodes of postoperative AF in the 93 patients (50.5%), consistent with the results in two other studies, which reported a post-valvular heart surgery prevalence of 37–60% and 30–50%, respectively [18,19].

This study is part of a project whose aim is to test the feasibility of exercise-based rehabilitation with digital support in patients after valve-operated endocarditis. Accordingly, every one of three patients with heart valve surgery after infectious endocarditis develops a sedentary lifestyle and report low levels of physical and mental health. A barrier to a more active life could be caused by patient health concerns of cardiac arrhythmias which are common in this patient group [20,21]. A smartwatch-based monitoring of heart rhythm may lessen this burden, increase the safety of exercise, and thereby promote a more active life in this patient group.

Future perspective

The use of wearable technology is progressively increasing globally, and smartwatches are the most popular devices among wearable sensors [22]. It is estimated to reach more than 400 million global shipments by 2026 [22]. Notably, the duration of hospital stay in patients after heart valve surgery has shortened in recent years, and exercise-based cardiac rehabilitation starts increasingly earlier. We are not aware of any studies that shows better prognosis for patients with AF screening strategy after heart valve surgery. However, several studies have shown that high-intensity interval training is more effective in increasing VO2max compared to moderate intensity. Monitoring of rhythm during physical training early after surgery can be of great use not only in terms of lessening the burden for the patients, but also for the therapist who is responsible for the intensity of the training. Therefore, accessible means of monitoring, detecting, and screening post-discharge AF is required. This study suggests that a single-lead smartwatch ECG might be positioned to contribute to closing this gap. Outpatient monitoring with the Apple Watch may provide these patients with a longer-term outpatient monitoring solution that they may not necessarily have access to otherwise.

Limitations

Although this was a single-centre study, patients were from a large and scattered populated geographical area. The ECG measurements had to be manually initiated to start recording a single-lead ECG. Additionally, most patients could not perform measurements independently except with the help of trained nurses or physiotherapists. However, the reason why only 69% performed the last single-lead ECG measurement was due to hospital logistics. Several of the patients without major complications who were not diagnosed with AF were transferred to local hospitals by air before the last measurement was taken. Patients with diseases such as Parkinson’s and other hand tremors had problems carrying out ECG measurements because of tremors. These patients experienced it as unpleasant and lost motivation to take measurements. Their ECG measurements were “Poor Recording” and were discarded from analysis. There are some challenges with smartwatch single-lead ECG measurements, and not everyone will be able to benefit from such devices. With the help from healthcare staff and/or good training, we believe that smartwatch devices can be handled correctly with good tolerance and satisfaction. Another limitation is that the smartwatches were updated continuously, resulting in measurements being taken with some different software versions. On the other hand, this fact represents the real world of smartwatches.

Lastly, the follow-up duration of this study was two to three postoperative days. Longer-term monitoring of our cohort may provide more varied data on single-lead ECG measurements after heart valve surgery.

Conclusion

The detection of AF using the smartwatch single-lead ECG had high sensitivity and specificity. Our study results show that the Apple 5 single-lead ECG could be a useful tool for detecting AF, and might have important diagnostic potential in the postoperative phase in patients after heart valve surgery. However, a technological barrier is that not all patients are able to perform such a measurement without support in a postoperative setting.

Acknowledgements

We thank the Department of Physiotherapy for study support, and the Department of Cardiothoracic and Vascular Surgery, University Hospital of North Norway in Tromsø for the help in data collection. We would like to thank Editage (www.editage.com) for English language editing.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Data for this article is collected in Service for Sensitive Data (TSD), which is Norwegian platform for collecting, storing, analyzing and sharing sensitive data in Norway. Data can be shared upon reasonable request to the corresponding author.

Additional information

Funding

This study was supported by a grant from the Northern Norway Regional Health [Helse Nord RHF grand number HNF 1574-21].