Learning curve analysis of radiofrequency ablation for benign thyroid nodules

Abstract

Background

Radiofrequency ablation (RFA) has been increasingly accepted as an alternative to surgery in the treatment of symptomatic benign thyroid nodules. However, the learning curve of thyroid RFA has yet to be defined. We hypothesized a temporal relationship between proficiency of the skill and midterm volume reduction.

Methods

Consecutive patients who underwent RFA and had at least a six-month follow-up were identified from an institutional database. The cumulative sum (CUSUM) analysis was applied to visualize the learning curve on the adjusted volume reduction rate (VRR).

Results

A total of 102 nodules in 93 patients were included in the analysis. Linear regression revealed that nodule composition was the main predictor of the VRR. The composition-adjusted VRR increased with the chronological treatment order. The series was divided into three phases based on inflection points of the CUSUM analysis: the initial learning phase (case 1–20), consolidation phase (case 21–65), and proficiency phase (case 66–102). In the later phase, more solid nodules were treated, power output used by the operator was higher, and RFA treatment time was longer.

Conclusion

The treatment efficiency of thyroid RFA was associated with technical proficiency, suggesting the presence of a learning curve effect.

Keywords:

• Learning curve
• CUSUM
• radiofrequency ablation
• volume reduction rate
• thyroid nodules

Introduction

The prevalence of thyroid nodules in the general population is as high as more than 60% as detected by sensitive imaging [1]. The majority of thyroid nodules are clinically insignificant benign lesions and can be followed by observation. However, some nodules that grow or cause symptoms would require treatment. Conventionally, thyroidectomy is the standard of care for symptomatic nodules. Nonetheless, thyroidectomy is associated with a small but not negligible risk of complications, such as nerve injury and hypoparathyroidism. In addition, patients may need thyroid hormone replacement when hypothyroidism occurs postoperatively.

In recent years, several image-guided thermal ablation procedures have been developed to reduce thyroid nodule volume, paralleled by improvement in symptoms [2]. In a randomized study, radiofrequency ablation (RFA) achieved a significantly larger nodule volume reduction than laser ablation [3]. Although RFA is generally an effective and safe procedure, physicians need extra dedicated training to master this new technology because the skillset of RFA is different from those of diagnostic ultrasound and fine-needle aspiration (FNA) biopsy [4]. In this regard, a learning curve needs to be defined to guide fellowship training.

To evaluate the effect of operator experience, Russ and colleagues divided patients into three equal groups according to their chronological treatment rank [5]. They found a measurable learning curve in RFA regarding technique efficacy until 90 patients. Certainly, there was a degree of arbitrariness in this division. The cumulative sum (CUSUM) analysis has been applied to the evaluation of learning curves of various procedures [6]. In this study, we hypothesized a temporal relationship between the surgeon's mastery of RFA and patient outcome. Given the rarity of RFA-associated complications, the main outcome measure to determine proficiency was midterm volume reduction.

Methods

Study cohort

This study (21MMHIS070e) was approved by the institutional review board of MacKay Memorial Hospital, and patient consent was waived owing to the retrospective nature of the study.

We reviewed our prospectively maintained database to search for patients who underwent RFA for benign thyroid nodules between April 2019 and November 2020. The inclusion criteria for this study were as follows: (i) age 20 or older; (ii) normal thyroid function before RFA; and (iii) benign thyroid nodules treated by RFA. The exclusion criteria for this study were as follows: (i) RFA for lesions other than benign thyroid nodules; (ii) RFA for treatment of recurrent nodules from remnants of previous thyroid surgeries; and (iii) insufficient follow-up of less than 6 months.

In our institution, evaluation and management of patients with thyroid nodules were based on ultrasonographic features and FNA cytology as per current guidelines [7]. Thyroid Imaging, Reporting and Data System (TI-RADS) was used for risk stratification and guidance in decision making regarding FNA and follow-up [8,9]. Patients who had benign thyroid nodules causing compressive symptoms or cosmetic issues were provided with the option to undergo RFA, which was not reimbursed under the national health insurance scheme.

Ultrasound-guided ablation

Neck ultrasound was performed using a high-frequency linear probe to document the presence of thyroid nodule(s) [10]. The location, size (three-dimension measurements), and composition of each nodule were recorded. Nodule volume was calculated using the ellipsoid volume formula as follows: length × width × depth × 0.524. Nodule composition was analyzed using the proposed classification system: solid (≤10% of fluid component), predominantly solid (11-50% of fluid component), predominantly cystic (51-90% of fluid component), or cystic (>90% of fluid component) [11].

All RFA procedures were performed by a single endocrine surgeon (C.-Y. Kuo) with five years of experience in thyroid ultrasound and ultrasound-guided FNA. Written informed consent was obtained from every patient before the procedure. Local anesthesia was administered at the skin puncture site and the perithyroidal, subcapsular space with the patient in a supine position with slight hyperextension without any sedation. Ablations were performed using a commercial RFA generator (RF Medical, Seoul, South Korea) and 18-gauge thyroid-dedicated electrodes with a 0.5 to 1.0 cm active tip. The trans-isthmic approach and moving-shot technique were performed in all patients as recommended by experts [12]. An RF electrode was inserted via the isthmus in the midline-to-lateral direction to approach the target nodule. The location of the electrode tip was continuously monitored on real-time ultrasound during the whole procedure until a maximum ablated volume was achieved.

Postablation assessment

Adverse events related to the procedure were assessed after the completion of RFA. The thyroid-stimulating hormone, free T4, and thyroid autoantibodies were determined and compared preoperatively and postoperatively. Treatment efficacy was evaluated by ultrasound follow-up performed at 1, 3, 6, and 12 months after RFA. The volume reduction rate (VRR) was calculated as: (baseline volume – post-treatment volume)/baseline volume × 100%. To control for bias due to differences in the length of follow-up, the postablation assessment was censored six months after RFA.

Cumulative sum analysis

The CUSUM is an analytic technique used for detecting the change and has been applied for quantitative evaluation of the learning curve [13]. Patients were ordered chronologically according to the date of surgery. The CUSUM is the running total of differences between the individual VRR value and the mean VRR value of all patients. This recursive process continued to the CUSUM for the last patient. The results were plotted graphically in the CUSUM chart, which represented the trend of data deviation from the mean value. Scatter plots of CUSUM against consecutive cases were fit with polynomial curves generated from nonlinear regression. The turning point of slope change indicated proficiency or mastery of the skill. The nonparametric Pettitt test was further performed for changepoint analysis. If potential changepoint(s) were identified, clinical parameters and outcomes were compared between different phases.

Given the possibility of preferential selection of simpler cases during the operator's initial experience with RFA, adjustment of CUSUM analysis was conducted to balance heterogeneity between different learning phases. We adopted a scheme similar to the risk-adjusted CUSUM described previously [14]. Predicted volume reduction of different nodule compositions was estimated using the multivariate regression model, and the CUSUM chart was updated with a likelihood-based scoring.

Statistical analysis

Simple and multiple linear regression analyses were used to determine the relationship between the VRR and clinical variables [15]. Comparisons between different phases were performed using Fisher's exact test, Cochran-Armitage trend test, or Jonckheere–Terpstra test, as appropriate. Probability values less than 0.05 (two-sided hypotheses) were regarded as statistically significant.

Results

Demographics and baseline characteristics

During the study period, RFA was performed in 107 benign thyroid nodules in 98 patients. Five patients whose six-month follow-up data were not available were excluded from further analysis. Therefore, data of 102 nodules in 93 patients (82 women and 11 men) were analyzed. The mean age of the patients was 45 years (range, 21 to 78). In nine patients, two separate nodules were treated during the same procedures. Four patients had two nodules with the same thyroid lobe. The remaining five patients had the nodules at different locations (e.g., in the right thyroid and the isthmus) or in a bilateral distribution.

The mean baseline nodule volume was 24.7 ml (range, 0.9 to 98.9). In all, 58 were solid; 18 predominantly solid; 20 predominantly cystic; and six cystic. For nodules with ultrasonographic features of TI-RADS 1 (benign) or 2 (not suspicious), a single FNA or core-needle biopsy (CNB) was performed before RFA. Otherwise, at least two separate FNAs or CNBs were needed to confirm a benign diagnosis.

Treatment efficacy

At six months post-RFA, the mean VRR of 102 nodules was 55.9% (range, −23.5 to 98.8%). We conducted a linear regression analysis to identify clinical variables that were associated with the VRR. Interestingly, sequential case number was not associated with the VRR (Table 1). The only predictor of the VRR after RFA was nodule composition. As expected, cystic nodules were associated with a higher VRR.

Table 1. Simple and multiple linear regression analyses of clinical variables and six-month volume reduction rate following radiofrequency ablation (RFA) for benign thyroid nodules (n = 102).

	Univariate		Multivariate
	β	p Value	β	p Value
Chronological order (decile)	−0.029	0.771
Age	0.219	0.027	0.158	0.112
Female	0.014	0.889
Nodule Location
Right	−0.293	0.149
Isthmus	Reference
Left	−0.238	0.241
Baseline volume	−0.148	0.138
Composition
Solid	Reference		Reference
Predominantly solid	0.078	0.434	0.092	0.351
Predominantly cystic	0.228	0.024	0.1932	0.055
Cystic	0.268	0.007	0.207	0.040
Active tip length	−0.112	0.269
RFA power	−0.080	0.431
RFA time	−0.254	0.011	−0.186	0.061

Learning curve

To account for the possibility that treatment efficacy might be confounded by differences in the distribution of nodule composition during various phases of clinical practice, we calculated the composition-adjusted VRR for each nodule. Notably, the adjusted VRR gradually increased along with the chronological treatment order (Figure 1). As a sensitivity analysis, we performed a subgroup analysis focusing on solid nodules with VRRs unadjusted. As shown in Figure 2, a similar trend of slowly increasing VRR was noted. These findings supported our hypothesis that proficiency or mastery of the skill was associated with better treatment efficacy.

Figure 1. The composition-adjusted volume reduction rate (VRR) plotted against the chronological treatment order in the whole study cohort. The solid red line represents the curve of best fit for the plot.

Figure 2. The volume reduction rate (VRR) plotted against the chronological treatment order in 58 solid thyroid nodules. The solid red line represents the curve of best fit for the plot.

Using the CUSUM chart, the learning curve was drawn by plotting the cumulative sequential differences between each adjusted VRR value and the average over time (Figure 3). We identified two inflection points of the learning curve at the 20th and 65th cases, respectively.

Figure 3. Cumulative sum (CUSUM) graph for the composition-adjusted volume reduction rate (VRR). The solid red line represents the curve of best fit for the plot.

Based on the results of the learning curve analysis, all patients were divided into three phases: initial learning phase (case 1–20), consolidation phase (case 21–65), and proficiency phase (case 66–102). Comparisons of clinical data and RFA parameters between phases are presented in Table 2. The proportion of solid nodules significantly increased in the proficiency phase. Additionally, longer (10 mm) active tips of RF electrodes and higher power output were used by the operator during the later phase. A significantly longer treatment time in the later phase of the learning curve was spent ablating thyroid nodules.

Table 2. Characteristics of each learning phase of radiofrequency ablation (RFA) for benign thyroid nodules (n = 102).

	Initial learning phase (case 1-20)	Consolidation phase (case 21–65)	Proficiency phase (case 66–102)	p Value
Age (years)	50 ± 17	43 ± 11	48 ± 9	0.486
Sex				1.000
Male	2 (10%)	5 (11%)	4 (11%)
Female	18 (90%)	40 (89%)	33 (89%)
Nodule location				0.596
Right	10 (50%)	25 (56%)	18 (49%)
Isthmus	3 (15%)	2 (4%)	2 (5%)
Left	7 (35%)	18 (40%)	17 (46%)
Baseline volume (ml)	23.6 ± 16.2	27.1 ± 24.8	22.3 ± 16.0	0.570
Composition				0.015
Solid	10 (50%)	20 (44%)	28 (76%)
Predominantly solid	4 (20%)	9 (20%)	5 (14%)
Predominantly cystic	4 (20%)	12 (27%)	4 (11%)
Cystic	2 (10%)	4 (9%)	0 (0%)
Active tip				<0.001
5 mm	1 (5%)	0 (0%)	0 (0%)
7 mm	18 (90%)	34 (76%)	13 (35%)
10 mm	1 (5%)	11 (24%)	24 (65%)
RFA power (W)	38.3 ± 4.4	42.2 ± 8.3	58.5 ± 15.4	<0.001
RFA time (min)	14.0 ± 9.7	21.1 ± 9.1	22.7 ± 8.4	<0.001
VRR at 6 mo (%)	56.3 ± 23.9	53.6 ± 25.7	58.4 ± 16.7	0.701
Composition-adjusted VRR (%)	51.7 ± 19.5	49.1 ± 23.3	64.1 ± 19.8	0.009

Values are expressed as number (percentage) or mean ± standard deviation. VRR: volume reduction rate.

Complications and regrowth

Two patients had minor complications in the consolidation phase. One had transient vocal cord palsy recovered spontaneously. The other had allergic reactions to the povidone-iodine prep. The median thyroid-stimulating hormone levels were 1.15 and 0.94 μlU/mL before and after RFA, respectively. There was no remarkable change in thyroid function after RFA.

At six months post-RFA, two patients (one in the consolidation phase and the other in the proficiency phase) underwent a second-session RFA for regrowth of thyroid nodules.

Discussion

Studies with long-term follow-up have shown that RFA leads to a durable shrinkage of benign thyroid nodules with an improvement in subjective symptoms [16,17]. Of interest, RFA is recommended as the first-line treatment modality for solid and predominantly solid thyroid nodules by the Korean guidelines [18]. In the long run, the VRR after RFA can be as high as 92.2% (95% confidence interval, 71.9% to 98.2%) [19]. However, it remains unknown whether the treatment efficacy was associated with operator experience.

In the present study, we used midterm volume reduction as a proxy for proficiency in thyroid RFA. Previous studies have shown that a significant volume reduction was evident six months after ablation, and results were stable over time [20]. Therefore, the VRR at six months was used as a measure of technical proficiency in this study. By extrapolation from other procedures [21], we reasoned that the efficiency of treatment might increase with case volume, indicating a learning curve effect. Indeed, we found that the adjusted VRR slowly increased over time (Figures 1 and 2). Our results are consistent with earlier findings that the treatment efficacy of RFA can be used as a proficiency marker [5].

In this consecutive, unselected series, more solid nodules were treated in the proficiency phase. It implies that the physician used more reasonable criteria for the selection of suitable candidates. Furthermore, the choice of RF electrode tip and energy was at the discretion of the operator. At the later phase, as more experience is gained, the operator might be more comfortable using longer tips and higher energy. The difference in choosing RF electrodes and settings may partly reflect maturity during the learning period. Intriguingly, although longer active tips of RF electrodes and higher power output were utilized, treatment time was longer in the later phase. A shorter procedure time may be considered a surrogate for proficiency with the technique and systematization of some operative procedures. By contrast, a longer RFA time might represent more challenging cases. The operator may spend more time attempting to reach a higher initial ablation ratio (IAR), which quantifies the amount of ablation after RFA and is highly correlated with volume reduction [22,23].

We did not choose the complication rate as an outcome measure. According to the literature, the average complication rate of thyroid RFA was around 4.6% (95% confidence interval, 2.5% to 8.4%) [19]. Nonetheless, the major complication was zero in our series, and the minor complications were uncommon. Additionally, in agreement with previous reports, we found that RFA had a minimal effect on thyroid function [24]. From a clinical standpoint, the regrowth rate may be adopted as an indicator of technical proficiency. It was estimated that the regrowth rate at 12 months ranged from 0% to 34% [17]. Nonetheless, the mean timing of regrowth was 39.9 months in a previous study [25]. Accordingly, a longer follow-up is necessary to determine whether proficiency with the technique could decrease the regrowth and/or re-ablation rate.

We recognize several limitations of the study. First, this was a single-center, single-operator study. Whereas it assures the validity and consistency of data, the generalizability of the findings is also limited. Second, the operator in this study is an endocrine surgeon, who has expertise mainly in neck and breast procedures. It is unclear whether the results can be extrapolated into nonthyroidal procedures. Additionally, the density of practice may vary considerably among institutions. It may have some impact on the learning curve. Finally, the information on IAR was not available in this study. We could not ascertain whether longer treatment time was associated with higher IAR, and therefore, higher VRR at midterm follow-up.

In summary, based on the CUSUM analysis, the learning curve for RFA of benign thyroid nodules consisted of three phases. The first 20 cases represented the initial learning phase, the next 45 cases represented the technical consolidation phase, and the 65th case was the turning point after which proficiency of the RFA technique was accompanied by increased volume reduction.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The publication was supported by a research grant [MOST-109-2314-B-195-018-MY3] from the Ministry of Science and Technology of Taiwan.