Hamstring tendon autograft better than bone patellartendon bone autograft in ACL reconstruction A cumulative meta-analysis and clinically relevant sensitivity analysis applied to a previously published analysis

Abstract

Background Current debate on treatment options for anterior cruciate ligament (ACL) reconstruction complicate the choice between hamstring and bone patellartendon bone autografts. We hypothesized a priori that cumulative meta-analysis (a form of sensitivity analysis) might show that the evidence for reduction of morbidity by hamstring grafts could have been reached at an earlier time. Furthermore, we hypothesized a priori that modern state-of-the-art hamstring graft fixation technique would give similar results regarding stability as bone-patellar tendon-bone autografts.

Methods We performed a cumulative meta-analysis and sensitivity analysis based on femoral graft fixation techniques to compare hamstring autograft and bonepatellar tendon-bone autografts in ACL reconstruction derived from a previously published meta-analysis.

Results Cumulatively, that hamstring autograft reduces anterior knee pain had already reached statistical significance in 2001 (relative risk 0.49 (95%CI: 0.32–0.76; p = 0.001, I2 = 0%)). The modern endobutton hamstring graft fixation technique (2 studies) yielded similar stability in the Lachman test as bone-patellar tendon-bone grafts, with a relative risk of 1.1 (95%CI: 0.82–1.5; p = 0.6, I2 = 0%). Exclusion of the endobutton group explains the increased laxity in the hamstring graft group.

Interpretation Cumulative meta-analysis strengthens the evidence for reduced morbidity using hamstring tendon autograft for anterior cruciate ligament reconstruction. Sensitivity analysis focusing on state-of-the-art hamstring graft fixation techniques further weakens the evidence that bone-patellar tendon-bone autografts provide better stability.

The number of meta-analyses is increasing in the surgical literature (Bhandari et al. 2001). In the orthopedic literature, evidence often needs to be obtained from randomized controlled trials with small sample sizes (Lochner et al. 2001). Metaanalyses provide a useful tool to give doctors up-to-date knowledge in order to help them in clinical decision making (Guyatt et al. 2002). Surgical trials differ from pharmaceutical trials; in the latter, the effects of treatment providers on the therapy play a less important role (Boutron et al. 2003, Devereaux et al. 2005). Surgical technique is the caveat (Devereaux et al. 2005).

Recently, the importance of a well-conducted meta-analysis for a commonly discussed orthopedic dilemma was elucidated (Biau et al. 2006). This study showed lower morbidity for patients with an anterior cruciate ligament reconstruction using a hamstring autograft. Although the authors conducted sensitivity analysis in their meta-analy-sis, we think this tool was not used optimally. We will illustrate this with the data available from their manuscript and provide stronger evidence for their conclusions. We hypothesized a priori that cumulative meta-analysis (a form of sensitivity analysis) could show that their conclusion, the reduction in morbidity by the use of hamstring grafts, could have been reached at an earlier time. Furthermore, we hypothesized a priori that modern state-of-the-art hamstring graft fixation technique would have similar effects on stability as bone-patel-lar tendon-bone autografts, thus, questioning the poor evidence that patellar tendon autografts offer better stability. The rationale for choosing this subgroup was adapted from previous studies claiming superior femoral fixation with endobuttons and subsequent superior stability (Prodromos et al. 2005). Thus, our hypothesis was set a priori. We hypothesized that sensitivity analysis could clarify this statement. Accordingly, we illustrate here that surgical meta-analysis can benefit from sensitivity analysis based on previously published differences in surgical techniques used in the primary studies.

Methods

Data abstraction and quantitative data synthesis

We abstracted the data as presented in the published meta-analysis by Biau et al. (2006). We conducted a cumulative meta-analysis using subgroups in RevMan 4.2.7 software (Cochrane Collaboration) and analyzed the data with RevMan Analysis Software (version 1.0.2) with the same statistical methods as described in the source meta-analysis. Next, we performed sensitivity analysis to assess the size of the treatment effect on the outcome of the Lachman test for studies with modern hamstring autograft fixation techniques (endobutton) versus studies using older fixation techniques (interference screws, staples, plates, or washers).

For the test of heterogeneity between studies, we calculated the I²statistics, which describe the percentage of total variation across studies that is due to heterogeneity rather than chance (Higgins et al. 2003). A value of 0% for I²indicates no observed heterogeneity. As the value of I²increases, the heterogeneity between studies becomes more evident. We also tested the heterogeneity of results across the studies using a Cochran Q test, and a p-value of greater than 0.1 was taken to indicate no significant heterogeneity (Higgins et al. 2003).

Results

Cumulative meta-analysis on anterior knee pain (Figure 1)

In the paper by Biau and co-workers, we identified 14 primary studies with data available on anterior knee pain from 1991 to 2005 (Marder et al. 1991, Aglietti et al. 1994, 1997, 2004, Callaway et al. 1994, Aune et al. 2001, Eriksson et al. 2001a, b, Feller et al. 2001, Ropke et al. 2001, Beynnon et al. 2002, Shaieb et al. 2002, Ejerhed et al. 2003, Feller and Webster 2003, Hantes et al. 2004, Ibrahim et al. 2005, Laxdal et al. 2005, Biau et al. 2006). Our cumulative meta-analysis showed that statistical significance that hamstring autograft reduces anterior knee pain had already been reached in the year 2001. The relative risk in 1997 was 0.60 (95%CI: 0.26–1.4; p = 0.2; test for heterogeneity: Chi²= 0.09, df = 1 (p = 0.8), I²= 0%) based on 3 available studies at that time. In 2001, 3 further studies were available, resulting in a relative risk of 0.49 (95%CI: 0.32–0.76; p = 0.001; test for heterogeneity: Chi²= 1.1, df = 4 (p = 0.9), I²= 0%). In the following years, 8 other studies became available without changing the conclusion reached in 2001. In 2005, the relative risk was 0.57 (95%CI: 0.44–0.74; p < 0.001; test for heterogeneity: Chi2 = 5.05, df = 11 (p = 0.9), I2 = 0%). The tests for heterogeneity did not reach statistical significance over the years.

Figure 1. cumulative meta-analysis.

Sensitivity analysis of the hamstring tendon autograft femoral fixation technique (Figure 2)

The modern endobutton hamstring graft fixation technique (2 studies) yielded similar stability in the Lachman test as bone-patellar tendon-bone grafts, with a relative risk of 1.1 (95%CI: 0.82–1.5; p = 0.6; test for heterogeneity: Chi²= 0.06, df = 1 (p = 0.8), I²= 0%). Exclusion of the endobutton group explains the increased laxity in the hamstring graft group. The relative risk was 1.3 (95%CI: 1.1–1.7; p = 0.03; test for heterogeneity: Chi²= 2.3, df = 4 (p = 0.7), I²= 0%) when comparing 6 studies using older fixation techniques. Thus, the studies using older and possibly less optimal fixation techniques made significantly greater contributions to the overall improved stability of bone-patellar tendon-bone grafts, with a relative risk of 1.2 (95%CI: 1.0–1.5; p = 0.04; test for heterogeneity: Chi²= 3.2, df = 6 (p = 0.8), I²= 0%). The tests for heterogeneity did not reach statistical significance.

Figure 2. Sensitivity analysis for modern versus older fixaion techniques.

Discussion

Our analysis shows that sensitivity analysis can further clarify conclusions from meta-analysis based on an a priori hypothesis. Especially in meta-analysis involving surgical trials, differences in surgical techniques need to be accounted for and can be analyzed in subgroups as shown in our example. Our sensitivity analysis on fixation technique further weakens the evidence that bonepatellar tendon-bone autografts give better stability when using state-of-the-art femoral fixation techniques for the hamstring tendon autograft. Our cumulative meta-analysis further strengthens the evidence that hamstring tendon autografts reduce anterior knee pain.

Combining knowledge of surgical techniques and sensitivity analysis in meta-analyses can strengthen the clinical usefulness of the results. Prior to pooling the data statistically, reviewers need to identify potential sources of inter-study differences (Bhandari et al. 2002). Although Biau did not find statistical heterogeneity, clinically relevant differences between studies did skew the results. The cooperation between surgeons (clinicians) and epidemiologist will facilitate development of clinically important a priori hypotheses to explain differences between studies. Surgical techniques with seemingly small differences may infiuence the results when data are pooled without sensitivity analysis. This requires further exploration.

Of course, our study has limitations. Firstly, the primary data were abstracted from a meta-analy-sis that abstracted the data from primary studies; this may have increased the chance of human error. Secondly, there were only two studies included in the analysis of endobutton fixation subgroup that might have underestimated the treatment effect of this technique. Future randomized controlled trials of the endobutton technique and updated metaanalysis will clarify this issue. Furthermore, we analyzed the data available for the Lachman test. This test is only one aspect in the clinical evaluation of the lax knee, and may be irrelevant as a patient-important outcome. Unfortunately, data on validated patient-orientated outcome instruments are unavailable.

Our study does, however, illustrate the usefulness of sensitivity analysis as an addition in the complicated nature of surgical meta-analysis. Our findings do not advocate overzealous utilization of subgroup or sensitivity analysis without an a priori hypothesis based on clinical expertise. It must be stressed that post hoc “data dredging” can lead to fiawed conclusions and can mislead readers (Guyatt et al. 2002).

In summary, cumulative meta-analysis strengthens the evidence for reduced morbidity using hamstring tendon autograft for anterior cruciate ligament reconstruction. Sensitivity analysis focusing on state-of-the-art hamstring graft fixation techniques further weakens the evidence that bonepatellar tendon-bone autografts provide better stability.

Contributions of authors

RWP: initiated and designed the study, abstracted the data, participated in the data analysis, and drafted the manuscript. FF: analyzed the data and edited the manuscript. MB: codesigned the study and edited the manuscript. All authors approved the final version of the manuscript.