Prospective study for establishing minimal clinically important differences in patients with surgery for lower extremity metastases

Abstract

Background

The clinical relevance of patient-reported outcomes score changes is often unclear. Especially in patients undergoing surgery due to lower extremity metastases – where surgery is performed in the palliative setting and the goal is to optimize functional mobility, relieve pain and improve overall quality of life. This study assessed the minimal clinically important difference (MCID) of Patient-Reported Outcomes Measurement Information System (PROMIS) Pain Interference, Cancer-specific Physical Function, and Global (Physical and Mental Health) in patients treated surgically for impending or completed pathologic fractures.

Methods

Patients undergoing surgery for osseous metastasis of the lower extremity because of an impending or completed pathologic fracture were consecutively enrolled in this tertiary center study. Patients completed the three PROMIS questionnaires preoperatively (n = 56) and at postoperative follow-up (n = 33) assessment one to three months later. Of the 23 patients that did not complete the postoperative survey, 5 patients died within 1–3 months and 18 patients were alive at 3-months but did not respond or show up at their postoperative consult. Thirty-one patients (94%) of the 33 included patients reported at least minimal improvement and two patients (6.1%) no change 1–3 months after the surgery based on an anchor-based approach.

Results

The PROMIS MCIDs (95% confidence interval) for Pain Interference was 7.5 (3.4–12), Physical Function 4.1 (0.6–7.6), Global Physical Health 4.2 (2.0–6.6), and Global Mental Health 0.8 (−4.5–2.9).

Conclusion

This prospective study successfully defined a MCID for PROMIS Pain Interference of 7.5 (3.4–12), PROMIS Physical Function of 4.1 (0.6–7.6), and Global Physical Health of 4.2 (2.0–6.6) in patients with (impending) pathological fractures due to osseous metastases in the lower extremity; no MCID could be established for PROMIS Global Mental Health. Defining a narrower MCID value for each subpopulation requires a large, prospective, multicenter study. Nevertheless, the provided MCID values allow guidance to clinicians to evaluate the impact of surgical treatment on a patient’s QoL.

Level of evidence

Level II Diagnostic study

Keywords:

• Minimal clinically important difference
• anchor-based
• PROMIS
• lower extremity metastases
• pathologic fracture
• prophylactic fixation

Introduction

Articulating the potential benefit of surgery can be difficult yet a definition of success after surgery is important to help set expectations of the outcome for the patient, family and the clinician. This definition of ‘success’ has progressed in the field of orthopedic surgery from disease centric outcomes such as bony union, joint motion and survival, to patient centric outcomes such as quality of life (QoL) assessments completed by the patient [1]. Quality of life is particularly relevant when the quantity of life is diminished by terminal illness such as patients with (impending) pathological fractures due to lower extremity metastases – where surgery is performed in the palliative and prophylactic setting and the goal is to optimize functional mobility, relieve pain and improve overall QoL [2,3].

The minimally clinical important difference (MCID) is the threshold for when a patient has experienced a minimal clinically relevant change. Thus, the MCID defines that change in an outcome score that is perceived as beneficial [4,5]. For example, a patient changing 1 point on a 10-point VAS score after a treatment may be significant (especially in large trials) but would not be considered clinically important by most patients [6]. The Patient-Reported Outcomes Measurement Information System (PROMIS) Global, PROMIS Cancer-specific Physical Function, and PROMIS Pain Interference are three lower extremity measures that have been scored for a wide variety of oncologic conditions [7,8]. However, no MCIDs are determined for these questionnaires in patients treated surgically for metastatic bone disease of the lower extremity, specifically impending pathologic fractures. As stated earlier, this information is important for this population as surgery is often indicated for palliative purposes. Apart from managing expectations for clinicians and patients during the treatment course, establishing the MCID for PROMIS is expected to aid in the assessment of clinical significance of QoL changes in clinical trials and sample sizes estimates for future studies.

The purpose of the present study was to establish the MCID value of three PROMIS questionnaires – Pain Interference, Cancer-specific Physical Function, and Global (Physical and Mental Health) – in patients treated surgically for impending or completed pathologic fractures secondary to metastatic bone disease of the lower extremity.

Methods

This prospective cohort trial was approved by our institutional review board prior to study initiation. All patients attending the orthopedic oncology clinic in a tertiary hospital were approached for the study in the consecutive months between April 1^st 2017 and December 18^th 2018. Inclusion criteria for patients were (1) 18 years of age or older, (2) surgical procedure for impending or completed pathologic fracture, and (3) metastatic bone lesion of the lower extremity. Metastatic disease was defined as metastases from solid primary tumors, multiple myeloma, or lymphoma as confirmed by pathology reports [9]; lower extremity was considered as the femur, tibia, and fibula. Exclusion criteria included (1) under 18 years of age, (2) a lack of English proficiency or mental status prohibiting consent for research participation, (3) primary bone lesions, and (4) revision procedures, defined as any subsequent procedure after the index surgery of the pertaining metastatic lesion. Patients to be enrolled in this study would otherwise be undergoing lower extremity surgery, regardless of their participation in this study.

All enrolled subjects were asked to complete the same battery of QoL questionnaires before surgery and one to three months after surgery at their post-operative follow-up appointment. The post-operative survey consisted of the same questionnaires, with the addition of a seven-point Likert global satisfaction anchor question, which was used as the anchor for the assessment of the patients’ global perceived effect to calculate the MCID (Supplementary Appendix 1) [4]. Necessary information on clinical characteristics and QoL was obtained from electronic medical records.

PROMIS measures

Three patient reported outcomes were used from PROMIS: (1) PROMIS Pain Interference SF 6a v1.0, (2) PROMIS Global 10 v1.1, and (3) PROMIS Cancer-specific Physical Function v1.1 (Supplementary Appendix 2). All these instruments have been previously validated [8]. The PROMIS Pain Interference short form 6a (six items) measures the effects of pain over the last seven days on relevant aspects of one’s life completed by the patient. The PROMIS Global Health short form v1.1 (eight items) allows for the assessment of Mental Health (four items) and Physical Health (four items). After previous comparison of several questionnaires for functional outcome in patients with musculoskeletal tumors by our group, the PROMIS Cancer-specific Physical Function was established as the most useful. This was due to ‘its reliability over a wide range of ability levels, validity, brevity, and good coverage through computerized adaptive testing (CAT)’ [7]. CAT was used to allow for more efficient physical function assessment [10]. All questionnaire answers were transformed to validated scores with the following ranges: Pain Interference (41–76), Physical Function (15–73), Global Physical Health (16–67), and Global Mental Health (21–67) [8]. A higher score indicates more of the symptoms or health status measured.

Statistical analysis

Descriptive statistics were used to characterize the study population. The included patients (those who completed the questionnaires at both timepoints), patients that did not complete the preoperative questionnaire, and patients that did not complete the postoperative questionnaire were compared for baseline differences with the Kruskal–Wallis test for continuous and Fisher’s exact test for categorical variables. The following baseline characteristics were compared: gender, age, body mass index, race, Eastern Cooperative Oncology Group (ECOG) performance status established by a clinician at the preoperative and postoperative follow-up visit one to three months after surgery [11], primary tumor, presence of multiple bone metastases, presence of visceral metastases, impending or complete pathologic fracture, history of chemotherapy or local radiotherapy to lower extremity metastasis (yes/no), Katagiri survival score [12], Modified Charlson Comorbidity score [13], and time in years between diagnosis of primary tumor and date of surgery. All variables were collected by a research fellow blinded to the outcome and study population. Graphical examination was used to evaluate performance of the anchor question. Despite being potentially contacted anywhere between one and three months, each patient contributed a single datapoint to the postoperative outcome measure. The questionnaires were administered by REDCap [14]. All statistical analyses were performed using Stata 13.0 (StataCorp LP, College Station, TX, USA).

MCID methods

The anchor-based approach was used to determine the MCID and 95% confidence intervals (CI) were provided [4,6]. This method compares the mean change between baseline and postoperative outcome measure to a second, external ‘anchor’ question as a reference. On the seven-item anchor question of response to surgery, a response of ‘a little better’ or ‘better’ was considered to be a minimal clinical improvement based on clinical expertise by the treating senior orthopedic surgeons (Supplementary Appendix 1). This categorization of the anchor question created two groups of patients experiencing (1) worsening or no change (‘much worse’, ‘somewhat worse’, ‘a little worse’, and ‘the same’), and (2) at least minimal improvement (‘a little better’, ‘somewhat better’, and ‘much better’) based on the anchor question after surgery. Naturally, only questionnaires with complete pre- and postoperative data can be included in this method. To provide supporting data that the MCID determined by the anchor-based approach exceeded the measurement error, the anchor-based results were compared to the mean baseline and mean postoperative standard error of measurement (SEM) [15,16]. The SEM is a statistical measure that represents the reliability of patients’ scores on a QoL assessment tool – a patient rating the depression level as 7 of 10 and later as 8 of 10, without any actual change in the perceived depression level. The calculated anchor-based MCID value being higher than this SEM, reflects that the MCID value is not due to chance or random variation, but due to a real change (e.g., surgical treatment).

To account for the sample and method dependent variations, we provided a range of MCID values with corresponding percentages of the total score in the Supplementary material by including the distribution-based approach [17], rather than an absolute single threshold from the anchor-based method [18]. Both the anchor and the distribution-based approach have limitations and no consensus exists on the preferred methodology [6]. Therefore, both results are provided in the supplemental material until better uniform guidelines exists for identifying an appropriate MCID analysis. Sample size limited a sub analysis for whether MCID estimated values differed according to baseline clinical or demographic characteristics.

Results

Sixty-five consecutive patients were approached to participate in this study. Six (9.2%) patients declined participation and three (4.6%) did not complete the preoperative surveys due to logistical issues, leaving 56 (86%) patients that completed the preoperative survey. Thirty-three of the 56 (59%) patients completed the postoperative survey: 15 (27%) did not respond or show up at their postoperative consult but were alive at three months, five (9%) died within 1–3 months, and three (5.4%) declined participation. The 33 patients who completed both surveys were included for MCID analysis for whom no missing values were recorded (Figure 1).

Figure 1. Flowchart of enrolled patients.

The median patient age was 68 years (interquartile range [IQR] 59 − 70, Table 1). The indication for surgery was an impending pathologic fracture in 27 (82%) and pathologic fracture in 6 (18%). Additional analyses for baseline characteristics were provided between the non-preoperative responders (n = 9), non-postoperative responders (n = 23), and included patients (n = 33). Non-responders had a higher Katagiri survival score and lower three- and six-month survival rates (Supplementary Appendix 3).

Table 1. Sociodemographics and clinical characteristics of the included patients (n = 33).

Variables	Median (IQR)
Age	68 (59–70)
BMI	27 (24–31)
Modified Charlson Comorbidity Index	6 (6–6)
Duration of primary diagnosis until metastatic operation (years)	2.8 (0.5–8.2)
	n (%)
Men	14 (42)
Ethnicity
Caucasian	32 (97)
Hispanic	1 (3.0)
Preoperative ECOG score
0–2	29 (88)
3–4	4 (12)
Primary tumor
Breast cancer	7 (21)
Kidney cancer	6 (18)
Lung cancer	4 (12)
Thyroid cancer	2 (6.1)
Melanoma	2 (6.1)
Myeloma	2 (6.1)
Other^a	10 (30)
Visceral metastases
Yes (lung, liver, or brain)	15 (45)
No	18 (55
Multiple bone metastases
Yes	17 (52)
No	16 (48)
Previous local radiotherapy
Yes	5 (15)
No	28 (85)
Previous systemic therapy
Yes	23 (70)
No	10 (30)
Katagiri survival score^b
0–3	18 (55)
4–7	14 (42)
6–10	1 (3.0)
Fracture type
Impending	27 (82)
Pathologic	6 (18)
Fracture location
Femur	31 (94)
Tibia	2 (6.1)
Operative treatment type
Intramedullary nailing	17 (52)
Endoprosthetic reconstruction	13 (39)
Dynamic hip screw	2 (6.1)
Plate-screw fixation	1 (3.0)
Postoperative
Reoperation	2 (6.1)
ECOG at 1–3 months
Worsened	3 (9.1)
Same	20 (61)
Improved	10 (30)
Mobility at 1–3 months
Unassisted	8 (24)
Cane/crutch	3 (9.1)
Walker	22 (67)
Survival^c
3-months	31 (94)
6-months	24 (73)

IQR: interquartile range; BMI: body mass index; ECOG: Eastern Cooperative Oncology Group.

^aThis category includes colorectal cancer 1 (3.0%), cholangiocarcinoma 1 (3.0%), chordoma 1 (3.0%), lymphoma 1 (3.0%), epithelioma 1 (3.0%), ovarian cancer 1 (3.0%), prostate cancer 1 (3.0%), head and neck cancer 1 (3.0%), sarcoma 1 (3.0%), and unknown 1 (3.0%).

^bSurvival prediction at 12 months for patients with bone metastases: low-risk group (score of ≤3), survival rate > 80%; the intermediate-risk group (score of 4–6), survival rate 30–80%; and the high-risk group (score of 7–10), survival rate ≤10%.

^cLoss to follow-up at 3 months was 0 patients, and at 6 months 2 patients (6.1%).

MCIDs

For the 33 patients that completed both pre- and postoperative survey, 70% patients (n = 23) reported ‘much better’, 15% patients (n = 5) ‘somewhat better’, 9.1% patients (n = 3) ‘a little better’, 3.0% (n = 1) patients ‘a little worse’, and 3.0% patients (n = 1) reported being ‘much worse’ at 1–3 months after surgery based on the anchor question. This corresponds with 31 patients (94%) reporting at least minimal improvement (e.g., ‘a little better’ or better) and two patients (6.1%) reporting deterioration (Figure 2). The score distribution for the questionnaires at both time points, the mean change between the pre- and postoperative questionnaire, and the anchor-based MCID were determined (Table 2).

Figure 2. Improvement in percentages according to response on a seven-point Likert anchor question (n = 33). The cutoff considered to be minimally improved is at “A little better”.

Table 2. Comparison of pre- and postoperative scores and MCID estimates.

	Mean (SD)		Mean change (SD)	Anchor-based MCID (95% CI)
	Preoperative Postoperative
PROMIS Pain Interference^a	65 (8.6)	58 (9.1)	−7.4 (11)	7.5 (3.4–12)
PROMIS Physical Function^b	30 (7.4)	34 (8.5)	4.5 (9.4)	4.1 (0.6–7.6)
PROMIS Global Physical^b	35 (6.9)	39 (7.7)	4.4 (6.2)	4.3 (2.0–6.6)
PROMIS Global Mental Health^b	47 (7.6)	46 (9.1)	−1.0 (9.7)	0.8 (−4.5–2.9)

MCID: minimal clinically important difference; SD: standard deviation; CI: confidence interval; PROMIS: Patient-Reported Outcomes Measurement Information System. No missing values were recorded. The pre- and postoperative mean are based on 33 patients; and the anchor-based MCID on 31 patients (the patients that improved based on the anchor question).

^aHigher score represents a worse health status; a greater degree of the pain interference symptoms is present.

^bHigher score represents better health status.

The MCID values calculated with the anchor-based approach in patients undergoing surgery for an (impending) pathological fracture due to lower extremity bone metastases were: PROMIS Pain Interference 7.5 [95% CI: 3.4−12], PROMIS Physical Function 4.1 [95% CI: 0.6−7.6], PROMIS Global Physical Health 4.3 [95% CI: 2.0−6.6], and PROMIS Global Mental Health 0.8 [95% CI: −4.5−2.9]. All MCIDs values, except Global Mental Health, were greater than the mean of SEM both at preoperative and postoperative assessment, indicating that the estimate of MCID exceeded measurement error. In other words, the established MCIDs, except Global Mental Health, in the PROMIS scores were deemed to be due to surgical treatment, not random variation. Combining the anchor-based and distribution-based approach, MCID ranges were as follows: PROMIS-Pain Interference, 2.9 to 7.5 points (8.3–21% of the total score); PROMIS-Physical Function, 2.5 to 4.2 points (4.3–7.2% of the total score); PROMIS Global Physical Health, 2.1 to 5.9 points (4.0–11% of the total score); and PROMIS Global Mental Health, 0.8 to 6.0 points (1.7–13% of the total score; Supplementary Appendix 4).

Discussion

Patients undergoing surgery for impending or pathologic fracture of lower extremity bone metastases often have poor QoL due to invalidating ambulatory status and pain [2]. Understanding the MCID allows the determination of the benefits of surgical treatments and interpretation of group-level data as opposed to assessing individual patients when conducting research. The anchor-based approach determined that a decrease of the following values corresponds to a MCID: PROMIS-Pain Interference 7.5 [95% CI: 3.4–12], PROMIS-Physical Function 4.1 [95% CI: 0.6 − 7.6], and PROMIS Global Physical Health 4.3 [95% CI: 2.0–6.6]. No MCID could be established for PROMIS Global Mental Health. To our knowledge, this is the first study establishing the MCID for three PROMIS questionnaires in patients surgically treated for (impending) pathologic fractures due to osseous metastases of the lower extremity [6].

Only one other similar study by Yost et al. [15] established MCIDs in the 10-item PROMIS Physical and Pain Interference for patients with advanced cancer-stage (III and IV) undergoing chemotherapy, radiation therapy, or both. This study included 101 patients with 23 clinically relevant, self-reported anchors and determined that a 4.0–6.0 change in both questionnaires represented clinically meaningful change. Although most similar to this study, the range of change in Yost et al. [15] is lower than our current findings. This could be explained due to differences in clinical anchors used, number of included patients, different item versions of the questionnaires and patient population (i.e., severity of treatment; surgical treatment versus chemotherapy and/or radiation). The MCIDs in this study for the pain and physical surveys were relatively high, which is reflected by the fact that these patients are experiencing a major decrease in mobility and an increase in pain interference scores at baseline due to their (impending) pathologic fracture [2]. Thus, by performing surgical stabilization the impact on physical function and pain interference can greatly improve, giving the large changes that were observed in this cohort. This would be beneficial in a patient population that has a 3-month survival rate of 50% as immobility and pain are the greatest concerns on patients’ minds [1,19,20]. Also, the MCID values may be narrowed toward a smaller value in the wide confidence interval range when the sample size is increased.

The anchor-based approach did not yield a usable estimate for the MCID of the PROMIS Global Mental Health. The MCID estimates included zero and was not different than the SEM. This may be explained because changes in mental QoL often occur in the earlier phase of diagnosis [21]. Over time, patients can adapt to their disease, which can contribute to the stabilization of mental QoL scores over time regardless of a treatment or not [15]. In particular, patients undergoing surgical treatment for bone metastases are in a more advanced stage of their disease and their mental health status could have changed – to their current mental status – in an earlier stage of the disease course (patients had a median of 2.5 years (IQR 0.5–8.2) from primary tumor diagnosis until the surgery related to this study) [21]. Furthermore, the included patients often had concurrent treatments for various other sequelae of their primary or metastatic disease, which could have impacted their mental status when contributing to the surveys. As a result, the favorable outcome of the surgery in terms of physical function and pain interference related to this study may not be reflected in a changed mental status, which could explain the non-detectable MCID. This has been observed in prior oncology studies where despite less pain and overall better QoL, patients experience similar mental health pre- and post-treatment [22,23].

This study has certain inherent limitations. First, the sample size is relatively small, though most prospective skeletal metastases series are small because of the low incidence and poor survival [19,24]. This is noted in the wide confidence interval, which is comparable with similar sample size studies including oncologic patients [25,26]. Analysis of a larger, multicenter cohort can narrow this interval down. Despite the wide interval of MCIDs, a MCID greater than the SEM was observed in three of the four questionnaires. This reflects that the MCID is not due to chance or random variation, but due to a real change (e.g., surgical treatment). One may argue that this real change can also be due to confounding factors and not surgical treatment. The sample size limited a sub analysis for whether MCID estimated values differed according to baseline clinical or demographic characteristics. For example, completed pathological fractures are known to be associated with worse outcomes as compared to impending fractures [27,28]. This may correspond with smaller MCIDs as these patients experience less postoperative improvement in pain and mobility as compared to impending pathologic fractures.

Second, 32 of the 65 (49%) eligible patients declined, had incomplete data, were lost to follow-up, or died, and this could introduce bias into our results (Supplementary Appendix 3). A retrospective analysis of the causes of patients not completing the surveys preoperative or postoperative demonstrated that their Katagiri survival score and three and six-month survival rate were worse than the included patients; the other characteristics such as preoperative ECOG score, ECOG improvement, and mobility at one to three months were comparable. These non-respondents may have experienced more severe postoperative deterioration compared with the included patients, and this could explain our low percentage of non-improved patients (6.1%; 2 of 33). However, this did not affect our anchor-based MCID analysis since this approach only takes into account the patients that report an improvement on the anchor question [4]. Despite having no effect on our MCID analysis, the low number of non-improved patients withheld us from using a receiver-operating characteristics curve to develop a MCID threshold. However, differences were found between three of the four MCID values and the mean of SEM, which supports validity of our MCID values since the MCIDs estimates exceeded measurement error [4].

Third, the choice of follow-up intervals may have affected the MCID value – was to three months an appropriate timepoint to determine MCID for metastasis surgery, as compared to say, for example six or 12 months? Since patients with extremity metastatic disease undergoing surgery have a three-month survival rate of 50%, we chose the follow-up intervals at one to three months to minimize loss to follow-up prior to potential marked improvement and allow for a rehabilitation period following the surgery [19,24]. Also, we believe one to three months is an appropriate time point as these patients need to see improvement in a short time horizon that is proportional to their life expectancies and a recent study reports that QoL is restored within 6 weeks in patients with femoral metastases undergoing endoprosthesis [29]. Additional analysis showed no difference between the patients who completed the postoperative scores at one month and three months in different amounts of improvement on PROMIS scores. Fourth, this was a tertiary institution study, in a single region of the country, and may not reflect practice in other regions. No uniform guidelines exist for treating impending or completed pathological fractures and this can result in different surgical techniques, timing of treatment, and philosophy on whether or not to treat in this complicated patient population. Fifth, an anchor-based approach was used to determine the MCID, which is open to criticism [30]. The rating scale of the anchor question requires an arbitrary choice of cutoff points to define degrees of clinical improvement and may be affected by recall bias [31]. It also does not take into account the statistical characteristics of a group’s baseline, which the distribution-based approach does [4]. At present, both the anchor and the distribution-based approach have limitations and no consensus exists on the preferred methodology [6]. Both results are provided in the supplemental material until better uniform guidelines exists for identifying an appropriate MCID analysis. However, we recommend using the anchor-based results to ensure consistent application of the MCID in practice and homogeneity between studies [4,15,32,33].

Future research – in order to address the above limitations – should include a larger sample from a multicenter cohort and provide stratified QoL results by demographic and clinical characteristics such as primary tumor type, preoperative ECOG and comorbidities in order to elucidate a more definite MCID for each subpopulation. Nevertheless, this prospective study established valid and usable MCIDs through a rigorous anchor-based method. To our knowledge, this is the first prospective study determining MCIDs in patients treated surgically for impending or completed pathologic fractures secondary to metastatic bone disease of the lower extremity. As recently recommended by Janssen [34], prospective studies such as this current study that ‘include homogenous samples like lower-extremity bone metastases, and assess multiple QOL and physical function questionnaires to calculate corresponding MCIDs, focusing on anchor-based MCIDs’ are necessary to improve and evaluate treatment goals in this complicated patient population. By using the provided MCIDs as benchmark, this study provides valuable information in managing expectations for clinicians and patients during the treatment course and the assessment of clinical significance of QoL changes in clinical trials.

In conclusion, this prospective study successfully defined a MCID for PROMIS Pain Interference of 7.5 (3.4–12) points, PROMIS Physical Function of 4.1 (0.6–7.6) points, and Global Physical Health of 4.2 (2.0–6.6) points in patients with impending or complete pathological fractures due to osseous metastases in the lower extremity; no MCID could be established for PROMIS Global Mental Health. These MCID values can be used for managing expectations for clinicians and patients during the treatment course, interpreting group-level data as opposed to assessing individual patients when conducting research and aid in the assessment of clinical significance of QoL changes in clinical trials. Future research should include a large, multicenter and provide stratified MCID results by demographic and clinical characteristics such as primary tumor type, fracture type and comorbidities in order to elucidate a more definite MCID for each subpopulation.

Ethical approval

This study was approved by our institutional review board.

Acknowledgements

The authors would like to thank D. Hayden from Harvard Biomedical Consultation for his support in the statistics.

Disclosure statement

Each author certifies that he or she has no commercial associations (e.g., consultancies, stock ownership, equity interest, patent/licensing arrangements, etc.) that might pose a conflict of interest in connection with the submitted article.