Prospective evaluation of relationships between radiotherapy dose to masticatory apparatus and trismus

Abstract

Aims: This feasibility study aimed to identify relationships between radiation doses to the masticatory apparatus as a combined block or as individual subunits with changes in trismus following radiotherapy.

Material and methods: Twenty patients from a single center were recruited prospectively as part of a randomized trial comparing proactive exercises in the management of trismus. Patients with stage III/IV oral cavity or oropharyngeal squamous cell cancers received intensity-modulated radiotherapy with concurrent systemic therapy. All patients had trismus prior to radiotherapy. Maximal inter-incisor distance (MID) was measured pre- and 6 months from the start of radiotherapy. Bilateral muscles of mastication: medial and lateral pterygoids (MP and LP), masseters (M), temporalis (T), temporomandibular joint (TMJ) were contoured on CT images. The block comprised all muscles excluding the TMJ below the orbital floor. Mean dose, equivalent uniform dose (EUD) and V35–V60 Gy were compared with change in MID.

Results: In six patients, the MID deteriorated at 6 months from the start of radiotherapy compared with 14 whose MID improved. No significant association was observed between age, gender, smoking, alcohol status, exercise compliance, cisplatin, tumor site, stage, V35–V60 Gy or EUD with change in MID. A clinical outlier was excluded. Without the outlier (n = 19), a significant association was seen between mean dose and change in MID at 6 months for the ipsilateral block (p = .01), LP (p = .04) and M (p < .01). All patients where trismus deteriorated at 6 months received mean doses >40 Gy to the block.

Conclusion: Higher mean radiation doses to the ipsilateral block, LP and M were significantly associated with deterioration in trismus. Limiting dose to these structures to ≤40 Gy for tumors not invading the masticatory muscles may improve treatment-related sequelae. The ipsilateral block, LP and M should be studied further as possible alternative avoidance structures in radiotherapy treatment planning.

Introduction

Trismus or ‘locked jaw’ is defined as an ‘inability to fully open the mouth’. It is a common treatment-related effect in patients with head and neck cancer resulting in significant morbidity. The prevalence of trismus varies considerably. This variability is due to patient and physician under-reporting, study differences in clinicopathologic and treatment factors, and the lack of a universally agreed definition [1–3].

Trismus is caused by impaired function of the muscles of mastication secondary to benign or malignant processes and their associated treatment. The muscles include: medial (MP) and lateral pterygoids (LP), masseter (M), temporalis (T) as well as the temporo-mandibular joint (TMJ). The paired muscles of mastication assist with chewing through their attachment onto the mandible. The MP, M and T close the jaw whilst the LP opens the jaw. Injury to these muscles will result in reduced function and range of mandibular motion leading to reduced nutrition, impaired oral hygiene and difficulty speaking [4]. The most widely used definition for trismus is a maximal inter-incisor distance (MID) of ≤35 mm [5].

There are a number of patient, tumor and treatment-related factors which contribute to trismus [1,6]. The radiation-induced pathogenesis involves fibrosis and atrophy of the mastication muscles secondary to ischemia [7]. The effect of radiation is not immediate but one that progresses over months to years following treatment. The severity of radiation-induced trismus appears to be related to the total dose received and the volume of tissues within the radiation field [8]. Significant dose–response relationships have been found but the literature varies considerably on what dose constraints should be used. There is also no consensus as to which of the mastication muscles should be defined as an organ at risk with a view to avoidance planning [6,9,10]. Rao and van der Molen [11,12] reported that doses to the pterygoids and masseter muscles were the most robust predictors for the development of trismus whilst other studies showed doses to the TMJ and the pterygoids were important [11–13]. A limitation of published studies is the use of retrospectively collected data and mouth opening measurements based on dichotomized data [14,15].

Given these limitations a feasibility study was carried out using prospectively collected data to identify and compare different dose parameters to a combination of these muscles defined as a block and as individual mastication muscles. Mouth opening measurements on a continuous scale were used to increase statistical power.

Material and methods

Patients

The study population comprised patients recruited in Manchester into the multi-center phase 3 randomized controlled Trismus trial [16]. The trial compared proactive exercises using therabite (platon medical) versus standard wooden spatulas in patients with stage III/IV oral cavity/oropharyngeal cancers to improve trismus. Patients randomized to either intervention were asked to follow a protocol of exercises beginning 2–3 weeks prior to starting radiotherapy. These included five sessions per day of five openings/closing per session with a 30 s stretch for each opening [17]. Patients had stage III/IV squamous cell carcinomas of the oral cavity/oropharynx and were treated with concurrent radiotherapy and systemic therapy to the primary tumor from February 2013 to January 2015. Of the 22 patients enrolled in Manchester, two patients were excluded due to undergoing primary surgery with mandibulectomy. The remaining 20 underwent ipsilateral, bilateral neck dissections or hemi-glossectomies as shown in Supplementary Table S1.

Patients received 60–66 Gy in 30 fractions (2.0–2.2 Gy per fraction) over 6 weeks. A rotational-based intensity-modulated radiotherapy treatment (IMRT) plan was calculated using the pinnacle treatment planning system (Pinnacle version 9.6; Philips Radiation Oncology Systems, Andover, MA) with target delineation performed on axial CT images. No dose constraints were applied to the muscles of mastication. All patients had concurrent chemotherapy with cisplatin or cetuximab if cisplatin was contraindicated.

Evaluation of trismus and dose

Patients enrolled in the trial had trismus defined as a sense of jaw tightening self-reported by each patient prior to radiotherapy. For this study, MID measurements were taken prospectively at baseline and 6 months from the start of radiotherapy. The averages of two MID measurements were taken at baseline and at 6 months. The change in MID for each patient was calculated.

The block was defined as the MP, LP, M and T muscles below the orbital floor to exclude the cranial component of the T. Mean dose, equivalent uniform dose (EUD) and V35–V60 Gy were calculated for the contoured volumes of the block and each individual muscle on the ipsilateral and contralateral side. The EUD is a method of reporting radiotherapy dose distributions taking account of non-linearity of tissue dose–response whilst not attempting to make predictions of absolute outcome [18].

An in-house contouring atlas was designed for the masticatory apparatus to aid contouring. Absorbed doses were re-calculated for the block and masticatory apparatus using dose volume histograms. Data were extracted via a script to calculate dose volume histograms using in-house software.

Statistics

Analysis was performed using graph pad prism version 6 (Graph pad software), SPSS version 23 (IBM SPSS statistics) and Microsoft Office excel version 2010 . Data analysis compared patient, tumor and treatment-related factors with percent change in MID at 6 months from baseline. Non-parametric tests of Fisher’s exact, chi-squared and chi-squared test for trend were used. Correlations between the different muscles and dose parameters were calculated using Spearman’s rank and linear regression models. A p value of <.05 was considered statistically significant.

Results

The 20 patients reviewed all had established trismus at baseline. In six, the change in MID deteriorated from a median baseline of 29 mm (17–34) pre-radiotherapy to 18 mm (6–29) 6 months from the first radiation treatment. In 14 patients, the MID improved from a median baseline of 16 mm (8–34) to 31 mm (9–39) at 6 months. At the time of the analysis no patients had recurred with residual disease. Supplementary Table S1 shows the distribution of patients in the deterioration and improvement groups in relation to a number of parameters. Patients with a deterioration in MID had more involved lymph nodes (p = .04). However, the distribution of exercise frequency did not differ between the deterioration and improvement group (p = 1.00), and there were no other statistically significant differences in relation to patient, tumor and treatment-related factors (Supplementary Table S1). Although not statistically significant, mean doses received by the block and individual ipsilateral muscles were higher in the deterioration group with the exception of the temporalis muscle (Supplementary Table S1). A clinical outlier was detected within the improvement group. Further analyses were performed with and without the outlier.

Negative correlations were observed between mean doses to the block and individual ipsilateral muscles with changes in MID. The correlation improved once the outlier was excluded across the masticatory apparatus (Table 1). The size of this effect increased with the outlier excluded and reached statistical significance for the ipsilateral block (β = 6.26, p < .01), MP (β = 6.61, p < .01), LP (β = 4.15, p < .01) and M (β = 4.58, p = .02) muscles. Figure 1 illustrates a change of 4–7% in mouth opening per gray of dose to the relevant structures.

Figure 1. Scatter plots showing the relationships between the percentage change in maximal inter-incisor distance (MID) 6 months following the start of radiotherapy and mean doses to the muscles of mastication. The clinical outlier is shown by the red point. A (ipsilateral block), B (ipsilateral medial pterygoid), C (ipsilateral lateral pterygoid), D (ipsilateral masseter), E (ipsilateral temporalis) and F (ipsilateral TMJ). Data is given for 19 patients.

Table 1. Correlations between the mean radiation dose to the muscles of mastication with change in maximal inter-incisor distance.

	With outlier (n = 20)		Without outlier (n = 19)
	r^s value^a	p value	r^s value	p value
Ipsilateral block	−0.42	.07	−0.58	.01
Ipsilateral MP	−0.27	.25	−0.42	.09
Ipsilateral LP	−0.32	.16	−0.41	.04
Ipsilateral M	−0.44	.05	−0.61	<.01
Ipsilateral temporalis	−0.23	.65	−0.52	.39

^a Calculated using Spearman rank correlation.

MP: medial pterygoids; LP: lateral pterygoids; M: masseters.

Discussion

This feasibility study using prospective data showed an association between mean dose to the ipsilateral block and individual masticatory muscles with the development of trismus. All patients had established trismus prior to radiotherapy measured on a continuous scale. The change in MID 6 months from the start of radiotherapy is a similar metric to that used by Joyce van der Geer et al. [2] who observed a peak in the prevalence of trismus at 6 months.

In our study, there were no statistical significant associations observed between changes in trismus and age, gender, tumor stage or site, concurrent cisplatin or frequency of proactive exercises during and after treatment. Even though not statistically significant (p = 1.00), a relationship was observed between the use of proactive exercises during and after treatment, in those whom the MID improved the frequency of daily exercises performed was higher. Mean doses to the ipsilateral LP (r^s = −0.41, p = .04) and M (r^s= −0.61, p < .01) were significantly associated with change in MID. There was also a correlation between mean dose to the ipsilateral MP but this did not reach statistical significance (r^s= −0.42, p = .09). The relationship between trismus and mean dose to the ipsilateral MP is similar to published data by Kent et al. [13]. In a study of 40 patients, Kent et al. [13] identified a 45% prevalence in trismus with mean doses to the pterygoids of ≥55 Gy [13]. In our study, mean dose to the ipsilateral M was most correlated with trismus. This is similar to other retrospective studies. In a study of 139 patients, a significant relationship between mean doses to the ipsilateral masseter of 60 Gy was found [19] contrasting with a study of 55 patients where doses of V20–V40 Gy to the masseter were associated with the development of trismus [12].

In one patient referred to as a clinical outlier, the MID improved at 6 months despite mean doses in excess of 60 Gy received by all masticatory muscles. This outlier was the only HPV positive, stage T4b tumor in the cohort. The large primary base of tongue tumor had invaded into the pterygoid muscles resulting in trismus at diagnosis. The contoured GTV was much larger at 106 cm³ than the rest of the cohort (median 15 cm³, range 3–106 cm³). This patient had very few risk factors for developing late effects being a nonsmoker, absence of concurrent platinum-based chemotherapy and no surgery. Post-treatment scans confirmed a complete response with no current evidence of recurrence 12 months following treatment. The improvement in trismus can be explained by tumor response and improved pterygoid muscle function.

Although there are known differences in muscle architecture and function between the pterygoid muscles, an association was found between mean dose to the MP and LP with trismus. This can be explained by the close proximity of the pterygoid muscles and synergistic function to open and close the jaw [20]. The mean dose received by the ipsilateral LP was lower than the MP in those where by trismus improved (39 Gy versus 57 Gy). This is in keeping with Hsieh et al. [21] who suggested a mean dose constraint of <42 Gy to the LP to reduce the risk of trismus. The mean doses to the ipsilateral MP, LP, M and TMJ excluding the clinical outlier were greater in those where trismus deteriorated at 6 months from the start of radiotherapy compared to those where trismus improved. There was no significant correlation observed between V35 and V60 Gy with changes in trismus across all paired muscles and the block. Our observation contrasts with another study in 124 patients where doses of V40–V60 Gy to the ipsilateral masseter were associated with statistically significant changes in trismus [14]. The difference in the dose–response relationships between the two studies may be attributed to the small sample size and short follow-up period of 6 months in our study compared with a median follow-up of 66 months in the study by Lindblom et al. [14] In our study there was a weaker and non-significant association seen between the EUD and change in MID compared with the mean dose. The EUD was calculated to evaluate potential non-linear dose effects within a muscle, but the results are less strongly correlated with changes in trismus than the mean dose.

To our knowledge, only the effect of dose on the masticatory muscles as individual subunits has been explored previously. An avoidance dose for the masticatory apparatus has not been agreed due to disparity within the literature. Our study is the first to describe and evaluate the concept of a block as an alternative OAR. The block includes the MP, LP, M and T below the floor of the orbit to exclude the cranial component of the T. These muscles were chosen due to their similarities in function and anatomical locations. There was a modest significant correlation (r^s= −0.58, p = .01) between mean dose to the ipsilateral block and change in MID. Patients with a deterioration in trismus 6 months from the start of radiotherapy received mean doses >40 Gy to the ipsilateral block, MP, LP and M. Significant correlations were seen between mean dose and changes in MID with the ipsilateral block, LP and M. Limiting mean dose ≤40 Gy to the ipsilateral block, LP and M for tumors not invading masticatory muscles in those with established trismus prior to radiotherapy and whom underwent proactive exercises could be considered. Mean dose ≤40 Gy is considerably lower than that reported for each individual muscle both in our study and in the published literature. The lower mean dose constraint for the ipsilateral block can partly be explained by the larger volume of muscle within the block compared to the muscles as separate entities. The concept of grouping muscles together as a block would help remove uncertainties as to which masticatory muscles should be avoided. It would also improve our understanding of the clinical significance of radiotherapy to the regions in the muscle interface such as fat, fascia or nerves of which little is known. Image based data mining has illustrated this point through highlighting an area adjacent to the masseter that has a dose–response relationship [22]. Avoidance structures such as the block may help reduce the severity of radiation-induced trismus but require validation in larger studies.

Limitations to the study

Although this is the first article to our knowledge that uses prospective pre- and post-radiotherapy MID measurements on a continuous scale to evaluate dose effects on organs at risk, the study is small. All patients had subjective jaw tightening prior to the start of radiotherapy and underwent proactive exercises during and after treatment. The results are hypothesis generating and require validation in a larger study. Our findings need to be verified in a larger sample and matched control cohort with longer follow-up. A larger study would enable development of a normal tissue complications probability (NTCP) model and recommendations for mean dose constraints to use in radiotherapy planning. However, our work does show that further studies using MID methodology would be useful.

Conclusions

In this prospective study, higher mean radiation doses to the ipsilateral block, LP and M were significantly associated with trismus. The findings suggest that limiting mean dose to the ipsilateral block, LP and M to ≤40 Gy for tumors not invading the masticatory muscles may reduce treatment-related morbidity. The ipsilateral block, LP and M should be studied further as alternative OARs and possible avoidance structures in radiotherapy planning in future studies. This suggestion requires validation in a larger study with longer follow-up.

Acknowledgments

The authors acknowledge support from the Trismus trial NIHR RfPB trismus trial portfolio ID [13415].

Disclosure statement

No potential conflict of interest was reported by the authors.