Assessing patients’ perceived bother from the gastrointestinal side effects of radiotherapy for localized prostate cancer: Initial questionnaire development and validation

Abstract

Background. The present study reports on the development and first steps of validation of the Gastrointestinal Side Effects Questionnaire (GISEQ), a measure of patient-reported gastrointestinal symptoms following local radiotherapy to the prostate. The questionnaire design provides a novel approach of assessment of side effects of prostate radiotherapy, by enabling measurement of patient-perceived change in symptoms. Material and methods. The eight-item GISEQ was administered to 130 prostate cancer patients referred to radiotherapy. Patients completed the GISEQ at four, eight and 15 weeks after start of radiotherapy. The psychometric properties including validity, reliability, responsiveness and feasibility were evaluated. The EORTC QLQ-C30 and QLQ-PR25 were chosen as comparative measures. Results. Expert opinion supported content validity. For concurrent validity, correlation between the GISEQ and matching items in the EORTC questionnaires was moderate but significant (r > 0.41, p < 0.001). The responsiveness was adequate, indicated by changes in GISEQ scores over time corresponding to the effects of radiation. Internal consistency was satisfactory (overall Cronbach's α> 0.70). Sensitivity and specificity for items diarrhea, constipation and blood in stools ranged from 50% to 100% and from 68% to 100%, respectively. All items had a floor effect above 15%. The response rates ranged from 85% to 92% and missing items was < 0.8%, indicating good feasibility. Conclusions. The GISEQ showed satisfactory internal consistency and adequate content validity, concurrent validity and responsiveness. It is brief, easy to use and can be quickly evaluated, making it useful not only for research but possibly also for clinical settings. Modification of response scale and extension of items are potential improvements. Further work is needed to strengthen the psychometric qualities of the GISEQ and to evaluate its clinical use and potential effects of response shift and recall bias.

Prostate cancer is the second most prevalent cancer in men worldwide [1]. Patients are affected by a number of disease- and treatment-related symptoms that have a negative impact on their health-related quality of life (HRQOL). Prostate radiotherapy can lead to acute and long-term gastrointestinal side effects, which may be due to proctitis [2,3]. The risk of radiation-induced proctitis is dose-volume dependent and symptoms of radiation proctitis may include diarrhea, abdominal pain and bloated abdomen, urgency, mucus discharge, rectal bleeding and sometimes constipation [4,5]. Such symptoms may persist in some patients and the reported incidence of chronic radiation proctitis ranges from 2% to 20%, depending on the radiation technique and the dose-volume distribution to the rectum [6, 7].

Management of gastrointestinal side effects can only be efficient and effective if based on reliable and valid symptom assessments. Research or clinical outcome assessments can be based on physician or patient ratings. Physician ratings of symptom occurrence cannot accurately reflect patient perception in its entirety [8], and the true incidence of radiation proctitis is likely to be underestimated because toxicity scales do not include assessment of a wide variety of symptoms [9]. It is increasingly common that assessment of treatment side effects also include patient-reported measures, which provide estimates of the impact of treatment on the patient's life, including the consequences of treatment and the patient's actual symptom experience [10]. A recent review of prostate cancer modules provided a detailed summary of existing patient-reported instruments [11]. Six modules on HRQOL and prostate cancer were reported, such as the EORTC Prostate Cancer Module (later renamed QLQ-PR25), the UCLA Prostate Cancer Index, the EPIC and the QUFW94 (later renamed PCSS) [12–15]. All six modules consisted of items on toxicity concerning intestinal, urinary and sexual function, but the measurement procedures varied considerably. The modules also differed in terms of number of items used, item dimensions (function and/or bother) and the intended time interval for the questions. The psychometric properties ranged from acceptable to not satisfactory. Hence, there is currently no consensus on the best method for measuring the side effects of radiotherapy for prostate cancer.

Patient-reported assessments pose challenges regarding several aspects. Patients experience the impact of side effects differently and evaluation of the results of patientreported measurements of side effects should be made with regard to their perceptual and subjective nature. If measurements are repeated in order to evaluate the effect of treatment or interventions, change scores may be calculated by subtracting the pretreatment assessment from the posttreatment assessment. However, such calculations require extra time and administrative work. Also, conventional change scores may be interfered by a concept known as ‘response shift’ [16] whereby patients’ perception of a construct may change over time as an adaptive response to a changed health status. The calculation of conventional change scores may disagree with an individual's perception of gastrointestinal symptoms, in magnitude and possibly also in direction, if response shift occurs between assessment points [17]. There is also recent evidence that prostate cancer patients’ response shift may be two-directional, both adaptive and mal-adaptive [18]. These aspects together illuminate a challenge for many existing prostate cancer instruments of gastrointestinal toxicity, as they were not designed to take account of these factors.

Our research and clinical experience indicates that it is important to account for the patients’ pretreatment conditions when analyzing radiation toxicity, because pretreatment symptoms may be a predictive factor for radiation-induced side effects [19,20], and the patient's pretreatment experience of symptoms may affect his perception of side effects which in turn may be affected over time. Available prostate modules of toxicity that assess merely the present symptom prevalence are unable to discern whether the patient's reported symptoms are persistent pretreatment conditions or actual side effects emerging as a consequence of radiotherapy. In light of this, we saw a need to develop a questionnaire that would account for the impact of pretreatment conditions and would be less vulnerable to response shift. The concept of ‘patient-perceived change’ was used, whereby the patient was asked to report the current situation in comparison to his current perception of the situation prior to treatment [21]. An advantage of using this concept is that the impact of changing internal standards, coping and adaptation (i.e. response shift) that may occur over time is eliminated from the assessment. The questionnaire design with retrospective reporting means that the assessment of change can be performed by a single questionnaire at one point in time. This approach gives relative subjective scores which emphasize any perceived change in symptom burden during radiotherapy and its aftermath. To our knowledge, there is no existing instrument to this patient category based on that approach and design. The new questionnaire would thereby complement existing measures of absolute scores.

We set out to develop an as sensitive as possible questionnaire capable of efficiently assessing patient-perceived change in a wide range of relevant bothersome gastrointestinal side effects from prostate radiotherapy. Preferably it would be sensitive to symptoms specifically induced by radiotherapy and quickly identify patients who experience worsening levels of symptoms. It is important to establish that the new questionnaire fulfils this objective and provides accurate and meaningful results. Thus, the aim of the present study was to evaluate the validity and reliability of the Gastrointestinal Side Effects Questionnaire.

Material and methods

Participants

The current questionnaire was administered to 130 patients recently diagnosed with localized prostate cancer referred to radiotherapy in the Department of Oncology at Uppsala University Hospital, as part of the data collection in a randomized controlled trial (RCT) [22]. The RCT studied the effect of a dietary intervention on acute gastrointestinal side effects and other aspects of HRQOL from baseline (prior to radiotherapy onset) up to two months after completed radiotherapy (T0–T3). The results of the RCT did not show any statistically significant interventional effect, and so it was feasible to combine the intervention group and control group in the statistical analysis of the questionnaire validation process. The methods, materials and ethical considerations of the RCT have been described previously [22].

Measures

The Gastrointestinal Side Effects Questionnaire (GISEQ) contained eight disease-specific items that concerned bother from diarrhea, constipation, blood in stools, mucus discharge, abdominal cramps, abdominal pain, intestinal gas and flatulence. Item questions read ‘To what extent have you been bothered by… during the past week, compared to before radiotherapy?’ Answers were given on a numerical rating scale anchored from 0 (‘To the same or a lesser extent’) to 10 (‘To a much larger extent’). GISEQ also contained two open-ended questions that read; ‘Have you had any other gastrointestinal problems, not present prior to radiotherapy?’ and ‘Have you used any medications for gastrointestinal problems (prescription, non-prescription or natural remedies) during or after radiotherapy?’.

The European Organization for Research and Treatment of Cancer Quality of Life Questionnaire C30 (EORTC QLQ-C30) (version 3) and the prostate-specific module EORTC QLQ-PR25 [12,23] were chosen as the comparator measures in the evaluation of GISEQ's psychometric properties. The two EORTC questionnaires contain a total of six single items on bowel symptoms (QLQ-C30: diarrhea, constipation. QLQ-PR25: limitation in daily activities, unintentional release (leakage) of stools, blood in stools, bloated feeling in the abdomen). The EORTC questionnaires were administered at all points of assessment (T0–T3) in the previously mentioned RCT. When analyzing concurrent validity, sensitivity and specificity of the GISEQ, three EORTC single-items were considered to match GISEQ items (constipation, diarrhea and blood in stools). The absolute scores of 1–4 of the three selected items from the EORTC questionnaires were transformed to conventional change scores (follow-up scores minus baseline scores) in order to look for the correlation to the relative follow-up scores reported in the GISEQ. The approach to use the individual item level of bowel symptoms reported in the QLQ-PR25 (i.e. blood in stools) has been recommended in a previous study [12].

A second questionnaire was developed and administered at baseline (T0) in the previously mentioned RCT, with the objective of identifying patients with gastrointestinal bother even prior to radiotherapy onset. The GISEQ Pre-Radiotherapy (GISEQ-PR) is an eight-item patient-reported assessment of pretreatment gastrointestinal status and its design has been previously reported [22]. Answers apply to the status in the last week and are given on a numerical rating scale anchored from 0 (‘Not at all’) to 10 (‘To a very large extent’).

Procedure

The development and validation of the GISEQ involved five steps. First, inclusion of relevant items covering gastrointestinal side effects was based on a literature review and consultation with experts with extensive experience in the care and research of the present patient category. Second, a provisional list of items was selected for the GISEQ, with eight items and one open-ended question. Third, a panel of prostate healthcare professionals read over the items and assisted with the wording to help ensure the relevancy of content and breadth of coverage of the questionnaire. The panel consisted of an oncologist, a nurse and a dietician, all of whom had extensive experience from the oncology setting. The panel evaluation resulted in the addition of one open-ended question regarding usage of medications for gastrointestinal conditions. Fourth, the revised GISEQ was administered to the study sample at four weeks (T1, i.e. in the middle of the radiotherapy treatment period), eight weeks (T2, i.e. one week after radiotherapy completion), and 15 weeks (T3, i.e. two months after radiotherapy completion). The four time points for administration and assessment were chosen based on the expected variations of side effects over time correlating with the effects of radiation. Consequently, symptoms and bothering side effects were expected to occur in the first few weeks after radiotherapy, be at its worst at T1 and/or T2 and then decrease again at T3. All patient-reported data (GISEQ, GISEQ-PR and the EORTC questionnaires) were collected as self-administered paper-format questionnaires. The questionnaires were filled in by the patients at regular visits to the radiotherapy unit, or at home and returned in pre-paid envelopes. Fifth, analyses of GISEQ's psychometric properties were conducted after the study period, in order to explore its feasibility as a clinical assessment tool in the target population.

Analysis

The statistical analyses were performed using the SPSS version 20 software package. All p-values were two-tailed and the level of statistical significance was set at p < 0.05. The selection of characteristics for the evaluation of psychometric properties was based on Streiner and Norman's description of ideal measurement characteristics (Supplementary Table I, available online at http://informahealthcare.com/doi/abs/10.3109/0284186X.2012.819994) [24]. Also, feasibility was assessed on the basis of response rate, missing items and floor/ceiling effects. The response rate was determined as the percentage of participants who completed the questionnaire out of the total number of patients that accepted participation and were included at baseline. The proportion of items missing was defined as the percentage of items that were missing out of the total number of items received for each questionnaire. Floor or ceiling effects were considered to be present if more than 15% of patients reported the lowest or highest possible score, respectively [25].

Table I. Feasibility for the GISEQ, including response rate, missing items and floor/ceiling effects at T1–T3.

Response rate^a Missing items		T1 4 weeks n = 119		T2 8 weeks n = 111		T3 15 weeks n = 113
		91.5% 0%		85.4% 0–0.8%		86.9% 0%
		Floor effect	Ceiling effect	Floor effect	Ceiling effect	Floor effect	Ceiling effect
GI1	Diarrhea	57.1	2.5	55.5	1.8	57.5	0.9
GI2	Constipation	65.5	1.7	71.8	0	74.3	0
GI3	Blood in stools	89.1	0.8	90.1	0.9	98.2	0
GI4	Mucus discharge	63.9	1.7	59.5	3.6	70.8	1.8
GI5	Abdominal cramps	79.0	1.7	79.3	0	85.8	0
GI6	Abdominal pain	65.5	1.7	70.0	0.9	83.2	0.9
GI7	Intestinal gas	31.1	0.8	28.8	5.4	37.2	0.9
GI8	Flatulence	29.4	1.7	27.9	3.6	33.6	0.9

Floor effect: The percentage of patients who reported the lowest possible score. Ceiling effect: The percentage of patients who reported the highest possible score.

^aResponse rate is the proportion of patients that completed the GISEQ out of the total number of participants entering the study at baseline (n = 130 at T0, i.e. in the week prior to radiotherapy onset).

Validity

Evaluation of face and content validity was based on input from an expert panel (please see step 1–3 above). The patient sample also contributed to the evaluation of content validity, as patients’ responses to the open-ended question on any other gastrointestinal problems were evaluated based on the viewpoint that the responses captured patients’ perspectives on relevant but inadequately covered items. When assessing concurrent validity, the Spearman’s rank order correlation coefficient was used to determine the degree of relationship between GISEQ item scores (diarrhea, constipation and blood in stools) and change scores of matching items in the QLQ-C30 and QLQ-PR25. The cut-off value for positive correlation was set at the statistically significant correlation coefficient rho > 0.41, as values ranging from 0.41 to 0.6 are usually regarded as moderate correlations [26]. Moderate correlations were predicted, indicating that the GISEQ and the EORTC questionnaires assessed related but different outcome constructs owing to the different wording of the questions. To assess the responsiveness of the GISEQ, scores were dichotomized to 0 (unchanged or improved compared to pretreatment) and 1 (increased bother compared to pretreatment), and then analyzed with McNemar's test regarding differences in proportion of patients with increased bother between T1, T2 and T3. Additionally, the Wilcoxon rank-sum test evaluated change in scores on the GISEQ from T1 to T2 and from T2 to T3 in patients with increased bother, as means for assessing the responsiveness of the GISEQ.

The hypothesis that pretreatment gastrointestinal symptoms may be associated to the gastrointestinal toxicity attributable to radiotherapy was explored by evaluating descriptive data of the GISEQ at T1–T3. At T1–T3, patient data were divided in two groups based on a dichotomization of the patients’ pretreatment gastrointestinal status, i.e. ‘no bother pretreatment’ (score 0) versus ‘bother pretreatment’ (score 1–10). The Fisher's exact test was used for the evaluation of differences between the two groups regarding the proportions of patients reporting bothering symptoms. Differences in proportions were analyzed for patients with score 0, score ≥ 3 (i.e. score 3–10) and score ≥ 5 (i.e. score 5–10), respectively.

Reliability

Internal consistency of the GISEQ items and whether internal consistency improved with any single item removal were estimated using Cronbach’s α, where a level of 0.70 or higher was considered desirable [25].

Sensitivity and specificity

The sensitivity and specificity of the GISEQ were determined using matching items in the QLQ-C30 and QLQ-PR25 as the comparator measures. The GISEQ item scores and the change scores of the EORTC questionnaires were dichotomized to 0 (unchanged or improved compared to pretreatment) and 1 (worsened compared to pretreatment). Estimates for sensitivity and specificity as well as positive-predictive value (PPV) and negative-predictive value (NPV) were calculated from cross-tabulation of the dichotomized scores.

Results

The response rate of the questionnaire ranged from 85.4% to 91.5% at T1–T3, where non-responses were due to incomplete assessment or total withdrawal (Table I). Missing items was < 0.8%. One patient had three unanswered missing items at T2, all other responding participants answered all items at T1–T3. The proportion of patients who responded with the lowest possible score on any item ranged from 27.9% to 98.2% (Table I). The proportion of patients who responded with the highest possible score on any item ranged from 0% to 5.4%.

Validity

The expert panel considered the GISEQ to be satisfactory regarding face validity as well as content relevance and coverage. In regard to the open-ended question on any other gastrointestinal problems, the patients mentioned rectal pain and/or irritation (n = 3), increased stool frequency (n = 6), feeling of having to pass stools when urinating (n = 4), leakage problems (n = 6) and problems of urgency (n = 4). There was a positive correlation (rho > 0.41, p < 0.001) between matching items from the GISEQ and QLQ-C30 at all assessments for the items diarrhea and constipation (Table II). For blood in stools, the correlation coefficient was below the cut-off value for positive correlation at the T2 assessment only (Table II). Responsiveness data revealed that the proportion of patients who reported an increased level of bother (i.e. score 1–10) on the items GI1 to GI6, at T1, T2 and T3 were 11–43%, 10–45% and 2–42%, respectively (Table III). For the items GI7 intestinal gas and GI8 flatulence, the proportion of patients was markedly higher: 69–71% at T1, 71–72% at T2, and 63–66% at T3. There were no statistical differences in the proportions of patients reporting bother between T1 and T2 or between T2 and T3 for any of the items, with the exception of GI6 abdominal pain between T2 to T3, where the proportion was lower at T3 (p < 0.02). The change in distribution of scores at T1 to T3 was examined, using the first quartile (Q1) and the third quartile (Q3). The item scores among patients with bother were stable or increased between T1 and T2 for six items (T1: Q1 = 1.0–2.0 and Q3 = 3.5–5.0; T2: Q1 = 1.0–2.0 and Q3 = 2.0–5.5). Between T2 to T3, the item scores were stable or decreased (T3: Q1 = 1.0 and Q3 = 2.0–4.0). Changes in the distribution of scores for patients with bother (i.e. score 1–10) were statistically significant between T2 and T3 for all items (p < 0.03), but not between T1 and T2.

Table II. Concurrent validity of three GISEQ items, using Spearman's rank order correlation coefficient to evaluate the degree of relationship with matching items in the EORTC QLQ-C30 and QLQ-PR25.

GISEQ item/EORTC item	T1 4 weeks n = 119	T2 8 weeks n = 110	T3 15 weeks n = 113
Diarrhea	0.52	0.55	0.49
Constipation	0.62	0.43	0.42
Blood in stools	0.70	0.30	0.71

All bivariate correlations between GISEQ items and EORTC QLQ-C30 and QLQ-PR25 items are statistically significant, with a p-value < 0.001.

GISEQ items: GI1 diarrhea, GI2 constipation, GI3 blood in stools. EORTC items: Q17 diarrhea in QLQ-C30, Q16 constipation in QLQ-C30, Q42 blood in stools in QLQ-PR25.

Table III. Responsiveness analysis with Wilcoxon rank-sum Test of the GISEQ scores: from T1 to T2 and from T2 to T3.

	Symptoms at T0 Baseline		Worsening symptoms at T1 4 weeks				Worsening symptoms at T2 8 weeks				Worsening symptoms at T3 15 weeks				Wilcoxon rank-sum Test p-value		McNemar Test p-value
	%	n	%	n	Q1	Q3	%	n	Q1	Q3	%	n	Q1	Q3	T1 to T2	T2 to T3	T1 to T2	T2 to T3
GI1 Diarrhea	19	23	43	51	1.0	5.0	45	49	1.5	5.5	42	48	1.0	3.0	n.s.	0.02	n.s.	n.s.
GI2 Constipation	17	21	34	41	1.0	5.0	28	31	2.0	4.0	26	29	1.0	2.0	n.s.	0.01	n.s.	n.s.
GI3 Blood in stools	5	6	11	13	1.0	4.0	10	11	1.0	2.0	2	2	1.0	–	n.s.	0.02	n.s.	n.s.
GI4 Mucus discharge	6	7	36	43	1.0	4.0	41	45	1.0	5.0	29	33	1.0	3.0	n.s.	0.01	n.s.	n.s.
GI5 Abdominal cramps	10	13	21	25	1.5	5.0	21	23	1.0	4.0	14	16	1.0	2.8	n.s.	0.02	n.s.	n.s.
GI6 Abdominal pain	11	14	34	41	1.0	3.5	30	33	1.5	4.0	17	19	1.0	3.0	n.s.	0.01	n.s.	0.02
GI7 Intestinal gas	56	69	69	82	2.0	5.0	71	79	2.0	5.0	63	71	1.0	4.0	n.s.	0.03	n.s.	n.s.
GI8 Flatulence	60	74	71	84	2.0	5.0	72	80	2.0	5.0	66	75	1.0	3.0	n.s.	0.01	n.s.	n.s.

‘Symptoms’ and ‘Worsening symptoms’ indicates score ≥ 1 on a 0–10 scale of gastrointestinal bother. Total number of patients at T0: n = 124, T1: n = 119, T2: n = 110, T3: n = 113. The Wilcoxon rank-sum Test evaluated change in scores from T1 to T2 and from T2 to T3, for patients with worsening symptoms (i.e. scores 1–10) at T1–T3.

n.s., not significant; Q1, the 25th percentile; Q3, the 75th percentile.

A greater proportion of patients with gastrointestinal bother pretreatment reported an increase in bother during and at the end of radiotherapy (i.e. T1–T2) compared to patients with no gastrointestinal bother pretreatment. This is implicated in that, for five of eight GISEQ items, the proportion of patients who reported no increase in symptoms at T1–T2 was significantly smaller in the group of patients with gastrointestinal bother pretreatment compared to patients without gastrointestinal bother pretreatment (p < 0.05, Supplementary Table II, available online at http://informahealthcare.com/doi/abs/10.3109/0284186X.2012.819994).

Reliability

The overall Cronbach's α was 0.76, 0.82 and 0.73 for the GISEQ from T1, T2 and T3, respectively. Improvement of internal consistency was seen with deletion of GI2 constipation at T1 (0.77) and T2 (0.84).

Sensitivity and specificity

The sensitivity of the individual GISEQ items in determining ‘worsening symptoms’ ranged from 79–92% for diarrhea, 88–100% for constipation, and 50–100% for blood in stools (Table IV). The specificity ranged from 68–71% for diarrhea, 79–82% for constipation, and 92–100% for blood in stools. The PPV in this sample ranged from 27–100%, and the NPV ranged from 91–100%.

Table IV. Sensitivity and specificity as well as PPV and NPV for GISEQ items in determining ‘worsening symptoms’.

	Sensitivity %	Specificity %	PPV %	NPV %
T0 (n = 122–123)
GI1/Q17 Diarrhea	85 (17/20)	94 (97/103)	74	97
GI2/Q16 Constipation	75 (9/12)	89 (99/111)	43	97
GI3/Q42 Blood in stools	100 (2/2)	97 (116/120)	33	100
T1 (n = 114–117)
GI1/Q17 Diarrhea	79 (23/29)	68 (60/88)	45	91
GI2/Q16 Constipation	92 (24/26)	81 (74/91)	59	97
GI3/Q42 Blood in stools	80 (8/10)	96 (100/104)	67	98
T2 (n = 102–108)
GI1/Q17 Diarrhea	92 (22/24)	71 (55/78)	49	97
GI2/Q16 Constipation	88 (14/16)	82 (75/92)	45	97
GI3/Q42 Blood in stools	50 (3/6)	92 (91/99)	27	97
T3 (n = 109–111)
GI1/Q17 Diarrhea	92 (22/24)	71 (60/85)	47	97
GI2/Q16 Constipation	100 (7/7)	79 (82/104)	24	100
GI3/Q42 Blood in stools	100 (2/2)	100 (107/107)	100	100

Conventional change scores of matching items in EORTC QLQ-C30 and QLQ-PR25 were used as the comparator measures. Prior to analysis of sensitivity and specificity, scores were dichotomized to 0 (unchanged or improved symptoms) and 1 (worsening symptoms).

NPV, negative-predictive value. PPV, positive-predictive value; GISEQ items: GI1 diarrhea, GI2 constipation, GI3 blood in stools. EORTC items: Q17 diarrhea in QLQ-C30, Q16 constipation in QLQ-C30, Q42 blood in stools in QLQ-PR25.

Discussion

Current prostate cancer instruments of gastrointestinal toxicity may not be able to take sufficient account of aspects such as response shift and intra-individual changes in the meaning of HRQOL. The GISEQ captures a new dimension in the assessment of gastrointestinal side effects from radiotherapy, in that it is based on the measurement of patient- perceived change in bothering gastrointestinal symptoms. By its retrospective question wording and relative subjective scores, the GISEQ makes it possible to assess change in gastrointestinal status from the pretreatment situation by the use of a single questionnaire at one point in time. Through this approach, the patient's perception of bothersome gastrointestinal symptoms can easily be assessed and a change for the worse regarding radiation side effects quickly becomes evident.

The GISEQ was initially intended and developed to evaluate the effects of the intervention in the previously mentioned RCT. In light of our experience from the RCT and the validation procedure, the questionnaire could possibly also be suitable for clinical settings as an initial screening tool of side effects from radiotherapy. The ability to identify patients who experience worsening levels of symptoms and radiation-induced side effects is important and useful in both research and clinical settings. The GISEQ could possibly provide a better understanding of the cause of the gastrointestinal symptoms and thereby influence the choice of appropriate management and enhance the care. Patients with pretreatment symptoms (e.g. inflammatory or functional bowel disorders) could be referred to the appropriate specialist, whilst efforts for patients with radiation-induced side effects could be directed towards radiotherapy- related factors.

The responsiveness of the GISEQ to the change in bother from gastrointestinal side effects was satisfactory. For most of the items, the progress of bother correlated well with the effects of radiation. However, the GISEQ did not prove to be as responsive to the change in the proportion of patients with bother. This was most likely due to the small number of events, with more than 50% of the patients reporting no bother during an at the end of radiotherapy for six of eight items. In comparison to reported levels of gastrointestinal side effects from medical practitioner-based toxicity scales, the GISEQ appears to detect patients with gastrointestinal bother to the same or a higher degree [27,28]. Moderate correlation between the GISEQ items and the EORTC QLQ-C30 and QLQ-PR25 items was expected in the evaluation of concurrent validity, as the questionnaires assess conceptually different issues, i.e. bother due to symptoms versus symptom prevalence. The conversion of the EORTC questionnaire absolute scores into change scores seemed to be the most appropriate way to handle the discrepancy in the wording of questions. The lack of evaluation of concurrent validity, sensitivity and specificity of the five items for abdominal pain, abdominal cramps, mucus discharge, intestinal gas and flatulence, due to the absence of matching items in the comparator measures EORTC QLQ-C30 and QLQ-PR25, is a limitation to the study.

The tendency of low PPV may contribute to a risk of detecting rather many ‘false positives’. If GISEQ would be used as a screening tool for clinical purposes, it is suggested that patients detected with increased gastrointestinal bother would undergo a more extensive assessment. In line with this reasoning, false positive identifications are preferable to false negative identifications. However, this validation study is based on patients and data in a research setting and not from a clinical setting. Although we believe that the GISEQ's would likely do very well as a screening tool, the validation process reported here was not based on that intent and so any assumptions on related properties is only speculative at this time. The NPV were exceptionally high which means that the GISEQ performs as well as the comparator measures and rarely misclassifies patients with no side effects. Also, both PPV and NPV are directly related to the prevalence of gastrointestinal bother. Assuming all other factors remain constant, the PPV will increase with increasing prevalence; and NPV decreases with increase in prevalence. Hence, it is likely that the small number of events affected the outcomes of NPV and PPV. Moreover, the divergent questionnaire design of the GISEQ and the EORTC questionnaires with differing question wording, scoring etc, may have hampered the analysis of sensitivity and specificity.

The approach of assessing patients’ perception of change in the GISEQ is based on the idea that the most appropriate perspective from which to rate subjective bother from side effects is the patient's perspective of change held at the current point in time. This strategy has the advantage of diminishing the confounding factor of ‘response shift’ compared to conventional change score calculation [21]. A method for calculating patient-perceived change through the use of a ‘then test’ has been suggested, but invalidating effects of recall bias was indicated [21,29]. In GISEQ, a retrospective comparison is comprised in the wording of the questions, where the patient takes his current perspective of pretreatment status into account when rating present symptoms. Thus, elements of a then test and calculation may be omitted. Asking the patient to rate current gastrointestinal symptoms in comparison to his current perspective of pretreatment status at a single assessment results in a straightforward indication of patientperceived change, that is amenable to use in both research and clinical assessments. However, inaccuracy of recall could potentially confound longitudinal evaluation of patient-perceived change, and it has been implied that prostate cancer patients do not accurately recall pretreatment status when asked more than one year after treatment [30]. The relative contribution of the response shift versus recall bias phenomena to the GISEQ in this population and over this time frame currently remains unknown. These issues are important and warrant further investigation of the quality and usefulness of the questionnaire.

The GISEQ has good precision of measurement, indicated by satisfactory Cronbach's αs. Comparing the reliability of the GISEQ to that of other similar instruments, the Swedish Prostate Cancer Symptom Scale reported Cronbach's αs of 0.83 and 0.55 in the two intestinal symptom scales [11]. The GISEQ was not tested for reproducibility, which was a limitation of the present study. This was partly due to the difficulty of administering a test-retest to assess gastrointestinal bother while patients were undergoing radiotherapy. For test-retests, the appropriate time interval between two test occasions is usually two to 14 days [24]. In this case, it would likely have resulted in inconsistent measures over time and low test-retest reliability for the GISEQ, simply because of the expected increasing levels of side effects of radiation during the time interval between test occasions and the accompanying risk of divergent assessments.

Content validity evaluation often includes reports from the literature, expert opinion, and patient input derived from qualitative research [31]. In the current study, the evaluation was based on expert opinion and patients’ replies to the open-ended questions in the GISEQ. The patients’ perspectives were considered equally important in the evaluation because they have first-hand experience of bothering side effects of radiotherapy. Based on their replies, items on bother associated with increased stool frequency, urgency and stool leakage could possibly be added to the eight existing items. Still, the number of patients that suggested additional problems ranged from three to six, which is lower than the frequencies of all eight existing items in GISEQ (n ≥ 13). A recent review identified six available prostate cancer modules, in which the number of items in the bowel dimensions ranged from one to 14, the median number of items being eight [11]. Evaluation of the content coverage of the GISEQ based on characteristic symptoms of radiation proctitis would also suggest urgency as possible additional item [4,7].

Overall, the GISEQ rendered good results concerning feasibility with high response rates and very few missing items. The GISEQ seem to be acceptable and clear to the intended patient category. Future research and analyzes could examine whether the GISEQ may be appropriate to use also for other groups of patients, i.e. primarily patients undergoing pelvic radiotherapy. However, the retrospective comparative reporting in GISEQ may be cognitively challenging for some patients. The patients must first check their level of bother from side effects at the current assessment, recollect the level of bother at the pretreatment assessment and then mentally compare the two and estimate the difference. The risk that some patients may not easily recollect their pretreatment gastrointestinal status and rather report their symptoms heuristically should be considered.

Unfortunately, patients did not respond to the full range of options on the response scale in the GISEQ. The floor effects were most likely influenced by the fairly low prevalence of gastrointestinal side effects in general, shown both in the GISEQ as well as in the EORTC QLQ-C30 and QLQ-PR25. About one half of patients were free from bowel symptoms during radiotherapy, and those with symptom occurrence reported relatively low levels of bother and symptoms. The labeling of end-anchor score and the 11-step response scale may also have affected the floor effect. It has been proposed that the number of steps in a response scale should be in the region of five to seven [24]. Hence, further evaluation of the response scale in the GISEQ may be appropriate. Possibly, the floor effect of the GISEQ could also partly be explained by patients’ use of medications for gastrointestinal conditions. In the open-ended question, seven different types of medications (antidiarrheals, acid suppressants/antacids, bulkforming/fibre laxatives, osmotic laxatives, flatulence medications, hemorrhoid medications, nausea medications) were mentioned by a total of 19 patients. Such medications may have affected the level of symptoms, and consequently the assessment of bother from gastrointestinal side effects.

Our results confirm earlier findings that pretreatment symptoms may be associated to increased radiation-induced side effects. This further highlights the importance of taking symptoms present prior to treatment into account when analyzing radiation toxicity. Although the result must be interpreted with caution because of the small number of events, it indicates the importance of identifying, distinguishing and managing patients with gastrointestinal bother both before and during radiotherapy.

The initial content of the GISEQ was vetted through clinicians rather than patients, which is a limitation to the study. Ideally, patient input on content should have been more pronounced during the development process. The GISEQ was developed in 2005, and at the time the importance of patients’ perspectives in the development of patientreported measures was not as widely recognized as it is today. With a growing emphasis on patient-centred care in recent years, patient input derived from qualitative research is nowadays acknowledged as being equally as important as literature reports and expert opinion in content validity evaluation [31]. In retrospect, it would have been preferred to invite irradiated prostate cancer patients or representatives from patient organizations should to take part in the process of development and validation. Another methodological weakness of this study is the actual answering procedure of the data collection process, where the questionnaires were collected directly adjacent to visits at the radiotherapy unit. Although seemingly convenient, it may have enhanced risk of social desirability bias when patients filled in questionnaires during the visits. It should also be stressed that the decision to dichotomize data was done only because of the skewness in data from this patient sample. Obviously, it would have been preferable to report data from all steps of the scale.

To conclude, the management of patients’ symptom burden from treatment is one of the main areas of the oncology clinical practice. In order to identify patients in need of intervention, it is important to asses the subjective state of bother from side effects in a valid, reliable and preferably time-effective fashion. Asking patients to rate their gastrointestinal symptoms in comparison to their current perspective of pretreatment status results in a straightforward indication of radiationinduced side effects. However, further research on occurrence of response shift and recall bias is needed. Based on the psychometric evaluation presented here, the GISEQ adequately reflects the issue of gastrointestinal bother and the items show good homogeneity. The questionnaire is also easy to use and can be quickly evaluated, making it useful both for research purposes and as an assessment tool in the clinical management of radiation-induced side effects. Further work is needed to strengthen the psychometric qualities and to evaluate clinical use of the GISEQ with respect to response shift and recall bias. Future potential improvements of content and coverage of the GISEQ include review and modification of items and response scales, with emphasis on patients’ perspectives captured through focus groups or semi-structured interviews with irradiated prostate cancer patients. Such improvement will hopefully allow enhanced evaluation of concurrent validity, content validity and internal consistency.

Notice of correction

The version of this article published online ahead of print on 19 Aug 2013 contained an error in Table 1 on page 5. The item “GI5 Intestinal cramps“ should have read “GI5 Abdominal cramps“ and the item “GI6 Intestinal pain“ should have read “GI6 Abdominal pain“. The error has been corrected for this version.

Declaration of interest: No conflict of interest to declare. The authors alone are responsible for the content and writing of the manuscript. This work was supported by the Cancer and Traffic Injury Fund, the Research Foundation of the Department of Oncology at Uppsala University, the Swedish Cancer Society, Uppsala County Council and Uppsala University. The granting institutions had no role in the design of the study, data collection, analysis, interpretation or writing of the manuscript.