Is QUOTE-IBD a valid questionnaire for measurement of quality of care in IBD? A validation study of the Swedish version

Abstract

Background: Quality of care has gained increased attention in IBD. The questionnaire Quality of Care Through the Patient’s Eyes – Inflammatory Bowel Disease (QUOTE-IBD) was the first published validated IBD-specific quality of care questionnaire. The aim of this study was to validate the Swedish version of the QUOTE-IBD.

Methods: Adult outpatients (n = 400) at a gastroenterology clinic in the south-east of Sweden were asked to fill in the questionnaire. For evaluation of construct validity, patients also responded to one global item for each health care dimension in the QUOTE-IBD, as well as for their overall experience of quality of health care.

Results: All quality of care dimensions (QI) correlated significantly (p < .05) with their respective global dimensional item (r = 0.016–0.43), except for accommodation (r = –0.02. Test–retest (n = 32) gave significant results for all the dimensions r = 0.31–0.80 (p < .05), except for accommodation (–0.15, p = ns).

Conclusions: The construct validity of the Swedish version of QUOTE-IBD is moderate. This indicates that the QUOTE-IBD may not fully cover the health care aspects important to patients. The high number of item non-response for Performance may be related to the questions being too specific, which may also contribute to the moderate level of construct validity. The reliability is moderate and the internal consistency is good.

Keywords:

• Inflammatory bowel disease
• validation
• quality of care
• QUOTE-IBD

Introduction

Inflammatory bowel disease (IBD) comprises Crohn’s disease (CD) and ulcerative colitis (UC). IBD are chronic diseases, often with an early onset in life. Symptoms can be burdensome and influence social life, including relationships, studies and working life [1,2]. Patients with active IBD experience lower health-related quality of life compared to a healthy population [3–5]. The self-image can be affected and the disease often rise feelings of powerlessness due to the un-predictability of the disease [1].

For patients with IBD, frequent contact with health-care providers is crucial to maintain remission and a better control of the disease [6,7]. Therefore, quality of care is vital to ensure the best possible outcome for these patients [8]. It has been shown that IBD-specialized clinics have better patient outcomes regarding treatment-related complications and follow-up routines compared to general practitioners [7]. Early intervention and best practice therapy may reduce complications that require hospitalization and/or surgery [9]. This suggests that a high quality of care may prevent worsened disease and thus increase patients’ quality of life [10].

The interest in quality of care in patients with chronic diseases has increased [8], but a quantitative measure of quality of care among patients with IBD was missing until the development of Quality of Care Through the Patient’s Eyes – Inflammatory Bowel Disease (QUOTE-IBD) [11]. The basic questionnaire QUOTE was developed by The Netherlands Institute for Health Services Research (NIVEL) specifically for chronically ill, elderly and disabled people who use specialized health-care services. The foundation of QUOTE is based on Donabedian’s ‘structure-process-outcome’ model defining quality of care. The model defines ‘structure’ as characteristics of the health-care system, ‘process’ implies the actions performed by the health care and the patient, and ‘outcome’ refers to the patient’s subsequent health status; the end result of care . Access to care, performance of care, and the patient’s satisfaction are focused upon [12].

NIVEL further developed QUOTE with disease-specific questions for chronic diseases, such as chronic pulmonary disease, HIV, diabetes, rheumatism and IBD. For IBD, disease-specific questions were developed through a qualitative patient-oriented consultation study focusing on issues important for patients with IBD. These questions were added to the questionnaire in the first study by van der Eijk et al. [11]. The IBD-specific questionnaire is validated in several European languages (Dutch, Danish, English, Greek, Italian, Norwegian and Portuguese) [13–16]. Earlier studies validating QUOTE-IBD have shown moderate to high validity and reliability [14,15].

Aim

The aim of this study was to validate the Swedish version of QUOTE-IBD.

Methods

Study design

This is a validation study performed as a cross-sectional postal survey.

Questionnaire

QUOTE-IBD contains 10 generic questions and 13 IBD-specific questions on patients’ assessment of quality of care. The questions comprise two categories: Importance, the weight that the patient has given to various care aspects and Performance, the experiences regarding the function of medical practices and health-care staff for each aspect of care. These two combined together measure the Quality Impact as a score [11].

Responses are given on four-point Likert scales. In the category Importance, the alternatives are Not important, Fairly important, Important or Extremely important. Questions in Performance are answered by: No, Not really, On the whole, yes or Yes. A factor analysis of the questionnaire yielded eight dimensions: competence, courtesy, accessibility, information, continuity of care, accommodation, autonomy and costs [11].

It is proposed that patients do not answer questions about just one single experience of health care, but about longer time periods of disease or overall perception. Therefore, this questionnaire should focus on the previous 12 months [11].

The QUOTE-IBD questionnaire has previously been translated from English into Swedish and back-translated into English by bilingual translators. Small adjustments to the Swedish health care system were made. For example, in Sweden, many IBD patients in stable remission have their annual routine checkup with a nurse instead of a physician, which is why some questions were re-phrased.

Participants

Patients (n = 400), 18 years or older, diagnosed with CD or UC and scheduled to an IBD-specialized clinic at a university hospital in the south-east of Sweden between February and June 2014, were invited to participate in the study.

Data collection

The patients were sent information by postal mail about the study through an information letter along with a written consent form, the QUOTE-IBD questionnaire and a pre-paid envelope. Reminders were sent two and four weeks after the first dispatch. Thirty-two random patients participated in a test–retest approximately two weeks after they answered the first questionnaire. Medical records of all participants were surveyed by the first author for diagnosis, year of diagnosis, disease activity and concomitant chronic diseases.

Ethical considerations

Ethical approval was obtained from the Regional Ethical Review Board, Linköping, Sweden (Dnr 2012/214-32). The study was carried out in accordance with the Declaration of Helsinki [17]. The study information described the purpose and method of the study, its confidential and voluntary nature, and that it was possible to quit study participation at any time without explanation. All patients were invited to contact the principal investigator if they had any questions about the study.

Statistics

Comparisons between categorical data were calculated with chi-square test. Two-group comparisons with continuous variables with normal distribution were calculated with t-test. All group comparisons on continuous ordinal data were calculated with the Mann–Whitney U-test, and comparisons between three or more groups were calculated with the Kruskal–Wallis test. Multiple comparisons were adjusted with the Bonferroni method.

For calculation of Performance, the first and second response alternatives on the four-point Likert scales were combined, as well as the third and fourth response alternatives [11].

The correlation between the global dimensional item for each dimension and the Performance, and Quality Impact score for each dimension was calculated with Spearman’s correlation coefficient. Score for Quality Impact was calculated through the formula (quality impact  =  10 – (importance × performance)) [11]. VAS scores were calculated as millimeters of the scales, where 0 mm meant the worst experience and 100 mm indicated the best experience. Total care was calculated as mean values of all eight dimensions in Performance and Quality Impact.

Internal consistency was estimated as Cronbach’s alpha for the 13 IBD-specific questions for comparisons with earlier studies. In this study, Cronbach’s alpha was also measured for the Importance and Performance. Test–retest for Quality Impact as well as Performance was calculated with Spearman’s correlation coefficient.

A conformational factor analysis was performed on the QUOTE-IBD to see if the original dimensions could be verified. The principal component analysis was used with varimax rotation for factor extraction. The number of factors was selected with latent root criterion (Eigen value >1). Each item was affiliated to the factor on which it exerted the greatest loading. Factor loadings below 0.40 were not considered.

All statistical analyses were performed with SPSS version 22 for Windows (SPSS Inc., Chicago, IL). A p value  < .05 was set as significant.

Results

A total of 234 patients participated in the study, giving a response rate of 60.7%. Forty-five percent had CD, 55% were women and the median age was 49.6 years (Table 1). There was no significant difference between the participants and the non-responders regarding diagnosis, age or sex.

Table 1. Demographic data.

Health-related data %	UC n = 124	CD n = 110	p Value
Female/male	56.2/42.7	53.6/46.3	.74
Smoker, yes/no	8.3/90.3	13.7/86.3	.73
Relapse last 6 months	23.3	20	.85
Relapse last 6–12 months	34.2	23.6
No relapses last 12 months	42.5	56.3
Age, mean/SD	50.2/18.4	49.1/16.0	–
Occupation %			.19
Employed	55.8	57.0
Student	9.2	5.6
Unemployed	5.0	1.9
Long-term sick-leave	0.8	4.7
Retired	25.0	22.4
Other	4.2	8.4
Highest finished education %			.98
Primary school	21.7	20.6
High school	40.0	40.2
University	38.3	39.3
Medical
Other chronic disease, yes/no %	30.0	29.1	.88
Disease duration mean years/SD	12.6/12.9	20.9/18.4	.001

UC: ulcerative colitis; CD: Crohn’s disease; SD: standard deviation.

The questionnaire was completely filled in by 56% of the respondents, and the nine global health care items used for validation were responded by 82% of the participants.

Construct validity assessed with Spearman’s correlation coefficient between Performance and the global items for each of the dimensions were (excluding costs and autonomy as in the original article [11]) significant for five of the QI-dimensions (0.43–0.02) and Performance (0.49–0.02). Data for accommodation were not significant (Table 2).

Table 2. Total score as well as dimensional scores of QUOTE-IBD and VAS-scales for quality impact and performance scores.

Care dimension	Quality impact score	Performance
Total care	0.42**	0.45**
Information	0.40**	0.42**
Courtesy	0.43**	0.43**
Competence	0.16*	0.25**
Accessibility	0.36**	0.49**
Continuity of care	0.38**	0.41**
Accommodation	–0.02	–0.02

Spearman’s correlation coefficients.

* p ≤ .05.

** p ≤ .01.

Cronbach’s alpha coefficient was 0.78 for the 13 IBD-specific questions. Cronbach’s alpha for the Performance part of the questionnaire was 0.86, and for Importance 0.89.

Test–retest (n = 32) for Performance gave significant results for the Total care score and all dimensions, except for Accommodation, whereas QI demonstrated significant results for all dimensions but Accommodation, Autonomy and Total care score (Table 3).

Table 3. Test–retest (n = 32) mean values for quality impact (IQ) and performance.

Dimension	QI Baseline mean	QI Follow-up mean	QI Spearman’s r	Performance Baseline mean	Performance follow-up mean	Performance Spearman’s r
Total care	8.5	8.7	0.57	0.9	0.8	0.77**
Information	7.8	8.0	0.80**	0.76	0.79	0.93**
Courtesy	8.4	8.9	0.73**	0.80	0.89	0.62**
Competence	8.3	8.3	0.54**	0.78	0.84	0.59**
Accessibility	8.0	8.7	0.51*	0.73	0.83	0.79**
Continuity of care	8.7	8.2	0.54*	0.84	0.86	0.47*
Accommodation	9.1	9.4	–0.15	0.87	0.96	–0.11
Autonomy	9.3	8.8	0.31	0.93	0.93	0.56**
Cost	8.9	9.3	0.80**	0.89	0.93	0.80**

* p < .05.

** p ≤ .01.

There were no significant differences between the UC and CD patients in Importance, Performance or Quality Impact score.

The item nonresponse in the Performance part varies between 1.7% and 22%. The three questions with the highest internal drop-out were: ‘My physician makes sure I see a specialist within two weeks after being referred to one’ (21.4%), ‘My physician communicates with other health- and social care providers about services I require’ (22.2%) and ‘my physician makes sure there is a competent stand-in when he/her is not available’ (20.5%). The internal drop-out for Importance varied between 0.4% and 3%.

In Importance, few of the respondents answered that aspects of health care were ‘not important’ (median 1.3%). The item with the highest percentage for ‘Not important’ was for the question ‘Health-care personal should not keep me in the waiting room more than 15 minutes’ (9.4%).

To confirm the dimensions of the QUOTE-IBD, a principal component factor analysis with varimax rotation for factor extraction was used. Questions were spread slightly different than the original article (Table 4). For example, one dimension contains questions from the dimensions Continuity of care, Competence, accessibility and courtesy.

Table 4. Mean values for all dimensions.

Care dimension	I	P	QI
Total care	7.41	0.16	8.84
Information	8.12	0.29	7.56
Courtesy	7.79	0.18	8.61
Competence	7.67	0.21	8.48
Accessibility	7.01	0.22	8.38
Continuity of care	7.26	0.24	8.28
Accommodation	7.17	0.04	9.73
Autonomy	7.97	0.09	9.34
Costs	6.39	0.05	9.71

I: importance ranging from 0 (least important) to 10 (very important); P: performance ranging from 0 (best performance) to 1 (poor performance).

Discussion

IBD is a chronic disease that sometimes has a significant impact on the patient’s quality of life and often associated with a frequent use of healthcare [18]. Thus, an adequate quality of care plays an important role for patients living with chronic diseases such as IBD. Therefore, the need for a valid and reliable instrument to assess quality of care in IBD is well founded.

The original QUOTE-IBD questionnaire has shown to be valid and reliable [11]. Responses to questionnaires assessing patient’s perception are influenced by cultural context. The aim of our study was to develop a valid and reliable Swedish version of QUOTE-IBD.

This study shows moderate correlations between the global quality of care items and the total and dimensional scores for Performance, and the Quality Impact score, indicating moderate construct validity for the QOUTE-IBD. The correlations in this study were similar to the Spanish validation study (n = 103), slightly higher compared to the Portuguese validation study (n = 114), but lower than the Dutch validation study (n = 162) and significant lower than the Greek validation study (n = 50) [15,16].

The moderate correlations in this study as well as in the Spanish and Portuguese studies imply that the QUOTE-IBD questions do not in full cover the experience that patients express in their responses on the global items. It is unclear what aspects this gap stands for and why the size of the gap varies between the different validation studies. It could be due to socio-cultural differences on the view of health and sickness and patient’s expectations on health-care. It could also be linguistic factors about how the questions in the QUOTE-IBD questionnaire is formulated and understood by patients. The patients in this study considered the Questions in the QUOTE-IBD as important. The item nonresponse in the Importance part was only between 0.4 and 3%. However, it could be that patients had not experienced what was asked for during the specified time span, or had additional issues they also considered important. The questions may be too specific and a response alternative that ‘the item is not relevant’ is lacking. That may explain the high item non-response for the Performance part, where the item nonresponse was up to 22.2%.

The high item non response in performance was also seen in the Portuguese validation study where only 59% completed the questionnaire in full [16]. The reason for this could thus be low content validity and this in turn might explain the moderate construct validity.

The questions with high item nonresponse mutually concern either a specific profession and/or a specific healthcare event. The responders may not have experienced that event or not had contact with that specific profession in the last 12 months. The lack of an alternative for ‘not relevant to me’ obstructs the matter. A solution to investigate is rephrasing of the questions to be more general. For example, could the item ‘my physician makes sure there is a competent stand-in when he/her is not available’ be rephrased to ‘I feel that there is a competent stand-in even when my physician is not available’.

Test–retest resulted in a lower correlation compared to both the Greek and Dutch studies [11,15], higher than the Portuguese study [16], but close to the Spanish results [14]. In this study, 32 patients participated in the test–retest which gives the Swedish questionnaire moderate reliability. The dimension Accommodation deviates from baseline to follow up. The reason for this has not been clarified, but if the responder does not consider the aspects asked for relevant, the answer may vary from time to time.

There are a few issues of the QUOTE-IBD’s suitability for clinical practice; the complicated and cumbersome calculations with scales running opposite ways, the merging of a four-point Likert scale into two answers in Performance as well as the large item nonresponse.

In this study, few responders chose ‘not important’ for areas in importance, which opens up for further discussion about the whole point of importance as part of the questionnaire. Questions about courtesy, such as the importance of having confidence in the physician or nurse, is rarely ‘not important’. The significance of questions about importance in general has been questioned before [19]. One important aspect when using questionnaires to measure patient-reported experience is the burden posed on the patients when asking them to answer questions that are not relevant [19]. This was also seen in a study by Pihl Lesnovska et al. [20] which concluded that all questions (except two) in the QOUTE were given the highest rating of importance by all patients. This is not surprising as the questions were included in the questionnaire because they are known to be of importance to patients [20]. It should therefore not be necessary to repeat the importance questions for each aspect in the questionnaire. However, in matters where individual patients find various aspects to be different, importance to them, this diversity on the importance is seen in questions such as decision-making [21–23]. Several obstacles have been identified for shared decision-making, including health personnel’s communication skills and lack of time. Younger patients have been shown to prefer more collaboration with physicians in decision-making, whereas older patients wanted the physician to make decisions independently of the patient [24]. In addition, Henderson and Shum [25] have shown that patients wish for more participation in the decision-making process if the medical condition is assessed as less severe. Patients with a high-level education preferred a more active participatory role in medical decision-making [24].

Strengths and limitations

There is no standard questionnaire for quality of care in IBD to compare the QOUTE-IBD with. The validation of the QUOTE-IBD in the current study and previous validation studies of the questionnaire therefore rest on external construct validity where the total and dimensional QUOTE-IBD scores are compared with corresponding global items responded on VAS.

All patients in the study had IBD, but many of them also had ongoing comorbidities. These patients are most likely patients at other clinics as well, and therefore it is possible that they have answered the QUOTE-IBD from a general impression of quality of care, especially when it comes to questions about assessments on health-care in various aspects, as they may have more than one health-care contact. Therefore not only in this study, but in general it may be difficult to fully interpret the result when measuring the quality of care among patients with numerous care contacts.

In conclusion, this study shows that the Swedish version of QUOTE-IBD has moderate validity. Altogether, the questionnaire needs development to achieve sufficient psychometric qualities.

Acknowledgements

The authors thank all patients for participation.

Disclosure statement

There are no conflicts of interest.

Additional information

Funding

This project was funded by the Department of Gastroenterology, Linköping University, Sweden.