Excellent adherence and no contamination by physiotherapists involved in a randomized controlled trial on reactivation of COPD patients: a qualitative process evaluation study

Abstract

Objective

To assess the adherence of physiotherapists to the study protocol and the occurrence of contamination bias during the course of a randomized controlled trial with a recruitment period of 2 years and a 1-year follow-up (COPE-II study).

Study design and setting

In the COPE-II study, intervention patients received a standardized physiotherapeutic reactivation intervention (COPE-active) and control patients received usual care. The latter could include regular physiotherapy treatment. Information about the adherence of physiotherapists with the study protocol was collected by performing a single interview with both intervention and control patients. Patients were only interviewed when they were currently receiving physiotherapy. Interviews were performed during two separate time periods, 10 months apart. Nine characteristics of the COPE-active intervention were scored. Scores were converted into percentages (0%, no aspects of COPE-active; 100%, full implementation of COPE-active).

Results

Fifty-one patients were interviewed (first period: intervention n = 14 and control n = 10; second period: intervention n = 18 and control n = 9). Adherence with the COPE-active protocol was high (median scores: period 1, 96.8%; period 2, 92.1%), and large contrasts in scores between the intervention and control group were found (period 1: 96.8% versus 22.7%; period 2: 92.1% versus 25.0%). The scores of patients treated by seven physiotherapists who trained patients of both study groups were similar to the scores of patients treated by physiotherapists who only trained patients of one study group.

Conclusion

The adherence of physiotherapists with the COPE-active protocol was high, remained unchanged over time, and no obvious contamination bias occurred.

Keywords:

• physiotherapy
• guideline adherence
• compliance
• bias
• randomized controlled trial
• chronic obstructive pulmonary disease

Introduction

Chronic obstructive pulmonary disease (COPD) is a systemic disease characterized by the progressive development of irreversible airflow limitation, leading to impaired muscle strength and exercise capacity.¹ Physiotherapeutic exercise programs given by specialized physiotherapists are a well-established part of the treatment of patients with COPD and are incorporated in rehabilitation programs,^2,3 self-management programs^4–6 and in near home rehabilitation programs.⁷ The reported effectiveness of these programs, however, varies considerably. Besides the program content, lack of adherence with the protocol by physiotherapists⁸ and occurrence of contamination bias may partly explain such variability in outcomes.

Effectiveness of health interventions can be seriously harmed by incomplete and incorrect execution of protocols by health care providers.⁹ Within randomized controlled trials assessment of protocol adherence is indicated, because protocol nonadherence may be one of the explanations for diminished or negative study outcomes. Assessment of adherence will give more insight in the delivery of the intervention under study. Whereas assessments of physiotherapists’ guideline adherence have been reported before,^10–14 assessments of physiotherapist’s adherence with study protocols are rare.

Besides protocol adherence, the occurrence of contamination bias can harm the effectiveness of an intervention.¹⁵ This bias results from cross-exposure between study arms (eg, when a control group is [partly] exposed to the intervention of interest in a randomized controlled trial) and can occur especially if randomization is performed at the patient level.¹⁶ As a result of randomization at the patient level, health care providers with knowledge of the study intervention can be involved in the treatment of intervention and control patients at the same time. This increases the risk of exchange of intervention elements. Evaluation of contamination bias may be advisable in randomized controlled trials, since this bias can lead to a reduction of the intervention effect.¹⁶

In the COPE-II study, an intensive, standardized community-based physiotherapeutic exercise program (COPE-active) was evaluated.⁴ Evaluation of contamination bias was of particular interest in this trial because the control group received usual care, which in some cases included regular physiotherapy, and randomization was performed at the patient level. As a consequence, physiotherapists could treat patients of both the intervention group and the control group.

We have developed and applied a practical procedure to assess physiotherapists’ adherence and contamination bias in the COPE-II study. Because adherence and the amount of contamination bias may have changed during the study, the degree of adherence of physiotherapists with the COPE-active protocol and the presence of contamination bias were evaluated during two separate periods within the COPE II-study (May–August 2005 and July–September 2006).

Methods

The design, inclusion criteria, intervention and outcome of the COPE-II study have been described previously.⁴ In the COPE-II study 159 outpatients with COPD were recruited (November 2004–July 2006). The study was approved by the Medical Ethics Committee of Medisch Spectrum Twente, Enschede, The Netherlands.

All patients attended a self-management education course, but only patients of the intervention group participated in the COPE-active program for a maximum period of 11 months. This training period per patient was divided in two parts: a ‘compulsory’ 6-month and a subsequent optional, but recommended, 5-month period. After the first 6-month period, patients had the opportunity to continue with the COPE-active program for another 5 months on a voluntary basis. Patients in the control group were allowed to receive regular physiotherapy as a part of their usual care treatment. Initiation of prescribed regular physiotherapy was also allowed.

The detailed content of the standardized COPE-active intervention has been described previously in an online repository. ⁴ The content of usual care physiotherapy cannot be described precisely, because its frequency and content were not standardized and varied considerably between patients. Frequency of treatment ranged between 1–3 sessions per week and whereas most sessions were directed towards training of respectively exercise capacity and muscle strength, the type of exercises and intensity differed considerably.

All physiotherapists who participated in the COPE-II trial were working in private physiotherapy practices in catchment areas of the Department of Pulmonary Medicine of Medisch Spectrum Twente, a large teaching hospital in Enschede, The Netherlands. They had all attended a national COPD course prior to the COPE-II study and were experienced in caring for COPD patients. Before the start of the COPE-II study, physiotherapists had to participate in an additional three-session course (11 hours in total) to refresh their knowledge about COPD in general and to standardize the content of the COPE-active-program. Physiotherapists were instructed to treat control patients according to the standards that were applied prior to the COPE II-study. So, physiotherapists were trained to use the new treatment, which they had to withhold knowingly, in the case of control patients.

Patients instead of physiotherapists were interviewed to avoid socially desirable responses by physiotherapists.¹⁷ Interviews were performed by one of the two independent interviewers during two periods in the study: period 1, May–August 2005 and period 2, July–September 2006. Interviewers were not blinded for the study group allocation of patients. Patients who were receiving physiotherapy (either COPE-active or regular physiotherapy) during one of these periods were asked for an interview. The goal of the interviews was to examine to what extent the COPE-active protocol was applied by their physiotherapists.

All patients were interviewed at home using identical, semistructured questionnaires. They were blinded for the purpose of the study and were unaware of the intended content of the COPE-active protocol. To distinguish patient adherence from protocol adherence by the physiotherapists, patients were not asked what activities they were actually performing, but what they were instructed to do. A total of 34 questions were scored. Although the questions were open in nature, a dichotomous score was attributed: (0) “not performed according to the COPE-active protocol” or (1) “performed according to the COPE-active protocol.”

Thirty-four questions were classified in ten different categories describing the features of the COPE-active training. During the study it became clear that one category “the choice for duration or interval training” could not be validly determined by only interviewing patients. Therefore, it was decided to determine the COPE-active score with nine aspects (30 questions) (Table 1). Adding together all aspect scores led to an overall score per patient ranging from 0 (no exposure to any aspect of COPE-active) through 9 (full exposure to the COPE-active-protocol). Because it is reasonable to assume that after 6 months of physiotherapy, the therapy is no longer aimed at improving exercise capacity and muscle strength but at maintaining the improvements achieved, it was decided to leave the two aspects regarding structural increase in intensity of the exercises (aspect 4 and 7) out of the overall score for patients with physiotherapy after 6 months. Thus, in patients who had been training for more than 6 months at the time of the interview, the overall score ranged from 0–7. Finally, all overall scores were converted into percentages, so scores of patients receiving physiotherapy for different lengths of time could be compared: 0% (no aspects of COPE-active) through 100% (full implementation of the COPE-active-protocol).

Table 1 The nine aspects of the COPE-active program, with number of items per aspect, the score per item, and the maximum score per aspect

Aspect	Description	Number of items per aspect	Score per item	Maximum score per aspect
1	Frequency of physiotherapeutic sessions (twice a week in the first 6 months; once a week thereafter)	1	1	1
2	Duration of physiotherapeutic sessions (60 minutes)	1	1	1
3	Type of exercises in physiotherapeutic sessions (cycling, walking, walking stairs, lifting [optional: push/pull])	5	0.25	1*
4†	Increasing intensity of exercises	5	0.25	1*
5	Number of exercise sessions at home per week (at least one)	1	1	1
6	Type of exercises at home (one strength exercise of the lower extremities + one strength exercise of upper extremities + one exercise for endurance training: walking or cycling)	7	0.14	1
7†	Increasing intensity of exercises at home	7	0.14	1
8	Within muscle strength training session: three repeating series	2	0.5	1
9	Use of Borg scores	1	1	1
	Total	30		9

Notes:

* Extra exercises (aspect 3: push/pull; aspect 4: push/pull, rowing, etc) could be scored, resulting in a surplus value of 0.25. Final aspect score cannot exceed 1;

† in patients receiving physiotherapy for more than 6 months these aspects have been excluded because they are no longer applicable.

Data analysis

Data analyses were performed using SPSS software (version 12.0; SPSS Inc, Chicago, IL). Descriptive statistics were used to compare the scores of the two study groups at two different periods in time and the scores per physiotherapist. Between-group differences were tested with the Mann–Whitney U test.

Results

In COPE-II 159 patients were included (COPE-active, 80; Control, 79). Sixty-seven patients (87.0%) participated in the COPE-active program. Twenty-five control patients (32.9%) received regular physiotherapy during the 12-month follow-up (Figure 1).

Figure 1 Flow diagram of patient progress through the COPE II-study.

Fifty-three of the included 159 patients were receiving physiotherapy (COPE-active or regular physiotherapy) during one of the two study periods in which the interviews were performed. No appointment for an interview could be made with two patients (COPE-active, n = 1; regular physiotherapy, n = 1). Therefore, 51 patients were finally interviewed (period 1: COPE-active, n = 14; regular physiotherapy, n = 10; period 2: COPE-active, n = 18; regular physiotherapy, n = 9). The 51 patients were trained by 18 different physiotherapists, of whom 15 were participating in the COPE-II study.

In both periods, patients in the COPE-active groups reported very high scores on protocol aspects (median scores 96.8% and 92.1%, respectively) (Figure 2). A small and nonsignificant reduction could be observed over time (P = 0.44). Combining scores of the two periods, all but two of the COPE-active patients scored between 80% and 100%. The two patients scoring below 80% (78.7% and 61.9%) were both trained in the second time period. In both periods there was a clear and significant difference between the COPE-active group (median scores 96.8% and 92.1%, respectively) and the group receiving regular physiotherapy (median scores 22.7% and 25.0%, respectively) (P < 0.001) (Figure 2).

Figure 2 The median, 25th and 75th percentile of total scores and outliers (*) of patients receiving regular physiotherapy and COPE-active in two different time periods within the COPE-II study.

In Figure 3 all the individual patients’ scores (n = 51) are linked to the corresponding 18 physiotherapists. No overlap in scores was detected between the COPE-active group and patients receiving regular physiotherapy. Seven physiotherapists (1, 4, 5, 6, 7, 11, and 12) trained patients of both study groups. The median scores of patients treated by these latter physiotherapists (COPE-active: 91.7%, n = 17; regular physiotherapy: 23.1%, n = 13) did not deviate significantly from the scores of patients treated by physiotherapists who only treated patients of one study group (COPE-active: 100.0%, n = 15; regular physiotherapy: 26.4%, n = 6; P > 0.05).

Figure 3 Individual patient’s scores of the COPE-active group (○) and the group receiving regular physiotherapy (△) per physiotherapist.

Notes: Seven physiotherapists (1, 4, 5, 6, 7, 11, and 12) trained patients of both study groups and three physiotherapists were not participating in the COPE-II study (13, 17, and 18). In four cases two patients trained by the same physiotherapist had exactly the same score (*).

The contribution of the different aspects to the total scores of both study groups for two subsamples (patients with <6 months physiotherapy and patients with >6 months physiotherapy at the time of the assessment for the current analyses) is presented in Figure 4. In both subsamples of COPE-active patients, all aspects contributed to the total score. In the regular physiotherapy group, the contribution of the different aspects to the score was less consistent over time.

Figure 4 Relative contribution (%) of each of the nine aspects constituting the (mean) COPE-active score, for both the COPE-active patients and those receiving regular physiotherapy.

Notes: Data are divided in patients receiving less than six months (maximal contribution of 9 items) and more than six months (maximal contribution of 7 items) of physiotherapy. *(1) Number of physiotherapeutic sessions per week; (2) duration of a physiotherapeutic session; (3) type of exercises in physiotherapeutic sessions; (4) increasing intensity of physiotherapeutic sessions; (5) number of exercise sessions at home; (6) type of exercises at home; (7) increasing intensity of exercises at home; (8) number of repeating sessions in training; (9) use of Borg-scores.

Discussion

This study suggests that the COPE-active protocol was appropriately performed by the physiotherapists in patients of the COPE-active group, and that physiotherapists maintained this high level of adherence during the study. Furthermore, the difference between the scores of the COPE-active group and the group receiving regular physiotherapy was marked and did not diminish during the COPE II-study.

The main reasons for not achieving the maximum score of 100% in the COPD active group can be found in the scores regarding home work sessions. A plausible explanation is that some physiotherapists did not give home exercise instructions. Another explanation might be bias introduced by patient report (ie, patient nonadherence). Patients can claim that no instructions for exercise at home were given, because they did not act on these instructions. Interviewers have tried to create a noncondemning and confidential atmosphere by not asking questions about whether or not exercises were really performed to avoid these socially desirable answers. A final explanation might be found in individual patient characteristics that limit full implementation of a protocol (eg, frequency of COPD exacerbations, comorbidities).¹⁰

The adherence levels of physiotherapists in the COPE- active group are high compared with adherence levels reported in previously published studies.^10–14 A study regarding an active implementation strategy in low back pain patients reported an overall adherence of 42%.¹⁰ A prospective cohort study among Dutch physiotherapists showed that adherence to the clinical guideline osteoarthritis of hip and knee varied between 46% and 100%.¹⁸ This difference in adherence is not surprising, though, as these two studies involved implementation interventions that typically aim to promote adoption of new treatment guidelines among a population of professionals. Our study included a selected sample of professionals who were motivated to participate in an effectiveness trial of an innovative treatment, and consequently can be regarded as adopters of this new treatment. Other explanations for the high adherence level of physiotherapists in our study may be the use of a thorough training which was compulsory for physiotherapists, and the awareness among physiotherapists that there was a reasonable chance that the content of their treatment would be evaluated (quality control).

Exposure of patients in the usual care group to the COPE-active treatment was largely limited to two aspects: “duration of physiotherapeutic session” and “type of exercises within physiotherapeutic sessions.” This was not unexpected because training aimed at improvement of exercise capacity in patients with COPD will frequently last 60 minutes and regularly incorporate walking and/or cycling. Therefore, these two aspects seemed not to be very unique for the COPE-active program and retrospectively it can be concluded that they did not contribute to the contrast between COPE-active and regular physiotherapy. The negligible exposure of usual care patients to the other seven COPE-active aspects, as well as the low and relatively stable overall COPE-active score of these patients over time, underline the absence of contamination bias. Moreover, the difference in scores between both study groups was as obvious in practices treating patients of both study groups, in which the risk of contamination would be higher, as in practices treating patients of only one of the two study groups.

The absence of contamination bias is probably a result of instructions given to physiotherapists prior to the study, namely to continue with treatment of control patients according to standards they applied before they were trained in the COPE-active protocol. As a result, physiotherapists may have avoided application of any COPE-active protocol-like strategies that they would normally have applied. This may have enlarged the differences between the two study groups.

Previous studies have used procedures such as self-report, ¹² individual patients’ forms,^10,14 and patient registration software¹¹ to assess physiotherapists’ adherence. Self-report of adherence may be subject to bias.¹⁹ We chose to interview patients instead of physiotherapists to avoid social desirability. Patients were blinded for the purpose of this study and were unaware of the detailed content of the COPE-active protocol. Therefore, it is unlikely that respondents in this study provided biased reports. It should be noted that the interviewers in this study were not blinded for the study group allocation of patients. So, interviewer bias (intentional or unintentional prompting by the interviewer, which affects the patient’s response) might have occurred, but cannot explain the vast contrast between both study groups.

The results of this evaluation give more insight into the delivery of the COPE-active intervention in the COPE-II study. Many randomized controlled trials lack such a process evaluation, leading to a black box concerning the effectiveness of the intervention. Especially in the case of negative effects, one has to be sure that the intervention was provided according to protocol and that the effect was not diluted due to contamination bias. The results of the evaluations described in this paper imply that the adherence of physiotherapists with the studied protocol was excellent. No obvious contamination bias occurred and therefore did not influence the final COPE-II results.⁴ The method applied seems suitable to assess how well treatments (both experimental and control) are adhered to in RCTs.

Acknowledgments

We wish to thank the patients and physiotherapists who took part in this study and furthermore gratefully acknowledge the data managers Betty Rinsma and Petra Meerlo. Additionally, we thank Kristel Broekkamp for interviewing patients.

Disclosure

The authors report no conflicts of interest in this work.