The challenge of recruiting in primary care for a trial of telemonitoring in asthma: an observational study

Abstract

Aim

Achieving target recruitment in randomized controlled trials (RCTs) is challenging. This paper compares our experience of recruiting for an RCT with the predictions made in our proposal.

Methods

Participating UK primary care practices searched their computer databases to identify patients (12 years and over) with asthma who may be poorly controlled. Postal invitations were sent to all patients identified. Respondees were prescreened by phone, to assess their asthma control and establish their mobile phone suitability. Potentially eligible patients were booked for a trial recruitment visit.

Results

We recruited 288 patients (2.4% of those invited) across 32 practices, with a total list size of 311,926 patients. This compares to our predicted recruitment of 312 patients from a population of 72,000 patients in six to eight practices. In addition to the recognized problem of poor response rates, the major challenges were insufficiently discriminating computer searches and incompatibilities between mobile phone handsets, networks and the asthma application.

Conclusion

Our data have implications for clinicians, managers, and researchers in primary care. Researchers in this area may wish to consider our data when designing their recruitment strategies. Improved coding of asthma morbidity data in clinical practice would ease identification of poorly controlled patients, both for clinical interventions and recruitment to trials. If telehealth is to become mainstream, there needs to be standardization of applications, operating platforms, and network capabilities.

Keywords:

• asthma
• primary care
• telehealth
• recruitment
• randomized controlled trials

Introduction

Achieving target recruitment in randomized controlled trials (RCTs) is often extremely difficult,¹ with some of the most effective strategies for improving recruitment considered ethically controversial (eg, use of opt-in rather than opt-out recruitment, telephoning nonresponders) or significantly affecting trial design (eg, open rather than blinded placebo-controlled trials). Factors, such as pressure of time, may affect recruitment in primary care, and trials are often jeopardized due to the inability to enter sufficient patient numbers.² The impact of strategies directed at researchers (eg, increasing contact with recruitment sites) or participants (eg, presenting trial information on videos or computer presentations) is variable. In addition, stringent eligibility criteria may substantially reduce the pool of potentially eligible participants.^3–5

Our “Can Your Mobile Phone Help Your Asthma” (CYMPLA)⁶ trial hypothesized that by integrating asthma monitoring with the day-to-day use of the patients’ own mobile phone, we would improve engagement with self-management and thus improve control.⁶ Key eligibility criteria, therefore, were that the patients should have poorly controlled asthma (defined as Asthma Control Questionnaire [ACQ] score ≥ 1.5)⁷ and have a mobile phone compatible with the t+ asthma application (OBS Medical, Abingdon, UK) used in the trial.

To achieve our required sample size of 125 completing, in each arm of the trial, we aimed to recruit 312 patients with poor asthma control.⁶ Our proposal stated that this could be achieved by approaching 1880 patients, identified as potentially eligible, from the asthma registers of six to eight practices (total list size approximately 72,000 patients). Previous exploratory work suggested that at least 45% of the population with asthma was poorly controlled. This paper reports the reality of recruiting for the trial.

Method

The CYMPLA trial was conducted in UK primary care in 2008–2009, with ethics approval from Hertfordshire Research Ethics Committee and governance approval from all relevant primary care trusts. Recruitment occurred over 9 months (July 2008–March 2009).

Recruiting practices

Practice recruitment was assisted by local Primary Care Research Networks (PCRNs), which were established by the National Institute of Health Research and funded by the Department of Health and were “dedicated to expanding clinical research in primary care where the majority of patient contacts take place.” The PCRN also provided funding to practices to cover the costs of participating in the trial. Our initial projections (Figure 1) led us to believe that we would be able to recruit our target population in the area covered by Norfolk and Waveney Primary Care Trust (PCT). We showcased the CYMPLA project at a PCT educational event for general practitioner (GP) practices and supplemented this with proactive practice recruitment by the PCRN facilitators. Practices were also actively recruited by our researchers (SM and SDM) by telephone and practice visits.

Figure 1 These flowcharts summarize and compare our experience of recruiting for an RCT with the predictions made in our proposal.

Abbreviation: RCT, randomized controlled trial.

Searching primary care electronic records and patient identification

Participating practices searched their computer databases to identify patients (aged ≥ 12 years) with asthma. Initially, we used a data extraction tool designed to search practice databases for indices of poor control (eg, frequent exacerbations, overuse of short-acting bronchodilators) to identify the potentially eligible population. The search strategy is explained in Table 1. In practices where it was not possible to run the search tool (because of incompatible computer systems or unorthodox or inadequate coding of events), participating practices used in-house search facilities. Although we searched for evidence of poor control, we were unable to apply rigid criteria because of the variable coding strategies in different practices. Exclusion criteria (other significant lung disease, under specialist care for severe/difficult asthma, unable to communicate in English or use a mobile phone, other significant social/clinical problems) were checked by inspecting the paper/computer records and then by the patient’s own GP. The practice posted invitations to all potentially eligible patients.

Table 1 Computer search strategy

	Search criteria
Demography	Age 12 years or over, AND
Diagnosis	Asthma diagnosis ever, AND
	EXCLUDE: diagnosis of COPD, chronic bronchitis, emphysema, bronchiectasis, pulmonary fibrosis, pneumoconiosis, lung cancer, OR sarcoidosis
Poor control	Prescription for asthma treatment (short-acting bronchodilator, inhaled steroid, add-on therapy) in the previous 6 months Prescription for prednisolone 5 mg tablets in the previous 12 months, OR
	Acute exacerbation of asthma in the previous 12 months
	EXCLUDE IF: code for asthma resolved
Practice specific criteria	Where practices routinely recorded useful additional data (eg, asthma-related admissions/A&E attendances/morbidity scores) these could be included

Abbreviation: COPD, chronic obstructive pulmonary disorders.

Prescreening potentially interested patients

Patients who expressed interest in participating by returning the reply paid response form were phoned by a researcher (SM or SDM) and prescreened against our eligibility criteria, using a standardized protocol. We checked that their mobile phone was compatible with the t+ asthma application and that they subscribed to a compatible network. By asking the six questions in the ACQ, we established that their asthma was poorly controlled asthma (defined as an ACQ ≥ 1.5).⁷

Trial recruitment visit

Potentially eligible patients were invited to a trial recruitment visit in their own practice where eligibility was confirmed. All consenting patients satisfying the inclusion and exclusion criteria were entered into the trial and allocated to mobile phone or paper-based monitoring.

Results

The actual recruitment process for the CYMPLA trial compared with our predicted schedule is given in Figure 1. Automated computer searches were insufficiently discriminating, identifying 3.6% of the practice population as potentially having poorly controlled asthma, instead of the anticipated 2.6%. A total population of 13,101 (mean [standard deviation, SD] age 48.2 [17.9], 63% female) was identified of whom 1020 were excluded by their practice. The response rate to our postal invitation was 8.4%, about a quarter of the 34% predicted in our proposal. Accordingly, we had to increase the number of participating practices fourfold. Of the 1016 patients who expressed an interest in participating, 623 (59%) patients were excluded by telephone prescreening, 470 of them because their asthma was well controlled, 124 had an incompatible mobile phone handset or network, and 29 for other reasons (such as being under specialist respiratory care).

The trial recruitment visit normally occurred within a week after the telephone prescreening. In that time 37 (9.4%) of the 393 patients booked for a trial recruitment visit had improved their control and were found to be no longer eligible. There was a statistically significant reduction in ACQ score between pre-screening and baseline (median [interquartile range] change was 1.0 [0.5–1.42], P < 0.001). This is illustrated for the 37 patients in Figure 2. This was a much larger number than we had expected. We had not set up any mechanism to record the reasons why people became well controlled, but anecdotally we received comments such as “doing the questions made me realize that my asthma was not properly controlled” and “as I was going into a trial I thought it was best if I started to take my inhalers.” A further 47 did not attend, and 21 were excluded for other reasons (such as relocation). This left a total of only 288 patients (2.4% of those invited) for recruitment and randomization: 145 to the mobile group and 143 to the paper group. The demography was similar to the whole potentially eligible population (mean [SD] age 48.1 [17.9], 64% female). After nine postrandomization withdrawals we were able to include 139 patients in each group for the intention to treat analysis.⁶

Figure 2 This graph shows the improvement in ACQ scores between telephone prescreening and baseline in 37 patients. Change in ACQ 1.0 [0.5–1.42], P < 0.001.

Abbreviations: ACQ, Asthma Control Questionnaire; CYMPLA, Can Your Mobile Phone Help Your Asthma trial.

We encountered two technological problems: handsets that were incompatible with the t+ asthma application (n = 110) and mobile phone networks that did not carry or whose subscriber tariffs did not include data carriage (n = 69). To address this, we offered to lend phones to those with compatible networks; 55 people accepted this arrangement.

Discussion

In contrast to the optimistic predictions in our protocol, our data demonstrate the challenges facing researchers recruiting for trials, and this represents a significant cost in terms of time and resources. In addition to the recognized problem of poor response rates, our experience highlights the limitations of using routinely collected data to search for potentially eligible patients and identified an additional difficulty imposed by the need to ensure compatibility of technology in a rapidly developing field. Furthermore, our screening instrument, the ACQ, seemed to provoke a change in behavior, inducing some people to restart their medications. In other words, this could be interpreted as a screening mechanism becoming an intervention in its own right.

Interpretation

Our findings reinforce the need to pilot the recruitment process in order to make realistic estimates and to design trials accordingly.⁸ We overestimated the specificity with which computer searches could identify patients with poorly controlled asthma. Our estimation (2.6% of the total list size) was based on a reported prevalence of active asthma in UK practice of 5.8%,⁹ combined with previous work that showed that 45% of people on asthma registers were poorly controlled.¹⁰ If the 44% of people found to be well controlled at the telephone prescreening call were representative of the total population invited, the rate of poor control in our study was 2.4% of the total population, which compares well with our estimate. Computer searches are limited by the quality of coding of routinely collected coded data, such as measures of control (morbidity scores, exacerbations) and data input of hospital admissions, emergency department attendances, and other out-of-hours contacts. In the UK, the recording of the Royal College of Physicians’ three questions¹¹ is now included as an indicator in the Quality and Outcome Framework,¹² which should enhance the accuracy of future searches. At the time of our study this was not the case.

The response rate of 8.4% to our mailed invitation was low compared to that obtained in an asthma trial (on which we based our estimate) that recruited 30% of the eligible population to a trial involving a face-to-face or telephone asthma review.¹³ That trial, however, included a telephone reminder to nonresponders from their own practice, a strategy known to increase recruitment rates¹ but which is often regarded by ethics committees as unacceptable coercion. In retrospect, it is likely that the context influenced the interest generated. Monitoring asthma with a mobile phone for 6 months with at least two review consultations was a substantial commitment and may have been less appealing than a single (possibly telephone) consultation.

Prescreening patients for poor control using the ACQ reduced the workload of the recruitment clinics and saved fruitless visits for patients by excluding ineligible participants. However, almost 10% of people whose ACQ was ≥1.5 at prescreening became well controlled by the time they attended for their baseline check about a week later. While this may represent the natural variation of asthma, it is possible that asking the morbidity questions during the prescreening conversation may have acted as an intervention in its own right. Anecdotally, a number of trial patients reported that being asked specific questions encouraged them to review their own asthma management. It is well recognized that patients underestimate their symptoms,¹⁴ and the simple expedient of routinely asking morbidity questions at an asthma review may facilitate improved compliance with treatment strategies.¹⁵ This may be an advantage in a clinical context but presents a dilemma for researchers who have to balance the advantages of reducing the workload of recruitment with the disadvantage of potentially influencing the patients’ asthma care. A further 47 patients (12% of those booked for trial recruitment) did not arrive for their appointment between screening and the recruitment visit. The reasons for this level of attrition are not known.

We were unable to recruit a significant proportion of interested patients because of compatibility problems with the patients’ mobile phones or networks. We were aware from our pilot work,¹⁶ that the t+ application was not compatible with some old or very new handsets, but we also experienced unexpected problems with new versions of previously compatible phones. We arranged to lend phones to those willing to use our handsets, although that did not overcome the network incompatibility that prevented some users from transmitting data. If telehealth is to become a mainstream reality, there needs to be standardization of applications, operating platforms, and network capabilities. The increased use of web-enabled phones using stable operating systems since the inception of this study should reduce this problem in any future work.

Strengths and limitations

In line with the principles of pragmatic research, we designed our trial to be as inclusive as possible with minimal exclusion criteria in order to maximize applicability of results obtained to unselected primary care populations.⁵

We were recruiting for a trial of mobile phone technology for people with poorly controlled asthma, and our experience may not be directly applicable to research in other health care systems, in other disease areas, or using other technology, although many of the issues (such as accuracy of coding, maximizing response rates, incompatibility and rapidly developing technology) are likely to apply in other contexts.

Conclusion and implications

Our experience of recruiting for the CYMPLA trial offers some key messages for researchers, funders of research, and clinicians. It is interesting to speculate whether we would have been awarded the grant had we given a more realistic appraisal of the recruitment process, at the time of application.

Applying evidence-based strategies for maximizing patient recruitment is essential but is probably insufficient to ensure efficient and effective identification of participants for clinical trials. Better coding of routine data in primary care and more sophisticated data extraction software could improve the focus of computer searches. The suggestion that asking standard morbidity questions may stimulate improved control is not only of significance to researchers considering prescreening for trial eligibility but also supports the clinical use of such tools as part of routine reviews. Our experience of incompatibility between telecommunication packages offers a salutary warning both to researchers in this field and also to health care systems investigating technological solutions to monitoring long-term conditions.¹⁴

Acknowledgments

This study was funded by Asthma UK. HP is supported by a Primary Care Research Career Award from the Chief Scientist’s Office of the Scottish Government. The Primary Care Research Networks in Norfolk and Yarmouth, East Kent, North of England, and Essex and Hertfordshire identified and recruited practices. We thank the practices, practice nurses, and administrative staff for their active participation and the patients who gave their time to participate in the trial. Professor Amanda Lee was the trial statistician. We thank Dr Andrew Wilson and Neil Kendle for serving on the ITSC and Dr Brian McKinstry and Dr Chris Burton who offered advice as collaborators. ISRCTN number: NCT00512837.

Contributorship: DR initiated the idea for the study and with HP led the development of the protocol, securing of funding, study administration, data analysis, interpretation of results, and writing of the paper. DP is a grant holder who contributed to development of the protocol, securing of funding, study administration, data analysis, interpretation of results, and writing of the paper. SM and SDM recruited practices and undertook the data collection. All authors had full access to all the data and were involved in interpretation of the data. SM wrote the initial draft of the paper, to which all the authors contributed. DR and HP are study guarantors.

Disclosure

The authors declare that they have no conflicts of interest.