Letter to the Editor

Re Letter to the Editor Re: Compalati E, Canonica GW. Efficacy and safety of rupatadine for allergic rhinoconjunctivitis: a systematic review of randomized, double-blind, placebo-controlled studies with meta-analysis. Curr Med Res Opin 2013 Jul 5

Dear Editor,

We read with great interest the letter of Drs Berger and Gudehithlu1, raising important criticism against our recent systematic review on rupatadine for the treatment of allergic rhinoconjunctivitis2, and we have the pleasure to clarify the issues pointed out, that may be summarized in: (1) missing use of explicit overall trial quality rating system and its incorporation in performing the analysis; (2) failed evaluation of trial internal validity because judged as ‘satisfactory’, despite some shortcomings in the original study reports, and mistakes in assessing the use of ITT analysis; (3) likely overestimation of the favorable treatment effect; (4) inappropriate pooling of trials with fundamental different clinical and qualitative characteristics and not addressing the same question.

Concerning the first point, in our systematic review the risk of bias of individual studies and evaluation of their internal validity, as pre-specified in the methods section, was assessed with the Cochrane RevMan tool, providing reviewers’ judgments about each item for risk of bias presented as percentages across all included studies and thus showing an immediate snapshot of the main deficiencies in the quality of trial conduction or reporting (plot in Figure 2). In the text we reported for each trial the circumstances in which we judged the risk of bias unclear or high for the different categories considered. We did not provide an explicit overall trial quality rating system because, although numerous tools exist to the scope, we followed the approach of the Cochrane Collaboration handbook, which explicitly recommend against the use of scales yielding a summary score3. Paraphrasing the Collaboration’s words, in fact, while this approach offers appealing simplicity, it is not supported by empirical evidence and calculating a summary score inevitably involves assigning weights to different items in the scale that are difficult to justify3,4,5. Furthermore, scales have been shown to be unreliable assessments of validity and they are less likely to be transparent to users of the review3. In addition scales have a strong emphasis on reporting rather than conduction3,6, an aspect that seems weakly considered by the letter concerning the second point raised. Finally formal statistical Bayesian methods for adjusting meta-analyses for biases are a subject of current research and are not sufficiently well developed to be currently recommended for use in Cochrane reviews7. For these reasons in standard meta-analysis the effect estimates are not usually recommended to be weighed by the quality score. On the other hand, considering the very small number of included studies and the consistency of their results, we were discouraged from restricting meta-analysis or stratifying studies on the basis of their risk of bias.

Concerning the second point, we did not make a secret of the individual study limitations, that may affect their weight, in fact we claimed for example that “Concerning selection bias, the risk is largely unclear because none of the studies provided information on the procedures for allocation concealment and only three described how the random sequence was generated”. On the other hand the criticism of the letter is somewhat justifiable because it reflects our concerns reported in the discussion, concerning the fact that the CONSORT statement criteria for the methodology of trial reporting, although strongly desirable, are seldom adopted in the majority of published studies, also owing to editorial policies that frequently discourage technicalities. In addition we clearly warned that some studies were available in abstract form only, but we believe that when a small number of studies is available on one topic, the exclusion of grey literature and low quality studies is not a good idea. The only way we had to address this aspect was the direct collection of additional information from the authors of the original articles or the manufacturers, although we are aware that this may represent a bias, as we clearly stated in the discussion. Although Drs Berger and Gudehithlu may have different views, we did not consider the apparent variation in the studies’ robustness and quality as able to significantly affect the summary estimations and we judged their internal validity not outstanding but at least rather satisfactory, however maintaining the transparent approach of showing in Figure 2 that an unclear risk of bias exists within trials for different items.

These considerations move directly to the third point raised by the letter. In fact we concluded that “… an appreciable risk/benefit ratio for rupatadine emerges from the DBPC RCTs” and that “The estimations of effect … appear robust and sustained by a reasonable quality of evidence”. We don’t think this went beyond the available evidence because we clearly referred in the discussion to the judgments supporting our conclusions, here summarized: (I) no substantial limitations in study design and execution, (II) summary estimations of moderate effects consistent for direction and extent along all the endpoints explored, (III) precision and robustness of the pooled estimations based on a noteworthy sample size and a relatively narrow variability, (IV) unlike risk of severe publication bias, (V) lack of fragility of the results in the sensitivity analysis. In addition we did not provide implications for practice since we believe that these are often dependent on specific circumstances and values that must be taken into consideration.

On the other hand we also claimed the possible limitations of our work that can eventually downgrade the quality of the evidence: “our risk of bias assessment identified some shortcomings in trial reporting, mainly concerning the procedures for allocation concealment, random sequence generation and blinding strategies … In addition, in some studies selective outcome reporting was apparent for the efficacy and safety endpoints, thus partially downgrading the quality of the evidence, but our overall judgment is inclined to consider their internal validity adequate”. In conclusion, we are aware of some limitations, but we overall judged them likely to be unable to affect the summary results in a significant way and counterbalanced by some factors reinforcing the evidence, as mentioned above. We are also aware that evidence is not something fixed, but something moving with the contribution of new primary research, and for these reasons we considered the evidence provided by our systematic review as ‘reasonable’, avoiding any pre-accepted level of classification, but quite confident that further research is not very likely to strongly change our estimate in a short time.

With specific reference to Fantin and colleagues’ study8, the letter is right in affirming that this trial did not follow the ITT principle, because 542 patients were analyzed instead of 543, however this trivial failure was related to the group of patients receiving cetirizine that was not considered in our systematic review. In the Guadano study9, the ITT approach was at least unclear, although mentioned in the original article for the primary efficacy analysis, in fact our systematic review did not exclude an overall certain risk of attrition bias, but we did not judge the extent of this drawback as misleading. (Out of 250 randomized to three groups, six noncompliant patients were excluded from the analysis and one was excluded because lack of baseline information data. These seven patients were equally distributed across the treatment groups.) We prefer not to enter the topic of the flawed randomization procedures in specific trials in order to avoid falling into a debated theoretical field going beyond our scope. On the other hand, although the risk of unmasking due to side effects is a well known problem for many medical interventions, we prefer a more cautious approach rather than concluding a failed masking procedure in this study, anyway in our review the item allocation concealment was judged as unclear.

Concerning the fourth point, we believe that the example given by the letter, of the inappropriateness of combining treatment for headache and insomnia or telephone numbers, does not fit at all with our case of a treatment with antihistamine for allergic rhino-conjunctivitis. In fact, all the included studies in our review were designed to respond to the same ‘PICOS’ question: to address in randomized double-blind trials whether rupatadine is superior to placebo in relieving the allergic symptoms in patients with allergic rhino-conjunctivitis. We suspect the words of our introduction were misunderstood, where we wrote “… the above mentioned researches, performed with different study design and sample size, drug dosages, comparative intervention, patients’ age, treatment duration and allergen exposition”, with reference to the previously mentioned “Numerous studies [that] have documented the clinical efficacy of rupatadine 10 mg in seasonal and perennial AR and its safety profile”, including not only the double-blind placebo controlled randomized studies. Conversely, concerning these latter, the object of our review, the observed common sources of between-study differences regarding patient features, clinical setting, methods of delivery treatment or allergen exposure and duration are obvious, because trials have not been conducted according to a common protocol, and are unlikely to relevantly affect the treatment outcome, as expected from the experience of clinical practice and several trials in the field.

We fully agree with the letter that meta-analysis should be considered when a group of studies is sufficiently homogeneous in terms of participants, interventions and outcomes to provide a meaningful summary, but in our circumstance the studies had comparable characteristics, despite some common variability in factors that have never been clearly shown to affect the intervention effects. This consideration is also supported by our estimation of low inter-study statistical heterogeneity, apparent for most of the outcomes explored throughout the interpretation of forest plots and the I² statistics. Conversely the impact of these covariates requires evaluation in a sensitivity analysis, where we remarked the consistency of trial results (except for the studies conducted in challenge exposition only for the instantaneous symptoms assessment obviously), providing important corroboration of the generalization of the treatment effect.

In conclusion, for all these reasons, we strongly disagree with the criticism that our analysis is a meta-response to a meta-question. The letter proposes that “specificity is the key, and so fundamentally different studies should not be combined”. In addition to the fact that our studies are not so fundamentally different, we wish to report the words of the Cochrane Handbook for Systematic Review10:

“It is often appropriate to take a broader perspective in a meta-analysis than in a single clinical trial. A common analogy is that systematic reviews bring together apples and oranges, and that combining these can yield a meaningless result. This is true if apples and oranges are of intrinsic interest on their own, but may not be if they are used to contribute to a wider question about fruit. For example, a meta-analysis may reasonably evaluate the average effect of a class of drugs by combining results from trials where each evaluates the effect of a different drug from the class.”

Transparency

Declaration of funding

The authors received no payment in preparation of this manuscript.

Declaration of financial/other relationships

E.C. has disclosed that he has no significant relationships with or financial interests in any commercial companies related to this study or article. G.W.C. is member of the Uriach scientific advisory board.

CMRO peer reviewers may have received honoraria for their review work. The peer reviewers on this manuscript have disclosed that they have no relevant financial relationships.