Letter to the Editor

Re: Compalati E, Canonica GW. Efficacy and safety of rupatadine for allergic rhino-conjunctivitis: a systematic review of randomized, double-blind, placebo-controlled studies with meta-analysis. Curr Med Res Opin 2013 Jul 5. DOI: 10.1185/03007995.2013.837386.

Dear Editor,

In a recent article published in CMRO, Compalati and Canonica1 conducted a meta-analysis and concluded that there is a favorable risk–benefit ratio of rupatadine to treat rhino-conjunctivitis. Any conclusion based on a meta-analysis is only as reliable as the primary studies upon which it is based, so this is where we focus our attention. The trials included in this analysis were evaluated based on 1) random sequence generation; 2) allocation concealment; 3) masking; 4) completeness of outcome data; and 5) selective outcome reporting. However, there was no explicit overall trial quality rating system for the ten studies. Moreover, there was no mention of how trial quality figured into the weights of the studies; presumably, then, the trial quality did not factor into the weights at all. That is, a fatally flawed trial would have the same influence over future medical decisions and policies as a pristine and rigorous trial.

The fact that trial quality did not factor into the weights may reflect the possibility that all trials were of equal quality. Along these lines, we note that the authors1 did remark that “their internal validity was rather satisfactory”. Recalling the manner in which the trials were evaluated, this statement would suggest that all ten studies were satisfactory for randomization, allocation concealment, masking, data completeness, and outcome reporting. However, further scrutiny reveals that none of the studies had information on allocation concealment and that only three of the ten studies described how the random sequence was generated. Moreover, though Compalati and Canonica1 clearly state that all studies based their outcomes assessment on an intent-to-treat (ITT) analysis, this is actually not the case, as evidenced by the fact that in one trial2 the analysis incorrectly labeled as ‘ITT’ was based on 542 patients, and not on the full set of 543 that were randomized. Also, since Figures 1 and 23 had no indication of the number of patients that were included in the analysis, we have no way to know if these were ITT analyses or not. We believe clinicians should err on the side of caution when interpreting these results4, as the use of ITT analysis, which is often misrepresented, is at best unclear here, and allocation concealment may not have been as ‘satisfactory’ as the authors1 claimed.

Regarding random sequence generation, we note that there is a precedent5 for trials labeled as randomized to actually be non-randomized, and we have also already noted that for 70% of the studies, the precise method of randomization was not fully divulged. What about the other three trials? Here also there may be problems to report. For example, one article3 stated that patients were assigned sequentially in balanced blocks. This is a flawed randomization procedure6, and should never be used in any trials that have even the possibility of being unmasked. This consideration calls into question the masking in this study. In Table 23 we find that somnolence occurred in only two placebo patients but in 14 patients taking rupatadine. Therefore, we believe that the trial was not masked satisfactorily, and the lack of masking, coupled with the use of permuted blocks, precludes the possibility of allocation concealment, since future allocations can be easily predicted6. Another study2 mentioned only that it was “computer-generated randomized schedule provided by the sponsor of the study”. In other words, there is absolutely no description whatsoever of how the randomization was conducted. Once again, it would be unwise to assume that this trial was randomized properly.

Given what we know about the (lack of) quality of the included studies, how certain can we really be that there is a favorable outcome of rupatadine to treat rhino-conjunctivitis? The honest answer is that this favorable outcome may be overstated. Beyond that, even if all ten studies had been conducted properly, the authors stated that all ten studies had different study designs, sample sizes, drug dosages, comparative interventions, patients’ ages, treatment durations, and allergen exposures1. If one treatment for rashes is 90% effective, another treatment for headaches is 90% effective, and a third treatment for insomnia is 0% effective, then do we declare that the ‘average’ drug is 60% effective and, on this basis, proceed as if all three are efficacious? We need to stop to ask if the quantities we are averaging should be averaged, or, indeed, if they are even addressing the same question. The ten studies considered here do not seem to be addressing the same question, so we have a meta answer to a meta question.

Real patients are interested in traditional questions, and not meta questions. Which treatment should I take for this condition, as opposed to which treatment is optimal in a Bayesian sense if I had some random distribution of symptoms. Specificity is the key, and so fundamentally different studies should not be combined. Moreover, even if multiple studies do address the same question, they still should not be combined if some are of higher methodological quality than others7. In this case, the best studies are the most reliable, and the others serve only to skew the results.

Transparency

Declarations of funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Declaration of financial/other relationships

Both of the authors affiliated with this letter declare no conflict of interest.

CMRO peer reviewers may have received honoraria for their review work. The peer reviewers on this manuscript have disclosed no relevant financial relationships.