Response to the publication by Ueberall and Mueller-Schwefe

Dear editor

While updating our Cochrane review on tapentadol for chronic musculoskeletal pain,¹ we found a study by Ueberall and Mueller-Schwefe.² It passed the initial screening phase by using words such as “blinded” and “randomly”, methodologically positive characteristics most often seen in randomized controlled trials (RCTs). On further analysis, we found that it is not an RCT and therefore not eligible for our Cochrane review. We nonetheless remained interested, since the authors clearly made methodological options that, despite sounding pondered, rigorous, and methodologically desirable, in fact add little to nothing in terms of quality or rigor. In fact, this quasi-rigor can be fully appreciated when assessing the study using the Cochrane model for risk of bias in non-randomized studies of interventions.³

We consider that the decision to randomize was key to the overall assessment of this study. What the authors did was randomly select which participants were going to be studied, and which would not. What may have been expected is to randomly allocate participants to treatment with either tapentadol or oxycodone, as is done in an RCT.

We believe that the risk of bias due to confounding is moderate. Although all important confounding domains were appropriately measured and controlled for, we would still expect some degree of confounding, as is the case for most observational research.

The bias stemming from classification is low, since the intervention status is well defined. Also, the risk of bias due to deviations from the intended interventions (also called performance bias) is low, since any deviations would reflect usual practice of neurologists, and for this the authors should be commended.

Importantly, the attrition rates in this study were similar in the tapentadol and oxycodone groups (22.6% vs. 23.4%, respectively), unlike what was found in other relevant studies.⁴ Overall, the bias due to missing data (also called attrition bias) was low.

The bias due to measurement in outcomes is another cause for serious concern, since the outcome measures in this study were subjective, and both the study participants and researchers knew who was receiving tapentadol and who was receiving oxycodone.

Finally, the bias due to selection of the reported results is moderate, since we are not aware of a publically available study protocol, despite the fact that the outcomes studied are internally and externally consistent.

Overall, in our opinion, this study is at a serious risk of bias, meaning that it has important methodological problems, and that the results should be put into the context of its biases. The risk of bias in measurement of outcomes (also called detection bias) will nearly always be problematic for any study in the area of pain research, since the outcomes are mostly very subjective and blinding of participants is very difficult. What is not commonly seen, in any field of research, is the author’s choice to artificially select a subset of participants to study, with no apparent reason for doing so. The choice to do so gravely undermines a study that would otherwise have been more informative, as in the case of a true RCT.

Disclosure

The authors report no conflicts of interest in this communication.

Authors’ replyCorrespondence: Michael A Ueberall, Institute of Neurological Sciences, Nordostpark 51, 90411 Nuernberg, Germany, Tel [phone], Fax [phone], Email [email protected]

Dear editor

Thank you for the opportunity to reply to the letter of Duarte et al and to discuss their objections concerning our analysis of data from the German Pain Registry.

As described in our publication,¹ the data for our analysis were not prospectively gathered during a double-blind randomized controlled trial (RCT), but retrospectively taken from the German Pain registry – a large database that provides detailed data (based on validated patient questionnaires and physician-recorded information, etc.) on the routine treatment of pain patients for health care research purposes. Patients with low back pain with a neuropathic component in whom a new treatment with either oxycodone/naloxone or tapentadol was initiated within a specific period were identified by predefined inclusion and exclusion criteria (as described in our paper). From patients who fulfilled the enrollment criteria for analysis, we took a random sample (based on an a priori defined randomization list) to guarantee an unbiased data selection for analysis and evaluated the effects of both treatments over a 12-week observation period in a blinded fashion.

As Duarte et al wrote, this process has some risk for a selection bias and therefore we performed a complementary analysis comparing the baseline characteristics of those patients whose data were selected vs. those whose data were not selected (section “Randomization effects” in our paper) and found only minor and insignificant between-group differences (also for the outcome data).

We have chosen this special kind of data analysis, because we were not interested in another replication of scientific RCT data but interested in those reflecting the efficiency of both drugs under real-life conditions, where the decision for treatment is neither dictated by distinct inclusion and exclusion criteria nor a random list, but only by individual patient needs and the intention of a physician to prescribe the most appropriate treatment available. In addition to that, treatments were tailored and maintained according to the specific responses of the individuals treated and this obviously resulted in significant differences with respect to distinct outcome parameters and endpoint analyses vs. those of the RCTs mentioned by Duarte et al (especially the attrition rates of patients on treatment), as we have discussed extensively in our paper.

Due to the fact that our analyses were based on routine data documented during routine practice under daily life conditions (sampled primarily for the purpose to document the response of a patient to a specific pharmacological treatment), physicians and patients of course knew what kind of treatment has been given. However, statistical analyses were done in a blinded fashion (i.e., without any knowledge on the drug status of distinct patients) to guarantee unbiased results.

For these (and a number of further) reasons, we strongly disagree with the conclusion by Duarte et al about the methodological shortcomings of our work and the allegation of a “serious risk of bias”. In contrast to those RCTs, the members of the Cochrane group usually prefer our approach focused on routine data of real-life treatments performed to improve the suffering of individual patients. Patient selection, procedures/treatments, and biometrical strategies were transparently reported and data of both comparators were equally handled, minimizing the risk of a positive discrimination of one treatment vs. the other. Finally, the subsets of patient data analyzed were randomly selected not to increase but to reduce the risk of bias, and our own analyses to prove this concept provided no signal that this approach failed.

We believe that this kind of analysis has not only the potential to complement the reports from conventional RCTs but also to overcome their obvious shortcomings. We strongly believe in the concept of RCTs for regulatory and approval purposes, but we (along with many other practitioners) are increasingly afflicted by doubts that if RCTs (at least the way they are realized today) are of any help to answer the question which kind of treatment works best for individual patients under real-life conditions and which size/kind of effect can be achieved. To gain further insight into the real-life effects of analgesic drug treatments, new and alternative health care research approaches are therefore necessary. If our approach and the current controversy of its pros and cons help to promote a sincere discussion regarding the best strategy for future patient-centered studies, we have achieved our main objective and we thank all our colleagues who have helped us on this way.

Disclosure

MAU and GHHM-S are physicians and independent of any significant/relevant financial or other relationship to the sponsor, except for minor reimbursements for occasional lecture or consulting fees. Both are honorary members of the management boards of the German Pain Association and the German Pain League. The German Pain Registry is hosted by an independent contract research organization by order of the German Pain Association and under the control of the Institute of Neurological Sciences and collects standardized data from daily routine medical care since January 2000. During the last 2 years, MAU received financial support and/or expenses in form of research money, consultancy fees, and/or remunerations for lecture activities from Almirall, Archimedes, Grünenthal, Kyowa Kirin, Menarini, MSD, Mucos, Mundipharma, Pfizer, Pharm-Allergan, Shionoghi, and TEVA. During the last 2 years, GHHM-S received financial support and/or expenses in form of research money, consultancy fees, and/or remunerations for lecture activities from Allergan, Almirall, Grünenthal, Kyowa Kirin, Mundipharma, Pfizer, Pharm-Allergan, Shionoghi, and TEVA. The authors report no other conflicts of interest in this communication.