Interpreting discordant indirect and multiple treatment comparison meta-analyses: an evaluation of direct acting antivirals for chronic hepatitis C infection

Figures & data

Figure 1 Potential sources of discordance and types of discordance that should be investigated when evaluating indirect and multiple treatment comparisons.

Abbreviation: PICO, population, intervention, control, outcomes.

Table 1 Potential sources of discordance among the indirect and multiple treatment comparisons assessing direct acting antivirals

	Indirect and multiple treatment comparisons
	Cooper et al^Citation22	Cooper et al^Citation21	Cure et al^Citation23	Kieran et al^Citation24
Clinical question (PICO)
Are the defined patient populations similar?	Adult patients with chronic hepatitis C genotype I infection	Adult patients with chronic hepatitis C genotype I infection	Adult patients with chronic hepatitis C genotype I infection	Adult patients with chronic hepatitis C genotype I infection
Are the interventions similar? Are the control interventions similar?	•Boceprevir + peginterferon alpha + ribavirin versus peginterferon alpha + ribavirin	•Boceprevir + peginterferon alpha + ribavirin versus peginterferon alpha + ribavirin	•Boceprevir + peginterferon alpha + ribavirin versus peginterferon alpha + ribavirin	•Boceprevir + peginterferon alpha + ribavirin versus peginterferon alpha + ribavirin
	•Telaprevir + peginterferon alpha + ribavirin versus peginterferon alpha + ribavirin	•Telaprevir + peginterferon alpha + ribavirin versus peginterferon alpha + ribavirin	•Telaprevir + peginterferon alpha + ribavirin versus peginterferon alpha + ribavirin	•Telaprevir + peginterferon alpha + ribavirin versus peginterferon alpha + ribavirin
		•Peginterferon alpha-2a + ribavirin versus peginterferon alpha-2b + ribavirin	•Peginterferon alpha-2a + ribavirin versus peginterferon alpha-2b + ribavirin
Are the chosen outcomes similar?	Primary outcome: Sustained virologie response	Primary outcome: Sustained virologie response	Primary outcome: Sustained virologie response	Primary outcome: Sustained virologie response
Study selection and inclusion criteria
Are the considered study designs similar?	Randomized controlled trials	Randomized controlled trials	Randomized controlled trials	Randomized controlled trials
Are databases searched similar?	MEDLINE, Embase, Cochrane Library, AMED, CINAHL, TOXNET, Development and Reproductive Toxicology, Hazardous Substances Data Bank, PsyclNFO, Web of Science	MEDLINE, Embase, Cochrane Library, AMED, CINAHL, TOXNET, Development and Reproductive Toxicology, Hazardous Substances Data Bank, PsyclNFO, Web of Science	MEDLINE, Embase, Cochrane Library, Center for Review and Dissemination	MEDLINE, Embase, Cochrane Library, Science Citation Index
Are the intervention/comparator arms used in the primary analysis similar?	Trials assessing only standard duration therapy and response guided therapy of boceprevir and telaprevir	Trials assessing only standard duration therapy and response guided therapy of boceprevir and telaprevir	Trials only assessing approved treatment durations of boceprevir and telaprevir	Trials only assessing approved treatment durations of boceprevir and telaprevir
Are the included trials of comparable quality?	Not assessed	Not assessed	Assessed using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions^Citation27 The authors deemed all studies to be of acceptable quality and sufficiently homogenous	Assessed using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions.^Citation27 In a sensitivity analysis, the authors excluded those deemed to have an increased risk of bias. The results of this analysis were not different from those of the primary analysis
Outcomes definition and measurement
Are the outcomes defined similarly?	Primary outcome:	Primary outcome:	Primary outcome:	Primary outcome:
	•Sustained virologic response (undetectable HCV-RNA at the end of the 24-week posttherapy follow-up period)	•Sustained virologie response (undetectable HCV-RNA at the end of the 24-week posttherapy follow-up period)	•Sustained virologie response (undetectable HCV-RNA at the end of the 24-week posttherapy follow-up period)	•Sustained virologie response (undetectable HCV-RNA at the end of the 24-week posttherapy follow-up period)
	Secondary outcomes:	Secondary outcomes:
	•Relapse (the reoccurrence of HCV-RNA within the 24-week posttherapy follow-up period)	•Relapse (the reoccurrence of HCV-RNA within the 24-week posttherapy follow-up period)
	•Treatment discontinuation (the discontinuation of all assigned drugs during the set treatment period)	•Anemia (defined as hemoglobin <100 g/L)
	•Anemia (hemoglobin <100 g/L)	•Neutropenia (an absolute neutrophil count <0.75 × 10^Citation9/L)
	•Neutropenia (an absolute neutrophil count <0.75 × 10^Citation9/L)	•Thrombocytopenia (a platelet count <l50,000/mL)
	•Thrombocytopenia (a platelet count <l50,000/mL)	•Rash (any, as reported by site investigators)
	•Rash (any, as reported by site investigators)	•Pruritus (any, as reported by site investigators)
	•Pruritus (any, as reported by site investigators)
Are the methods used to measure the outcomes similar?	Primary outcome:	Primary outcome:	Primary outcome:	Primary outcome:
	•COBAS TaqMan HCV assay (Roche)	•COBAS TaqMan HCV assay (Roche)	•COBAS TaqMan HCV assay (Roche)	•COBAS TaqMan HCV assay (Roche)
Statistical approach
Are the statistical approaches used similar?	Adjusted indirect comparison	Bayesian multiple treatment comparison (odds ratio and credible intervals)	Bayesian multiple treatment comparison (odds ratio and credible intervals)	Bayesian multiple treatment comparison (odds ratio and credible intervals)
Statistical model and heterogeneity
Are the statistical models and exploration of heterogeneity similar?	Random effects model	Random effects model when more than 1 trial arm was available and fixed effects model when only 1 trial arm was available	Fixed effects model (a secondary analysis for the random effects model to the base-case analysis)	Fixed effects model
Effect measures
Are the measures and statistics for establishing comparative superiority or inferiority similar?	Relative risk	Odds ratio	Odds ratio	Odds ratio
Funding source
Who funded each study?	Merck	Merck & Co	Janssen	Not stated

Abbreviations: HCV-RNA, hepatitis C virus RNA; PICO, population intervention control outcomes.

Figure 2 Indirect treatment comparison (Panel A) and multiple treatment comparison (Panel B) meta-analyses of boceprevir and telaprevir.

Notes: The solid lines between interventions represent direct evidence. The dashed lines between interventions represent indirect evidence.

Table 2 Discordance for the primary outcome (sustained virologic response) among the indirect and multiple treatment comparisons assessing direct acting antivirals

	Indirect and multiple treatment comparisons
	Cooper et al^Citation22	Cooper et al^Citation21	Cure et al^Citation23	Kieran et al^Citation24
Analysis of treatment naïve patients
Are there discrepancies in the direction of effect?	•Favors boceprevir (48-week SDT)	•Favors telaprevir (48-week SDT)	•Favors telaprevir (RGT)	•Favors telaprevir (RGT telaprevir vs combined regimens boceprevir)
	•Boceprevir and telaprevir equivalent (RGT)		•Favors telaprevir (RGT telaprevir vs 48-week SDT boceprevir)
		•Favors telaprevir (RGT)
Are there discrepancies in the magnitude of effect?	•Small effect (48-week SDT)	•Small effect (48-week SDT)	•Small effect (RGT)	•Small effect (RGT telaprevir vs combined regimens boceprevir)
	•No effect (RGT)	•Small effect (RGT)	•Small effect (RGT telaprevir vs 48-week SDT boceprevir)
Is there discordance among claims of statistical significance?	•Not significant (48-week SDT)	•Not significant (48-week SDT)	•Not significant (RGT)	•Not significant (RGT telaprevir vs combined regimens boceprevir)
	•Not significant (RGT)		•Not significant (RGT telaprevir vs 48-week SDT boceprevir)
		•Not significant (RGT)
Analysis of treatment experienced patients
Are there discrepancies in the direction of effect?	•Favors telaprevir (48-week SDT)	•Favors telaprevir (48-week SDT)	•Favors telaprevir (48-week SDT)	•Favors telaprevir (48-week SDT telaprevir vs combined regimens boceprevir)
			•Favors telaprevir (48-week SDT telaprevir vs RGT boceprevir)
Are there discrepancies in the magnitude of effect?	•Small effect (48-week SDT)	•Small effect (48-week SDT)	•Small effect (48-week SDT)	•Small effect (48-week SDT telaprevir vs combined regimens boceprevir)
			•Moderate effect (48-week SDT telaprevir vs RGT boceprevir)
Is there discordance among claims of statistical significance?	•Not significant (48-week SDT)	•Not significant (48-week SDT)	•Not significant (48-week SDT)	•Not significant (48-week SDT telaprevir vs combined regimens boceprevir)
			•Significant (48-week SDT telaprevir vs RGT boceprevir)
Subgroup analysis of treatment experienced prior relapsers
Are there discrepancies in the direction of effect?	•Favors telaprevir (48-week SDT)	–	•Favors telaprevir (48-week SDT)	•Favors telaprevir (48-week SDT telaprevir vs combined regimens boceprevir)
			•Favors telaprevir (48-week SDT telaprevir vs RGT boceprevir)
Are there discrepancies in the magnitude of effect?	•Small effect (48-week SDT)	–	•Moderate effect (48-week SDT)	•Moderate effect (48-week SDT telaprevir vs combined regimens boceprevir)
			•Moderate effect (48-week SDT telaprevir vs RGT boceprevir)
Is there discordance among claims of statistical significance?	•Not significant (48-week SDT)	–	•Not significant (48-week SDT)	•Significant (48-week SDT telaprevir vs combined regimens boceprevir)
			•Significant (48-week SDT telaprevir vs RGT boceprevir)
Subgroup analysis of treatment experienced prior partial responders
Are there discrepancies in the direction of effect?	–	–	•Favors boceprevir (48-week SDT)	–
			•Favors boceprevir (48-week SDT telaprevir vs RGT boceprevir)
Are there discrepancies in the magnitude of effect?	–	–	•Small effect (48-week SDT)	–
			•Small effect (48-week SDT telaprevir vs RGT boceprevir)
Is there discordance among claims of statistical significance?	–	–	•Not significant (48-week SDT)	–
			•Not significant (48-week SDT telaprevir vs RGT boceprevir)

Notes: We arbitrarily defined the magnitude of effect sizes as follows: Small: Odds Ratio (OR) 1.00–1.99; RR 1.00–1.49. Moderate: OR 2.00–4.99; RR 1.50–2.99. Large: OR ≥ 5.00; RR ≥3.00.

Abbreviations: RGT, response guided therapy; SDT, standard duration therapy; vs, versus.

Table S1 Approved treatment labels for boceprevir and telaprevir in Europe and North America

	Europe		North America
	Boceprevir	Telaprevir	Boceprevir	Telaprevir
Treatment naïve	PR 1–4 → BPR 5–28 or PR 1–4 → BPR 5–36 → PR 37–48*	TPR 1–12 → PR 13–24 or TPR 1–12 → PR 13–48*	PR 1–4 → BPR 5–28 or PR 1–4 → BPR 5–36 → PR 37–48*	TPR 1–12 → PR 13–24 or TPR 1–12 → PR 13–48*
Treatment experienced – prior relapsers	PR 1–4 → BPR 5–36 → PR 37–48	TPR 1–12 → PR 13–24 or TPR 1–12 → PR 13–48*	PR 1–4 → BPR 5–36 or PR 1–4 → BPR 5–36 → PR 37–48*	TPR 1–12 → PR 13–24 or TPR 1–12 → PR 13–48*
Treatment experienced – prior partial responders	PR 1–4 → BPR 5–36 → PR 37–48	TPR 1–12 → PR 13–48	PR 1–4 → BPR 5–36 or PR 1–4 → BPR 5–36 → PR 37–48*	TPR 1–12 → PR 13–48
Treatment experienced – prior null responders	PR 1–4 → BPR 5–48	TPR 1–12 → PR 13–48	PR 1–4 → BPR 5–48	TPR 1–12 → PR 13–48
All cirrhotic patients	PR 1–4 → BPR 5–48	TPR 1–12 → PR 13–48	PR 1–4 → BPR 5–48	TPR 1–12 → PR 13–48

Note: * denotes response guided therapy.

Abbreviations: PR, Peginterferon–ribavirin; BRP, Boceprevir + peginterferon-ribavirin; TPR, Telaprevir + peginterferon–ribavirin.

CooperCDruytsEThorlundKBoceprevir and telaprevir for the treatment of chronic hepatitis C genotype 1 infection: an indirect comparison meta-analysisTher Clin Risk Manag2012810513022442631

 PubMed Web of Science ®Google Scholar

CooperCLesterRThorlundKDirect-acting antiviral therapies for hepatitis C genotype 1 infection: a multiple treatment comparison meta-analysisQJM2013106215316323159839

 PubMed Web of Science ®Google Scholar

CureSDielsJGavartSBianicFJonesEEfficacy of telaprevir and boceprevir in treatment-naive and treatment-experienced genotype 1 chronic hepatitis C patients: an indirect comparison using Bayesian network meta-analysisCurr Med Res Opin201228111841185623016967

 PubMed Web of Science ®Google Scholar

KieranJSchmitzSO’LearyAThe relative efficacy of boceprevir and telaprevir in the treatment of hepatitis C virus genotype 1Clininical Infectious Diseases12013562228235

 Web of Science ®Google Scholar

HigginsJPTGreenSThe Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]The Cochrane Collaboration2011 Available from: http://www.cochranehandbook.orgAccessed May 6, 2013

 Google Scholar

LuGAdesACombination of direct and indirect evidence in mixed treatment comparisonsStat Med200423203105312415449338

 PubMed Web of Science ®Google Scholar