The impact of clinical heterogeneity on conducting network meta-analyses in transthyretin amyloidosis with polyneuropathy

Abstract

Objective: The comparative safety and efficacy of tafamidis, patisiran and inotersen treatments for transthyretin amyloidosis with polyneuropathy (ATTR-PN) has not been evaluated in clinical trials. In the absence of head-to-head evidence, indirect treatment comparisons such as network meta-analyses (NMAs) can be performed to evaluate relative effects of treatments. This study aims to assess the feasibility of conducting an NMA of available therapies for ATTR-PN patients.

Methods: Pivotal trials for three approved ATTR-PN treatments, tafamidis (Fx-005), patisiran (APOLLO) and inotersen (NEURO-TTR), were compared in terms of study design, baseline population characteristics, outcome definitions and baseline risk. These assessments of heterogeneity informed the decision to perform Bayesian NMAs.

Results: Despite similar study designs, clear differences in eligibility criteria between trials were accompanied by imbalances in baseline population characteristics considered to be plausible effect modifiers, such as disease stage and previous treatment. Of the outcomes assessed, only quality of life and adverse events were similarly reported in all trials. Neuropathy outcomes were not evaluated consistently between trials.

Conclusions: An NMA of ATTR-PN treatments was not feasible, given the observed cross-trial heterogeneity. This decision highlights the importance of careful consideration for clinical heterogeneity that may threaten the validity of indirect comparisons.

Keywords:

• Feasibility assessment
• indirect treatment comparison
• inotersen
• patisiran
• tafamidis
• transthyretin amyloidosis with polyneuropathy

Introduction

Transthyretin amyloidosis with polyneuropathy (ATTR-PN) is a rare and fatal neurodegenerative disease resulting from an inherited autosomal dominant mutation in the transthyretin (TTR) gene¹. The mutation leads to TTR protein misfolding and extracellular deposition of amyloid fibrils in peripheral and autonomic nerves, as well as organs such as the kidneys and the heart^1,2. While more than 120 TTR mutations have been documented in ATTR-PN patients, the most common mutation is the replacement of valine with methionine at position 30 of the amino acid sequence (the Val30Met mutation)^1,3,4. Disease phenotype is dependent on the specific mutation in TTR, and these mutations vary in prevalence by region^5,6. The Val30Met mutation is prevalent in a number of endemic foci including Portugal, Sweden and Japan⁷. In ATTR-PN, sensorimotor and autonomic neuropathy cause progressive sensory disturbances, which typically begin in the toes and ascend to the knees and hands¹. The prognosis of patients with ATTR-PN is poor; death usually occurs within a decade of symptom presentation and is often associated with progressive neuropathy, secondary infections or cachexia⁸.

In 2011, the European Medicines Agency (EMA) approved tafamidis (Vyndaqelⁱ) as the first pharmacological treatment for Stage 1 ATTR-PN¹. Currently approved in 41 countries⁹, tafamidis is a small molecule that stabilizes the tetrameric structure of TTR by occupying thyroxine-binding sites on the protein¹⁰. Stabilization of TTR by tafamidis inhibits amyloid formation thus slowing the progression of ATTR-PN. Tafamidis was evaluated against placebo in a phase II/III, double-blind, randomized controlled trial (RCT) in patients with Stage 1 ATTR-PN (Fx-005¹⁰, NCT00409175). No differences were observed between the tafamidis and placebo groups for the primary endpoints of proportion of responders based on the Neuropathy Impairment Score of the Lower Limbs (NIS-LL; 45.3% vs 29.5%; p = .068) and change in the Norfolk Quality of Life – Diabetic Neuropathy total score (TQOL; 2.0 vs 7.2; p = .116) in the intention-to-treat population¹⁰. However, both endpoints were met in the pre-specified efficacy-evaluable population (NIS-LL: 60% in tafamidis group vs 38.1% in placebo group; p = .041; Norfolk TQOL: 0.1 in tafamidis group vs 8.9 in placebo group; p = .045)¹⁰. Overall tolerability was indicated by adverse event and serious adverse event rates, which were balanced between placebo and tafamidis groups. In both groups, less than 10% of patients withdrew due to adverse events¹⁰.

Two additional drug therapies were approved by the EMA, Health Canada and the United States Food and Drug Administration (FDA) in 2018. Patisiran (Onpattroⁱⁱ) is an RNA interference (RNAi) agent that controls gene expression by cleaving target messenger RNA (mRNA) bound to the RNA-induced silencing complex, thereby reducing the production of mutant and wild-type TTR¹¹. Patisiran was evaluated in a phase III, double-blind, placebo-controlled RCT in patients with ATTR-PN (APOLLO¹¹, NCT01960348). A significant difference was observed between the patisiran and placebo groups for the primary endpoint of change in modified Neuropathy Impairment Score + 7 (mNIS + 7; −6.0 vs 28.0; p < .001)¹¹. Adverse event and serious adverse event rates were balanced between placebo and patisiran groups. Less than 10% of patients in both groups withdrew due to adverse events¹¹. Inotersen (Tegsediⁱⁱⁱ) is a 2’-O-methoxyethyl-modified antisense oligonucleotide inhibitor of hepatic TTR protein production¹². Inotersen was evaluated in a phase III, double-blind, placebo-controlled RCT in patients with Stage 1 or 2 ATTR-PN (NEURO-TTR¹², NCT01737398). Significant differences were observed between the inotersen and placebo groups for the primary endpoints of change in modified Neuropathy Impairment Score + 7 Ionis (mNIS + 7_Ionis; 5.8 vs 25.5; p < .001) and change in Norfolk TQOL (1.0 vs 12.7; p < .001)¹². Adverse event and serious adverse event rates were 1% higher and 10% lower in the placebo group versus the inotersen group. In the inotersen and placebo groups, 14% and 1.6% of patients withdrew due to adverse events¹².

Although tafamidis, patisiran, and inotersen were each reported to be more effective than placebo for the treatment of patients with ATTR-PN, the relative efficacy and safety of these therapies is not clear. Indirect treatment comparisons (ITCs) such as network meta-analyses (NMAs) are increasingly being used to inform decision-making regarding the relative efficacy and safety of treatments that have not been compared directly in clinical trials. Published examples of pairwise indirect comparisons between ATTR-PN treatments do exist^13,14. Yet, there is an absence of direct evidence comparing all three ATTR-PN therapies. In this case, an NMA to compare these treatments represents another way in which the relative safety and efficacy of each ATTR-PN therapy could be reliably assessed^15–22. However, it is necessary to assess whether it is feasible to perform valid ITCs based on the available clinical trial evidence. The validity of such analyses relies on whether there are systematic differences among the studies included in the treatment comparison networks, especially patient or disease characteristics that are treatment effect modifiers^19,23–25. Failure to account for these differences can lead to misleading comparisons of treatment effect^22,26. The aim of this study was to assess the suitability of synthesizing the results of multiple trials in a unified analysis to compare the safety and efficacy of tafamidis, patisiran and inotersen for patients with ATTR-PN.

Methods

Identification and selection of studies

The three therapies of interest were pre-specified (tafamidis, inotersen and patisiran). A targeted literature search was performed in November 2018 to confirm the identification of their pivotal publications and to identify any additional English-language published RCTs evaluating the efficacy and safety of tafamidis, patisiran and inotersen for patients with ATTR-PN. Regular update searches were also performed to identify RCTs. The population of interest was adult (≥18 years of age) men and women with a diagnosis of hereditary ATTR-PN. Patients with any disease stage harboring a TTR mutation (i.e. not limited to Val30Met) were within the scope of this search. The population was defined broadly to capture all available evidence for trials that included tafamidis, patisiran and inotersen.

For each identified study that met the selection criteria, details were extracted using a standardized data extraction form designed in Microsoft Excel (Microsoft Corporation, Seattle, USA) and included: general publication details (e.g. authors, study location, year, journal, NCT number), study design (e.g. phase, blinding, allocation, study duration), inclusion/exclusion criteria, study population characteristics, interventions (e.g. dosing regimens, route of administration), baseline population characteristics (e.g. sample size, sex, age, race, geographic region, modified body mass index [mBMI], disease duration, disease stage, TTR mutation burden, and Norfolk Quality of Life – Diabetic Neuropathy [QOL-DN] total score [TQOL]), study-specific outcome definitions for the available outcomes of interest, and efficacy and safety endpoint data (timepoints/timeframe of assessments, results). Data extraction was performed by one reviewer and validated by a second independent reviewer to verify data accuracy. Any disagreements in assessment of the data were resolved through discussion until consensus was reached.

Assessing the feasibility of a network meta-analysis for patients with ATTR-PN

Systematic differences between treatment groups may compromise the validity of an NMA; therefore, it was prudent to assess the degree of heterogeneity in patient characteristics and study design between trials. Specifically, differences in plausible treatment effect modifiers were evaluated across trials. Clinical heterogeneity in terms of treatment and outcome characteristics, and the study and patient characteristics were assessed to ensure similarity. All primary and secondary outcomes evaluated in the pivotal tafamidis (Fx-005), patisiran (APOLLO) and inotersen (NEURO-TTR) trials were of interest^10–12. These outcomes included measures of neuropathy impairment, quality of life, nutritional status and safety (Supplementary Table 1). For neuropathy assessments, only total scores were considered in this feasibility assessment. Evidence network diagrams illustrating the structure of the network for each clinical outcome of interest were developed and differences were assessed in terms of baseline risk within and between trials.

An important step in the feasibility assessment involved determining whether there were sufficient data to explain or adjust for differences in plausible treatment effect modifiers using meta-regression. Gagnier et al. suggest that there should be approximately ten trials when working with summary or aggregate data and caution that fewer studies may be associated with a higher type 1 error rate^27,28. Therefore, the number of studies was considered when determining whether meta-regression was feasible.

The decision to perform an NMA was based on clinical evidence from the pivotal RCTs, primarily the differences between the studies that were considered likely to affect the treatment comparisons or make some comparisons inappropriate^18,27.

If an NMA was deemed feasible, a Bayesian NMA was considered using the methodology outlined by the National Institute for Health and Care Excellence (NICE) Decision Support Unit (DSU) Technical Support Documents (TSD)^24,29. Traditional ITCs, such as NMA, rely on summary or aggregate data derived from comparative studies; however, advanced techniques involving individual patient data (IPD) have emerged to address evidence networks when there is heterogeneity among studies. These techniques, such as matching-adjusted indirect comparisons (MAICs), are designed to reduce cross-trial differences in baseline patient characteristics, reduce sensitivity to effect measures, resolve differences in study outcome definitions and allow comparison of clinically relevant dosages. As such, MAICs were also considered using the methods outlined in the NICE DSU TSDs³⁰.

Results

Study identification

The literature review identified three unique RCTs, each of which evaluated the efficacy and safety of ATTR-PN therapies versus placebo: tafamidis (20 mg once daily; oral)¹⁰, patisiran (0.3 mg/kg once every 3 weeks; intravenous)¹¹ and inotersen (300 mg once weekly; subcutaneous)¹². None of the included studies directly compared one ATTR-PN drug with another.

Study design and treatments

The included RCTs used double-blind, parallel-group, and placebo-controlled study designs evaluating men and women with hereditary TTR amyloidosis with peripheral neuropathy. Trial durations were 15 months for NEURO-TTR and 18 months for Fx-005 and APOLLO^10–12. Despite differences in duration of follow-up, the studies were similarly designed (Table 1).

Table 1. Study designs and interventions investigated in ATTR-PN trials.

Study	Phase	Study design	Sample Size^a	Intervention			Comparator
				Treatment	Dose	Regimen	Treatment	Regimen
Fx-005	II/III	RCT, DB, active comparator, PC	128	Tafamidis (Vyndaqel)	20 mg	PO, qd	Placebo	PO, qd
APOLLO	III	RCT, DB, active comparator, PC	225	Patisiran (Onpattro) with steroids	0.3 mg/kg body weight	IV, q3w	Placebo with steroids	IV, q3w
NEURO-TTR	II/III	RCT, DB, active comparator, PC	173	Inotersen (Tegsedi)	300 mg	SC, q1w (3 doses in week 1)	Placebo	SC, q1w (3 doses in week 1)

^aRandomized patients.

Abbreviations. DB, Double blind; IV, Intravenous; PC, Placebo-controlled; PO, Oral; q1w, Once weekly; q3w, Once every 3 weeks; qd, Once daily; RCT, Randomized controlled trial; SC, Subcutaneous.

All treatment regimens were clinically relevant in the treatment of ATTR-PN; however, differences in the route of administration, treatment doses and/or schedules were identified (Table 1). Tafamidis and matching placebo were administered orally¹⁰, inotersen and matching placebo were administered subcutaneously¹², and patisiran and matching placebo were administered intravenously with concomitant steroid pre-medications to prevent infusion-related reactions¹¹. Since there was a limited number of RCTs included per treatment, a meta-regression was not deemed to be feasible to adjust for differences in treatment regimens.

Study eligibility criteria

All included patients were diagnosed with ATTR-PN, aged 18 or older (with similar maximum ages), and had similar ranges of functional impairment as defined by their Karnofsky Performance Status scores (Table 2). All trials excluded patients with previous liver transplantation. Differences were identified with respect to disease stage, genetic mutation presentation and prior treatment exposure. Disease stage eligibility criteria were defined in NEURO-TTR (permitting stages 1–2 patients)^10,12, whereas the APOLLO and Fx-005 trials did not specify disease stage eligibility criteria^10,11. Fx-005 included only patients with Val30Met mutations in TTR¹⁰, whereas APOLLO and NEURO-TTR enrolled patients with any TTR mutation^11,12. Finally, APOLLO and NEURO-TTR eligibility criteria specified a requirement for washout of prior tafamidis and diflunisal therapies^11,12. Fx-005 patients could not have received prior ATTR-PN therapy as no other ATTR-PN therapies were approved at the time of study design¹⁰. Patients prescribed diflunisal would have been ineligible, as chronic use of non-steroidal anti-inflammatory drugs (NSAIDs) was a specific exclusion criterion in Fx-005.

Table 2. Summary of key eligibility criteria in ATTR-PN trials.

Study	Population	Age range (years)	Mutation in TTR	Previous treatment for ATTR-PN	Karnofsky performance status^c (0–100)	Previous liver transplantation
Fx-005	ATTR-PN (Stages undefined)	18–75	Val30Met	No history of chronic NSAID use (defined as >3–4 uses per month)^a	≥50	Excluded
APOLLO	ATTR-PN (Stages undefined)	18–85	Mixed	Washout (3 months) for antisense oligonucleotide therapy^b Washout (2 weeks) for diflunisal, tafamidis, doxycycline or tauroso-deoxycholic acid	≥60	Excluded
NEURO-TTR	ATTR-PN (Stages 1 and 2)	18–82	Mixed	Washout (2 weeks) for tafamidis or diflunisal	≥50	Excluded

^aThis criterion applies to patients prescribed the NSAID and TTR stabilizer diflunisal prior to enrollment.

^bIncludes inotersen (the active drug in the NEURO-TTR trial).

^cClassifies functional impairment, where a score of 100 represents no evidence of disease and a score of 0 represents death³¹.

Abbreviations. ATTR-PN, Transthyretin amyloidosis with polyneuropathy; NSAID, Non-steroidal anti-inflammatory drug; TTR, Transthyretin; Val30Met, Valine to methionine point mutation at position 30.

Outcome definitions

Differences in the RCTs were also identified in terms of outcome definition. Notably, neuropathy impairment was evaluated differently in all three RCTs: Fx-005 used NIS-LL¹⁰, APOLLO used mNIS + 7¹¹, and NEURO-TTR used mNIS + 7_Ionis¹². These assessments differed with respect to their specific evaluation of neuropathy impairment (i.e. motor strength, sensation, reflexes, nerve conduction and autonomic function)³². For example, autonomic function was assessed through blood pressure tests in the mNIS + 7 and through heart rate response to deep breathing in the mNIS + 7_Ionis, but is not assessed in any way by the NIS-LL³². The weakness component of the Neuropathy Impairment Score (NIS-W) was reported alone in both APOLLO and NEURO-TTR trials and scored identically in mNIS + 7 and mNIS + 7_Ionis assessments, whereas the weakness component score of the NIS-LL used in Fx-005 was restricted to the lower limbs^10–12. Nerve conduction was reported in Fx-005 using two summated nerve tests, which assessed large and small fiber nerves of the lower limbs only¹⁰. The mNIS + 7 assessments used in APOLLO and NEURO-TTR contained similar nerve conduction tests, but scoring differences were noted despite the nerves assessed in each mNIS + 7 being identical^11,12,32. A meta-regression to adjust for differences in the NIS definitions was not feasible given the limited number of studies.

Data for the Norfolk QOL-DN – a 35-item patient-reported questionnaire providing a total quality of life score (TQOL) – were reported in all three RCTs³³. Minor numerical differences in scoring were identified in the Norfolk TQOL assessment between studies. Total scores ranged from −2 through 138 in Fx-005, which is consistent with the published report where the assessment was first adapted for use in ATTR-PN patients^10,33. Conversely, APOLLO and NEURO-TTR scores ranged from −4 through 136, which is consistent with its reported use in diabetic neuropathy^11,12,34,35. Given the assessments were identical with respect to relative range and scoring weights, the Norfolk TQOL data were considered comparable across trials after the Fx-005 numerical scores were appropriately adjusted.

With regards to other outcomes, mBMI was reported similarly in both Fx-005 and APOLLO but was not reported in NEURO-TTR^10–12. Reporting of adverse events (AEs) due to any cause, serious AEs (SAEs) and withdrawals due to AEs was similar in all three RCTs.

Evidence network

The included RCTs formed a connected network of evidence with pairwise connections for the following outcomes: Norfolk TQOL, AEs due to any cause, SAEs and withdrawals due to AEs (Figure 1(A)). To form a connected network, differences in trial durations were assumed to be negligible. Of the 525 randomized patients included in the network, the APOLLO trial accounted for 43% (n = 225)¹¹, NEURO-TTR for 33% (n = 172)¹² and Fx-005 for 24% (n = 128)¹⁰. The evidence network for mBMI included Fx-005 and APOLLO (Figure 1(B))^10,11.

Figure 1. Evidence networks in ATTR-PN trials. (A) Network of RCTs for the Norfolk TQOL and safety outcomes (any adverse events, serious adverse events, and withdrawals due to adverse events). (B) Network of RCTs for the mBMI outcome. Abbreviations. INO, Inotersen; mBMI, Modified body mass index; PAT, Patisiran; PBO, Placebo; RCT, Randomized controlled trial; TAF, Tafamidis; TQOL, Total Quality of Life Score.

Baseline patient characteristics

Differences in the distribution of disease-related characteristics were identified across the treatment comparisons. Characteristics with notable differences included disease stage, disease duration, TTR mutation type and exposure to prior ATTR-PN therapies (Figure 2). All patients in Fx-005 presented with Stage 1 disease¹⁰, whereas 46% and 67% of the patients in APOLLO and NEURO-TTR, respectively, were enrolled with Stage 1^11,12. Over half of patients in APOLLO and 33% of patients in NEURO-TTR had Stage 2 disease^11,12. Duration since symptom presentation was shortest in APOLLO and longest in NEURO-TTR^10–12. The average disease duration for patients enrolled in NEURO-TTR was approximately 5.3 years¹², more than twice as long as patients enrolled in APOLLO (2.5 years)¹¹, while those in Fx-005 had average disease duration of 3.4 years¹⁰. All patients in Fx-005 were treatment-naïve¹⁰, whereas more than half of the APOLLO and NEURO-TTR populations received prior ATTR-PN therapies^11,12. All patients in Fx-005 had the Val30Met TTR mutation¹⁰, whereas this mutation was only present in 43% and 52% of the patient populations in APOLLO and NEURO-TTR^11,12, respectively, with the remainder of the population harboring other TTR mutations. The distribution of non-Val30Met mutations was variable across APOLLO and NEURO-TTR trial populations^11,12. The second most common mutation in the APOLLO population was Ala97Ser (9.3%)¹¹, whereas the second most common mutation in NEURO-TTR was Thr60Ala (12.8%)¹².

Figure 2. Distribution of baseline disease-related characteristics across ATTR-PN trials. Abbreviations. ATTR-PN, Transthyretin amyloidosis with polyneuropathy; TTR, Transthyretin; Val30Met, Valine to methionine point mutation at position 30.

Beyond the differences identified in disease-related characteristics, differences were also detected with regards to sex, age, race and geographical region (Figure 3). The proportion of male patients was approximately 30% and 20% greater in APOLLO and NEURO-TTR, respectively, than in Fx-005^10–12. The mean age was approximately 20 years older in APOLLO and NEURO-TTR than in Fx-005^10–12. A smaller proportion of Caucasian patients was also noted for the APOLLO trial than for Fx-005 and NEURO-TTR^10–12. Additionally, there were no North American patients enrolled in Fx-005¹⁰, compared to 21% and 48% in APOLLO and NEURO-TTR, respectively^11,12.

Figure 3. Distribution of baseline demographic characteristics across ATTR-PN trials. Abbreviations. ATTR-PN, Transthyretin amyloidosis with polyneuropathy; Eur., Europe; N. Am., North America; S. Am., South America; W. Eur., Western Europe.

Assessing plausible effect modifiers

The impact of clinical characteristics on treatment effect was assessed in pre-specified subgroups in the APOLLO and NEURO-TTR trials. A clinical characteristic was identified as a plausible effect modifier based on the magnitude of difference in treatment effect associated with the characteristic. Disease stage and prior ATTR-PN therapy exhibited the most prominent treatment effect differences in both APOLLO and NEURO-TTR, for both Norfolk TQOL and neuropathy endpoints^11,12. Although only numerical differences were observed, lack of statistical significance was expected as the trials were not powered to detect treatment differences in subgroup analyses.

Differences in baseline risk

The included RCTs were conducted between 2007 and 2013. Therefore, there was a possibility that differences attributable to changes in clinical practice over time could influence the treatment effect, which would warrant baseline risk adjustment (i.e. placebo response). For example, patients enrolled in Fx-005 were all treatment-naïve because of the lack of ATTR-PN therapies approved at the time of the trial¹⁰. Conversely, APOLLO and NEURO-TTR did not exclude patients with exposure to prior TTR stabilizers but required a washout of any TTR stabilizer therapy before study initiation^11,12.

In this network, there was a common treatment arm (i.e. placebo) that allowed for assessment of baseline risk. Two outcomes were reported consistently across all three trials: Norfolk TQOL and AEs; mBMI was reported consistently in Fx-005 and APOLLO only^10,11. Figure 4 presents the individual study results for Norfolk TQOL, mBMI and AEs per treatment for the included RCTs. This illustration suggests that the baseline risk among the included treatments was inconsistent, which could be associated with the dissimilarities in baseline patient characteristics that are plausible treatment effect modifiers. The changes in Norfolk TQOL score from baseline to study end varied greatly across the included RCTs, ranging from 7.2 to 14.4^10–12. Although not reported in NEURO-TTR¹², mean mBMI change was approximately 3.5 times greater in APOLLO compared to Fx-005^10,11. Rates of AEs due to any cause were similar across the included RCTs; however, the rate of SAEs was more than five times greater in APOLLO and almost three times greater in NEURO-TTR than in Fx-005^10–12. Placebo arm withdrawals due to AEs were high in APOLLO compared to the other trials^10–12.

Figure 4. Placebo response for commonly defined outcomes across ATTR-PN trials (ITT populations). ^†Small numerical difference in range reported for the Norfolk TQOL scale between studies. Abbreviations. AE, Adverse event; ATTR-PN, Transthyretin amyloidosis with polyneuropathy; ITT, Intention to treat; LS, Least squares; mBMI, Modified body mass index; NR, Not reported; SAE, Serious adverse event; TQOL, Total Quality of Life; WDAE, Withdrawal due to adverse events.

Decision to perform a network meta-analysis

The limited dataset of only three studies and lack of loops in the evidence network hindered the ability to synthesize data in an NMA. The included RCTs formed a connected open network of single-study connections through placebo as the common treatment arm, but differences identified in terms of the treatment regimens and outcome definitions also limited the feasibility of conducting valid indirect comparisons. Additional differences with respect to plausible treatment effect modifiers identified within the published data were considered to threaten the overall validity of an NMA, given the risk of introducing bias into the evidence base. Furthermore, the slowing of neuropathy impairment as a measure of efficacy could not be compared across trials, as components and scoring definitions were vastly different for each assessment. These differences inherently limit the clinical relevance of an NMA comparing ATTR-PN therapies. An NMA incorporating meta-regression to adjust for each of these differences was not possible because of the small number of studies. Based on the publicly available data, it was decided that an NMA was not feasible to compare the identified ATTR-PN therapies.

Discussion

The results of this study demonstrate the limited feasibility of performing a valid NMA to compare the relative efficacy of tafamidis, patisiran and inotersen for patients with ATTR-PN. It was not possible to achieve a connected network for changes in neuropathy impairment, which was a primary outcome in each of the three included RCTs, because of differences in outcome definitions and assessments used. A connected network was possible for Norfolk TQOL, AEs due to any cause, SAEs and withdrawals due to AEs; however, differences were identified in terms of clinical characteristics between trials, specifically with respect to plausible treatment effect modifiers. Some variation in treatment regimens, treatment doses, outcome definitions and baseline risk between trials was observed. As such, there was a substantial risk that these differences may introduce bias, threatening the overall validity of an NMA. It was therefore determined that an NMA was not feasible, which reflects a limitation of the available data.

Consistent with the findings from this feasibility assessment, a number of limitations pertaining to indirect comparisons in ATTR-PN were identified by the Institute for Clinical and Economic Review (ICER) in a recent report comparing patisiran and inotersen, based on data from the APOLLO and NEURO-TTR trials³⁶. In addition to issues with generalizability of the findings from either trial, ICER noted that the clinical relevance of changes in polyneuropathy as measured by the mNIS + 7 and the Norfolk TQOL was limited, given that thresholds for meaningful clinical change have not been established for these assessments³⁶. Furthermore, differences in the primary outcome measures and trial population (e.g. race, geographic region, disease severity) precluded comparison of the APOLLO and NEURO-TTR trials^11,12,36.

In our analysis of outcome definitions, we determined that the only efficacy outcome reported similarly across all three trials was the Norfolk TQOL score. Although APOLLO and NEURO-TTR evaluated changes in neuropathy impairment, as it is a critical disease-specific efficacy outcome for ATTR-PN, neither trial used the NIS-LL, Σ3 NTSF, or the Σ7 NTs assessments that were used in Fx-005^10–12. Instead, the trial-defined measure of neuropathy used in APOLLO was the mNIS + 7¹¹, while NEURO-TTR used a further modified version termed the mNIS + 7_Ionis¹². Unlike the mNIS + 7 assessments, the NIS-LL score does not contain nerve conduction or autonomic function domains³². Although the remaining domains of the NIS-LL (weakness, reflexes and sensation) are contained in both variations of the larger mNIS + 7 assessments, these tools include additional components not restricted to the lower limbs³². Thus, detailed mNIS + 7 or mNIS + 7_Ionis information from the APOLLO and NEURO-TTR trials could allow for derivation of the NIS-LL score^11,12,32. As this level of detail was not provided in the publicly available trial reports of APOLLO and NEURO-TTR^11,12, our feasibility assessment determined that the outcome could not be compared. Using unpublished data from the APOLLO trial, Planté-Bordeneuve et al. were able to synthesize NIS-LL scores for the APOLLO population in a recently published ITC^11,14,32. While the ITC investigated the relative treatment effect of tafamidis and patisiran, evidence is lacking to generate a consensus on the overall comparability of all three ATTR-PN therapies simultaneously¹⁴. Furthermore, the Planté-Bordeneuve et al. ITC has important methodological shortcomings that have previously been reported^14,37.

Most differences in baseline characteristics between the trial populations suggest that patients in Fx-005 were younger and had higher overall functioning than those in APOLLO or NEURO-TTR^10–12. Specifically, the mean patient age was 20 years lower in Fx-005 than in the other trials, the proportion of Stage 1 patients was approximately twice as high in Fx-005 (comprising all patients) as the other trials, and the mean baseline Norfolk TQOL score was approximately 20 points and 10 points higher in APOLLO and NEURO-TTR (14% and 7% of the entire range) than in Fx-005^10–12. Interestingly, this was not aligned with disease duration at baseline, which was lowest in the APOLLO population and greatest in the NEURO-TTR population^10–12. Although comparison of baseline neuropathy impairment based on original RCT data is not possible because of the differences in reporting, the baseline NIS-LL score for the APOLLO trial reported by Planté-Bordeneuve et al. was more than three times greater than that of the Fx-005 population^10,14. Taken together, it is clear that large differences in baseline characteristics between Fx-005 and either APOLLO or NEURO-TTR can be attributed to selection for a different status of patient, rather than a random result. Furthermore, traditional unadjusted NMAs comparing targeted therapies rely on summary or aggregate data that do not account for cross-trial differences. Therefore, we suggest the need for a rigorous approach to minimize the bias among comparisons of efficacy. As such, we advise caution regarding the interpretation of the Planté-Bordeneuve et al. results given the conclusion of the current feasibility assessment.

Results of the feasibility assessment indicated that an NMA was not feasible due to the extent of clinical heterogeneity between trials, particularly with respect to plausible effect modifiers. According to NICE, definitive evidence of effect modifier status is not necessary – rationale for considering a standard ITC inappropriate may be based on information suggesting that effect modifier status is plausible³⁰. The magnitude of difference observed in subgroup analyses from APOLLO and NEURO-TTR was sufficient to suggest effect modification due to variation in several disease and demographic characteristics. Collectively, the plausibility of multiple effect modifiers threatens the validity of unadjusted NMAs in ATTR-PN. In light of this, we suggest that a population-adjusted anchored indirect comparison such as an MAIC is more appropriate than standard methods for comparisons in ATTR-PN³⁰. Leveraging IPD, cross-trial differences may be minimized by adjusting for plausible treatment effect modifiers, such as disease stage, disease duration, TTR mutation type and exposure to prior ATTR-PN therapies, which is not possible with summary-level data only. Access to IPD from APOLLO and/or NEURO-TTR trials, which have more broadly defined patient populations than Fx-005, would be required for such pairwise comparisons. Recently, Obici et al. reported results of an MAIC comparing patisiran and inotersen¹³. The resultant effective sample size in such analyses would have to be taken into consideration when interpreting results, as low effective sample size may undermine the validity of MAICs despite correcting for between-trial differences. In general, our assessment indicates that there are limited options for conducting reliable indirect comparisons in ATTR-PN due to the extent of heterogeneity across trials. As interest in these therapies remains, it is likely that RCT evidence that addresses comparative safety and efficacy will be sought by decision makers for this rare disease population. Although a head-to-head comparison study would be ideal, the rarity of the disease and availability of efficacious therapies would likely preclude the conduct of such a study. In addition to exploring alternative ITC methods that leverage patient-level data, future research could look at conducting NMAs in ATTR-PN that incorporate real-world studies of effectiveness.

Several limitations of this assessment should be considered. Different placebo arms were used in each trial but were assumed to be sufficiently comparable in the interest of completing the full feasibility assessment. Additionally, assessment timepoints were not identical across trials but were considered equivalent, which could be a stronger source of heterogeneity. If an ITC had been conducted, the potential impact of these differences on resultant treatment effects is not clear. Finally, it should be noted that a formal systematic literature review was not conducted to inform this assessment. Given the rarity of ATTR-PN and the limited number of approved therapies, a targeted structured search was deemed sufficient for the purpose of identifying all relevant trials. Despite these limitations, the approach to assess the feasibility of an NMA was conservative and sufficiently detailed such that the underlying assumptions were systematically explored and that the risks (and benefits) of indirectly comparing treatment effects from RCTs were presented transparently.

Conclusion

A detailed assessment of cross-trial heterogeneity should be conducted when undertaking ITCs. Moreover, analysts should take care with selecting a methodology to adjust for cross-trial differences when possible. We have presented approaches to assess the impact of heterogeneity between trials using a case study in ATTR-PN. There was considerable clinical heterogeneity among the included RCTs in terms of patient characteristics, outcome definition and study design, thus limiting the reliability of any ITCs of tafamidis, patisiran and inotersen trials in patients with ATTR-PN. To circumvent these limitations, future studies should consider the use of alternative strategies such as population-adjusted methods for the indirect comparison of ATTR-PN treatments.

Transparency

Declaration of funding

This research was supported by Pfizer Inc. Pfizer Inc. is the manufacturer of tafamidis.

Author contributions: I.A.S., E.M.S., D.T. and C.C. were involved with the analysis of the data and drafting of the manuscript. L.A. and M.S. assisted with the interpretation of the data and critically reviewed for importance of intellectual content for the work. All authors approved the final version of the manuscript.

Declaration of financial/other relationships

I.A.S., E.M.S., D.T. and C.C. have disclosed that they are employees of Eversana, who were paid consultants to Pfizer Inc. in connection with the development of this manuscript. C.C. has disclosed that he is also a shareholder of Eversana. L.A. and M.S. have disclosed that they are employees of Pfizer Inc. and hold stock and/or stock options. CMRO peer reviewers on this manuscript have received an honorarium from CMRO for their review work but have no relevant financial or other relationships to disclose.

Acknowledgements

None reported.

Notes

Notes

i Vyndaqel is a registered trade name Pfizer Inc., New York City, New York, USA).

ii Onpattro is a registered trade name Alnylam Pharmaceuticals, Cambridge, MA, USA.

iii Tegsedi is a registered trade name Akcea Therapeutics, Boston, MA, USA.