Economic evaluations of exome and genome sequencing in pediatric genetics: considerations towards a consensus strategy

Abstract

Objective

Next Generation Sequencing (NGS) is increasingly used for the diagnosis of rare genetic disorders. The aim of this study is to review the different approaches for economic evaluations of Next Generation Sequencing (NGS) in pediatric care used to date, to identify all costs, effects, and time horizons taken into account.

Methods

A systematic literature review was conducted to identify published economic evaluations of NGS applications in pediatric diagnostics, i.e. exome sequencing (ES) and/or genome sequencing (GS). Information regarding methodological approach, costs, effects, and time horizon was abstracted from these publications.

Results

Twenty-eight economic evaluations of ES/GS within pediatrics were identified. Costs included were mainly restricted to direct in-hospital healthcare costs and varied widely in inclusion of sort of costs and time-horizon. Nineteen studies included diagnostic yield and eight studies included cost-effectiveness as outcome measures. Studies varied greatly in terms of included sort of costs data, effects, and time horizon.

Conclusion

Large differences in inclusion of cost and effect parameters were identified between studies. Validity of outcomes can therefore be questioned, which hinders valid comparison and widespread generalization of conclusions. In addition to current health economic guidance, specific guidance for evaluations in pediatric care is therefore necessary to improve the validity of outcomes and furthermore facilitate comparable decision-making for implementing novel NGS-based diagnostic modalities in pediatric genetics and beyond.

Keywords:

• Next generation sequencing
• exome sequencing
• genome sequencing
• health technology assessment
• economic evaluations
• children
• genetics

JEL Classification Codes:

• I00
• I
• I15
• I1
• I
• O10
• O1
• O

Introduction

Worldwide, approximately 350 million people are affected by rare disorders, of which 50% had an onset during childhood and 80% have a genetic origin^1,2. The diagnosis of rare genetic disorders in children is challenging and time consuming due to, among other things, the rarity of the individual disorder, the variability of the clinical manifestations, the genetic heterogeneity, and the deficiencies in laboratory testing. Recent developments in Next Generation Sequencing (NGS) have made it possible to investigate all protein-coding regions or even entire genomes in one single time, i.e. by implementing exome sequencing (ES) or genome sequencing (GS), respectively³. These developments have resulted in an increase in genetic diagnoses and a shortened time-to-diagnosis for patients with expected genetic disorders^4,5.

These rapid technological developments are often of clinical relevance, and subsequently studies on economic impact are increasingly being performed to assess the (added) value of new diagnostic modalities. To ensure valid decision-making and high-quality studies, it is essential that these studies adhere to respective health economic guidelines. Unfortunately, a recent study has indicated that there is currently a lack of adherence to these “basic health economic” guidelines in economic evaluations of pediatric genetics⁶.

Unfortunately, adherence to these “basic health economic” guidelines does not 100% ensure the validity and quality of evaluations. Recent studies in other areas have indicated the need for additional, so-called disease-specific guidance, in order to ensure right choices are made on a disease level with regard to inclusion of cost data, outcome parameters, and length of the analysis⁶. In addition to adherence to existing guidance it is therefore also essential to adhere and or to include (newly) developed guidance published in scientific journals, guidance which increases disease specific uniformity of methods and outcome measures. This would not only facilitate the ability to share and combine health economic data, but would possibly also lead to a decrease in research waste. An excellent example of such disease-specific guidance is the publication of Buchanan et al.⁷, in which they outline characteristics which should be included in economic evaluations.

Moreover, next to a complete and comparable collection of costs and adherence to existing guidelines, there is ongoing discussion about the inclusion of effect measures for genetic technologies. In the field of health economics, combining costs and effects into one outcome measure, such as cost per life year gained or cost per quality adjusted life year gained, is routinely used and subsequently decision-makers need to decide on reimbursement⁸. However, using a uniform effect measure, i.e. an outcome measurement which makes it easier to compare or combine studies, is very challenging within the field of pediatric genetics, as, most often, obtaining a genetic diagnosis does not immediately lead to treatment options.

Although recent reviews have indicated that challenges with comparability are prevailing in economic evaluations of pediatric genetics, they did not outline whether differences in included costs and effects are prevalent and which costs and effects were included by the individual studies^6,9. Also assessment on inclusion of disease-specific recommendations is often lacking.

An overview of all parameters used so far within previous economic evaluations is currently lacking. We therefore aim to review all economic evaluations of ES and/or GS performed for pediatric onset genetic disorders, identifying all costs, effects, and time horizon (i.e. time over which data was collected) included, and providing the differences in approaches taken. Approaches are compared to disease-specific guidance, as recently recommended^7,10.

Methods

Study design and search strategy

A systematic literature review was conducted (14 February 2021) to identify published economic evaluations, in which costs and effects are compared (model-based, prospective, and retrospective) of the clinical applications of ES and/or GS as a diagnostic tool for rare genetic disorders in a pediatric setting. Hereto, the following search strategy was applied: (sequence analysis OR high throughput nucleotide sequencing OR next generation sequencing OR whole genome sequencing OR whole exome sequencing) AND (costs and cost analysis OR cost-effectiveness) AND (children OR infant OR pediatric OR paediatric). The databases used for the search were PubMed/MEDLINE and EMBASE.

Two independent reviewers validated the published articles based on the following inclusion criteria: (i) the full-version of the article was available; (ii) the article was published in English; (iii) ES and/or GS are part of the economic evaluation described; and (iv) the study population consists of children aged 18 years or younger. The independent reviewers participated in both data extraction and article screening. Both reviewers independently selected studies suitable for data analysis based on the inclusion criteria. Discrepancies were resolved by discussing the article in question. There were no restrictions considering year of publication, since the role of NGS in genetic diagnostics (ES and/or GS) has only become more evident since 2009. Of all included studies, reference lists were reviewed to identify additional studies.

Data extraction

Characteristics including study population, health condition, sample size, comparison of genetic tests, time horizon, type of included costs (for example costs related to diagnostic testing or non-medical costs) and effects (i.e. outcome measures) were extracted and summarized to create an overview of all economic evaluations included in this study. Time horizon was defined as the duration over which costs and effects were included. For each included study, the final conclusion was extracted.

Data analysis

The included costs were compared to the cost components as stated by Drummond et al.¹⁰ and Buchanan et al.⁷ to determine which aspects are currently missing in the evaluations. Drummond et al.¹⁰ outlined that the following costs should be included: (i) costs within the healthcare sector, consisting of all medical costs directly resulting from the intervention and costs incurred during life years gained; (ii) costs for the patient and family, for example travel expenses, own contributions, time spending costs (e.g. time spent on informal care, time needed to undergo treatment or lost working hours due to an intervention) or costs of informal care; and (iii) costs in other sectors, which can be costs incurred in sectors outside the healthcare system, for municipal services, education, or voluntary work.

Buchanan et al.⁷ defined eight cost components, which were specifically attributable to the evaluation of genomic technologies: costs related to (i) patient recruitment (e.g. publicity and education of patients); (ii) blood or tissue sample collection; (iii) sample testing; (iv) data analysis; (v) communication of test results; (vi) actions taken based on tests results; (vii) training and infrastructure (e.g. costs related to staff training); and (viii) indirect costs. In order to judge which effects should be measured during an economic evaluation, an overview of all included effects was created.

Results

Studies identified

Based on our database searches, we identified 313 studies in Pubmed/MEDLINE and 494 studies in EMBASE, resulting in 807 unique studies. Of these, 28 studies (3.5%) fulfilled our inclusion criteria. Based on the PRISMA reporting guidelines¹¹, an overview of the complete selection procedure is shown in Figure 1. Manual inspection of the reference lists from these 28 studies did not yield any new studies.

Figure 1. Overview of systematic database search.

In Table 1, the characteristics of the included studies are summarized. The rare disorders of the children for which the child received ES/GS (i.e. intellectual disability, epilepsy, autism spectrum disorder) and the cohort sizes varied (IQR = 40–300, median = 101). Nine studies (32.1%) included a scenario analysis^{13,16,18,23–26,28,37} and six (21.4%) investigated the implementation of GS instead of, or in addition to, ES^{12,21,24,26,36,39}. Time horizons were also different for the included studies, and there was no consensus regarding the moment a study started nor the moment a study ended (Table 1). In more detail, five studies (17.9%) had a time horizon of a fixed duration (range = 7–24 months)^{12,23,24,29,38} and six studies (21.4%) included the complete diagnostic trajectory^{16,25,31,35–37}. There were eight studies (30.8%) which started at a certain moment in time (i.e. first visit in hospital, onset of symptoms or moment of inclusion) until a diagnosis was found^{13,15,19,20,22,28,30,32}. Five studies (17.9%) looked at what had happened after ES was performed^{21,26,27,34,39} and the remaining four studies (15.4%) included a time period from the onset of symptoms or first visit to the hospital until ES was initiated^14,17,18,33. The final conclusions of the included studies are shown in Table 1.

Table 1. Overview of characteristics of studies included in this review.

Study	Country	Health condition	Total sample size	Genetic test	Time horizon	Conclusion
Soden et al.^12	United States	Neurodevelopmental disorders	119	Standard ES versus rapid GS	First 33 months of diagnostic trajectory	Sequencing exomes or genomes should become an early part of the diagnostic workup for NDD
van Nimwegen et al.^13	Netherlands	Neurological disorders	50	Standard diagnostic trajectory versus: (1) ES, by which ES replaces: All genetic tests currently used (2) ES, by which ES replaces: All genetic tests Repeated and burdensome tests 50% of the physician visits	From first physician visit in the university hospital until a diagnosis was found	Novel diagnostic strategies, including NGS, should be evaluated
Valencia et al.^14	United States	Genetic disorders	40	Standard ES in addition to or replacing standard diagnostic trajectory	All diagnostic tests performed before ES	Implementing ES is clinically useful and cost-effective
Joshi et al.^15	United States	Early onset epileptic encephalopathies	4	ES versus standard diagnostic trajectory	From inclusion until diagnosis was found	Cost savings if ES was performed early in diagnostic trajectory
Sabatini et al.^16	United States	Non-small cell lung cancer, sensorineural hearing loss, neurodevelopmental disorders	—	Three different diagnostic trajectories including ES	Complete diagnostic trajectory	Both sequencing technology and cost savings will improve over time, so cost-impact will be refined in the future
Nolan and Carlson^17	United States	Pediatric neurological disorders	53	ES versus secondary genetic and metabolic testing	All diagnostic tests performed before ES	ES could allow for more efficient and fruitful diagnostic neurological evaluations
Monroe et al.^18	Netherlands	Intellectual disability	17	ES versus standard diagnostic trajectory: Diagnosed patients: ES replaces all genetic costs (except aCGH and SNP arrays) and metabolic assessments Undiagnosed patients: ES replaces all genetic costs (except aCGH and SNP arrays) Both groups: assumption that ES results in cost reduction of 50% for health-care visits, imaging, biochemical investigations, and patient day admission	From first visit to hospital until initiation of ES	Early implementation of ES is relevant and cost-effective
Schofield et al.^19	Australia	Muscle diseases	56	Standard diagnostic trajectory versus molecular diagnostic trajectory (Neuromuscular gene panel or ES)	From referral of patients with suspicion of a diagnosis until genetic diagnosis or return of ES if no genetic diagnosis was found	Cost-effectiveness of implementing ES increases data reanalysis, testing first-degree relatives and parental reproductive outcomes are taken into account
Vissers et al.^20	Netherlands	Neurological disorders of suspected genetic origin	150	ES versus standard diagnostic trajectory	From first visit to hospital clinic until diagnosis was established	ES results in a higher diagnostic yield without increasing the costs
Hayeems et al.^21	Canada	Developmental delay	101	GS versus CMA	All clinical care activities prompted by CMA or GS	The clinical and economic consequences of implementing GS depend on the evaluation of downstream care and cost consequences
Stark et al.^22	Australia	Childhood syndromes	40	ES versus standard diagnostic trajectory, by implementing ES three different moments in time: (1) ES after basic & complex investigations (2) ES after basic investigations (3) ES immediately after admission	From age of onset symptoms until a diagnosis was established or an uninformative ES report was issued	Implementing ES early in the diagnostic trajectory significantly increases diagnostic rate and decreases cost per diagnosis
Tan et al.^23	Australia	Suspected monogenic disorders	40	ES versus standard diagnostic trajectory, by comparing four pathways: (1) Standard diagnostic trajectory without ES (2) Standard diagnostic trajectory with ES (3) ES applied at first clinical genetics assessment (4) ES at tertiary presentation	Children suspected of having a monogenic condition were followed for 7 months	ES is associated with a higher diagnostic yield and cost-effectiveness is maximized by implementing ES early in the diagnostic trajectory
Tsiplova et al.^24	Canada	Autism spectrum disorder	300	Three models, comparing: CMA and ES versus CMA CMA versus GS CMA and ES versus GS	Fixed time period of 5 years	The added value of ES and GS will increase over time, since costs will decrease and clinical benefit will increase
Dillon et al.^25	Australia	Suspected monogenic disorders	145	ES versus three disease-specific panels recommended by clinical experts	Complete diagnostic trajectory	ES has a broader coverage with no change in diagnostic costs
Yuen et al.^26	Canada	Autism spectrum disorder	1,000	Four models, comparing: CMA only CMA and ES ES only GS only	Two years, starting from time of autism spectrum disorder diagnosis	At the moment, ES or GS does not lead to cost savings, but can become cost-effective. Focus should not only be on diagnostic yield, but also on utility and the increasing need for genetic services
Stark et al.^27	Australia	Rare diseases	80	Re-analysis of ES results compared with ongoing standard diagnostic testing	From ES disclosure (February 2014) until the end of study (October 2016) with a minimum follow-up of 12 months	This study supports early implementation of ES
Howell et al.^28	Australia	Epilepsy	114	ES at different stages during diagnostic tract	From epilepsy onset until diagnosis found	Early implementation of ES leads to an increase in diagnostic yield associated with lower costs
Vrijenhoek et al.^29	Netherlands	Intellectual disability	370	ES at different stages during diagnostic tract	From first visit to hospital until a fixed moment in time	Implementing ES can be cost-effective
Stark et al.^30	Australia	Pediatric care	40	Rapid ES versus standard diagnostic trajectory	From age of onset symptoms until a diagnosis was established or an uninformative ES report was issued	Rapid ES is associated with high diagnostic and clinical utility and is cost-effective
Palmer et al.^31	Australia	Epileptic encephalopathy	32	ES versus standard diagnostic trajectory	Complete diagnostic trajectory	Use of ES is cost-effective and has a higher clinical utility
Demos et al.^32	Canada	Early-onset epilepsy	180	ES versus standard diagnostic trajectory	From genetic counseling until research validation of the variant	Performing ES early in the diagnostic trajectory affects diagnostic yield, clinical utility and potential cost-effectiveness positively
Radio et al.^33	Italy	Rare diseases	324	ES versus standard diagnostic trajectory	Onset of symptoms until an ES was performed	ES decreases the diagnostic trajectory and positively influences management of undiagnosed patients
Schofield et al.^34	Australia	Suspected monogenic disorders	80	ES versus standard diagnostic trajectory, by implementing after basic investigations	From age of onset symptoms until 473 days post-result	Use of ES is cost-effective
Yokoi et al.^35	Japan	Multiple congenital anomalies and intellectual disability	200	ES versus aCGH	Complete diagnostic trajectory	ES saves costs and time
Dragojlovic et al.^36	Canada	Suspected genetic disorders	498	Standard diagnostic trajectory including GS versus standard diagnostic trajectory without GS	Complete diagnostic trajectory until GS was performed	For economic models, longer time horizons need to be included. Especially for patients without a conclusive diagnosis, long-term costs may influence conclusions about the cost-effectiveness of GS
Kosaki et al.^37	Japan	Suspected monogenic disorders	360	ES at different stages during diagnostic trajectory	Complete diagnostic trajectory	ES as first tier test leads to cost-savings, but more research is needed to see whether this also applies to other countries
Smith et al.^38	United States	Suspected genetic disorders	368	ES versus standard diagnostic trajectory	One year, starting from the first genetics consult	More research is needed in order to define the most optimal use of ES
Yeung et al.^39	Australia	Complex, suspected monogenic disorders	92	GS versus standard diagnostic trajectory	Fixed time period of 3 years after diagnosis	GS leads to increased diagnostic yield and decrease in costs

Abbreviations. ES, exome sequencing; GS, genome sequencing; aCGH, microarray-based comparative genomic hybridization; CMA, Chromosomal microarray.

Included costs and effects

Table 2 shows a summary of the different cost categories according to both Drummond et al.¹⁰ and Buchanan et al.⁷ Regarding inclusion of internal costs (i.e. diagnostic and non-medical costs within the hospital of interest), external costs (i.e. costs outside or in another hospital), and additional costs, results are diverse: 14 studies (50.0%) included only internal costs^{12,14,15,18,24,25,29,31–33,35,37–39}, whereas 11 studies (42.3%) also took a part of the external costs into account^{13,16,17,19–22,27,28,30,34}; two studies (7.7%) also investigated nonmedical costs like traveling costs^23,36 and time spending costs for (parents of) the patient, such as time lost due to medical visits^26,36. Furthermore, 11 studies (42.3%) solely focused on costs of diagnostic testing^{12,14,16,17,19,20,22,24,25,32,35}. None of the studies took costs incurred during life years gained or costs related to other sectors (non-medical costs) into account.

Table 2. Included costs categories according to Drummond et al.¹⁰ and Buchanan et al.⁷.

	Soden et al.^12	van Nimwegen et al.^13	Valencia et al.^14	Joshi et al.^15	Sabatini et al.^16	Nolan and Carlson^17	Monroe et al.^18	Schofield et al.^19	Vissers et al.^20	Hayeems et al.^21	Stark et al.^22	Tan et al.^23	Tsiplova et al.^24	Dillon et al.^25	Yuen et al.^26	Stark et al.^27	Howell et al.^28	Vrijenhoek et al.^29	Stark et al.^30	Palmer et al.^31	Demos et al.^32	Radio et al.^33	Schofield et al.^34	Yokoi et al. ^35	Dragojlovic et al.^36	Kosaki et al.^37	Smith et al.^38	Yeung et al.^39
Included cost categories according to Drummond et al.^10
Costs within the healthcare sector
- Costs incurred during life years gained
Internal costs
- Costs genetic diagnostics	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
- Costs other diagnostics	X	X		X		X	X	X	X	X	X	X			X	X	X	X	X	X	X	X	X		X	X	X	X
- Other medical costs		X		X			X			X		X			X	X	X	X	X	X		X	X		X	X	X	X
External costs
- Diagnostic costs in different hospitals		X			X	X		X	X	X	X					X	X		X				X
- Other medical costs in different hospitals		X								X						X									X
Costs for patient and family
- Travel expenses												X													X
- Own contributions/co-payments
- Time spending costs															X										X
- Costs of informal care
Costs in other sectors
- Non-medical costs (e.g. municipal services, education or voluntary work)
Included cost categories according to Buchanan et al.^7
Patient recruitment
Sample collection	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
Sample testing	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
Data analysis	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
Communication of test results		X		X			X			X		X			X	X		X		X		X			X	X	X	X
Action taken based on test results		X		X			X			X		X	X			X		X										X
Training and infrastructure
Indirect costs												X			X										X

Compared to the cost components of Buchanan et al.⁷, all of the studies included costs related to sample collection, sample testing, and data analysis. Fourteen out of 28 studies (50.0%) took post-test counseling into account^{13,15,18,21,23,26,27,29,31,33,36–39} and nine out of 28 studies (32.1%) took action taken based on test result into account^{13,15,18,21,23,24,27,29,39}. Three studies (11.5%) included non-medical costs, such as travel expenses and time spending costs of patients and family^23,26,36. Costs related to patient recruitment and training and infrastructure were not investigated.

Table 3 shows a summary of the investigated effects. Nineteen studies (67.9%) included diagnostic yield^{12–14,17–20,22,23,25,27,29,31,32,34,35,37–39}. Two studies (7.7%) took the number of ongoing pregnancies and utilization of reproductive genetic services into account^27,34. Considering cost-related outcomes, included outcome effects varied: five studies (19.2%) focused on cost-effectiveness only^{12,14,19,22,23}, but the more recent publications included incremental costs per additional positive finding in (hypothetical) testing scenarios^24,25, incremental costs per additional diagnosis^26–28,31, or an incremental cost-effectiveness ratio (ICER)³⁴.

Table 3. Effect measures of the studies included in this review.

Year of publication	2014		2015		2016				2017						2018							2019			2020
Study		Soden et al.^12	van Nimwegen et al.^13	Valencia et al.^14	Joshi et al.^15	Sabatini et al.^16	Nolan and Carlson^17	Monroe et al.^18	Schofield et al.^19	Vissers et al.^20	Hayeems et al.^21	Stark et al.^22	Tan et al.^23	Tsiplova et al.^24	Dillon et al.^25	Yuen et al.^26	Stark et al.^27	Howell et al.^28	Vrijenhoek et al.^29	Stark et al.^30	Palmer et al.^31	Demos et al.^32	Radio et al.^33	Schofield et al.^34	Yokoi et al. ^35	Dragojlovic et al.^36	Kosaki et al.^37	Smith et al.^38	Yeung et al.^39
Clinical effect measures
Clinical effectiveness		X
Diagnostic yield		X	X	X			X	X	X	X		X	X		X		X		X		X	X		X	X		X	X	X
Diagnostic sensitivity		X
Duration of diagnostic trajectory			X																	X		X		X	X	X
Diagnostic rate							X																				X
Healthcare consequences											X																		X
Number of diagnoses																X		X
Number of ongoing pregnancies																	X							X
Number of reproductive genetic services used																	X							X
Differences in short-term resource use and wait time between genetic testing strategies																X
Survival																												X
Economic effect measures
Cost-effectiveness		X		X					X			X	X				X				X			X
Cost savings					X		X	X		X							X		X	X	X	X		X			X		X
Cost impact						X					X
Cost ranges						X																	X
Healthcare resource use			X														X
Associated direct medical costs			X
Costs in general														X	X			X	X						X	X	X	X	X
Incremental costs per additional positive finding in (hypothetical) testing scenarios														X	X
Cost per diagnosis																	X	X		X	X			X	X	X		X
Cost per patient																X	X	X		X	X	X	X	X					X
QALY																	X							X
Cost of reanalysis																	X							X
Incremental cost per additional diagnosis																X	X	X			X
Costs after diagnosis																			X
Yearly costs of diagnostic trajectory																										X
Total and yearly cost of diagnostic delay																							X
ICER																								X
Costs of parental reproductive costs																								X

Abbreviations. QALY, Quality-adjusted life year; ICER, Incremental cost-effectiveness ratio.

Discussion

Based on our inclusion criteria, to date 28 economic evaluations regarding NGS have been performed within the pediatric population. These economic evaluations were published between 2014 and 2020. Our review outlined the presence of an extensive variability in choice and usage of costs, effects, and time horizon in these economic evaluations. According to the cost categories defined by Drummond et al.¹⁰, costs included by the studies mainly focused on diagnostic costs. Only three out of 28 took other healthcare resources (i.e. travel expenses^23,36 and time spending costs^26,36) into account, while personal costs (i.e. costs of informal care, own contributions/co-payments, and non-medical costs) were not included at all. The cost categories as defined by Drummond et al.¹⁰ and Buchanan et al.⁷ focus on the costs related to (genetic) diagnostics and beyond diagnostics. However, no uniformity can be found between the cost taken into account by the different studies in this review. In order to increase uniformity, future studies should, at a minimum, follow their guidelines^7,10. Since it will be challenging to include non-medical costs, we suggest to also include at least non-diagnostic costs in addition to the diagnostic costs. If studies work to conform to this approach, this would also improve insight in and appreciation of the cost-effectiveness of ES and GS³⁶. This is also supported by a study of Vrijenhoek et al.²⁹ which stated that benefit of genetic diagnostics can be broader than the diagnostic test itself. Having a diagnosis can also influence future health, treatment can be started earlier or expensive surgeries can be prevented.

Considering time horizon, no uniform method was used to determine the start and end of a study. Most of the studies ended the economic evaluation when a final diagnosis was found or when ES was initiated. However, the total duration included by the studies varied widely. This confirms the need for a uniform approach to ensure comparability. A study of Dragojlovic et al.³⁶ concluded that longer time horizons need to be included. Especially in patients without a conclusive diagnosis, long-term costs may influence conclusions about the cost-effectiveness of a diagnostic test. It is not possible to judge whether studies which took a fixed time-period as time horizon also took the complete diagnostic trajectory into account. To capture the full benefit of ES and/or GS, at least the complete diagnostic trajectory should be taken into account. According to Dragojlovic et al.³⁶, a longer follow-up period of at least 2 or 3 years is needed to capture the health benefit following having or not having a diagnosis. The latter may even suggest that stratification according to the outcome of the genetic test should be performed, as differences may be expected.

Interestingly, there seems to be a shift in focus on how effects were studied over time. The first publications regarding evaluations of NGS in pediatric genetics include effects directly related to the diagnostic trajectory, for instance diagnostic yield or cost-effectiveness itself. In health economic evaluations, a quality-adjusted life year (QALY) is most often used as an effect measure to calculate the incremental cost effectiveness ratio (ICER)⁴⁰. Since the end of 2017, it is common to use the ICER used to investigate cost-effectiveness within the field of health technology assessment. Although Stark et al.²⁷ did investigate QALYs, this was not included in an ICER. The first two studies which calculated an ICER included additional positive findings in scenario analysis as an effect measure^24,25. Later on, effects were replaced by an additional number of diagnoses^26,28. A more recent study of Schofield et al.³⁴ included QALYs to calculate the ICER. Although valuable, making decisions on willingness to pay per additional diagnoses is difficult. Besides, also no conclusive diagnosis can have added value to both doctors and (parents of the) patients, for example, in the case of a severe disease. Expensive care might be continued if no severe diagnosis can be found. For decision-makers, QALYs are very well-known parameters in the decision-making process. Other outcome measures, such as clinical utility, are more difficult to base reimbursement decisions upon, although these better reflect the added clinical value within the pediatric population. Findings of this study again demonstrated that more discussions should be initiated with all decision-makers to gain complete insight in most valuable other outcome measures compared to the QALY to base a decision upon.

Remarkably, two studies included the number of ongoing pregnancies and the utilization of parents’ reproductive genetic services as effect measures^27,34. This result also suggests that including costs goes beyond including costs directly related to the patient (child) him/herself. That is, in daily practice, guidance based on genetic test results is often not limited to impacting (future) life decisions of the patient him/herself, but also affects the choices and healthcare costs of his/her (blood) relatives. In order to perform economic evaluations in NGS, it is important to include all relevant information. However, at the moment it is unclear how this relevant information can be defined and to what extent it is needed to involve effect measures related to the relatives of the child. These findings confirm the need of a uniform approach, including all cost aspects, which should be the minimum requirement to guide decision-making.

Recently, it has been outlined that non-adherence to current health economic checklists is a major issue in economic evaluations of NGS for pediatric patients⁶. Although Alam and Schofield⁶ make a valid point regarding non-adherence, they did not discuss that disease-specific guidance to perform an economic evaluation in a certain setting was lacking. Examples include for instance that the use of terminology by the included papers was very diverse (Table 3), but also that is was unclear what was meant by the terms in the absence of definitions, making comparisons and interpretation difficult. Whereas these at large may depend on the study perspective, also these were not clearly outlined. The end results are economic evaluations within pediatrics for which it is underdefined which costs to include or how to define these. Such practical, disease-specific guidance is needed to ensure the improvement of future economic evaluations in genetics and thereby ensure the correct collection of highly valid data which can be used by the entire scientific field. These improvements can be created by more guidance as many authors of economic evaluations take previous papers as an example when deciding upon inclusion of costs; high quality standards should therefore be developed and also journals should be made aware of the importance of so-called cross-validation.

It was unclear whether or not studies had the intention to perform a full economic evaluation. In order to create more uniformity and comparable studies, it is essential that studies indicate whether the objective was to conduct a full-economic evaluation and/or whether there was a transparent reason to include certain costs and or effects in the evaluation.

The variability in choice of costs and effects also indicates the lack of discussion and guidance with current decision-makers within the field. It is highly relevant that such a discussion takes place to ensure inclusion of the most valuable outcome measures for making decisions on reimbursement and timing of expensive diagnostics. Within pediatric genetics, disease-specific guidance as outlined in this review is needed. This guidance would increase the quality, reliability, and comparability of outcomes. Current initiatives such as the GEECS (Global Economics and Evaluation of Clinical Genomics Sequencing Working Group) are essential in this case⁴¹. Within GEECS, for instance, they focus on improving methods used for assessing the value of new genomic technologies. The lack of uniformity and consensus on outcome measures (as indicated in this study) indicates that guidance and consensus on outcome measures should have high priority to ensure and improve decision-making.

Finally, the majority of the included evaluations in recent reviews have had a focus on the implementation of ES^{13–20,22,23,25,27–35,37,38}, and only six studies investigated the implementation of GS^{12,21,24,26,36,39}. In order to help decision-makers with future decisions, especially in a fast-developing field such as medical genetics, it would be very relevant to outline which outcomes and individual input parameters are useful for future decisions on for instance implementation of GS in clinical practice. If achieved, this will also improve the power of evidence and better guide future research on cost-effectiveness studies towards those areas for which evidence is still largely unexplored. Within the field of medical genetics, much effort is being made to share knowledge on genetic causes of disease. Similar efforts should be made to push the field of economic analysis in genetics. Hereto, transparency in sharing outcomes and knowledge with regard to health economic evaluation is essential.

In general, it is challenging to use current health economic methods to capture the full benefit of NGS-related diagnostics. For instance, the inclusion of secondary findings, non-health benefits, and family spillover effects are difficult to incorporate in cost savings, let alone in one overarching number informing decision-makers. New approaches and methods should become available to fully capture, address, and evaluate the added benefit. Methodological research is essential in order to more precisely estimate the impact of genetic diagnostics and to improve well-informed/valid decisions in the near future.

This study has two (minor) limitations. First of all, only Pubmed/MEDLINE and EMBASE were searched. However, we also manually inspected the reference lists of the included studies to ensure that all relevant studies were included in this review. This did not lead to any new inclusions. Another possible limitation of this study was that we did not perform quality checks for the included studies. Although we did use the PRISMA reporting guidelines¹¹, no alternative quality checks were performed. However, we do not think that inclusion of extra databases or adding quality checks would result in different results and/or conclusions.

Conclusions

In conclusion, a large variety in choice of costing characteristics and outcome measures is present in economic evaluations of new genetic technologies in pediatric conditions. This variability, shown by the included studies, indicates randomness in methods for economic evaluations in NGS and hampers a reputable comparison between outcomes. We argue that an improvement in cost collection (duration and type of cost data) and standardization in outcome measure is necessary for valid decision-making in the field of pediatric genetics and beyond. In addition to collaboration on clinical outcomes and data sharing, we should strive for a uniform approach in health economics in genetics to fight research waste but even more to speed up and improve the decision-making process; now is the time.

Transparency

Declaration of funding

This work was financially supported by grants from the Netherlands Organization for Health Research and Development (ZonMw) [843002608 and 846002003 to L.E.L.M. Vissers and J.K. Ploos van Amstel]. The aims of this study also contribute to the Solve-RD project (to L.E.L.M. Vissers) which has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 779257.

Declaration of financial/other interests

The authors have no conflict of interest to declare. Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgements

The results of this study contribute to the goals of the RADICON-NL consortium, which aims to increase diagnosis for patients with rare diseases in a shortened time frame, thereby reducing the complexity and overall costs associated with obtaining this diagnosis.