Exploring the approach to parameter uncertainty in early economic evaluations of surgical technology – a systematic review

ABSTRACT

Introduction

The role of early economic evaluation (EEE) in the development of medical technology has been increasingly recognized; however, data on the use of EEE in surgical technology are sparse. The objective of this review was to explore the use of EEE in the development of surgical technologies, with emphasis on how uncertainty has been addressed.

Areas covered

A systematic review was conducted, and original articles employing any form of EEE of surgical technology were selected for review, with 10 studies included in the analysis. These studies demonstrated significant variation in the approach to managing parameter uncertainty, specifically regarding the type of analysis used and the inclusion of effectiveness parameters in sensitivity analysis. The conclusions drawn did not appear to factor in uncertainty in the models.

Expert opinion

Approaches to handling parameter uncertainty in previous EEEs of surgical technology have been limited, with some studies failing to address parameter uncertainty. In addition, EEEs do not appear to follow established guidelines with respect to the use of sensitivity analyses. It is important that EEEs of surgical technology address parameter uncertainty in order to draw more robust conclusions from the analysis and allow investors to consider this uncertainty when making investment decisions.

KEYWORDS:

• Cost-effectiveness
• early economic analysis
• economic evaluation
• expected value of perfect information
• headroom analysis
• medical devices
• modeling
• probabilistic sensitivity analysis
• sensitivity analysis
• surgical technology

Acknowledgments

The authors would like to thank Dagmara Chojecki for her expertise in developing the search strategy.

Article highlights

• There is a paucity of literature examining the role of early economic evaluation (EEE) in the development of surgical technology, and as such, well-defined approaches to conducting EEE of surgical technology are lacking.

• Understanding the maximum cost at which a novel technology is cost-effective at a given Willingness-to-Pay threshold (headroom analysis), is a valuable step in the early stages of surgical device development. However, less than half of the studies included in our analysis presented headroom results, suggesting that its utility in EEE of surgical technology may not be completely understood.

• Managing uncertainty in EEE of surgical technology can prove challenging, however a number of techniques can be employed to help mitigate these challenges. Probabilistic Sensitivity Analysis (PSA) is one tool that may serve this purpose. The data shown in our analysis suggest that PSA may be underutilized in EEE of surgical technology.

• Value of Information (VOI) analysis can be beneficial in surgical device development in a variety of ways; however, the results of our analysis suggest that the merits of VOI analysis may not be fully appreciated by those commissioning EEEs of early-stage surgical technology.

• Uncertainty in the wider care pathway is often overlooked in EEE of surgical technology, which may lead to conclusions that appear more certain than they are in reality.

• Uncertainty should be better incorporated into EEE of early-stage surgical technology following the same guidelines used for economic evaluations of late-stage technologies.

Declaration of interest

W Zhang acknowledges support by a Michael Smith Foundation for Health Research (MSFHR) Scholar Award. MSFHR had no role in the design, methods, data collection, analysis, data interpretation, or preparation of this manuscript. I Browne was a consultant for Voyage Biomedical Inc. and an advisor to Neuraura Biotech Inc. I Browne holds shares in Voyage Biomedical Inc., Neuraura Biotech Inc. and Protxx Inc. I Browne declares funding in part from BioTalent Canada. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Registration and data availability

This systematic review represents the culmination of work performed based on a non-registered protocol. The protocol, data extraction forms and complete data tables are available at https://doi.org/10.6084/m9.figshare.20445261.v1.

Additional information

Funding

This paper was supported in part by a grant from BioTalent Canada. BioTalent Canada had no role in the design, methods, data collection, analysis, data interpretation, or preparation of this manuscript. W Zhang acknowledges support by a Michael Smith Foundation for Health Research (MSFHR) Scholar Award. MSFHR had no role in the design, methods, data collection, analysis, data interpretation, or preparation of this manuscript.