Methodological similarities and variations among EQ-5D-5L value set studies: a systematic review

Abstract

Objectives

(1) To descriptively compare the selected elements of valuation methods for EQ-5D-5L value set studies, (2) to compare the characteristics of the value sets, and (3) to examine the associations between the selected elements of valuation methods and the EQ-5D-5L value sets.

Methods

A systematic literature search of EQ-5D-5L valuation studies from 1 January 2009 to 22 April 2021 was conducted in selected databases. Following the initial search, we also explored additional studies published during the completion of the final version of the manuscript. Similarities and variations for selected elements of valuation methods were descriptively compared. The relative importance of dimensions, utility decrements between the levels, and distribution of the utility scores were used to compare value sets. A meta-regression analysis examined the associations between the selected methodological elements and the utility scores and dimension levels of EQ-5D-5L.

Results

A total of 31 studies were included in this review. Methodological similarities centered around data collection and preference elicitation method. On the other hand, variations include sampling technique, sample size, and value set modeling. The variations in value sets based on the relative importance of dimension, decrement in utility score, and distribution of utility score across countries were observed. Although the distribution of the utility scores differed across countries, higher levels of each dimension tended to have a larger decrement in the utility scores. Mean utility scores for the experience-based value sets were higher than those estimated using stated choice methods. The selected methodological elements were not significantly associated with the mean predicted utility scores or most dimension-level coefficients.

Conclusions

EQ-5D-5L health state valuation methods and characteristics of value sets differed across studies. The impact of the variation of methodological elements on the value sets should be further investigated.

Keywords:

• EQ-5D-5L
• EuroQol
• value set
• tariff
• health utility
• health state valuation
• preferences

JEL Classification Codes:

• I10
• I1
• I
• I18

Introduction

EQ-5D is a standardized multi-attribute utility instrument (MAUI) introduced by the EuroQol group in the late 1980s to describe a health state, for which a value can be attached¹. It is composed of five dimensions, i.e. mobility (MO), self-care (SC), usual activities (UA), pain/discomfort (PD), and anxiety/depression (AD)². In the initial version (EQ-5D-3L), each dimension has three levels, i.e. no problems, some problems, and extreme problems/unable to, thus describing a total of 243 (3⁵) possible health states². The EQ-5D-3L also includes the patient’s self-rated health on EuroQol visual analog scale (EQ VAS) with values ranging from 0 (worst health state) to 100 (best health state), one can imagine^3,4. However, given the problems of lack of sensitivity (unable to detect small or moderate changes in a person’s health) and presence of ceiling effect (clustering of scores towards the higher end of measurement and little differentiation for mild health states), the EQ-5D with five levels (EQ-5D-5L) was introduced in 2009^4–6. These five levels include no problems, slight problems, moderate problems, severe problems, and extreme problems, describing 3,125 (5⁵) possible health states⁷. Five levels increase the sensitivity, reduce the ceiling effect, and improve the validity of the EQ-5D^5,8. The most severe level of the MO dimension was changed from “confined to bed” to “unable to walk about” in EQ-5D-5L, enhancing its applicability⁹. The instructions for the EQ VAS task of the EQ-5D-5L were simplified compared to EQ-5D-3L, making it easier to complete and score^2,10.

Additionally, a standardized protocol was developed to introduce uniformity in the valuation methods, to support best practices, promote a high level of quality control, and facilitate comparability of value sets for the EQ-5D-5L^11–13. The computer-embedded EQ-5D-5L valuation protocol, also known as EuroQol Valuation Technique version 1.0 (EQ-VT v 1.0)¹¹, consists of five elements; (1) A general welcome, (2) Introduction, where participants self-report their health on the EQ-5D-5L descriptive system and EQ VAS and background question, (3) Instruction and example of time trade-off (TTO) task, TTO valuation of 10 EQ-5D-5L states, and debriefing/structured feedback, (4) An instruction of discrete choice experiment (DCE) task, DCE valuation of seven paired comparisons, three debriefing questions, and (5) A general thank you and goodbye. The EQ-5D-5L valuation protocol also consists of an interviewer instruction manual.

Experience from earlier valuation studies suggested that, apart from the complexity of the task, random error, and learning effect, the interviewer effect and quick task completion (satisficing) from the respondent may lead to large numbers of inconsistent responses with a detrimental effect on the data quality¹³. To address this challenge EQ-VT v 1.0 was updated to EuroQol Valuation Technique version 2.0 (EQ-VT v 2.0) and, more recently, EuroQol Valuation Technique version 2.1 (EQ-VT v 2.1)¹³. Two major changes were introduced in EQ-VT v 2.0. First, three practice states followed by a wheelchair example and a feedback module were incorporated in the composite TTO (cTTO) task. Secondly, a quality control (QC) process was included after the end of the survey to review protocol compliance and interviewer effect¹³. Interviews were flagged based on four QC criteria: (1) if there was no explanation of the worse than dead task (lead time) in the wheelchair example, or (2) too short a time was spent on the wheelchair example (less than 3 minutes), clear inconsistencies were noted in the cTTO ratings, and if all 10 cTTO tasks were completed in less than 5 minutes¹³. Some countries employed the EQ-VT v 1.0, whereas those who conducted their valuation studies after the publication of the EQ-VT v 2.0 did so using the current version.

Despite the introduction of the standardized valuation protocol, value sets could still be artificially affected by the variations in the valuation methods or study designs that made a comparison of the preference value for health states across countries difficult^14–16. A study indicated that variation in preference elicitation technique (cTTO vs. DCE) or change in protocol might systematically affect the utilities¹⁷. Previously, some studies examined the differences in health preference across countries for EQ-5D-5L^14,18,19. For instance, Yang et al.¹⁸ suggested a considerable difference in the modeling and relative importance for EQ-5D-5L dimensions among the DCE-based preference studies in 11 Asian studies. Wang et al.¹⁹ concluded that the TTO-based utility values for seven Asian countries differed. Similarly, a study conducted by Olsen et al.¹⁴ suggests some similarities in EQ-5D-5L value sets across four of seven western countries (i.e. Canada, England, the Netherlands, and Spain). However, these studies only compared the characteristics of the value sets. To the best of our knowledge, methodological similarities and variations across all value set studies and the impact of such variation on the EQ-5D-5L value set has never been explored. A systematic comparison of EQ-5D-5L valuation methods, their value set, and the impact of methodological elements on the value sets can provide a better understanding of the common practice of valuation studies and promote informed comparison and use of health state values across countries. Such comparison would provide evidence for and against using value sets from another country if its own value set is not available. Additionally, the choice of valuation method may systematically influence economic analysis (i.e. cost-utility analysis), leading to erroneous decisions^14,20. Thus, the objectives of this study were (1) to descriptively compare the selected elements (i.e. general study characteristics, preference elicitation technique, value set modeling) of valuation methods for EQ-5D-5L value set studies, (2) to compare the characteristics of value sets based on the relative importance of the dimensions, relative utility decrement across levels and distribution of the utility scores, and (3) to examine the associations between the selected elements of valuation methods and the EQ-5D-5L value sets.

Methods

This systematic review followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines²¹.

Inclusion and exclusion criteria

This review included country-specific health state valuation studies that collected data from general populations using EQ-5D-5L to generate societal value sets until 22 April 2021, and EQ-5D-5L value studies published during the early versions of this manuscript. Exclusion criteria included studies that (1) generated EQ-5D-5L value sets by mapping EQ-5D-3L to EQ-5D-5L, (2) did not report any country-specific value sets, (3) assessed the feasibility of conducting health state valuation studies, (4) made comparisons across different methods, (5) compared the health state values derived from general and patient populations, (6) used primary data from valuation studies for secondary analyses, or (7) were not published in the English language.

Search strategy

Six bibliographic databases, including PUBMED, PsycINFO, CINAHL, EconLit, Cochrane Library, and Google Scholar, were searched from 1 January 2009 to 22 April 2021. Subject headings (MESH terms) and free texts (keywords) were used to reflect differences in indexing among databases. Key search terms included “EQ-5D-5L” alongside “valuation”, “value set”, or “tariff”. Targeted searches on individual websites were conducted to assess the grey literature. A hand search for references was performed to identify relevant articles. A detailed search strategy is presented in Supplement 1.

Study selection and data extraction

Two independent reviewers (NP and SMF) screened for titles and abstracts. Full-text articles were then critically reviewed by two independent researchers (NP and SMF), and a consensus was reached for rejection and retention decisions. Discrepancies between the reviewers were resolved by a consultation with the third reviewer (SN). Data extraction of the included studies was initially performed by one reviewer (NP).

A Microsoft Excel spreadsheet was developed to include the extracted data and was verified by the second reviewer (SMF). The Excel spreadsheet contained information regarding (1) general study characteristics – (a) sample: sampling method, number of responses analyzed, and (b) data collection: training for the interviewer, mode of data collection, EuroQol Valuation Technique (EQ-VT) used, QC procedure, hypothetical state (HS) versus own health state/experience (EB) approach used; (2) preference elicitation technique – (a) design: types of TTO, types of DCE, types of visual analog scales (VAS), number of valuation choices per respondent in each design, and (b) modeling approach: model used to derive value set; and (3) value set modeling – final model used to derive value set. Also, the reported value set and standard error for each country were extracted.

Quality assessment of included studies

All retained articles were assessed for methodological rigor using the Checklist for REporting VAluaTion StudiEs (CREATE) guideline²². The CREATE guideline consists of 21 items: descriptive system (two items), health states valued (three items), sampling (five items), preference data collection (two items), study sample (two items), modeling (five items), and value set (two items). Two reviewers (NP and SMF) independently evaluated the retained articles based on whether each item was present, and a consensus was reached. Any discrepancy between the two reviewers was resolved through discussion with the third reviewer (SN).

Comparison of EQ-5D-5L value sets studies

The similarities and variations of these selected methodological elements (i.e. sample, preference elicitation technique, value set modeling) of valuation methods were descriptively compared across studies. Given the difference in the valuation methods across countries, value sets derived from these methods were also compared. Value sets were compared based on the utility value for 3,125 health states derived from the scoring algorithm for each individual country or region. The relative importance of dimensions, utility decrements between the levels, and distribution of the utility scores were determined. The relative importance of dimension was estimated as the magnitude of utility decrement for each dimension. For example, the magnitude of utility decrement for MO was estimated as the difference between the first (11111) and fifth level (51111). This difference was then ranked from highest (most important) to lowest (least important) to derive the relative importance of dimension. Utility decrements between levels were estimated using a similar approach but for each level. The difference was then graphically presented, and the similarities and differences in utility change across countries were examined. Kernel density plots were constructed for the predicted utility of 3,125 health states which were then visually compared for distribution similarities and differences. Utility estimates were grouped across four continents, including Asia, Europe, America (North and South), and Africa, as countries within a similar geographical location, language, and cultural context, were expected to have some similarities in preferences.

Meta-regression analysis

To examine the effect of methodological elements or study characteristics on the mean of predicted utility scores and value set coefficients for the levels of EQ-5D-5L dimensions, we used meta-regression analysis^23,24. Potential sources of heterogeneity in observed utility values and value set coefficients were explored by using Cochran’s Q test and I² statistic in meta-regression analyses. Only valuation studies reporting both mean values and standard errors for value sets were included. The tested variables included sampling technique, sample size, the version of the standardized valuation protocol, QC procedure, and the final model used to derive the value set. Separately, each variable was regressed using a random-effects model to the mean of predicted utility scores and coefficients of the level of EQ-5D-5L dimensions. The models were fitted using R (4.0.3) packages “meta” with Restricted Maximum Likelihood Estimation (REML) and the Knapp-Hartung method. A two-sided p-value of <.05 was considered statistically significant.

Results

Retained studies

The PRISMA flow diagram (Figure 1) depicts the literature search/review tiers and rejection reasons. The initial search resulted in 963 studies. One additional study was identified during the reference search; however, it was removed at the title and abstract screening step since no full text was available in the English language²⁵. This final version of the manuscript contained two additional articles published following the first version of the manuscript^26,27. After screening for titles and abstracts, 50 articles were selected for the full-text review. Subsequently, 19 articles were removed based on the exclusion criteria. A total of 31 articles that determined the value sets using EQ-5D-5L were incorporated in the final version of the manuscript^{9,15,16,26–53}. Several other studies were in progress during our study period.

Figure 1. PRISMA flow diagram of study selection process. Abbreviations. cTTO, Composite Time Trade-Off; DCE, Discrete Choice Experiment; PAPRIKA, Potentially All Pairwise RanKings of all possible Alternatives; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; tTTO, Traditional Time Trade-Off; VAS, Visual Analog Scale.

Retained studies were from 29 countries or regions, including Canada⁵³, China⁴², Denmark³⁷, Egypt²⁸, England³³, Ethiopia⁵¹, France⁹, Germany^39,41, Hong Kong⁵², Hungary⁴⁸, Indonesia⁴⁶, Ireland³⁶, Japan¹⁵, Taiwan⁴⁰, Thailand⁴⁴, the Netherlands¹⁶, Malaysia⁴⁹, Mexico²⁷, New Zealand⁵⁰, Peru²⁹, Poland³⁵, Portugal³⁴, Korea³⁸, Spain⁴⁷, Sweden³¹, Uganda²⁶, Uruguay³⁰, the US^32,45, and Vietnam⁴³. The majority of these studies used EQ-VT (n = 27) to derive the value set, however, four studies^31,32,39,50 used other methods. Studies were published in the PharmacoEconomics (n = 9), Value in Health (n = 8), or Other (n = 14) journals. The majority of the studies (n = 26) were cofounded by the EuroQol.

Quality of the retained studies

A total of 26 studies^{9,16,26–31,33,35–38,40–49,51–53} reported 17 (80%) or more of the CREATE items. The remaining five studies^{15,32,34,39,50} reported 13–16 items (60%). The least frequently reported items were the response rate and model specification (details in Supplement 2).

EQ-5D-5L valuation methods across studies using EQ-VT

Table 1 summarizes the selected methodological elements of the included studies.

Table 1. Evidence table summarizing the methodological elements of EQ-5D-5L for the retained studies (n = 31).

Country	Publication year	General study characteristics						Preference elicitation technique and modeling						Value set
		Sample		Data collection				TTO			DCE
		Sampling	Analyzed^a	Training	Mode	Protocol version	QC^b	Design	C/R	Model^c	Design	C/R	Model^c	Final model
EQ-VT Version 2
Uganda^26	2022	Q	492	Y	CAPI	EQ-VT Lite	Y	cTTO	20	Tobit	—	—	—	Tobit
Denmark^37	2021	—	1,014	Y	CAPI	EQ-VT 2.1	Y	cTTO	10	Tobit	DCE	7	CLM	Hybrid
Egypt^28	2021	Q	974	Y	CAPI	EQ-VT 2.1	Y	cTTO	10	Heteroskedastic	DCE	7	CLM	Heteroskedastic
Mexico^27	2021	St	1,000	Y	CAPI	EQ-VT 2.0	Y	cTTO	10	Heteroskedastic	—	—	—	Heteroskedastic
France^9	2020	Q	1,048	Y	CAPI	EQ-VT 2.0	Y	cTTO	10	Tobit	DCE	7	CLM	Hybrid
Hungary^48	2020	Q	1,000	—	CAPI	EQ-VT 2.1	Y	cTTO	10	Tobit	DCE	7	—	Tobit
Peru^29	2020	St	1,000	Y	CAPI	EQ-VT 2.1	Y	cTTO	11	Tobit	DCE	34	ZBT	Tobit
Vietnam^43	2020	St	1,200	Y	CAPI	EQ-VT 2.1	Y	cTTO	10	Tobit	DCE	7	LM	Hybrid
Ethiopia^51	2019	St	1,048	Y	CAPI	EQ-PVT 2.1	Y	cTTO	10	OLS	DCE	7	CLM	Hybrid
Poland^35	2019	Q	1,252	Y	CAPI	EQ-VT 2.0	Y	cTTO	10	—	DCE	7	—	Hybrid
Portugal^34	2019	St	1,451	Y	CAPI	EQ-VT 2.0	Y	cTTO	10	Tobit	DCE	7	CLM	Hybrid
US^45	2019	Q	1,102	Y	CAPI	EQ-VT 2.0	Y	cTTO	10	Tobit	DCE	7	MLM	Tobit
Germany^41	2018	Q	1,158	Y	CAPI	EQ-VT 2.0	Y	cTTO	10	Tobit	DCE	13	CLM	Hybrid
Ireland^36	2018	St	1,160	Y	CAPI	EQ-VT 2.0	Y	cTTO	10	—	DCE	7	—	Hybrid
Malaysia^49	2018	St	1,125	Y	CAPI	EQ-VT 2.0	Y	cTTO	10	Mixed effect	DCE	7	CLM	Hybrid
Taiwan^40	2018	St	1,000	Y	CAPI	EQ-VT 2.0	Y	cTTO	10	Tobit	DCE	7	CLM	Hybrid
Hong Kong^52	2017	—	1,014	Y	CAPI	EQ-VT 2.0	Y	cTTO	10	Tobit	DCE	7	CLM	Hybrid
Indonesia^46	2017	St	1,054	Y	CAPI	EQ-VT 2.0	Y	cTTO	10	Tobit	DCE	7	CLM	Hybrid
EQ-VT Version 1
Spain^47	2018	St	973	Y	CAPI	EQ-VT 1.0	Re	cTTO	10	Linear	DCE	7	CLM	Hybrid
Thailand^44	2018	St	1,205	Y	CAPI	EQ-VT 1.1	Y	cTTO	10	Multilevel	DCE	7	CLM	Hybrid
China^42	2017	Q	1,271	Y	CAPI	EQ-VT 1.0	—	cTTO	10	Nonlinear	DCE	—	—	Non-linear
England^33	2017	R	912	Y	CAPI	EQ-VT 1.0	Re	cTTO	10	—	DCE	7	—	Hybrid
Canada^53	2016	Q	1,073	–	CAPI	EQ-VT 1.0	—	cTTO + tTTO	10	Linear	—	—	—	Linear
Japan^15	2016	St	1,026	Y	CAPI	EQ-VT 1.1	Y	cTTO	10	Linear mixed	DCE	7	CLM	Linear mixed
Korea^38	2016	St	1,080	Y	CAPI	EQ-VT 1.1	Y	cTTO	10	Linear mixed	DCE	7	CLM	Linear mixed
Netherlands^16	2016	St	992	Y	CAPI	EQ-VT 1.0	Re	cTTO	10	Tobit	DCE	7	CLM	Tobit
Uruguay^30	2016	St	794	Y	CAPI	EQ-VT 1.1	Y	cTTO	10	Robust	DCE	7	—	Robust
Others
New Zealand^50	2020	Re	2,468	—	SAOS	—	Y	—	—	—	DCE ( PAPRIKA)	—	—	—
Sweden^31^d	2020	R	25,867	—	SAPS	—	—	TTO	1	OLS	—	—	—	OLS
US^32	2018	Q	8,222	—	SAOS	—	—	—	—	—	DCE	20	ZBT	ZBT
Germany^39^d	2017	R	8,114	—	SAPS	N	—	—	—	—	—	—	—	GLM

Abbreviations. –, not used/unspecified/unclear; CAPI, face-to-face computer-assisted personal interviewing; CLM, conditional logit model; C/R represents choice/respondent; cTTO, composite time trade-off; DCE, discrete choice experiment; EB, experience based; EQ-PVT, EuroQol portable valuation technology; EQ-VT, EuroQol valuation technology; GLM, generalized linear model; HS, hypothetical state; LM, logit model; MLM, mixed logit model; OLS, ordinary least square; Q, quota sampling; QC, quality control; R, random sampling; Re, market research companies; Rp, representative; SAOS, self-administered online survey; SAPS, self-administered paper/postal survey; St, stratified sampling; tTTO, traditional time trade-off; VAS, visual analog scale; Y, yes; ZBT, zermelo-bradley-terry.

^aThe sample represents the final sample size analyzed.

^bQC represents the use of quality control procedure recommended by the EuroQol.

^cModel indicates the final model used to analyze the TTO/DCE data.

^dVisual analog scale (VAS) was used to derived value set based on own health experience.

Sampling technique and sample size

Most EQ-VT studies used a stratified sampling method (n = 15) – multistage stratified, stratified quota, stratified cluster, or stratified random sampling. However, other studies used diverse sampling techniques, such as quota sampling, random sampling, and unspecified. Sample sizes used to derive final value sets differed across studies. The majority of the studies (n = 21) had a sample size of at least 1,000 participants or higher. However, six studies had less than 1,000 participants, most (n = 4) of them used EQ-VT v 1, i.e. EQ-VT v 1.0 or EuroQol Valuation Technique version 1.1 (EQ-VT v 1.1).

Data collection

All EQ-VT studies (n = 27) used the computer-assisted personal interviewing (CAPI) method to collect data. In almost all these studies (n = 25), interviewers were trained. However, variations in the protocol version were observed: EQ-VT v 1.0 (n = 5), EQ-VT v 1.1 (n = 4), EQ-VT v 2.0 (n = 11), EQ-VT v 2.1 (n = 5), portable EQ-VT 2.1 (n = 1), and EQ-VT Lite (n = 1). The majority (n = 22) of these studies used EuroQol Group’s QC procedure to assess the quality of the data. The QC procedures in five studies^{16,33,42,47,53} were not specified or used market research companies; they all used the EQ-VT v 1.0.

Preference elicitation

All EQ-VT studies (n = 27) used a stated preference-based approach that asked respondents to value HS. Most of the studies (n = 25) used both cTTO and DCE to elicit preferences. One study used traditional TTO (tTTO) along with cTTO⁵³, while the other used a lite version of the protocol with cTTO²⁶. For the cTTO task, primarily, a total of 86 health states – 80 health states generated through simulation, five mild (only one dimension at level 2 and all others at level 1, e.g. “11112”), and one pit state (55555) – were divided into 10 blocks. Each block included 10 unique health states – eight random states, one mild state, and one pit state – and was presented to the respondents during the interview. The Lite version of the cTTO task consists of 91 health states with 20 health states per block²⁶. For the DCE task, most of the studies (n = 24) generated 196 health states and divided them into 28 blocks, resulting in seven pairs in each block. Each participant completed one block during the interview. No opt-out alternative was included in the DCE. One of the studies that used the EQ-VT v 2.0 added six choice tasks, including duration and a death alternative⁴¹, while participants in another study completed a DCE task with 10 latent-scale pairs and 12 matched pairs (34 choice tasks in total)²⁹. For one study that used tTTO along with cTTO, the number of tTTO tasks per respondent was unclear⁵³.

Of the 27 EQ-VT studies that used TTO as a preference elicitation technique, 13 studies used Tobit regression to develop their preference models^{9,16,26,29,34,37,40,41,43,45,46,48,52}. The other 14 studies used various regression techniques, including multilevel, linear, nonlinear, robust, linear mixed, heteroskedastic, ordinary least square, and mixed effect models. Among the 24 studies using DCE as a preference elicitation technique, 15 used a conditional logit model (CLM). Of the remaining nine studies, three studies used either the logit model (LM) ⁴³, mixed logit model (ML)⁴⁵, or Zermelo-Bradley-Terry (ZBT)²⁹ model to analyze DCE data, whereas six studies did not specify any model type.

Value set modeling

The hybrid regression model (n = 15) was most frequently used to generate final value sets^{9,33–37,40,41,43,44,46,47,49,51,52}. The other 12 studies used various modeling approaches, including non-linear, robust regression, linear mixed, Tobit, linear, and heteroscedastic models. Two studies also included interaction terms in their final value set models^38,53.

EQ-5D-5L valuation methods across studies that did not use EQ-VT

Sampling technique and sample size

Four studies did not use EQ-VT^31,32,39,50. The sample size for these studies ranges between 2,468 and 25,867.

Data collection

Data were collected using a self-administered online survey or paper/postal survey. Of four studies, one study described the QC procedure and used self-developed procedures, such as removing responses from participants who provided inconsistent responses to validity questions, etc.⁵⁰.

Preference elicitation

Two studies^32,50 used a stated preference-based approach while the other two studies^31,39 used revealed preference or participants EB to generate the final value sets. DCE or DCE with PAPRIKA only^32,50 was used to elicit preference from HS, whereas TTO and VAS³¹, or VAS only³⁹, were used to elicit preference from EB. For two studies that used DCE, one study elicited preference using 20 choice tasks and no opt-out option for each participant³², while another study used an adaptive DCE, and the number of choice tasks differed across participants⁵⁰. For a study that used TTO with VAS to elicit experience-based preference, one task each was presented to respondents³¹. The number of choice tasks for a VAS-based study remains unclear³⁹.

One study used ordinary least squares to model TTO based preference data³¹, whereas a DCE-based study used ZBT³². The specific regression techniques used to model TTO and DCE data in three studies were not specified. Two studies used VAS as a preference elicitation method and used a generalized linear model (GLM)³⁹ or ordinary least square (OLS)³¹ model to analyze preference data.

The value set modeling

OLS, ZBT, and GLM were used to generate final value sets among three studies. The modeling approach for the final value set in one study was not specified⁵⁰. Three studies included interaction terms in their final value set^31,32,39.

EQ-5D-5L value sets across countries (EQ-VT and non-EQ-VT)

The Asian continent or regions consisted of China, Hong Kong, Indonesia, Japan, Korea, Malaysia, Taiwan, Thailand, and Vietnam. The European continent included 11 countries, i.e. Denmark, England, France, Germany, Hungary, Ireland, Netherlands, Poland, Portugal, Spain, and Sweden. Canada, Mexico, Peru, Uruguay, and the US were grouped as American continents. New Zealand was also included in this group since it has geographical proximity to America, uses English as an official language, and is a developed country. The African continent included Egypt, Ethiopia, and Uganda. Among the retained studies, studies from Germany and the US had two value sets for their general populations, based on HS versus EB. EB-based studies for Germany and the US were published a year earlier than HS-based value sets. These studies were primarily conducted to elicit preference from the patients population to evaluate their own health³⁹ or improve the valuation method³². Since the VAS-derived value set of Sweden was not rescaled to 0 and 1, it was excluded from the comparison of the value sets across countries.

The relative importance of the dimensions

Table 2 summarizes the relative importance of the dimension across retained studies. MO was ranked as the most important dimension in all nine countries in Asian continents, followed by PD in six countries. SC and UA were the least important in eight out of nine countries. Among 12 studies from 11 countries in European continents, PD was ranked the most important dimension in seven studies and the second most important dimension in five studies. UA was the least important in nine studies. Among seven studies from five countries in the American continent, PD was the most important dimension in five studies, whereas UA was the least important in five studies. For three countries in the African continent, the relative importance of health state dimensions varied across countries.

Table 2. Summary characteristics of the value sets based on utility score (n = 31).

Country	Utility score^a				Dimension scores^b					Relative importance^c
	Mean	SD	Min	Max	MO	SC	UA	PD	AD
Asia
China^42	0.310	0.236	–0.391	1.000	0.345	0.253	0.233	0.302	0.258	MO > PD > AD > SC > UA
Hong Kong^52	0.113	0.314	–0.865	1.000	0.529	0.352	0.282	0.354	0.348	MO > PD > SC > AD > UA
Indonesia^46	0.112	0.314	–0.865	1.000	0.613	0.316	0.385	0.246	0.305	MO > UA > SC > AD > PD
Japan^15	0.449	0.155	–0.025	1.000	0.304	0.221	0.236	0.252	0.257	MO > AD > PD > UA > SC
Korea^38	0.451	0.159	–0.066	1.000	0.425	0.296	0.349	0.381	0.311	MO > PD > UA > AD > SC
Malaysia^49	0.331	0.239	–0.442	1.000	0.340	0.261	0.202	0.338	0.300	MO > PD > AD > SC > UA
Taiwan^40	0.036	0.338	–1.026	1.000	0.477	0.324	0.351	0.453	0.421	MO > PD > AD > UA > SC
Thailand^44	0.379	0.234	–0.421	1.000	0.371	0.250	0.248	0.256	0.295	MO > AD > PD > SC > UA
Vietnam^43	0.353	0.246	–0.512	1.000	0.376	0.231	0.299	0.367	0.239	MO > PD > UA > AD > SC
Europe
Denmark^37	0.267	0.338	–0.758	1.000	0.220	0.209	0.174	0.537	0.618	AD > PD > MO > SC > UA
England^33	0.383	0.226	–0.285	1.000	0.274	0.203	0.184	0.335	0.289	PD > AD > MO > SC > UA
France^9	0.424	0.269	–0.525	1.000	0.325	0.258	0.240	0.444	0.258	PD > MO > SC > AD > UA
Germany^41	0.343	0.308	–0.661	1.000	0.224	0.260	0.209	0.612	0.356	PD > AD > SC > MO > UA
Germany^39^d	0.448	0.149	0.104	0.920	0.143	0.190	0.208	0.361	0.214	PD > AD > UA > SC > MO
Hungary^48	0.251	0.318	–0.848	1.000	0.455	0.366	0.276	0.411	0.340	MO > PD > SC > AD > UA
Ireland^36	0.131	0.367	–0.974	1.000	0.344	0.287	0.187	0.510	0.646	AD > PD > MO > SC > UA
Netherlands^16	0.287	0.268	–0.446	0.953	0.203	0.168	0.192	0.415	0.421	AD > PD > MO > UA > SC
Poland^35	0.476	0.286	–0.590	1.000	0.314	0.264	0.205	0.575	0.232	PD > MO > SC > AD > UA
Portugal^34	0.366	0.263	–0.603	1.000	0.356	0.294	0.263	0.406	0.284	PD > MO > SC > AD > UA
Spain^47	0.351	0.241	–0.415	1.000	0.337	0.196	0.153	0.381	0.348	PD > AD > MO > SC > UA
Sweden^31^d	0.605	0.140	0.243	0.976	0.055	0.085	0.166	0.200	0.236	AD > PD > UA > SC > MO
America
Canada^53	0.399	0.202	–0.148	0.949	0.207	0.242	0.188	0.319	0.278	PD > AD > SC > MO > UA
Mexico^27	0.374	0.264	–0.596	1.000	0.270	0.259	0.276	0.458	0.334	PD > AD > UA > MO > SC
New Zealand^50^d	0.198	0.292	–0.83	1.000	0.350	0.370	0.340	0.381	0.389	AD > PD > SC > MO > UA
Peru^29	0.047	0.321	–1.073	1.000	0.473	0.355	0.347	0.476	0.422	PD > MO > AD > SC > UA
Uruguay^30	0.540	0.209	–0.264	1.000	0.311	0.286	0.244	0.283	0.190	MO > SC > PD > UA > AD
US^45	0.235	0.272	–0.573	1.000	0.322	0.261	0.255	0.414	0.321	PD > MO > AD > SC > UA
US^32^d	0.501	0.238	–0.287	1.000	0.250	0.262	0.211	0.324	0.240	PD > SC > MO > AD > UA
Africa
Egypt^28	0.151	0.359	–0.978	1.000	0.602	0.285	0.213	0.443	0.419	MO > PD > AD > SC > UA
Ethiopia^51	0.349	0.303	–0.718	1.000	0.360	0.222	0.272	0.406	0.458	AD > PD > MO > UA > SC
Uganda^26	0.081	0.395	–1.116	1.000	0.376	0.354	0.306	0.798	0.282	PD > MO > SC > UA > AD

Abbreviations. AD, Anxiety/Depression; MO, Mobility; PD, Pain/Discomfort; SC, Self-Care; UA, Usual Activities.

^aMean and standard deviation of utility score are for the 3,125 health states. Min = minimum utility value, Max = Maximum utility value.

^bDimension scores were calculated as the magnitude of relative utility decrements for a dimension. For example, in the case of Thailand, the dimension score for AD was estimated as the difference between the utility for full health (11111) = 1 subtracted by the utility level for the fifth level of AD (11115) = 0.7047 (i.e. 1 ‒ 0.7047 = 0.295).

^cRelative importance was estimated from the ordering of the magnitude of the dimension scores. For example, in the case of Thailand, MO (0.371) > AD (0.295) > PD (0.256) > SC (0.250) > UA (0.248).

^dUtility scores were derived from the experience-based value set.

Relative utility decrement across the level

The utility decrement reflects the general population preference for severity across health state levels from one level to another. Supplement 3(A) shows the trend in the utility decrement in level for countries across different continents. In general, some similarities were observed in the decreases in health state utility levels from full health (11111) to worst states in each dimension, e.g. MO (51111), SC (15111), UA (11511), PD (11151), AD (11115). The utility decrements were greater for higher levels across dimensions, e.g. from level 3 to level 5 compared to level 1 to level 3.

Also, variations in the utility decrements were observed across countries within the same continent. The utility decrements between level 1 to level 5 were similar across the Asian continent, except China, Japan, and Korea. Compared to other countries, the utility decrement from level 2 to level 3 across each dimension was relatively higher for China, while the utility decrement from level 3 to level 5 was relatively lower for Japan and the utility decrement from level 1 to level 3 was relatively higher for Korea. The utility decrements for the countries in European continent, especially Sweden, Germany, and Hungary, tended to differ. Compared to other countries in the European continent, the utility decrements from level 3 to level 5 for MO and PD dimensions were relatively lower for Sweden and Germany that derived EB value set. The highest utility decrement for Hungary was found in the MO dimension from level 3 to level 5. For the SC dimension, the utility decrement from level 3 to level 5 was relatively lower for Sweden but was higher for Hungary. For countries in the American and African continent, the utility decrements across levels and dimensions varied, with noticeably larger utility decrements for Peru moving from level 1 to level 5 across all dimensions. Uganda showed a relatively larger utility decrement for PD moving from level 3 to 5.

Distribution of the utility scores

Supplement 3(B) presents Kernel density plots of the utility scores across countries and continents. Three distinct preference patterns in Asia emerged. Japan and Korea seemed to have similar utility distributions. Similarly, Indonesia, Hong Kong, and Taiwan had similar preference distribution curves, as did Thailand, Malaysia, China, and Vietnam. The utility curves for Japan and Korea were pointed, and the mean utility score was higher, suggesting better health states among a larger number of individuals. On the other hand, the utility curves for Taiwan, Indonesia, and Hong Kong were flatter, with a large percentage of the respondents with the utilities of health states less than 0. The utility distribution curves for Thailand, Malaysia, China, and Vietnam were between the two groups. In the European continent, except for Germany, Sweden, and Ireland, the distributions of utility scores were similar. For Germany and Sweden that elicited EB value sets, the utility distributions for health states were different. They were steeper and had higher mean utility scores. The kernel distribution plots for Ireland and Hungary were flat, while the distribution curves for France and Poland were not smooth. The utility distributions for England and Spain seemed to be similar. In the American continent, the utility distributions of Mexico, New Zealand, Peru, and the US were similar. Still, they differed from the DCE-based utility distribution for the US and the utility distributions for Canada and Uruguay. Also, the distribution curves for Uruguay and Canada were pointed, with higher utility means. The utility distributions for Ethiopia and Egypt were similar, with Uganda having a slightly flatter distribution curve.

Factors associated with utility scores

Table 3 shows the meta-regression results. Only studies that used EQ-VT were included in the meta-regression analysis to minimize the amount of between-study heterogeneity, i.e. heterogeneity between studies eliciting HS versus EB. A total of 17 studies were included in the meta-regression analysis^{16,27,28,30,33–37,41–45,48,49,52}. Compared to protocol version 1, the standardized protocol version 2 tended to have a lower mean of predicted utility scores of 3,125 health states (β = –0.099, p = .067, I² = 0%) but was not statistically significant at the alpha level of .05.

Table 3. Results for meta-regression analysis (n = 17).

Variable	Sampling technique (stratified)^b		Sample size (1,000 or more)^b		Protocol version (EQ-VT 2)^b		QC procedure (Yes)^b		Final model (hybrid)^b
	Estimates	p-value	Estimates	p-value	Estimates	p-value	Estimates	p-value	Estimates	p-value
Mean utility^a	0.079	0.141	–0.084	0.168	–0.099	0.067*	–0.006	0.928	–0.013	0.819
MO2	–0.006	0.656	0.011	0.521	0.005	0.759	–0.001	0.952	0.012	0.397
MO3	–0.033	0.218	–0.006	0.844	0.014	0.650	–0.007	0.855	–0.020	0.479
MO4	–0.050	0.203	–0.002	0.968	0.030	0.491	–0.003	0.952	–0.027	0.513
MO5	–0.043	0.410	0.001	0.991	0.063	0.273	0.083	0.220	–0.021	0.701
SC2	–0.002	0.820	0.008	0.436	0.005	0.602	0.004	0.682	0.002	0.813
SC3	–0.012	0.297	–0.001	0.949	–0.004	0.757	–0.015	0.349	–0.021	0.063*
SC4	–0.023	0.322	0.014	0.612	0.019	0.460	0.004	0.886	–0.018	0.432
SC5	–0.017	0.469	0.042	0.131	0.046	0.067*	0.065	0.027**	–0.004	0.860
UA2	0.008	0.149	0.000	0.959	–0.005	0.474	0.000	0.964	–0.002	0.715
UA3	–0.001	0.950	0.005	0.737	–0.008	0.486	–0.019	0.171	–0.029	0.004**
UA4	–0.027	0.241	–0.010	0.706	0.011	0.678	–0.013	0.670	–0.048	0.026**
UA5	0.014	0.477	0.036	0.119	0.018	0.406	0.034	0.191	–0.014	0.485
PD2	0.007	0.418	0.008	0.467	0.003	0.739	–0.007	0.564	0.010	0.255
PD3	–0.004	0.772	0.015	0.412	0.011	0.495	–0.011	0.581	0.006	0.680
PD4	–0.040	0.189	0.018	0.631	0.045	0.179	–0.010	0.819	0.027	0.396
PD5	–0.065	0.203	0.062	0.305	0.137	0.008**	0.074	0.269	0.042	0.420
AD2	0.005	0.625	0.000	0.994	–0.001	0.962	–0.013	0.384	0.010	0.353
AD3	–0.010	0.675	–0.010	0.742	0.016	0.552	–0.011	0.744	0.000	0.995
AD4	–0.024	0.666	–0.010	0.883	0.034	0.576	–0.027	0.714	0.027	0.640
AD5	–0.016	0.805	0.027	0.721	0.081	0.234	0.028	0.740	0.037	0.561

Significance level:** 0.05, * 0.1.

Abbreviations. AD2, slight problems for the dimension Anxiety/Depression; AD3, moderate problems for the dimension Anxiety/Depression; AD4, severe problems for the dimension Anxiety/Depression; AD5, Unable/extreme problems for the dimension Anxiety/Depression; MO2, slight problems for the dimension Mobility; MO3, moderate problems for the dimension Mobility; MO4, severe problems for the dimension Mobility; MO5, Unable/extreme problems for the dimension Mobility; PD2, slight problems for the dimension Pain/Discomfort; PD3, moderate problems for the dimension Pain/Discomfort; PD4, severe problems for the dimension Pain/Discomfort; PD5, Unable/extreme problems for the dimension Pain/Discomfort; SC2, slight problems for the dimension Self-Care; SC3, moderate problems for the dimension Self-Care; SC4, severe problems for the dimension Self-Care; SC5, Unable/extreme problems for the dimension Self-Care; UA2, slight problems for the dimension Usual Activities; UA3, moderate problems for the dimension Usual Activities; UA4, severe problems for the dimension Usual Activities; UA5, Unable/extreme problems for the dimension Usual Activities.

^aUtility value for each of the 3,125 health states was derived from the scoring algorithm of each country. The mean and standard error of the utility value was then constructed.

^bReference for the variable included in the meta-regression analysis consists of: Sampling Technique (Other); Sample Size (less than 1,000); Protocol Version (EQ-VT 1); QC procedure (No); Final Model (Others).

Additionally, the protocol version, QC procedure, and final value set model were significantly associated with some levels across different dimensions, including SC5, UA3, UA4, and PD5. Compared to the studies using protocol version 1, the studies using protocol version 2 tended to have a higher value set coefficient for PD5 (β = 0.137, p < .01, I² = 99.96%). The studies using the EuroQol QC procedure tended to have a higher value set coefficient for SC5 (β = 0.065, p < .05, I² = 99.86%) than those that did not use this procedure. The studies using the hybrid model tended to have a lower value set coefficient for SC3 (β = –0.021, p < .063, I² = 90.23%), UA3 (β = –0.029, p < .01, I² = 85.87%), and UA4 (β = –0.048, p < .05, I² = 96.89%), as compared to those using other methods, however, SC3 was not statistically significant at the alpha level of 0.05. Except for mean utility, I² values were greater than 90% and had significant value for Cochran’s Q test (p < .001). However, the sampling technique and the sample size were not associated with the utility scores. Details for meta-regression analysis are presented in Supplement 4.

Discussion

This systematic review compared a group of selected elements of EQ-5D-5L valuation methods (i.e. general study characteristics, preference elicitation technique, the value set modeling) and value sets among value set studies from various countries. The impacts of the methodological elements on the derived value sets of EQ-5D-5L were also examined using meta-regression analyses. Based on the CREATE criteria, the methodological quality of these studies was of relatively high quality (met at least 13 of 21 criteria).

The retained studies frequently used the EQ-VT. The primary reason was the experiences accumulated during the EQ-5D-3L valuation led to the realization that a standardized approach for the EQ-5D-5L valuation could ensure data quality¹³. Improvement in the data quality would subsequently lead to improved quality of economic evaluation for health technology assessment, promote the comparison of the value sets across countries, and explore the influences of cultural and other factors on health state values. The similarities observed across the retained studies using the EQ-VT centered around data collection and preference elicitation methods. One explanation might be that these methodological components were less likely to depend on countries contexts after the studies chose to follow a protocol. However, it should be noted that some methodological variations might be inevitable given the different local contexts or different EQ-VT versions^14,24,54. However, such variations tend to decrease over time. One of the reasons might be that a feedback module was added to the EQ-VT v 2 (EQ-VT v 2.0 or EQ-VT v 2.1) to control data quality. Studies using this version might provide more robust data and fewer inconsistent responses. In other cases, avoidable variations could impact value sets. For example, the training times for interviewers varied from 4 hours⁹ to 3 days⁴⁹. Interviewer bias and framing effect could be higher if the training times for interviewers was shorter. Therefore, those studies with shorter training hours could potentially have more interviewers bias and framing effect than those with more hours of interviewers training, undermining data quality. Previous literature recommended the multiday training workshop with a detailed discussion of the interviewer script and demonstration, pilot study, and feedback opportunities¹².

Almost all retained studies using the EQ-VT used the cTTO with DCE as the preference elicitation method. One study using the EQ-VT v 1.0 used tTTO and cTTO to elicit preferences. A study reported that tTTO had a lower bound of negative infinity and required rescaling¹² and thus might impact the value set. However, this study was among the early wave of valuation studies. It was possible that, during this period, the impact of methodological variation on the value set was not properly established. Two studies using cTTO and DCE also included additional paired comparisons for respondents. The purpose of this addition was not specified^29,41. However, such addition was prone to produce low utility values and could increase the complexity of DCE tasks for participants^55,56.

On the other hand, the sampling method, sample size, and value set modeling varied among the retained studies. One explanation could be that different countries had different resources to determine sampling methods and sample sizes. Another reason might be that an earlier wave of valuation studies, despite using the most recent valuation protocol of the time, was unlikely aware of the impact of the sampling procedure on the value sets. All studies, except one using EQ-VT v 2, included about 1,000 samples. Countries with resources constraints, such as Uganda and Ethiopia, tended to use a portable version of the valuation protocol. On the other hand, the sample sizes of those studies that did not use the EQ-VT were much larger than 1,000. These studies were conducted in Germany, Sweden, New Zealand, and the US. Since these sample sizes were large, these studies used either online or paper-based surveys instead of CAPI. Germany and Sweden used an entirely different valuation approach, with respondents valuing a single state instead of a set of health states, to estimate the EB value set. Although this alternative approach offered certain benefits, it needed a different sampling approach to ensure generalizability or derive the country-specific value set. This might be why a high number of representative samples were selected. The variation of the value set modeling was expected due to the variation of data across the retained studies. Interestingly, the variation tended to occur more within the first wave of valuation studies using EQ-VT v 1, whereas almost all studies using EQ-VT v 2 used a hybrid model. One explanation could be that the hybrid model using both cTTO and DCE data would reduce logical inconsistency. Furthermore, two studies using EQ-VT v 1 included interaction terms in the final value set models. One of the reasons could be that these studies tried to optimize the models⁴⁵. However, previous studies showed the improvement in the model performance with interaction term was marginal^30,33,57,58.

Four other studies did not use the EQ-VT. They intended to derive value sets or develop a cost-effective valuation approach that was cognitively less burdensome to the respondents^31,32,39,50. EB valuation completely deviated from the quality-adjusted life-year (QALY) maximization approach to value the health states respondents currently experienced during the study. Additionally, regulatory context might also drive the choice of valuation method. For example, the QALY-based value set was preferred by the National Institute for Health Care Excellence (NICE) in the UK, whereas the Swedish Dental and Pharmaceutical benefits Agency preferred EB value sets. Also, this review implied that other country contexts, e.g. culture, resources, and need, should be considered when choosing a valuation method. For example, “full health” was not interpreted similarly in Arabic; thus, the cTTO based preference elicitation task might be interpreted differently, requiring an adaptation of the protocol to be used in a certain country, e.g. the United Arab Emirates, a Muslim majority country, which rarely valued any health states that were worse than death⁵⁹.

The comparison of the value sets across countries in this study was based on the relative importance of dimensions, utility decrements between the levels, and distribution of the utility scores. MO was the most important dimension among countries in the Asian continent, followed by the PD, whereas PD was the most important dimension across the majority of countries in the Europe and America continents. UA was the least important dimension across countries. This finding was consistent with previous studies by Yang et al.¹⁸ and Wang et al.¹⁹ for Asian countries and Olsen et al.¹⁴ for western countries. Constraints in the infrastructure, such as transportation systems, might be one of the possible reasons for the higher importance of the mobility dimension among Asian countries⁴⁶. Study findings also suggested that the utility decrements of the higher levels of each dimension tended to be larger across countries. One possible reason might be that the utility of the improvement from a poorer health state was likely valued higher. Additionally, the distribution of the utility scores varied across countries. People from different countries might value health states differently. While a previous study noted that differences in the health utilities were not explained by the national cultural values¹⁷, other studies reported that utility values were associated with cultural differences, e.g. reference⁶⁰. Therefore, a possible reason for variation in the distribution of utility could be due to the variation of valuation methods. One example found in this review was that the value sets derived from the EB method tended to be higher than those estimated using stated choice methods, consistent with the findings in a previous study⁶¹. Another example might be the language used in the translation of EQ-5D-5L, as it was not always possible to find identical terms⁶².

Interestingly, the meta-regression results showed that the EQ-VT v 2 protocol studies tended to have a lower value of mean predicted utility, although it was not statistically significant. Additionally, our findings suggested that differences in the methodological elements (EQ-VT version, QC procedure, and value set modeling) were significantly associated with the higher levels (SC5, UA3, UA4, and PD5) of some dimensions. These observations suggested that some elements of the valuation method could be the potential source of variation in value sets.

Future valuation studies and users of this value can benefit from this review. First, the significant impact of EQ-VT v 2 on the fifth level of SC and PD highlights that any change in methodological procedure (i.e. the introduction of QC procedure) might have an impact on the value sets. Secondly, future studies can review the summary of potentially differentiating factors, such as sample size, sampling methods, data collection procedure, preference elicitation methods, and modeling technique. Third, we also observe that utility for the EB values set had a higher mean utility distribution than those studies that derived utility values from EQ-VT. Among studies that used EQ-VT, utility values obtained with EQ-VT v 1 tended to be higher than the EQ-VT v 2. Future researchers and users of the value set thus should note that the tendency of higher utility due to these specific approaches would impact economic evaluation as such cost-utility analysis and eventually affect decision-making^14,20. Fourth, the variation in the value set across studies in countries with different cultural contexts suggested the need for country-specific value sets for cost-utility analysis.

This systematic review suffered from three major limitations. First, there was a potential selection bias regarding articles included in this review. Primarily, this was due to our inclusion and exclusion criteria. For example, only studies conducted among the general populations were included, resulting in removing any valuation studies focusing on patients. The second limitation was that only specific elements of valuation methods of the retained studies were included in this review. Thus, they might not capture all methodological variations of these studies. Another limitation was that each meta-regression analysis had only one predictor variable due to the limited sample size. In other words, the meta-regression results in this study might not be robust, and further research is needed when more valuation studies are available.

Conclusions

EQ-5D-5L health state valuation methods and characteristics of value sets differed across studies. The use of EQ-VT v 2 tended to improve design similarities to reduce heterogeneity. Methodological elements were unlikely associated with the predicted utility scores and coefficients of EQ-5D-5L dimensions. The impact of variation of methodological elements on the value sets should be further investigated.

Transparency

Declaration of funding

The authors received no financial support for this research.

Declaration of financial/other interests

The authors report there are no competing interests to declare. Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Author contributions

Concept and design: NP, SN. Acquisition of data: NP, SMF. Analysis and interpretations of data: NP, SMF, JQ, GK, NC, SN. Drafting of the manuscript: NP, SN. Critical revision of the paper for important intellectual content: NP, SMF, JQ, GK, NC, SN. Statistical analysis: NP. Provision of study materials or patients: Not applicable. Obtaining funding: Not applicable. Administrative, technical, or logistic support: Not applicable. Supervision: SN. Other: Not applicable.

Acknowledgements

The authors want to thank and acknowledge Ms. Adelia Grabowsky, Health Science Librarian and Associate Professor at Auburn University, for her time and expertise in developing the search strategy used in this systematic review.

Data availability statement

All data generated or analyzed during this study are included in this published article.