A systematic review of SF-6D health state valuation studies

Abstract

Background

The short-form 6-dimension (SF-6D) is a preference-based measure designed to calculate quality-adjusted life-year (QALY). Preference-based measures are standardized multidimensional health state classifications with preference or utility weights elicited from a sample of the population. There is a concern that valuations may differ between countries because of differences in culture, thus invalidating the use of values obtained from one country to another.

Objective

To conduct a systematic review of elicitation methods and modeling strategies in SF-6D studies and to present a general comparison of dimensions’ ordering among different countries.

Methods

We performed a systematic review of studies that developed value sets for the SF-6D. The data search was conducted in PubMed, ScienceDirect, Embase, and Scopus up to 8 September 2022. Quality of studies was assessed with the CREATE checklist. Methodological differences were identified, and the dimensions’ ordering of the selected studies was analyzed by cultural and economic factors.

Results

From a total of 1369 entries, 31 articles were selected. This corresponded to 12 different countries and regions and 17 different surveys. Most studies used the standard gamble method to elicit health state preferences. Anglo-Saxon countries gave more importance to pain, while other countries have physical functioning as the highest dimension. As the economic level increases, people care less about physical functioning but more about pain and mental health.

Conclusions

Value sets for the SF-6D are different from one country to another and there is a need to develop value sets for more countries to consider cultural and economic differences.

Keywords:

• SF-6D
• QALY
• health utilities
• preferences
• systematic review
• value set

JEL CLASSIFICATION CODES:

• I19
• I1
• I
• I31
• I3

1. Introduction

Economic methods offer a way to quantify health outcomes, which are becoming integral to the decision-making process in many countries¹. In this setting, preference-based measures (PBM) of health-related quality of life (HRQoL) can be used to generate health state utility values for the calculation of quality-adjusted life year (QALY)^1,2. Typically, QALY calculation is based on standardized multidimensional health state classification systems that are associated with country-specific preference weights (also called “tariffs” or “value sets”) elicited from a representative sample of the general population¹.

In the literature, various systematic reviews have contributed to the studies of tools designed to measure QALYs through different aspects such as methodology, history, and experimental tasks. Xie et al.³ compared EQ-5D valuation studies and focused on methodological reports. Devlin and Brooks⁴ reviewed the history and development of EQ-5D. Mulhern et al.⁵ conducted a structured review of discrete-choice experiments for health state valuation. In this review, we focused on the SF-6D preference-based measurement and the value sets that were elicited in various countries.

The SF-6D is derived from the Short-Form 36 (SF-36) health survey, which is one of the most widely used HRQoL measures worldwide⁶. A country-specific value set for SF-6D is currently available in some countries and regions. The original health state classification system SF-6Dv1 comprised six dimensions with four to six levels in each, including physical function (PF), role limitation (RL), social functioning (SF), pain (PN), mental health (MH), and vitality (VT).

In 2020, a new version, the SF-6Dv2 was developed, which has the same six dimensions as the SF-6Dv1, but with five to six levels each⁷. In this second version, descriptions changed significantly for all dimensions apart from SF where changes were minimal. As for the SF-6Dv1, the classification system for SF-6Dv2 was derived from the SF-36 and was not restricted to items on both the SF-36 and SF-12⁸. For details see supplemental Appendix 1.

From Europe, America and Asia, there have been an increased number of value sets, where preferences have been elicited for SF-6Dv1 and SF-6Dv2. On one hand, different survey design features were applied to the health state valuation processes, including different SF-6D versions, administration modes, elicitation methods, number of health states, and sample characteristics. Based on the data collected, researchers also implemented a variety of modeling techniques and regression approaches to estimate health state utilities. On the other hand, even with the same standard survey procedure, and similar modeling techniques, there is a concern that valuations may differ between countries because of differences in culture, economy, or other socio-economic factors. As discussed in Kharroubi⁹, “countries with different population compositions, work, cultures, and language, all of which can impact on the relative values given to different dimensions of health.” In this line, the French jurisdiction Haute Autorité de Santé¹⁰ recommends only considering value sets developed in the country in which the economic assessment is to be conducted.

Since the SF-6D is widely used, several country-specific scoring algorithms have been developed, but no general discussion among different countries has been presented. This article aimed to conduct a systematic review of elicitation methods and modeling strategies in SF-6D studies and to present a general comparison of dimensions’ ordering among different countries. We carried out a systematic review of SF-6D studies in the past 25 years and aimed to reach the following objectives: (1) review the current literature relating to the development of SF-6D value sets; (2) provide a detailed summary of the SF-6D methods used to value health states (i.e. similarities and difference in the survey design, the experiment task, and modeling methods); (3) explore the pattern of the value sets between different culture or economy.

2. Methods

2.1. Literature search and criteria

A systematic review was carried out. The search was conducted in PubMed, ScienceDirect, Embase, and Scopus, as well as in the gray literature through the ResearchGate page of John Brazier, inventor of the SF-6D, and the SF-6D webpage hosted by the University of Sheffield. This was done over a period time from the first article published by John Brazier¹¹ up to 8 September 2022. All published studies using the SF-6D (v1 and later v2) to generate value sets were scanned. The search terms were developed by the investigators with the help of a librarian, considering the keywords used to describe the SF-6D and the process of valuation (e.g. six-dimension health state short form, quality-adjusted life-year). A list of key terms and the search strategy are provided in supplemental Appendix 2.

In the first stage, titles and abstracts of articles were screened, and non-relevant articles were excluded. Then, in a second stage, we went through full-text articles for the value sets report. Analyzing the value set was our key target, consequently, articles without a value set were excluded from this systematic review. In this stage, a set of full-text articles were read carefully and assessed for detailed reviews. All articles were deliberately pinned down for the best model that was recommended by the authors. Also, all primary studies were filtered out for the analysis of the survey design. In each stage, the investigators screened articles. Data were extracted by one investigator and checked by another one. In case of disagreement, an arbitration was done by a collaborator.

Articles were included if they used the SF-6D (v1 or v2) to develop or directly inform the production of the value sets for generic or condition-specific preference-based measures. Articles could include any sample (e.g. the general population or patients). Articles were excluded if the value sets were not elicited in the study (e.g. mapping studies). This was because this review aimed to compare value set patterns among different countries. Qualitative studies were also excluded since the focus of the review was to report the econometric methods and models that could be applied for valuation. In several cases, the same datasets were published in multiple articles (e.g. in the original article and later in a comparison article). In this case, the golden rule was still whether new value sets for specific countries were elicited (e.g. using a new modeling method).

2.2. Assessing paper content and quality

Assessment of the quality of the reporting in the articles was guided by the Checklist for Reporting Valuation Studies (CREATE)³. This checklist includes 35 items assessing the key elements that should be reported for health state valuation studies. The valuation generally includes a descriptive system, health states valued, sampling, preference data collection, study sample, modeling, and scoring algorithm. The items were scored either yes or no. As the review focus was on the value sets and scoring algorithms, the included articles were scored by all 35 items (the mean score of the two reviewers is provided).

3. Results

3.1. General study information

The articles’ selection procedure was presented according to the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) flowchart¹² which addresses several conceptual and practical advances in the science of systematic reviews. Figure 1 shows a flowchart of the overall literature identification process. The search identified 1369 records. At the title and abstract screening stage, 1315 were excluded since they were irrelevant or duplicates. After this stage, 54 articles were selected for potential inclusion in the review. Following assessment of the full-text articles, a further 23 were excluded as they were not elicitation studies (i.e. no elicitation, no new value set, value set application, or comparison with other instruments; see the list of studies excluded in phase 2 in supplemental Appendix 3). This resulted in 31 articles being included in the review.

Figure 1. PRISMA flowchart up to 8 September 2022. Notes. *References refer to articles that were found from the reference list of the articles included.

Table 1 reports the characteristics of studies included as well as their quality according to the CREATE checklist. The quality was quoted as good, with a mean score of 27.3 (maximum score of 35). Table 1 lists the 31 articles in chronological order by year of publication. The year of publication demonstrated that this research field has been consistently explored since the pilot project (1998), the first version v1 (2002), and the last version v2 (2020). Throughout the past 25 years, various surveys from different continents have been conducted, including Europe, America, and Asia. Twenty-seven of 31 samples were from the general population, but other populations were represented in four studies: patients, health professionals, and university¹¹; university students^9,33; food-allergic patients or their parents³⁵. The majority of the articles (n = 18) were primary studies, which originally conducted the SF-6D elicitation survey in one country (region). Among these 18 primary studies, there were only 17 surveys because one¹³ used a sub-sample of another one⁶ to develop a new value set. Secondary studies (n = 13) included further analytical work on the primary data and methodological comparisons, thus producing new value sets developed from the original dataset.

Table 1. Main characteristics of SF-6D studies.

Authors	Country/region	Year of survey	Population	No. respondents	Version*	Survey mode	Elicitation method	Death or pits	No. health states	Primary/secondary data
Brazier et al. 1998^11	UK	NA	Health professions, students & patients	165	v0	Self-completion	VAS^a+SG	Death	59	Primary
Brazier et al. 2002^6	UK	NA	General	836	v1	Face-to-face	SG	Pits + death	249	Primary
Brazier and Roberts 2004^13	UK	NA	General	836	v1 SF12	Face-to-face	SG	Pits + death	241	Primary
Kharroubi et al. 2005^14	UK	NA	General	836	v1	Face-to-face	SG	Pits + death	249	Secondary
McCabe et al. 2006^15	UK	NA	General	836	v1	Face-to-face	Rank	Pits + death	249	Secondary
Kharroubi et al. 2007^16	UK	NA	General	836	v1	Face-to-face	SG	Pits + death	249	Secondary
Kharroubi et al. 2007^17	UK	NA	General	836	v1	Face-to-face	SG	Pits + death	249	Secondary
Lam et al. 2008^18	Hong-Kong	2002–2003	General	126	v1	Face-to-face	SG	Death	50	Primary
Brazier et al. 2009^19	Japan	2005	General	600	v1	Face-to-face	SG	Pits + death	241	Primary
Ferreira et al. 2010^20	Portugal	2006	General	140	v1	Face-to-face	SG	Pits + death	55	Primary
Ferreira et al. 2011^21	Portugal	2006	General	140	v1	Face-to-face	Rank	Pits	55	Secondary
Cruz et al. 2011^22	Brazil	2007–2008	General	469	v1	Face-to-face	SG	Pits + death	248	Primary
McGhee et al. 2011^23	Hong-Kong	2004–2006	General	641	v1	Face-to-face	SG	Pits + death	197	Primary
Méndez et al. 2011^24	Spain	2007	General	1020	v1	Face-to-face	Lottery equivalent	Death	78	Primary
Abellán Perpiñán et al. 2012^25	Spain	2007	General	1020 + 60	v1	Face-to-face	LE + SG	Death	78	Secondary
Craig et al. 2013^26	USA	2010	General	666	v1	Online	DCE	Death	NA	Primary
Kharroubi et al. 2013^27	Hong-Kong	2004–2006	General	641	v1	Face-to-face	SG	Pits + death	197	Secondary
Kharroubi et al. 2014^28	HK vs. UK	2004–2005; NA	General	641;836	v1	Face-to-face	SG	Pits + death	197;249	Secondary
Norman et al. 2014^29	Australia	NA	General	1160	v1	Online	DCE	Death	NA	Primary
Craig 2016^30	UK	NA	General	836	v1	Face-to-face	SG	Pits + death	249	Secondary
Kharroubi 2015^31	Japan vs. UK	2005; NA	General	600;836	v1	Face-to-face	SG	Pits + death	241;249	Secondary
Kharroubi 2018^9	HK & UK	2004–2005; NA	General	641;836	v1	Face-to-face	SG	Pits + death	197;249	Secondary
Jonker et al. 2018^32	Netherlands	NA	General	1775	v1 SF-12	Online	DCEtto	None	224	Primary
Kharroubi et al. 2020^33	Lebanon	2019	University	126	v1	Face-to-face	SG	Pits	49	Primary
Kharroubi 2020^34	Lebanon	2019	University	126	v1	Face-to-face	SG	Pits	49	Secondary
Mulhern et al. 2020^8	UK	NA	General	3014	v2	Online	DCEtto +  DCEtto with death	Death	About 600	Primary
Dufresne et al. 2021^35	Canada	2018–2019	Food-allergic patients OR parents	241 + 125	v2	Face-to-face OR online	TTO + DCE	Pits + death	76 + 140	Primary
Kharroubi et al. 2021^36	Japan & UK	2005; NA	General	600;836	v1	Face-to-face	SG	Pits + death	241;249	Secondary
Xie et al. 2020^37	China (Tianjin)	2018	General	533	v2	Face-to-face	TTO +  DCE/DCEtto	Pits + death	115 + 300	Primary
Wu et al. 2021^38	China (eight provinces)	2019	General	3401	v2	Face-to-face	TTO +  DCEtto	Death	295 + 600	Primary
Osman et al. 2021^39	China	2019	General	463	v2	Online	BWS	Death	About 1000	Primary
Authors	Best model	Pits in best model	MAE	No. Inconsistency^b	CREATE score	Highest dimension	Lowest dimension	Order of dimensions
Brazier et al. 1998^11	Fixed effect	0.455	NA	2	27/35	MH (−0.200)	VT (NS)	MH,PN,PF,RL,SF,VT
Brazier et al. 2002^6	Mean (10)	0.296	0.073	5	31/35	PN (−0.167)	RL (−0.050)	PN,MH,PF,VT,SF,RL
Brazier and Roberts 2004^13	Mean parcimonious (4)	0.345	0.079	0	29/35	PN (−0.137)	PF (−0.045)	PN,MH,VT,SF,RL,PF
Kharroubi et al. 2005^14	MCMC	0.203	0.103^c	–	21/35	–	–	–
McCabe et al. 2006^15	Mean rescaled	0.18	0.08	3	18/35	PF (−0.225)	RL (−0.063)	PF,PN,MH,SF,VT,RL
Kharroubi et al. 2007a^16	MCMC	0.203	0.089^c	–	22/35	–	–	–
Kharroubi et al. 2007b^17	MCMC covariates	0.209	–	–	24/35	–	–	–
Lam et al. 2008^18	OLS mean (3)	0.152	0.054	5	28/35	PF (−0.232)	RL (−0.067)	PF,PN,VT,SF,MH,RL
Brazier et al. 2009^19	Random effects	0.387	0.111	6	30/35	PF (−0.137)	VT (−0.055)	PF,PN,SF,RL,MH,VT
Ferreira et al. 2010^20	GEE (7)	0.373	0.209	8	28/35	PF (−0.214)	RL (−0.054)	PF,MH,VT,PN,SF,RL
Ferreira et al. 2011^21	Rank-ordered parcimonious (3)	0.124	0.1	0	27/35	PF (−0.349)	RL (−0.051)	PF,PN,SF,MH,VT,RL
Cruz et al. 2011^22	Random effects (3)	0.264	0.064	9	32/35	PF (−0.177)	VT (−0.083)	PF,PN,MH,SF,RL,VT
McGhee et al. 2011^23	OLS mean	0.315	0.057	0	26/35	PF (−0.178)	RL (−0.054)	PF,PN,MH,SF,VT,RL
Méndez et al. 2011^24	IPW adjusted	−0.515	0.167^c	4	28/35	PF (−0.451)	RL (−0.134)	PF,PN,VT,SF,MH,RL
Abellán Perpiñán et al. 2012^25	OLS mean (3)	−0.357	0.081	0	31/35	PF (−0.338)	RL (−0.70)	PF,PN,MH,SF,VT,RL
Craig et al. 2013^26	Stacked probit	0.013	–	0	26/35	PN (−0.221)	VT (−0.12)	PN,MH,PF,SF,RL,VT
Kharroubi et al. 2013^27	MCMC covariates	0.098	0.085^c	–	26/35	–	–	–
Kharroubi et al. 2014^28	Bayesian model	0.066;0.100	0.07^c	–	26/35	PF;PN	VT;SF	PF,MH,PN,SF,RL,VT(HK) PN,PF,MH,VT,RL,SF(UK)
Norman et al. 2014^29	Random effects (B)	−0.363	–	1	27/35	PN	RL	PN,PF,MH,VT,SF,RL
Craig 2016^30	ERUM	−0.029	–	0	26/35	PN (−0.248)	RL (−0.085)	PN,PF,MH,SF,VT,RL
Kharroubi 2015^31	Bayesian model	0.224;0.220	0.08;0.09	–	21/35	–	–	–
Kharroubi 2018^9	Bayesian model (HK/UK)	0.0708	0.045^c	10%	25/35	–	–	–
Jonker et al. 2018^32	MCMC (hyperbolic)	−0.44	–	0	29.5/35	PN (−0.38)	RL (−0.16)	PN,MH,VT,SF/PF,RL
Kharroubi et al. 2020^34	OLS mean	0.431	0.036	2	29.5/35	PF (−0.168)	VT (−0.024)	PF,SF,PN,MH,RL,VT
Kharroubi 2020^34	BR + RE	0.331	0.058^c	2	30.5/35	PF (−0.797)	VT (−0.246)	PF,SF,RL,PN,MH,VT
Mulhern et al. 2020^8	CL anchored model (3)	−0.574	NA	0	30/35	PN (−0.620)	RF (−0.102)	PN,MH,PF,SF,VT,RL
Dufresne et al. 2021^35	Hybrid (likelihood)	−0.119	NA	1	28/35	PN (−0.281)	SF/RL (−0.107)	PN,MH,PF,VT,SF/RL
Kharroubi et al. 2021^36	Bayesian models (2)	0.289	0.076^c	–	26.5/35	–	–	–
Xie et al. 2020^37	FE (TTO); Conditional logit (DCE)	TTO (−0.229); DCE (−0.105); DCEtto (−0.726)	TTO (0.1003)^d; DCE (0.1339)^d; DCEtto (0.162)^d	TTO (3); DCE (2); DCEtto (3)	30.5/35	TTO (PF:−0.410); DCE (PF:−0.364)); DCEtto (PN:−0.518)	TTO (RL:−0.086); DCE (SF:−0.067); DCEtto (SF:−0.058)	PF,PN,MH,SF,VT,RL(TTO) PF,PN,MH,RL,VT,SF(DCE) PN,PF,MH,VT,RL,SF(DCEtto)
Wu et al. 2021^38	RE (TTO); Conditional logit (DCE)	TTO (−0.277); DCEtto (−0.535)	TTO (0.0616)^d	0	33/35	TTO (PN:−0.427); DCEtto (PN:−0.541)	TTO (RL:−0.097); DCEtto (RL:−0.112)	PN,PF,MH,VT,SF,RL(TTO;DCEtto)
Osman et al. 2021^39	Anchored rescaled Hierarchical Bayes	0.197	–	0	33/35	PN (−0.2034)	VT (−0.1032)	PN,PF,MH,SF,RL,VT

Abbreviations. NA, not available; UK, United Kingdom; HK, Hong-Kong; NA, non-available; VAS, visual analog scale; SG, standard gamble; LE, lottery equivalent; DCE, discrete choice experiment; DCEtto, discrete choice experiment with a duration dimension; TTO, time trade-off; BWS, best-worst scaling; MCMC, Markov-chain Monte-Carlo; OLS, ordinary least square; GEE, generalized estimating equations; IPW, inverse probability weighting; ERUM, episodic random utility model; MAE, mean absolute error; TTO, time trade-off; DCE, discrete choice experiment; DCEtto, discrete choice experiment with a duration dimension; NS, not significant; BR, Beta regression; RE, random effect; FE, fixed effect; CL, conditional logit. *SF-6D version. ^aResults for VAS are not presented. ^bInconsistency indicates the number of coefficients on the dimension levels that are not decreasing when the health state gets worse. ^cValue is for RMSE (root mean square error). ^dValue is for MAD (mean absolute difference).

3.2. Types of designed experiments

Table 2 reports the key characteristics of the tasks and study designs used in primary studies. One study was conducted for the pilot version (v0) and two for v1 from the SF-12. Ten studies were conducted for the SF-6Dv1, and five for the SF-6Dv2. The majority of designs used “Death,” “Pits,” or “Pits and death” as the reference to anchor the data on a continuum. One study³² used a balanced range of health-state severities and introduced the net present value (NPV) to estimate QALY.

Table 2. Survey design characteristics in primary studies.

Characteristics	Number of studies	References^a
SF-6D version
v0	1	^11
v1	10	^6^,^18–20^,^22–24^,^26^,^29^,^33
v1 from SF-12	2	^13^,^32
v2	5	^8^,^35^,^37–39
Elicitation methods*
SG	7	^6^,^13^,^18–20^,^22^,^23^,^33
VAS	1	^11
Rank	2	^6^,^20b
Lottery equivalent (LE)	1	^24
DCE	4	^26^,^29^,^35^,^37
DCEtto	4	^8^,^32^,^37^,^38
DCEtto with death	1	^8
TTO	3	^35^,^37^,^38
BWS	1	^39
Use of death or pits
Death	8	^8^,^11^,^18^,^24^,^26^,^29^,^38^,^39
Pits	1	^33
Pits + death	7	^6^,^13^,^19^,^20^,^22^,^23^,^35^,^37
None	1	^32
Survey mode*
Self-completion	1	^11
Face-to-face	11	^6^,^13^,^18–20^,^22–24^,^33^,^35^,^37^,^38
Online	6	^8^,^26^,^29^,^32^,^35^,^39
Survey year
NA	5	^6^,^8^,^11^,^13^,^29^,^32
2002–2003	1	^18
2004–2006	1	^23
2005	1	^19
2006	1	^20
2007	1	^24
2007–2008	1	^22
2010	1	^26
2018–2019	1	^35
2018	1	^37
2019	3	^33^,^38^,^39
No. Respondents
<200	4	^11^,^18^,^20^,^33
201–500	3	^22^,^35^,^39
501–1000	5	^6^,^13^,^19^,^23^,^26^,^37
1001–2000	3	^24^,^29^,^32
>2000	2	^8^,^38

Notes. ^a[13] used a subsample from [6]. ^bRank data were analyzed in [15] and [21], respectively. *Some surveys used more than one method.

The number of intermediate health states elicited ranged from 49 to about 1000. Such a variation was mainly due to the sample size used that varied from 126 to 3401 respondents. Most studies (70%) were conducted with less than 1000 respondents and only two recent surveys^8,38 had above 3000 respondents. The administration mode was mostly face-to-face interviews (61%). However, online surveys became more popular in recent years. One study³⁵ both collected data from an in-person and online survey. The year of the survey shows that almost every year a new survey was conducted in one country (region).

The most used elicitation method was the standard gamble (SG), followed by the discrete choice experiment (DCE), the DCEtto (i.e. discrete choice experiment with a time duration dimension), and the time trade-off (TTO). Other methods also used by researchers were ranking, DCEtto with death, lottery equivalent (LE), and best-worst scaling (BWS). Supplemental Appendix 4 shows features of elicitation methods among countries (regions).

3.3. Data analysis and modeling

Table 3 displays the best functional form and modeling approaches used in the studies (as per the authors’ recommendation). Indeed, many studies explored more than one model, and the best model preferred by the authors of these studies was the model that had comparatively the best performance and was pinned down to report the value sets. These best models all corresponded to the main effects during model estimations.

Table 3. Modeling characteristics in SF-6D studies.

Best model	Regression approach	References
Individual fixed effect	Additive OLS	^11^,^37
Mean level model	Additive OLS (constant forced to unity)	^7^,^18^,^25^,^33
Parsimonious consistent model at mean level	OLS constrained	^13^,^23
	Rank-ordered logistic regression (ROLogit)	^21
Mean rescaled	Conditional logistic regression	^15
Random effects individual-level model	GEE (constant forced to unity)	^19^,^20^,^22
	Probit regression (with life expectancy)	^29
	Tobit regression	^38
Anchored model	Conditional logistic regression	^33^,^37^,^38
Bayesian non-parametric model	MCMC	^14^,^16^,^32
	MCMC covariates	^17^,^33
	Estimated function for the difference between HK & UK; Japan & UK	^28^,^31
	UK as informative prior to HK; Japan	^9^,^36
Beta regression with random effects	MCMC	^27
Anchored Hierarchical Bayes model	MCMC	^39
Inverse probability weighted (IPW) adjusted	Semi-parametrically estimated	^24
Episodic random utility model (ERUM)	Stacked probit	^26
	Semi-parametric	^30
Hybrid regression model	Likelihood	^35

Abbreviations. OLS, ordinary least square; GEE, generalized estimating equations; MCMC, Markov-chain Monte-Carlo; UK, United Kingdom; HK, Hong-Kong.

Based on the model used, various regression approaches were applied to predict the value sets. Econometric strategies used in these studies were unique or innovative, specially adjusted for certain survey elicitation methods. Generally, they can be categorized as parametric and non-parametric approaches.

The parametric models mainly focused on fixed effects and random effects. Later the parsimonious consistent, mean rescaled, or anchored were added during the model specification. The regression approaches included additive ordinary least squares (OLS)^6,18,25,33; constrained OLS^13,23; generalized estimating equations (GEE)^19,20,22; Probit regression²⁹; Tobit regression³⁸; Rank-ordered logistic regression (ROLogit)²¹; and conditional logistic regression^8,15,37,38.

The other three particular cases for parametric models that are different from the above traditional regression approaches were inverse probability weighted (IPW) adjusted²⁴, Episodic random utility model (ERUM)^26,30, and a hybrid regression model combining data from TTO and DCE³⁵. Their regression approaches were semi-parametric, stacked probit, and likelihood, respectively.

The non-parametric best models were mainly conducted through the Bayesian method. They included the Bayesian non-parametric model, Beta regression with random effects, and Anchored Hierarchical Bayes model. For this group of models, researchers usually conducted estimates through Markov-Chain Monte-Carlo (MCM). While four comparison studies applied the Bayesian methods, two studies were combining data from the UK with HK and Japan, respectively^28,31. Note that two other studies used data from the UK as informative prior to estimating HK and Japan value sets, respectively^9,36.

3.4. Dimensions’ ordering

As stated above, we considered the six-dimension orders based on the best model that was preferred by their authors. Furthermore, we explored the ranks of each dimension in the value sets from the view of culture and economy. To perform a comparison between countries (regions), we needed to pin down one value set respectively as many countries were surveyed more than once. As such, 12 primary studies were selected for the 12 countries (regions) under assessment. The selection criteria were specified in each case to limit heterogeneity in the comparison: the latest version of SF-6D (e.g. UK v2 was selected instead of UK v1); the largest sample size (e.g. the largest sample in HK²² was selected instead of another¹⁸ as well as for China³⁸ instead of others^37,39); the most widely used elicitation methods were also preferred (e.g. Portugal with SG²⁰ instead of Portugal with Rank²¹; Spain with SG²⁴ instead of Spain with lottery equivalent²⁵; for Lebanon using OLS³³ was selected instead of beta regression³⁴).

3.4.1. Cultural difference

In the literature, there were discussions of how countries could be compared based on common features such as western-eastern countries³⁸, European-Asian countries^31,36, and cultural or religious differences³³. To explore a comparatively measurable factor, we categorized the countries according to the Anglo-Saxon cultural sphere or not (i.e. other cultural spheres). The Anglosphere is a group of English-speaking (or with common language roots) nations that share common ancestral, historical, or cultural ties with England or the United Kingdom broadly and which today maintain close political, diplomatic, and military co-operation. After categorizing the data as “Anglo-Saxon countries” and “non-Anglo-Saxon countries” (details in Table 4), the dimensions’ orders in the value sets present some obvious patterns.

Table 4. Value set and dimension categorized by culture.

Anglo-Saxon country/region	PF	RL	SF	PN	MH	VT	Ref	Order of dimensions
UK v2	3	6	4	1	2	5	^8	PN,MH,PF,SF,VT,RL
USA v1	3	5	4	1	2	6	^26	PN,MH,PF,SF,RL,VT
Australia v1	2	6	5	1	3	4	^29	PN,PF,MH,VT,SF,RL
Netherlands^a v1	4/5	6	4/5	1	2	3	^32	PN,MH,VT,SF/PF,RL
Canada^b v2	3	5/6	5/6	1	2	4	^35	PN,MH,PF,VT,SF/RL
Non-Anglo-Saxon country/region	PF	RL	SF	PN	MH	VT	Ref	Order of dimensions
Hong Kong v1	1	6	4	2	3	5	^23	PF,PN,MH,SF,VT,RL
Japan v1	1	4	3	2	5	6	^19	PF,PN,SF,RL,MH,VT
Portugal v1	1	6	5	4	2	3	^20	PF,MH,VT,PN,SF,RL
Brazil v1	1	5	4	2	3	6	^22	PF,PN,MH,SF,RL,VT
Spain v1	1	6	4	2	5	3	^24	PF,PN,VT,SF,MH,RL
Lebanon v1	1	5	2	3	4	6	^33	PF,SF,PN,MH,RL,VT
China v2	2	6	5	1	3	4	^38	PN,PF,MH,VT,SF,RL

Notes. ^aNetherland people speak Flemish. However, this language shares a common root with English. ^bIn the Canadian sample, most respondents were from Quebec and responded in French, but they are evolving in a North American context.

The columns in red/green stand for the most/least important dimension in each category.

In the Anglo-Saxon countries, all countries have PN (pain) as the most important dimension (i.e. the dimension that is reduced the most in the HRQoL). On the other side, almost all countries (except the USA) have RL (role limitation) as the least important dimension. The USA has RL as its second least dimension, the least dimension being VT (vitality). Also, it is important mentioning that MH (mental health) was frequently second, which was not the case for non-Anglo-Saxon countries.

In the non-Anglo-Saxon countries, almost all countries (except China) have PF (physical functioning) as the highest dimension. However, China has PF as its second-largest dimension while the highest dimension was PN. For all non-Anglo-Saxon countries, RL (role limitation, average 5.4) and VT (vitality, average 4.7) were the least important dimensions.

Our review seemingly indicates a world divided in two between the Anglo-Saxon cultural sphere and the other cultural spheres. In the literature, researchers discussed it from a similar idea such as the English-speaking sphere and countries outside, whether eastern or western countries. For example, native English-speaking people seem to give more importance to pain than countries outside this cultural world³⁴, and populations of eastern countries may give more weight to PF, while those of western countries may have more preference for MH³⁸. It is also noticed that China³⁸ has the same order of dimensions as Australia²⁹. Without focusing on specific statistical tests, this systematic review would discuss it from two directions. One might be related to our selection process. Indeed, another data of China³⁹ that was not selected to represent this country, due to smaller sample size, reported a value set that was not the same as for Australia. In this view, the selection process will be presented later in the discussion section (sensitivity analysis). Another explanation would be that the features of the value set might relate to the growing influence of the Chinese culture in Australia⁴⁰, or maybe that it is more than a cultural difference from the Anglo-Saxon sphere.

3.4.2. Economic and social development

As some researchers who discussed the difference between their studies with other countries’ results, Wu et al.³⁸ mentioned that similarities and distinctions in the ranking of the dimensions may reflect cultural and socioeconomic factors, which are essential in shaping the preferences of populations.

As stated in the culture section, the Anglo-Saxon and non-Anglo-Saxon countries have different results in the order of the most and least important dimensions. However, two countries are somewhat different in their respective group: China and USA. Indeed, China’s most important dimension is Pain (PN), which is the same as in Anglo-Saxon countries; While USA’s least important dimension is Vitality (VT), which is one of the least important dimensions in non-Anglo-Saxon countries. As a consequence, some other factors may be related to these orders.

In this section, we hypothesized that the differences in dimensions’ order would be associated with the country-related economic and social development level. On one hand, the socio-economic development differences may link to the macroeconomic view of government expenditure. For example, Rodriguez and Nieves Valdés⁴¹ provided empirical evidence of the existence of a long-run causal relationship between GDP and health care expenditures. On the other hand, from the microeconomic view, individuals would consider how their health might influence their job or income. In this view, Strauss and Thomas⁴² provided a general discussion about the relationship between income, labor market, and health.

The first economic factor we explored is GDP per capita (see supplemental Appendix 5), the value sets selected present the following results: generally speaking, PF (physical functioning) decreases in the ranking as the economic level increases, while PN (pain) and MH (mental health) increase in ranking as the economic level increases. Specifically, PF (physical functioning) was ranked the most important (ranked 1st) in Lebanon, Portugal, Japan, and Hong Kong, while it was ranked 4.5 (the same as SF) in the Netherlands, which has a comparatively higher GDP per capita. Similarly, SF (social functioning) was ranked 2nd in Lebanon while it was ranked 5.6 (the same as RL) in Canada with a higher GDP per capita. However, For Mental Health (MH), Canada, USA, Netherlands, and Portugal have it in second while Lebanon has it in fourth rank. Among all economic levels, RL (role limitation) was the least or the second least (except for Japan, ranked fourth).

We also tried to study these orders by linking them to other socio-economic factors such as population, health expenditure per capita, life expectancy, and GDP growth. Details were reported in supplemental Appendix 6. When sorted by culture and then by GDP growth (%) in 2017 (prior to COVID-19) (see supplemental Appendix 6 Table A.6.4), the value set orders in non-Anglo-Saxon countries present a clear trend in favor of PF compared to other dimensions in the Anglo-Saxon country group. Within its cultural group, the importance of PF decreases as the GDP growth rate increases. It is, however, noteworthy to mention that this trend is sensitive to economic indicators and years.

4. Discussion

We performed a systematic review of studies that developed value sets for the SF-6D up to September 2022. Methodological differences were identified, and the dimensions’ ordering of the selected studies was analyzed by cultural and economic factors. A total of 31 articles were selected for value set estimation-related analysis, and among them, 17 primary studies were selected for survey design and characteristics summary. This global review covered 12 different countries and regions. To our knowledge, there was no previous review study on SF-6D that has been done in the literature.

In the distribution of values and the size of the coefficients, we discovered two features. One finding was that Anglo-Saxon countries gave more importance to pain, while most non-Anglo-Saxon countries (six out of seven) have physical functioning as the highest dimension. The other finding was that, as the economy increases, people care less about physical functioning but more about pain and mental health.

As indicated in the section on dimensions’ ordering, we filtered 12 studies from the retrieved studies. Studies were selected based on criteria such as the version of the SF-6D, sample size, and elicitation methods. A sensitivity analysis was conducted by replacing the chosen studies with the excluded ones and by reporting how the results would change.

Among these 12 countries, 3 countries had more than two primary studies. The value sets for these three countries were replaced by their alternative for the UK⁶, Hong Kong¹⁸, and China³⁹. For Portugal²¹, Spain²⁵, and Lebanon³⁴, researchers developed a second study based on the primary data, these studies may present a different value set and order as a result of applying different models. They were thus selected for the replacement, while the same studies were kept for the remaining countries. Specifically, half of the selected studies were replaced, and the countries (regions) in dark shadow in supplemental Appendix 6 Table A.6.5. were added. This new selection shows that the basic results found in Table 4 still hold based on the replacement of these available studies.

In Anglo-Saxon countries, when the UK was replaced, as compared to the version that was previously selected, the difference was in the order of VT and SF. Generally, the results were the same as in Table 4, with all countries having PN (pain) as the most important dimension, and all countries (except the USA) having RL (role limitation) as the least important dimension.

In the non-Anglo-Saxon countries, all countries except Japan and Brazil were replaced by other studies. However, the results that were reported in Table 4 still hold. PF (physical functioning) is the highest dimension (except in China). For all non-Anglo-Saxon countries, RL (role limitation, average 5.1) or VT (vitality, average 5.1) were the least important dimensions.

These differences found in dimensions’ ordering might a priori be explained by the difference in the valuation techniques used, the different regression methods and functional forms, or the actual preference differences between nationalities with different cultural and economic backgrounds. Although we attempted to compare studies with common features in survey design and regression methods, studies still have methodological differences, which do not preclude that the differences observed were not due to these features. However, the patterns observed were sufficiently obvious to suggest that cultural and economic differences could be a major factor to explain the differences in dimensions’ ordering and the size of coefficients.

Considering the heterogeneity found in the retrieved studies, we recommend, for the implementation of future studies regarding the development of value sets for the SF-6D, to consider a variety of estimation procedures (e.g. OLS mean, Bayesian, anchored) to help comparison with previous studies, as well as to consider in the design the use of more than one elicitation method (e.g. TTO, DCE, SG). We also recommend producing value sets for countries where this has not been done yet, since cultural differences matter, as highlighted in this review.

This systematic review has limitations. One of these is that we may have missed studies that were not published in the database we retrieved. However, for the search period, we took some measures such as setting a search alarm for keywords in the database, following the SF-6D major authors, and updating the selected articles occasionally up to September 2022, so that we are confident that our search strategy was sufficient to capture most of them.

The value set comparison in this study also has its limitation. Strictly speaking, the summary tables showed that the year of the survey was different, the versions used were mixed, and the elicitation methods were different. Based on the given studies, when selecting the studies for value set comparison, we tried to present studies with the most similarities in terms of methods (e.g. SG, OLS) and versions (e.g. v1 vs. v2).

Another limitation is the representativeness of the sample for the country or region. In this systemic review, we included and described all studies valuing the SF-6D health states from different countries and regions, but some of these studies might not aim to generate a value set for use in their specific country or region so that the study design and the representativeness of the study sample might not be designed for generating national value sets.

Finally, a limitation is related to the instrument used to assess the quality of studies included, the CREATE checklist³, which may not fully capture the risk of reporting biases.

5. Conclusions

This systematic review provided an up-to-date summary of the methodological features in the development of SF-6D value sets. It presented survey design characteristics, modeling characteristics, and survey regions. Moreover, a country representative dataset was selected, and a global value set comparison was presented. Generally, when considering culture and economy, the dimensions’ ordering presented a certain pattern. Anglo-Saxon countries gave more importance to pain than countries outside this cultural world. While most non-Anglo-Saxon countries have physical functioning as the highest dimension. As the economic level increased, people cared less about physical functioning but more about pain and mental health. Besides the summary of survey design and elicitation modeling, these findings could offer some socio-economic features for further research.

Transparency

Declaration of funding

This work was funded through a research fund of the Fondation de l’Institut universitaire en santé mentale de Montréal (IUSMM).

Declaration of financial/other relationships

The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Author contributions

Concept and design: TGP

Acquisition of data: TGP, LW

Analysis and interpretation of data: TGP, LW

Drafting of the manuscript: TGP, LW

Critical revision of the paper for important intellectual content: TGP, LW

Statistical analysis: TGP, LW

Obtaining funding: TGP

Administrative, technical, or logistic support: TGP

Supervision: TGP

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgements

We would like to present our sincere gratitude to Wilfried Gandji for his contribution to a previous part of this project. TGP is a member of the FRQS-funded Research center of the IUSMM and was a former employee of the CIUSSS de l’Estrie – CHUS.