Improving the quality of discrete-choice experiments in health: how can we assess validity and reliability?

References

• Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington (DC): The National Academy of Sciences; 2001.
   Google Scholar
• Epstein RM, Street RL. The values and value of patient-centered care. Ann Fam Med. 2011 Mar;9(2):100–103.
   PubMed Web of Science ®Google Scholar
• Frank L, Basch E, Selby JV. The PCORI perspective on patient-centered outcomes research. JAMA. 2014 Oct 15;312(15):1513–1514. DOI:10.1001/jama.2014.11100.
   PubMed Web of Science ®Google Scholar
• Domecq JP, Prutsky G, Elraiyah T, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014 Feb;26(14):89.
   Web of Science ®Google Scholar
• Bridges JF, Jones C. Patient-based health technology assessment: a vision of the future. Int J Technol Assess Health Care. 2007 Winter;23(1):30–35. DOI:10.1017/s0266462307051549.
   PubMed Web of Science ®Google Scholar
• Ho M, Saha A, McCleary KK, et al. A framework for incorporating patient preferences regarding benefits and risks into regulatory assessment of medical technologies. Value Health. 2016 Sep–Oct;19(6):746–750. DOI:10.1016/j.jval.2016.02.019.
   PubMed Web of Science ®Google Scholar
• Muhlbacher AC, Bridges JF, Bethge S, et al. Preferences for antiviral therapy of chronic hepatitis C: a discrete choice experiment. Eur J Health Econ. 2016 Feb 04. DOI:10.1007/s10198-016-0763-8.
   Web of Science ®Google Scholar
• European Medicines Agency (EMA). Benefit-risk methodology project. Work package 4 report: benefit-risk tools and processes; 2012. [cited 2017 Oct 12]. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Report/2012/03/WC500123819.pdf.
   Google Scholar
• Medical Device Innovation Consortium. A Framework for Incorporating Information on Patient Preferences Regarding Benefit and Risk into Regulatory Assessments of New Medical Technology; 2015 [cited 2017]. Available from: http://mdic.org/wp-content/uploads/2015/05/MDIC_PCBR_Framework_Web1.pdf
   Google Scholar
• Guo JJ, Pandey S, Doyle J, et al. A review of quantitative risk-benefit methodologies for assessing drug safety and efficacy-report of the ISPOR risk-benefit management working group. Value Health. 2010;13(5):657–666.
   PubMed Web of Science ®Google Scholar
• Hauber BA, Fairchild AO, Johnson FR. Quantifying benefit-risk preferences for medical interventions: an overview of a growing empirical literature. Appl Health Econ Health Policy. 2013;11(4):319–329.
   PubMedGoogle Scholar
• Mühlbacher AC, Juhnke C, Beyer AR, et al. Patient-focused benefit-risk analysis to inform regulatory decisions: the European Union perspective. Value Health. 2016;19(6):734–740.
   PubMed Web of Science ®Google Scholar
• Australian Pharmaceutical Benefits Advisory Committee (PBAC). Appendix 6 - Including non health outcomes in a supplementary analysis; 2016 [cited 2017]. Available from: https://pbac.pbs.gov.au/appendixes/appendix-6-including-nonhealth-outcomes-in-a-supplementary-analysis.html
   Google Scholar
• United States Food and Drug Administration (FDA). Patient Preference Information – Voluntary Submission, Review in Premarket Approval Applications, Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in Decision Summaries and Device Labeling; 2016 [cited 2017 Aug 23]. Available from: http://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm446680.pdf
   Google Scholar
• Ozdemir S. Improving the validity of stated-preference data in health research: the potential of the time-to-think approach. Patient. 2015 Jun;8(3):247–255. DOI:10.1007/s40271-014-0084-x.
   PubMedGoogle Scholar
• Adamowicz W, Swait J, Boxall P, et al. Perceptions versus objective measures of environmental quality in combined revealed and stated preference models of environmental valuation. J Environ Econ Manage. 1997;32(1):65–84.
   Web of Science ®Google Scholar
• Adamowicz W, Louviere J, Williams M. Combining revealed and stated preference methods for valuing environmental amenities. J Environ Econ Manage. 1994;26(3):271–292.
   Web of Science ®Google Scholar
• Gurmankin AD, Baron J, Hershey JC, et al. The role of physicians’ recommendations in medical treatment decisions. Med Decis Making. 2002 May-Jun;22(3):262–271.
   PubMed Web of Science ®Google Scholar
• MacLeod TE, Harris AH, Mahal A. Stated and revealed preferences for funding new high-cost cancer drugs: a critical review of the evidence from patients, the public and payers. Patient. 2016 Jun;9(3):201–222. DOI:10.1007/s40271-015-0139-7.
   PubMedGoogle Scholar
• Sacristan JA, Lizan L, Comellas M, et al. Perceptions of oncologists, healthcare policy makers, patients and the general population on the value of pharmaceutical treatments in oncology. Adv Ther. 2016 Nov;33(11):2059–2068.
   PubMed Web of Science ®Google Scholar
• Rakotonarivo OS, Schaafsma M, Hockley N. A systematic review of the reliability and validity of discrete choice experiments in valuing non-market environmental goods. J Environ Manage. 2016;183:98–109. DOI:10.1016/j.jenvman.2016.08.032.
   PubMed Web of Science ®Google Scholar
• Clark MD, Determann D, Petrou S, et al. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014 Sep;32(9):883–902. DOI:10.1007/s40273-014-0170-x.
   PubMed Web of Science ®Google Scholar
• De Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012 Feb;21(2):145–172. DOI:10.1002/hec.1697.
   PubMed Web of Science ®Google Scholar
• Ryan M, Gerard K. Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy. 2003;2(1):55–64.
   PubMedGoogle Scholar
• Ryan M, Scott DA, Reeves C, et al. Eliciting public preferences for healthcare: a systematic review of techniques. Health Technol Assess. 2001;5(5):1–186.
   PubMedGoogle Scholar
• Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making: a user’s guide. Pharmacoeconomics. 2008;26(8):661–677.
   PubMed Web of Science ®Google Scholar
• Muhlbacher A, Johnson FR. Choice experiments to quantify preferences for health and healthcare: state of the practice. Appl Health Econ Health Policy. 2016 Jun;14(3):253–266. DOI:10.1007/s40258-016-0232-7.
   PubMed Web of Science ®Google Scholar
• Ryan M, Bate A, Eastmond CJ, et al. Use of discrete choice experiments to elicit preferences. Qual Health Care. 2001 Sep;10(Suppl 1):i55–i60.
   PubMedGoogle Scholar
• Johnson FR, Mohamed AF, Ozdemir S, et al. How does cost matter in health-care discrete-choice experiments? Health Econ. 2011 Mar;20(3):323–330.
   PubMed Web of Science ®Google Scholar
• Lagarde M. Investigating attribute non-attendance and its consequences in choice experiments with latent class models. Health Econ. 2013 May;22(5):554–567. DOI:10.1002/hec.2824.
   PubMed Web of Science ®Google Scholar
• Lancsar E, Swait J. Reconceptualising the external validity of discrete choice experiments. Pharmacoeconomics. 2014 Oct;32(10):951–965. DOI:10.1007/s40273-014-0181-7.
   PubMed Web of Science ®Google Scholar
• Miguel FS, Ryan M, Amaya-Amaya M. ‘Irrational’ stated preferences: a quantitative and qualitative investigation. Health Econ. 2005 Mar;14(3):307–322. DOI:10.1002/hec.912.
   PubMed Web of Science ®Google Scholar
• Ozdemir S, Mohamed AF, Johnson FR, et al. Who pays attention in stated-choice surveys? Health Econ. 2010 Jan;19(1):111–118. DOI:10.1002/hec.1452.
   PubMed Web of Science ®Google Scholar
• Telser H, Zweifel P. Validity of discrete-choice experiments evidence for health risk reduction. Appl Econ. 2007;39(1):69–78.
   Web of Science ®Google Scholar
• Trochim W, Kane M. Concept mapping: an introduction to structured conceptualization in health care. Int J Qual Health Care. 2005 Jun;17(3):187–191. DOI:10.1093/intqhc/mzi038.
   PubMed Web of Science ®Google Scholar
• De Bekker-Grob EW, Hol L, Donkers B, et al. Labeled versus unlabeled discrete choice experiments in health economics: an application to colorectal cancer screening. Value Health. 2010 Mar-Apr;13(2):315–323. DOI:10.1111/j.1524-4733.2009.00670.x.
   PubMed Web of Science ®Google Scholar
• Muhlbacher AC, Rudolph I, Lincke HJ, et al. Preferences for treatment of attention-deficit/hyperactivity disorder (ADHD): a discrete choice experiment. BMC Health Serv Res. 2009;9:149. DOI:10.1186/1472-6963-9-149.
   PubMed Web of Science ®Google Scholar
• Kenny P, Hall J, Viney R, et al. Do participants understand a stated preference health survey? A qualitative approach to assessing validity. Int J Technol Assess Health Care. 2003 Fall;19(4):664–681.
   PubMed Web of Science ®Google Scholar
• Bijlenga D, Birnie E, Bonsel GJ. Feasibility, reliability, and validity of three health-state valuation methods using multiple-outcome vignettes on moderate-risk pregnancy at term. Value Health. 2009 Jul-Aug;12(5):821–827. DOI:10.1111/j.1524-4733.2009.00503.x.
   PubMed Web of Science ®Google Scholar
• Hollin IL, Caroline Y, Hanson C, et al. Developing a patient-centered benefit-risk survey: a community-engaged process. Value Health. 2016 Sep - Oct;19(6):751–757. DOI:10.1016/j.jval.2016.02.014.
   PubMed Web of Science ®Google Scholar
• Janssen EM, Segal JB, Bridges JF. A framework for instrument development of a choice experiment: an application to type 2 diabetes. Patient. 2016 Oct;9(5):465–479. DOI:10.1007/s40271-016-0170-3.
   PubMedGoogle Scholar
• Mark TL, Swait J. Using stated preference and revealed preference modeling to evaluate prescribing decisions. Health Econ. 2004 Jun;13(6):563–573. DOI:10.1002/hec.845.
   PubMed Web of Science ®Google Scholar
• Janssen EM, Hauber AB, Bridges JFP. Conducting a discrete choice experiment study following recommendations for good research practices: an application for eliciting patient preferences for diabetes treatments. Value in Health. 2017. DOI:10.1016/j.jval.2017.07.001
   Web of Science ®Google Scholar
• Veldwijk J, Essers BA, Lambooij MS, et al. Survival or mortality: does risk attribute framing influence decision-making behavior in a discrete choice experiment? Value Health. 2016 Mar-Apr;19(2):202–209. DOI:10.1016/j.jval.2015.11.004.
   PubMed Web of Science ®Google Scholar
• Rockers PC, Jaskiewicz W, Wurts L, et al. Preferences for working in rural clinics among trainee health professionals in Uganda: a discrete choice experiment. BMC Health Serv Res. 2012 Jul;23(12):212.
   Web of Science ®Google Scholar
• Kruk ME, Paczkowski MM, Tegegn A, et al. Women’s preferences for obstetric care in rural Ethiopia: a population-based discrete choice experiment in a region with low rates of facility delivery. J Epidemiol Commun Health. 2010 Nov;64(11):984–988. DOI:10.1136/jech.2009.087973.
   PubMed Web of Science ®Google Scholar
• Ho MP, Gonzalez JM, Lerner HP, et al. Incorporating patient-preference evidence into regulatory decision making. Surg Endosc. 2015;29(10):2984–2993. DOI:10.1007/s00464-014-4044-2.
   PubMed Web of Science ®Google Scholar
• Bridges JF, Mohamed AF, Finnern HW, et al. Patients’ preferences for treatment outcomes for advanced non-small cell lung cancer: a conjoint analysis. Lung Cancer. 2012 Jul;77(1):224–231. DOI:10.1016/j.lungcan.2012.01.016.
   PubMed Web of Science ®Google Scholar
• Kimberlin CL, Winterstein AG. Validity and reliability of measurement instruments used in research. Am J Health Syst Pharm. 2008 Dec 01;65(23):2276–2284. DOI:10.2146/ajhp070364.
   PubMed Web of Science ®Google Scholar
• McFadden D. Conditional logit analysis of qualitative choice behavior. In: Zarembka P, editor. Frontiers in econometrics. New York (NY): Academic Press; 1974. p. 105–142.
   Google Scholar
• Hougaard JL, Tjur T, Osterdal LP. On the meaningfulness of testing preference axioms in stated preference discrete choice experiments. Eur J Health Econ. 2012 Aug;13(4):409–417. DOI:10.1007/s10198-011-0312-4.
   PubMed Web of Science ®Google Scholar
• Bijlenga D, Bonsel GJ, Birnie E. Eliciting willingness to pay in obstetrics: comparing a direct and an indirect valuation method for complex health outcomes. Health Econ. 2011 Nov;20(11):1392–1406. DOI:10.1002/hec.1678.
   PubMed Web of Science ®Google Scholar
• Hollin IL, Peay HL, Bridges JF. Caregiver preferences for emerging duchenne muscular dystrophy treatments: a comparison of best-worst scaling and conjoint analysis. Patient. 2015;8(1):19–27.
   PubMedGoogle Scholar
• Doiron D, Yoo HI. Temporal stability of stated preferences: the case of junior nursing jobs. Health Econ. 2016Apr;14:802–809. DOI:10.1002/hec.3350.
   Web of Science ®Google Scholar
• Orme B. Including holdout choice tasks in conjoint studies. Washington: Sawtooth Software Inc.; 2015.
   Google Scholar
• Hauber AB, Gonzalez JM, Groothuis-Oudshoorn CG, et al. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR conjoint analysis good research practices task force. Value Health. 2016 Jun;19(4):300–315. DOI:10.1016/j.jval.2016.04.004.
   PubMed Web of Science ®Google Scholar
• Sen A. Internal consistency of choice. Econometrica. 1993;61:495–521.
   Web of Science ®Google Scholar
• Marshall D, Bridges JF, Hauber B, et al. Conjoint analysis applications in health - how are studies being designed and reported?: an update on current practice in the published literature between 2005 and 2008. Patient. 2010 Dec 1;3(4):249–256. DOI:10.2165/11539650-000000000-00000.
   PubMedGoogle Scholar
• Joy SM, Little E, Maruthur NM, et al. Patient preferences for the treatment of type 2 diabetes: a scoping review. Pharmacoeconomics. 2013;31(10):877–892.
   PubMed Web of Science ®Google Scholar
• Bridges JF. Stated preference methods in health care evaluation: an emerging methodological paradigm in health economics. Appl Health Econ Health Policy. 2003;2(4):213–224.
   PubMedGoogle Scholar
• Ryan M, Farrar S. Using conjoint analysis to elicit preferences for health care. Bmj. 2000 Jun 3;320(7248):1530–1533.
   PubMed Web of Science ®Google Scholar
• dosReis S, Castillo WC, Ross M, et al. Attribute development using continuous stakeholder engagement to prioritize treatment decisions: a framework for patient-centered research. Value Health. 2016 Sep–Oct;19(6):758–766. DOI:10.1016/j.jval.2016.02.013.
   PubMed Web of Science ®Google Scholar
• Coast J, Al-Janabi H, Sutton EJ, et al. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ. 2012;21(6):730–741.
   PubMed Web of Science ®Google Scholar
• De Bekker-Grob EW, Donkers B, Jonker MF, et al. Sample size requirements for discrete-choice experiments in healthcare: a practical guide. Patient. 2015 Oct;8(5):373–384.
   PubMedGoogle Scholar
• Lancsar E, Louviere J, Flynn T. Several methods to investigate relative attribute impact in stated preference experiments. Soc Sci Med. 2007 Apr;64(8):1738–1753. DOI:10.1016/j.socscimed.2006.12.007.
   PubMed Web of Science ®Google Scholar
• Bridges JF, Hauber AB, Marshall D, et al. Conjoint analysis applications in health–a checklist: a report of the ISPOR good research practices for conjoint analysis task force. Value Health. 2011 Jun;14(4):403–413. DOI:10.1016/j.jval.2010.11.013.
   PubMed Web of Science ®Google Scholar
• Johnson FR, Lancsar E, Marshall D, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force. Value Health. 2013;16(1):3–13.
   PubMed Web of Science ®Google Scholar
• Viney R, Lancsar E, Louviere J. Discrete choice experiments to measure consumer preferences for health and healthcare. Expert Rev Pharmacoecon Outcomes Res. 2002 Aug;2(4):319–326. DOI:10.1586/14737167.2.4.319.
   PubMedGoogle Scholar
• Lancsar E, Louviere J. Deleting ‘irrational’ responses from discrete choice experiments: a case of investigating or imposing preferences? Health Econ. 2006 Aug;15(8):797–811. DOI:10.1002/hec.1104.
   PubMed Web of Science ®Google Scholar
• Thurstone LL. A law of comparative judgment. Psychol Rev. 1927;34(4): 273–286.
   Google Scholar
• Kahneman D, Tversky A. Prospect theory: an analysis of decision under risk. Econometrica. 1979;47(2):263–291.
   Web of Science ®Google Scholar
• Loomes G, Sugden R. A rationale for preference reversal. Am Econ Rev. 1983;73(3):428–432.
   Web of Science ®Google Scholar
• Loomes G, Sugden R. Testing for regret and disappointment in choice under uncertainty. Econ J. 1987;97:118–129.
   Web of Science ®Google Scholar
• Durbin J, Watson GS. Testing for serial correlation in least squares regression: I. Biometrika. 1950;37(3–4):409–428.
   PubMed Web of Science ®Google Scholar