Discrete-choice experiments versus rating scale exercises to evaluate the importance of attributes

Abstract

Aim: To examine the difference between discrete-choice experiments (DCE) and rating scale exercises (RSE) in determining the most important attributes using a case study. Methods: Undergraduate health sciences students were asked to complete a DCE and a RSE. Six potentially important attributes were identified in focus groups. Fourteen unlabelled choice tasks were constructed using a statistically efficient design. Mixed multinomial logistic regression analysis was used for DCE data analysis. Results: In total, 254 undergraduate students filled out the questionnaire. In the DCE, only four attributes were statistically significant, whereas in the RSE, all attributes except one were rated four or higher. Conclusion: Attribute importance differs between DCE and RSE. The DCE had a differentiating effect on the relative importance of the attributes; however, determining relative importance using DCE should be done with caution as a lack of statistically significant difference between levels does not necessarily imply that the attribute is not important.

Keywords:

• discrete choice experiment
• Likert scale
• preferences
• rating scale
• relative importance of attributes

Financial & competing interests disclosure

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

No writing assistance was utilized in the production of this manuscript.

Key issues

• In decision making, one should be aware that the determination of the (relative) importance of attributes differs between rating scale exercises and discrete-choice experiments.

• Determining the relative importance of the attributes valued using discrete-choice experiments should be handled carefully as no statistical significant difference between levels does not necessarily mean that the attribute is not important.

• Building on Philips et al. (2012), it is reasonable to assume that rating scale exercises are more likely to determine the attitude of respondents toward attributes and that discrete-choice experiments are more focusing on preferences toward attributes when making a decision.