Repeated questioning in choice experiments: are we improving statistical efficiency or getting respondents confused?

Abstract

This paper investigates options to obtain additional information about individual preferences in conjoint choice experiments when the number of respondents and the number of choice cards per respondent are fixed. We examine best/worst (or best/second best) repeated elicitation and forced-choice questions. Using simulations, we find that best/worst (or best/second best) produce the best efficiency gains over one-shot elicitation. Forced-choice questions also improve the statistical efficiency of the key estimates. We also use data from two empirical studies deploying best/worst (or best/second best) and forced-choice questions and find that the one with best/worst may yield undesirable response effects. The empirical study that uses forced choice results in stable, well-behaved estimates of the key coefficients, with little evidence of heteroskedasticity.

Keywords:

• conjoint choice experiments
• stated preferences
• statistical efficiency
• utility variance
• complexity

Acknowledgements

I wish to thank Prof. Carlo Carraro, Chiara Zanandrea, FEEM, and the International Center for Climate Governance (ICCG), housed at FEEM, Venice, Italy, for financial and logistics support in organising the International Workshop on “Recent Trends in Non Market Valuation.” I also wish to thank workshop attendees, presenters and discussants, and especially Riccardo Scarpa, for their helpful comments. Finally, I am grateful to three anonymous reviewers for their constructive comments and suggestions on an earlier draft of this manuscript. All remaining errors are, of course, my own.

Notes

1. They conclude that the marginal utility of income is more stable over choice tasks for the group that was given the advance disclosure experiments, that there is evidence of precedent dependence whether or not advance disclosure was deployed (with advance disclosure group focusing on the best deals and the stepwise disclosure group focusing on the worst deals), and that the group with stepwise disclosure is likely to be more prone to failing credibility.

2. For example, the “true” VSL in a public program context is CAN$ 1 million.

3. We remind the reader that this is the case only when K = 3. One would need iterated best/worst elicitation if K > 3.

4. To examine efficiency, we compute the standard deviation of the VSL or VSH estimates across the 1000 replications, which provides a robust standard error around each individual estimate. The efficiency properties of the VSL and VSH estimates can also be examined using the theoretical variance-covariance matrix of the coefficients and the delta method. Here, we prefer Monte Carlo simulations because they allow us to check for evidence of any finite-sample bias.

5. This indirect utility function assumes that individual discount back risk reductions to the beginning of the decade over which the risk reductions are experienced.

6. We remind the reader that there are a total of five such triplets in each respondent's questionnaire.

7. In the Czech Republic, respondents had to be at least 18 years old.

8. Approximately half of the respondents were asked to value reductions in their own risk of dying. The other half were to value risk reductions for one of their children. If a respondent had more than one child within the specified age range, the computer selected a child at random among the eligible ones.

9. We ignore the effect of cause of death in this model and analysis.

Additional information

Notes on contributors

Anna Alberini

This paper was originally prepared for presentation at the International Workshop “Recent Trends in Non Market Valuation,” International Center for Climate Governance, Island of San Giorgio Maggiore, Venice, Italy, November 3–4, 2011.