Re-examining empirical evidence on stated preferences: importance of incentive compatibility

ABSTRACT

The stated preference (SP) methods use respondents’ stated choices made in hypothetical situations to infer their preferences for environmental and other public goods. These methods enable researchers to express the general public's preferences in monetary terms, and hence, to estimate the economic value of a change in the quantity or quality of the goods. However, a key question remains regarding SP methods’ validity: do the value estimates obtained from an SP study reflect respondents’ true preferences? Numerous empirical investigations have tested SP methods’ validity, but overall conclusions are mixed. We critically re-evaluate this evidence considering the issue of the necessary conditions for incentive compatibility of SP surveys. Our analysis shows that once theory-based conditions for incentive compatibility are taken into account, the available studies consistently show that the SP methods provide valid estimates of actual preferences. As a result, we argue that SP surveys must be made incentive compatible in order to observe consumers’ true preferences.

KEYWORDS:

• Stated preferences
• contingent valuation
• discrete choice experiments
• validity
• incentive compatibility

JEL CODES:

• Q51
• H4
• D6

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. Incentive compatibility is a mechanism–design property of an SP survey, which leads to that the optimal strategy for the survey respondent is to answer truthfully.

2. Subsequent advancements developed theoretical conditions under which the sequence of binary choice questions (Vossler, Doyon, and Rondeau 2012) as well as the open-ended and payment card formats can be incentive compatible (Vossler and Holladay 2016). However, as the revised conditions are new and their application requires satisfying several important assumptions, the existing tests of SP methods’ validity typically do not satisfy them. Thereby, we focus on the basic conditions as proposed by Carson and Groves (2007).

3. Obviously, non-incentive compatible surveys can also generate data that reflect true preferences. However, then, researchers have no theory-based expectations whether respondents answer in line with their actual preferences.

4. The crucial role of consequentiality has long been recognised. Hoehn and Randall (1987) emphasised that ‘a key assumption’ underlying the application of SP methods is respondents’ conviction about ‘some influence [of the survey results] on the eventual policy decision’. Instead, many existing SP studies rely on the so-called ‘epsilon truthfulness’ assumption, according to which a respondent who does not perceive any gain or loss from the way the survey is answered gives truthful responses (Rasmusen 1989). As innocuous as it appears, this is a very strong assumption, and the need to avoid it has been long been recognised (Kurz 1974).

5. A noteworthy exception is the requirement of consequentiality, as theoretical and empirical evidence of its importance continues to mount (e.g. Vossler and Evans 2009; Broadbent, Grandy, and Berrens 2010; Herriges et al. 2010; Vossler, Doyon, and Rondeau 2012; Vossler and Watson 2013; Carson, Groves, and List 2014).

6. The weighted dataset treated the mean SP to RP ratio from each study as one observation when the study provided multiple estimates.

7. These results should be treated with caution because the sample included only two studies that applied the hedonic price method.

8. Another point raised in the support of criterion validity tests based on actual market studies is respondents’ familiarity with the good. This, however, is not required by any of the incentive compatibility conditions.

9. Nearly all studies (except for two) referred to in this subsection are based on between-samples comparisons (see Tables A1–A3). Therefore, our discussion does not raise the issue of the role of split vs. within samples differences for the obtained results.

10. Cheap talk provides respondents with additional information before the valuation question. It reminds respondents about the hypothetical survey character, directly discusses the impact of hypothetical bias on self-reported values, and asks respondents to take into account this impact while stating their preferences.

11. Herriges et al. (2010) is a notable example of an out-of-laboratory study conducted in a public good context, contributing to the discussion on the SP methods’ validity, although not being a standard validity test. Instead of comparing consumers’ stated choices with behaviour in an actual payment setting, Herriges et al. (2010) investigated preferences of respondents perceiving an SP survey as consequential and as non-consequential. The authors found that the WTP of respondents who believed that the survey could be consequential was significantly different from respondents who perceived the survey as inconsequential. This contributes to the discussion on the role of consequentiality for truthful preference revelation.

12. Herriges et al. (2010) introduced the term ‘knife-edge result’ to refer to the requirement that for survey's consequentiality, the perceived probability of a survey being consequential needs to be at least marginally larger than zero.

13. In induced-value experiments, a researcher ascribes values to specific experimental actions or results, which are presented to subjects in the instructions (for example, a payoff to all participants if they jointly satisfy a certain condition). These actions or results have no value in and of themselves. In contrast, home-grown value experiments elicit respondents’ personal preferences, which they bring with them to the experiment, and thus, which existed prior to the experiment; these preferences are not induced by the experimental setting.

14. A possible exception is the experiment reported by Johannesson, Liljas, and Johansson (1998), who emphasised that respondents should provide the amount they would pay ‘here and now’ in contrast to the usual ‘would you ever pay...’.

15. Personal perceptions of consequentiality are typically measured through self-reports to a follow-up question regarding how strongly a respondent believes in real consequences resulting from an SP survey.

16. In contrast, Herriges et al. (2010) measured consequentiality perception through self-reports on a five-degree Likert scale; Hwang, Petrolia, and Interis (2014) used a four-degree scale; and Vossler, Doyon, and Rondeau (2012) assessed consequentiality perceptions on a six-degree scale.

17. A referendum setting is more likely to be incentive compatible, as it clearly states the provision rule (usually voting) and is typically linked with a payment mechanism such as a tax increase, which excludes the possibility of free riding.

18. In order to provide a good criterion for validity testing, the SP survey should perfectly mirror the naturally occurring referendum. This includes, among others, the wording used in the survey and the voting population. Obviously, these conditions are often difficult to be met by SP studies which may result in reservations towards their conclusions.

19. Loomis (2014) provides an extensive review of techniques used to minimize or eliminate hypothetical bias along with a discussion of their effectiveness.

20. For example, lab data is usually collected from respondents who are aware of the fact that they are participating in the experiment.

21. The following notation is used (applies to all tables in Appendix): SBQ – a single binary choice question, DBQ – a double-bounded binary choice question, SMQ-certainty – a single multinomial choice question with options representing different levels of payment certainty, B-SEQ – a binary choice sequence, SMC – a single multinomial choice question with options representing different goods/projects, M-SEQ – a multinomial choice sequence, OE – an open-ended question, PC – a payment card, PL – a payment ladder (a variation of PC with ordered values in which a respondent indicates a yes or no answer to each amount), MBQ – a multiple-bounded question (a variation of PC in which a respondent indicates payment certainty to each of monetary values listed).

22. ‘Divergence of SPs from criterion’ expresses the divergence of SP-based value estimates from values used as criterion/reference for comparison, typically obtained from real payment settings. Upward divergence means that an SP-based value estimate exceeds its real-payment counterpart.

Additional information

Funding

This study was supported by the National Science Centre of Poland under the grant Preludium [grant number 2014/15/N/HS4/01328]. Mikołaj Czajkowski gratefully acknowledges the support of the Polish Ministry of Science and Higher Education and the National Science Centre of Poland.