Comparing the effect of oath commitments and cheap talk entreaties in contingent valuation surveys: a randomised field experiment

ABSTRACT

Contingent valuation is a common methodology for eliciting preferences for non-market goods under hypothetical scenarios. Bias reduction strategies have been developed when evaluating low-cost realistic policy changes, including cheap-talk scripts, that alert respondents to tendencies to overstate values, and oath scripts, whereby respondents promise to answer valuation questions truthfully. This paper is the first large-scale experimental comparison of cheap-talk and oath commitments, amongst randomly-assigned respondents, in a field-setting using hypothetical voluntary donations. The data come from three general population surveys eliciting willingness to pay (WTP) for cultural institutions in England. We find limited and case-specific evidence regarding the effectiveness of cheap-talk and oath scripts in affecting stated values, which we attribute to realism and low cost of the proposals, which arguably diminishes hypothetical bias and produces realistic WTP values. We find evidence of the depressing effect of entreaty script on WTP or probability of paying in principle in only one of three case studies. Future research should replicate this experimental design with larger sample sizes and on non-voluntary payment mechanisms. Given the inconsistent findings across three large-scale experimental field studies, our recommendation is to include both cheap-talk and oath scripts where possible, and only cheap-talk where survey length is constrained.

KEYWORDS:

• Contingent valuation
• hypothetical bias
• entreaty script
• cheap talk
• oath script
• willingness to pay

Acknowledgements

We are grateful to colleagues at the UK Arts and Humanities Research Council, Department for Culture Media and Sport, and HM Treasury for their valuable inputs to the research as part of the Measuring Economic Value in Cultural Institutions project.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 It has conversely been argued that truth telling in the lab might be unlikely to be achieved when using voluntary payment mechanisms such as donations given the tendency of individuals to free-ride – and as such hypothetical bias could actually be amplified in lab experiments using real cash donations (Loomis 2014, provides an excellent overview of hypothetical bias).

2 Other causes of overestimation of values include ‘yea-saying’, or ‘response acquiescence’ (Loomis, Traynor, and Brown 1999), defined as the ‘tendency to subordinate outcome-based or “true” economic preferences in favor of expressive motivations when responding to CV questions (Blamey, Bennett, and Morrison 1999). Yea-saying may be potentially motivated by social desirability bias where survey participants respond in the way they think the interviewer wants to hear (this is particularly relevant in face-to-face surveys).

3 In all experimental branches respondents were asked to be realistic, were reminded of their household budgetary constraints, and the existence of alternative cultural institutions that they might wish to spend their money on (Champ and Bishop 2001; Cummings and Taylor 1999). Of all the respondents shown the oath script in branches (iii) and (iv), over 97% on average gave affirmative responses, promising to answer the questions in the survey as truthfully as possible.

4 https://www.nhm.ac.uk/

5 We calculate non-parametric mean and median WTP using the mid-point interval, i.e. the interval between the amount chosen on the card and the next amount up. Open-ended responses reported no values above £500. We did not, therefore, truncate the WTP values at any upper end value. Unanswered open-space responses were coded as missing data.

6 Survey scripts maintained a basic budget reminder, which would already have a constraining effect on WTP, reducing the likelihood of identifying significant difference between entreaty treatments. For this reason, we apply a lower bound for significant testing at the 90% confidence level.

7 Surprisingly, neither Carlsson et al. (2013) nor Jacquemet et al. (2013) found evidence of this effect in their sample, as they observed instead a decrease in numbers of zeros, suggestive of initial protest zero responses later reduced by the entreaty, along the lines of Atkinson et al. (2012).

8 In CS1 and CS2, payment card amounts ranged from £0 to £150. In CS3, payment card amounts ranged from £0 to £75.

9 Sample = conditional WTP (i.e. WTP of the sub-sample that stated they would be willing or maybe willing to pay in principle)

Additional information

Funding

This research was funded by the Arts and Humanities Research Council, and the UK Department for Culture Media and Sport as part of the Cultural Value Project.