http://orcid.org/0000-0001-7467-2318
![Sample our Engineering & Technology journals, sign in here to start your access, latest two full volumes FREE to you for 14 days]({"type":"image" , "src" : "/sda/5933/TOC-Subject-Sample-2021-Engineering-Tech.jpg"})
ABSTRACT
In this paper, we use paradata relating to the length of time respondents required in a self-administered online stated preference surveys. Although this issue has been previously explored, there is little guidance on how to identify and deal with ‘fast’ and ‘slow’ respondents. In this paper, we use scale-adjusted latent class models to address preference and variance heterogeneity and explore how class membership varies with response latency. To test our methodology, we use stated choice data collected via an online survey to establish German anglers’ preferences for fishing site attributes in Denmark. Results from our analysis corroborate that response latency has a bearing on the estimates of utility coefficients and the error variance. Although the results highlight the non-triviality of identifying fast and slow respondents, they signal the need to estimate a large number of candidate models to identify the most appropriate ‘fast’ and ‘slow’ thresholds. Not doing so is likely to lead to an inferior model and has repercussions for marginal willingness to pay estimates and choice predictions.
Acknowledgments
The authors would like to thank the Editor, Ken Willis, and anonymous referees for their helpful comments and suggestions. For data collection, funding from The Danish Ministry of Food, Agriculture and Fisheries is also gratefully acknowledged. The views expressed do not necessarily represent the views of the funder. Furthermore, Carsten Lynge Jensen is thanked for assisting with the data collection. Any errors or misinterpretations are solely the authors’ responsibility.
Disclosure statement
No potential conflict of interest was reported by the authors.
Notes
1. While the relatively easy access to paradata (such as time stamps) also applies to computer-assisted personal interviewing and computer-assisted telephone interviewing survey modes, using online surveys can facilitate the collection of additional paradata (such as keystrokes and mouse clicks).
2. We note that, while we opt for finite distributions of preferences and scale parameters, the models presented here could be extended to continuous representations. However, we favour the appeal of finite distributions since they are not constrained by distributional assumptions, but suggest that this is potentially an interesting extension to this modelling approach.
3. We note that we use paradata relating to the response latency of the panel of choice tasks. Of course, the latency associated with each choice task could instead be used, but in our subsequent scale-adjusted latent class models we are interested in explaining class membership at the panel (i.e. individual) level rather than cross section (i.e. observation) level. We also note that the overall response latency averages out idiosyncrasies unique to each task and is, arguably, a better construct of overall attention (Malhotra Citation2008). Since the time respondents spend on making their choices generally drops as they progress through the experiment (cf. Haaijer, Kamakura, and Wedel Citation2000; Rose and Black Citation2006), this also helps to disentangle the issue from the potential effects of learning and fatigue discussed in Campbell et al. (Citation2015). We are also mindful that our deterministic inclusion of response latency may be considered as a limitation compared to a latent variable approach. However, in this paper we are specifically interested in how class membership differs with response latency (as opposed to a latent variable of survey engagement). For this reason, we choose to include response latency in a deterministic fashion. Readers interested in a latent variable application of response latency are directed to Hess and Stathopoulos (Citation2011).
4. We appreciate that this grid searching is essentially a ‘trial’ and ‘error’ process and it may not be feasible in some settings as it significantly increases runtime. Nevertheless, it does provide a reliable way to determine appropriate ‘fast’ and ‘slow’ thresholds and it permits a comparison of model fit and willingness to pay estimates produced from a large number of possible definitions of ‘fast’ and ‘slow’.
5. Each respondent’s contribution to the MNL LL can be retrieved using .
6. Ideally, the time it takes for the webpage to load should be subtracted from the response latency measure. Unfortunately, exact load times for the specific survey webpage were not measured. Load time would, of course, depend on the survey webpage itself but also on the respondent’s internet connection. Based on the average internet connection speed in Germany of 8 mbps at the time of the data collection (SpeedTests.net Citation2011), we conjecture that our respondents typically experienced load times in the region of 1–2 seconds.
7. Differences are due to rounding.
8. With the estimation of such a large number of candidate models, model selection uncertainty becomes a concern. Given this uncertainty, and the fact that each of our different models provide different relative statistical fits, we admit that it does not seem sensible to ultimately select only one model. Nevertheless, we feel that the comparing of the baseline model to the best fitting model should give the richest and most reliable insight into the differences between ‘fast’, ‘medium’ and ‘slow’ respondents.
9. We note that we report the sample level expected value of marginal willingness to pay based on the unconditional class membership probabilities. Admittedly, reporting only a single value of marginal willingness to pay per attribute is at odds with our evidence of heterogeneity across anglers. However, it allows for a easier and more direct comparison.
10. Although, it should be noted that care is needed when comparing the marginal willingness to pay estimates of the MNL model against those produced from the two scale-adjusted latent class models since marginal willingness to pay is calculated conditional on belonging in classes 1–4 (i.e. the marginal willingness to pay estimates are less sensitive to respondents who chose completely randomly).
11. Once more, we note that this analysis retrieves sample level expected values based on the unconditional class membership probabilities.
12. Estimate based on the predicted 3.1 million overnight stays made by German tourists who engaged in recreational angling when on vacation in Denmark in 2008 (Jensen et al. Citation2010). This number is multiplied with an estimate for the average number of fishing trips per overnight stay as calculated based on supplementary questions in the questionnaire concerning respondents’ number of fishing trips and number of overnight stays during their last vacation in Denmark.