Masking data: a solution to social desirability bias in paired comparison experiments

Abstract

This paper deals with the reduction of evasive answering bias in paired comparisons studies by providing a masking mechanism to the judges. Specifically, the Warner (1965) masking design is applied in the application of Bradley and Terry (1952) paired comparison model. For estimating the worth parameters and preference probabilities, Bayesian method of estimation is applied. To study the behavior of Bayes estimates and the effect of masking parameter, simulation study is performed. It is observed that the preference ordering is not disturbed when the number of judges is moderate to large. Also, the preference ordering is observed to be robust with respect to the extent of masking, when the number of judges is large. These findings are also supported by a numerical study using real data.

Keywords:

• Masked data
• MCMC
• paired comparisons
• preference probability
• ranking
• randomized response
• social desirability bias