The importance and efficacy of controlling for social desirability response bias

ABSTRACT

The extant literature acknowledges social desirability response bias (SDRB) is a pervasive issue for research that uses survey data and proposes several approaches to mitigating the issue, including: self-administered questionnaires, indirect questioning, and direct measurement. The objective of this study is to provide empirical evidence related to the importance of controlling for SDRB and the efficacy of these approaches. Using a primary sample of 365 business majors, we find the most common methodologies used to control for SDRB, self-administered questionnaires and indirect questions, are insufficient. A direct measure of SDRB remains significant when these approaches are used, and the exclusion of this measure affects the significance of other variables. Less than 5% of accounting-and-ethics research using survey data includes a direct control for SDRB. Our study provides empirical evidence on a methodological issue addressed in few existing studies.

KEYWORDS:

• Social desirability response bias
• international studies

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

¹ Recent studies examining SDRB utilize participants from a single nation (Dalton & Ortegren, 2011; S.R. Valentine & Hollingworth, 2012), but personality characteristics differ across nationalities (Triki et al., 2017).

² While the focus of this study is accounting ethics research, this is not the only subset of the literature where it is important to control for SDRB. For example, research related to organizational behavior and information systems has documented the effects of SDRB (Constant et al., 1994; Robertson & Anderson, 1993), and Bernardi (2006, p. 49) concluded that SDRB “should be controlled for in research – especially in international studies.”

³ Bäckström et al. (2009) suggest that in addition to indirect wording, SDRB can be mitigated by altering the semantic content of items. By systematically making items less attractive without changing their meaning, authors can eliminate variation related to SDRB. Because this method is less common in the accounting and ethics literature, we do not test efficacy.

⁴ The “halo effect” is defined as the difference between the first- and third-person responses to an ethics question. The positive correlation between SDRB and the “halo effect” suggests indirect questions are better at controlling for SDRB than direct questions. Critically, this correlation does not imply indirect questions fully mitigate the effects of SDRB.

⁵ While Bernardi and LaCross (2004) and Bernardi et al.’s (2011) students were from the United States, Bernardi and Adamaitis (2007) students came from Australia, China, Ireland, Japan and the United States.

⁶ To ensure any differences between U.S. and Indonesian participants aren’t attributable to temporal differences, we obtained responses from a second group of participants in the U.S. at the same time we obtained responses from Indonesian participants. The U.S. participants in the secondary sample are from a different university than the one used to obtain the primary sample.

⁷ A number of the questions require students to have at least some working experience for the questions to even make sense. Given the option (or requirement in some majors) for the student to participate in an internship while in college, we believe that the students from both countries had sufficient experience to accurately respond to these questions.

⁸ While the Paulhus (1991) Impression Management Subscale is used in accounting research, because attitudes and perspectives of business professionals differ from the general public, biases and the measures used to capture bias may not be as effective across populations (Jidin & Monroe, 2017). It is therefore important to note that in other domains, the Paulhus instrument’s validity has been questioned. For example, while many authors have concluded personality judgments can be influenced by perceiver biases (Leising et al., 2021), others have found either limited effects (Paunonen & LeBel, 2012; Piedmont et al., 2000) or that the inclusion of a direct control for SDRB reduced the validity of the model (McCrae & Costa, 1983). These results have led authors to suggest that in certain settings the effects of SDRB are too specific to justify the routine use of SDRB controls (Borkenau & Ostendorf, 1992) and led others to develop alternative measures (Leising et al., 2016). Future research might examine whether these other measures are superior in the business domain.

⁹ Because the responses are reverse coded, a socially desirable response to odd statements is one or two and a socially desirable response to even statements is six or seven. The number of socially desirable responses are identified and then tallied for each participant, resulting in a score that can range from zero to 20.

¹⁰ Lalwani et al. (2006) suggest that different cultures and different genders may score high in one type of SDRB but low on another type. This limitation offers an opportunity for future research that could examine which type(s) of SDRB provides the best control for socially desirable responding.