Nudging is Ineffective When Attitudes Are Unsupportive: An Example from a Natural Field Experiment

Abstract

For security reasons, employees of a Dutch local government department needed to wear an identifying lanyard with their employee badge, but compliance with this policy was low. Two nudges to increase compliance were evaluated in a pre-registered natural field experiment using a pre-post design, and a qualitative survey. Bayesian inference provides insufficient support for the effectiveness of the nudges. While more respondents judged the nudges and the lanyard policy positively than negatively, there was substantial negative judgment and incomprehension for both with some employees finding the nudges paternalistic. We hypothesize that the nudges were ineffective because they failed to change attitudes about the policy, and because they led to reactance among some employees. Implications for nudging within organizations are discussed.

Government employees need to comply with policies for information security to protect sensitive information (e.g., addresses, health records, police operations) and mitigate risks of many sorts (e.g., privacy breaches, loss of trust, information loss). However, employees’ compliance cannot be taken for granted. Indeed, employee compliance and a lack thereof is a major concern for information security (e.g., Hwang et al., 2017; Warkentin & Willison, 2009). Following Weaver (2014), compliance is dependent on three broad and related categories: perceived external incentives (incentives and sanctions, monitoring, enforcement), willingness to comply (information and cognition problems, peer effects, attitude and beliefs problems), and capacity to comply (resource problems, autonomy issues).

For this research, compliance with an information security policy at a Dutch local government department regularly dealing with sensitive information was investigated and we aimed to increase compliance by testing interventions in the field. This policy aimed to help identify unauthorized individuals at the department’s office space. It required employees of that department to wear an identifiable lanyard with their employee badge around their neck when being present at the office space so that unauthorized individuals could be identified based on a missing lanyard. In aiming to increase compliance with that policy, the departmental management approached the Behavioral Insights Group Rotterdam (BIG’R; www.bigrotterdam.nl). BIG’R, like other behavioral insights teams around the world (Afif et al., 2018), enables institutionalized collaborations between behavioral scientists and public servants to pioneer the application of behavioral insights for public policy (John, 2014).

The main barrier to compliance were willingness problems. Conversations with employees (N = 8) revealed that forgetting to wear the lanyard was a relevant cognition problem, that wearing it separated the employees from peers working for other departments who did not need to wear it, and that some employees’ attitudes were unsupportive of the policy. The attitudes were unsupportive because employees did not believe in the effectiveness or necessity of the policy, for instance because they believed that they could recognize all employees even without the lanyard, or that the requirement to scan one’s badge to get access to the department’s office area was sufficient protection. As a result of this requirement, it was practical and often necessary for employees to carry the lanyard with them. Yet, they often had it in a pocket or bag, or held it in their hands instead of wearing it around their neck.

In developing interventions, BIG’R relied on nudging (Thaler & Sunstein, 2008) techniques. A nudge is “any aspect of the choice architecture that alters people’s behavior in a predictable way without forbidding any options or significantly changing their economic incentives” (Thaler & Sunstein, 2008). Nudges often exploit automatic cognitive processes (e.g., biases, heuristics) to stimulate behavior rather than engaging individuals in rational thought. The promises of nudges in the related field of safety compliance have been stressed elsewhere (Lindhout & Reniers, 2017) and reviews generally provide support for nudging techniques (Benartzi et al., 2017; DellaVigna & Linos, 2020; Hummel & Maedche, 2019). There is however little field research that examines nudging within organizations (Chapman et al., in press). In addition, nudging research is in many ways still in its infancy and more evidence from the field is needed to determine when, how, for whom, and to what extent nudging techniques work (e.g., Hummel & Maedche, 2019; Jachimowicz et al., 2019; Kosters & Van der Heijden, 2015; Lin et al., 2017; Marchiori et al., 2017).

We decided to trial multiple nudges to be able to address multiple barriers to compliance (i.e., forgetting and unsupportive attitudes; Weaver, 2015), and because the literature advises to take an integrated approach to compliance (via organizational culture) that could not be achieved with a single nudge alone (DeJoy, 2005; Neal et al., 2000, Sommestad et al., 2014). In an act of participatory research (Reason, 1994), ideas for nudges were brainstormed together with BIG’R employees and employees from the department (N = 6). Subsequently, the authors and BIG’R employees involved in this research discussed and selected the nudges to be tested. In this, we avoided nudges that would upgrade the lanyard (e.g., framing it as a perk) because this was assumed to further strengthen the perceived separation between the employees of the department and their peers from other departments who were not required or allowed to wear the lanyard.

The first nudge, here called head-start nudge after Thaler and Sunstein (2008), was a point-of-decision prompt that served to counteract forgetting to wear the lanyard. Point-of-decision prompts aim to disrupt habitual choices at the moment of making the choice and were hence believed to be well-suited to counteract forgetting. They were shown to affect, for instance, stair use (Soler et al., 2010; Nocon et al., 2010), healthy food choices (Cadario & Chandon, 2020), and hand hygiene (Caris et al., 2018; Weijers & de Koning, 2020). Specifically, stickers with an image of a person wearing the lanyard and text saying “You’re holding it already, now just wear it” were placed at all printers and access points to the department’s office space because there employees needed to scan their badges. The prompt exploited the insight that framing a task as begun but incomplete leads to an increased likelihood of task completion (Barasz et al., 2017; Nunes & Dreze, 2006; Ovsiankina, 1928). We assumed that with that reminder, employees would perceive scanning the badge and wearing it as part of a larger integrated task.

The second nudge, here called norm-awareness nudge, aimed to positively influence social norms because they were found to be an important predictor of compliance with information security policies (Bulgurcu et al., 2010; Guo et al., 2011) and because they are a powerful behavior change technique (Armitage & Conner, 2001; Cialdini & Goldstein, 2004; Miller & Prentice, 2016; Rhodes et al., 2020). Specifically, mirrors with a life-size print of the lanyard on it were placed at the department’s office space. Observing oneself in a mirror is a typical manipulation to increase awareness of oneself and the nudge exploited the insight that awareness of oneself also raises awareness for social norms and has a positive effect on being faithful to those norms (Diener & Wallbom, 1976; Gibbons & Wright, 1983; Hofmann & Heinrichs, 2002; Wicklund, 1979). Placing mirrors together with an image of the lanyard we thus assumed would increase conformity with the injunctive norm to wear the lanyard. Moreover, we assumed that introducing the norm-awareness nudge after the head-start nudge would also increase awareness for a changing descriptive norm (i.e., an increasing number of employees wearing the lanyard as a result of the head-start nudge). Such increasing descriptive norms were recently found to be effective in promoting sustainable behavior (Loschelder et al., 2019; Sparkman & Walton, 2017). Moreover, a similar combination of social norm information and mirrors was shown to have a positive effect on healthy food choices (Niculescu et al., 2016). By capitalizing on social norms this nudge was hypothesized to counteract unsupportive attitudes.

Initially, the two nudges were planned to be complemented by a third nudge exploiting messenger effects (Pornpitakpan, 2004). Specifically, it was planned that new employees of the department would receive the lanyard from the department head during a departmental meeting instead of picking it up from an administrative employee. We assumed that this would signal management commitment which has been identified as a major determinant of safety performance (e.g., Christian et al., 2009; Zohar, 1980) and compliance with information security policies (e.g., Chan et al., 2005; Hu et al., 2012). Importantly, the effect of management commitment on compliance seems to be mediated by employee attitudes (Hu et al., 2012) which led us to hypothesize that this nudge would counteract unsupportive attitudes. However, the department head explained to us during the experiment that this nudge could not be carried out after he had received negative reactions from employees concerning the head-start nudge. The department head wanted to avoid more negative reactions and escalation. Following Weaver (2014), we theorized that the negative reactions were related to autonomy issues and decided to investigate why the nudges led to some negative reactions in Study 2 using a survey among the department’s employees.

The guiding question for this research therefore was to what extent the nudges were able to affect compliance with the lanyard policy and how they were perceived. Approval for this research was obtained from the local ethics committee of the Department of Psychology, Education, and Child Studies at Erasmus University Rotterdam (approval number 19-040) and we obtained informed consent for Study 1 testing the effectiveness of the nudges from the head of the department because we collected data on department level, rather than individual level. For Study 2, we obtained informed consent from the respondents. Study 1 was registered online prior to inspecting the data (https://osf.io/sdtf5).

Study 1

Study 1 served to evaluate the effectiveness of the nudges. For this, we conducted a natural field experiment at the department’s office space and tested the following hypotheses. Note that these hypotheses have been rephrased in comparison to the preregistration document (e.g., deleting references to the messenger nudge) with their meaning unchanged.

H1: More employees will wear the lanyard correctly after implementation of the head-start nudge.

H2: More employees will wear the lanyard correctly after adding the norm-awareness nudge to the head-start nudge.

H3: The removal of all nudges, will not have an effect of the number of employees wearing their lanyard correctly at follow-up.

The department’s office space was a secured area which only employees from that department could enter after scanning their badge. For the experiment, the department was considered the study population and the only participant.

Methods

Context

At the start of the study, 265 employees belonged to the department and had access to the secured area. The employees were on average 44 years old, the majority (n = 171, 65%) was female, and a small subgroup of employees (n = 13, 9%) had a supervisory function. The content of the employees’ work regularly involved handling sensitive and personal information. Not all employees could work at the secured area at a time because only 137 individual workplaces were located there, mostly in one large open office space. No fixed workplaces were assigned within the secured area and if employees could not find a free workplace there, they could search for workplaces in a less secured office area next to the secured area. In this less secured area, employees were not required to wear the lanyard.

Procedure

The procedure is summarized in Figure 1. Measurement 1 (M1) in week 1 (May 6 to 10, 2019) served as a pre-measurement of the proportion of employees wearing their lanyard. In week 2, stickers with the head-start nudge were placed at the two printers and the nine badge scanners at the entries to the secured working area. Starting in week 4, the first post-measurement (M2) was carried out to evaluate the head-start nudge. Because of a bank holiday in week 4, M2 also included the first day of week 5. After M2, four mirrors with the lanyard on it were placed (week 5). In week 7, the second post-measurement (M3) was carried out to measure the incremental effect of the norm-awareness nudge. After M3, both nudges were removed and a follow-up measurement (M4) was conducted in week 14. Note again that the messenger nudge displayed in Figure 1 was not executed for reasons explained above.

Figure 1. Schematic display of the study procedure.

Measures

Compliance with the lanyard policy was measured by two human counters who walked the secured working area on a specified route during standard working hours using handheld tally counters.¹ Before the start of the study, all counters received instructions and training. The first counter, a research assistant conducting all measurements, counted all individuals present and those wearing the lanyard around their necks. Individuals wearing a uniform of support staff (e.g., cleaning personnel) were excluded from being counted. As supervisors could not be recognized by the research assistant, differing municipality employees accompanied the research assistant to count the supervisors present and those wearing the lanyard around their necks. Supervisors thus form a subgroup of the counts of all employees.

Each measurement encompassed ten counting instances during five consecutive working days. For each measurement the counters walked the secured area twice a day with a delay of minimum 2.5 hours and maximum 6 hours. It was not possible to rule out that some employees would be counted multiple times during a single counting instance (i.e., when an employee moved from an area already walked by the counters to an area not yet walked). However, the chance that more than a few employees were counted multiple times was low as the employees mostly conducted sedentary deskwork (hence little movement between workplaces) and counting instances lasted maximum five minutes. In addition, counters had a good oversight over the secured area and could notice when employees moved. Whenever someone had questions about the purpose of the counting, the counters told a cover story about the occupancy rate of the office area. For two counting instances from each measurement, the first author accompanied the two counters acting as a parallel counter to the research assistant. This allowed us to calculate the reliability of the counting method as Krippendorff’s alpha (Krippendorff, 2004) with an excellent value of .98. Krippendorff’s alpha is a common measure in content analysis for the extent of agreement between coders when coding unstructured data or observations.

Analytic strategy

The effectiveness of the nudges was evaluated for all employees including supervisors. We started the analysis with the detection of possible outliers in the compliance percentages. Outliers were empirically predefined as counting instances where the z-score of the percentage was larger than 2.58 (i.e., outside a 95% confidence interval for normally distributed data).

To investigate the effects of the nudges, we relied on Bayesian modeling which is often more flexible compared to standard (frequentist) procedures (Van de Schoot & Depaoli, 2014), thereby allowing us to test all hypotheses within one statistical framework that made use of all statistical information. The aim of this modeling was to generate posterior distributions through MCMC chains that integrated priors with the observed data. These posterior distributions were then used to infer point estimates for modeled parameters and intervals of interest. To do this, we relied on the model structure described in Kruschke (2015, pp. 251–260) and adapted the prior specification in the accompanying R code. We specified vaguely informative priors which meant that the posteriors (and thus our results) were almost completely informed by the observed data and only marginally by prior beliefs. The vaguely informative priors meant that compliance values very close to 0% or 100% were believed to be unlikely as full (in)compliance is rare. We provide a description of the model, the model code, and details concerning initialization of the MCMC chains in the Supplementary material. The primary interest of our analyses were the different proportions of employees complying with the lanyard policy during the four measurements. In the model structure, these proportions were represented by the ω_c parameters with c indicating the measurement (e.g., ω_M1 represents the proportion at measurement 1). They were defined as the modes (i.e., the most likely points) of the posterior distributions describing the compliance rate for measurement c. Subtracting the posterior distributions of compliance rates during different measurements from each other yielded new posterior distributions that described the change in the proportion of employees complying with the policy between measurements. Investigating these new distributions allowed us to inspect the effectiveness of the nudges. Specifically, we investigated the modes of these differences (denoted ω_{c – c’}) as point estimates for the change in compliance rates between measurements, and the 95% highest density intervals (HDI) of these distributions which included 95% of the most credible values around that mode. If the 95% HDIs did not include the null value, a change in compliance between measurements was inferred.

The analysis was conducted using R version 4.0.2 and JAGS. After initial observation of the MCMC chains (van Ravenzwaaij et al., 2018), we specified 1,000 adaptation steps, 200 burn-in steps, and 500,000 saved steps. Convergence of the chains was evaluated by ensuring that the value of the Gelman-Rubin criterion (Gelman & Rubin, 1992) was smaller than 1.01 for all parameters (Vehtari et al., 2020) and by visually inspecting the trace plots for all ω_c parameters. In addition, we ensured that the effective sample sizes (ESS) of the posterior distributions describing the compliance rates per measurement (from which the ω_c parameters were inferred) and the differences between these distributions (from which the ω_{c – c’} parameters were inferred) exceeded 10,000 which has been recommended as a minimum to obtain stable limits of 95% HDIs (Kruschke, 2015). This also applies to the tails of all these posterior distributions which is important to achieve precision for the limits of 95% HDIs (Vehtari et al., 2020).

Results

In total, individual employees were counted 2,246 times during the 40 counting instances. This meant that on average 56.15 (SD = 18.17) employees were counted per counting instance. Inspecting the percentages of employees wearing their lanyard, we found no evidence of outliers. Figure 2 displays the percentages of employees complying with the lanyard policy per measurement. At M1, we measured 47% of employees wearing their lanyard averaged across counting instances. From M1 to M2 there was an increase of six percentage points. Thereafter only subtle changes were observed. For supervisors, there was a step-wise increase of 23 percentage points from M1 to M4.

Figure 2. Percentage of employees wearing their lanyard per measurement (bold) and per counting instance (non-bold). Supervisors form a subgroup of employees. Due to the small number of supervisors per counting instance, only the average over all counting instances is displayed for them.

Turning to inference statistics and the results from Bayesian modeling, the diagnostic criteria indicated that all MCMC chains seemed to have converged. In Table 1, the summary statistics for the ω_c and the ω_{c – c’} parameters are displayed. Note that the modes did not differ from the means of the posterior distributions by more than 0.002. Inspecting the 95% HDI for ω_M2 – ω_M1, we did not find sufficient support of an increase in compliance from M1 to M2 as the interval [–.014, .147] includes the null value. Hence, we do not infer an effect of the head-start nudge. The same held when inspecting the 95% HDI for ω_M3 – ω_M2 as it also includes the null value [–.090, .066]. Consequently, we do not infer an effect of the norm-awareness nudge when added to the head-start nudge. There was also no effect of removing the nudges as the 95% HDI of ω_M4 – ω_M3 did include the null value [–.074, .081]. Note that the conclusions do not change when supervisors are excluded from the analysis.

Table 1. Summary statistics for the ω_c and ω_{c – c’} parameters.

Parameter	Mode	ESS	95% HDI
			low	high
ωM1	.470	189,382	.415	.525
ωM2	.537	238,188	.477	.596
ωM3	.525	271,881	.472	.577
ωM4	.530	309,958	.469	.588
ωM2 – ωM1	.066	181,538	–.014	.147
ωM3 – ωM2	–.012	279,745	–.090	.066
ωM4 – ωM3	.002	310,351	–.074	.081

ESS: effective sample size.

During the experiment, the department head received some negative reactions from employees who were irritated by the head-start nudge because they found it “too much.” In addition, the head-start nudges themselves received some negative reactions, for instance when the image of a model was glued to the sticker to cover the image of the person wearing the lanyard and it seemed like employees had removed some of the nudges (which were then replaced).

Discussion

This study provides insufficient support for the effectiveness of the nudges. There is no support for hypotheses 1 and 2. There is support for hypothesis 3 as the removal of the nudges did not affect compliance. During the experiment, one of the nudges was not executed because of negative reactions from employees concerning the head-start nudge. We could only speculate about the underlying reasons for these reactions. It seemed plausible, however, that autonomy issues (Weaver, 2014) played a role as nudges have been criticized for their paternalistic notion (e.g., Mitchell, 2004), and because the negative reactions could be interpreted as an attempt to restore one’s autonomy according to reactance theory (Brehm, 1966; Brehm & Brehm, 1981). Therefore, we decided to investigate the perception of the nudges in a second study.

Study 2

Study 2 encompassed an exploratory, cross-sectional survey amongst the department’s employees that served to investigate why the nudges led to some negative reactions and why the nudges were ineffective. In combining quantitative and qualitative methods, we endorsed a pragmatist approach to research (Johnson & Onwuegbuzie, 2004; Morgan, 2007).

Methods

Study population

Employees who had worked at the secured working area at least once a week during the experimental period were considered to belong to the study population because they could be assumed to have experienced the nudges. As we do not know for how many employees this requirement was fulfilled, we also do not know the exact size of the study population. In total, 142 employees agreed to participate and answered the survey (54% of all employees of the department). See Study 1 for the demographics of all employees.

Procedure

Seven weeks after the follow-up measurement (M4; see Figure 1), all employees of the department were invited via email by the department head to fill in the online survey. They were given two weeks’ time to fill in the survey, with a reminder sent after one week. Before employees could fill in the survey, they were asked to confirm that they had worked at the secured area where the experiment had been conducted at least once a week during the experimental period.

Measures

Compliance

As a self-reported compliance measure, respondents were asked how often they were wearing the lanyard at that moment on a scale from 1 (never) to 7 (always).

Lanyard policy

Respondents were asked what they thought about the lanyard policy using an open answer format.

Nudges

Respondents were asked separately for both nudges what they thought about them using an open answer format. Respondents were only asked this question for the nudge(s) that they remembered.

Paternalism of the nudges

Respondents were asked separately for both nudges to what extent they found them paternalistic on a scale from 1 (not at all paternalistic) to 7 (totally paternalistic). In addition, respondents were asked to explain their answer(s) using an open answer format, again, separately for both nudges. Respondents were asked these questions after they reported their general thoughts on the nudges and only for the nudge(s) that they remembered.

Counting instances

Respondents were asked using an open answer format what they thought about the counting instances.

Purpose of the counting instances

Using open answer formats, respondents were asked separately what they thought was the purpose of the counting instances, and if the purpose they assumed had changed some time after the first counting instance.

Reasons for negative reactions

Using an open answer format, respondents were asked what they thought were the reasons for some negative reactions to the nudges.

Improving compliance

Using an open answer format, respondents were asked to think of aspects that would increase their motivation to wear the lanyard.

All survey questions are provided in the Supplementary material.

Analytic strategy

The answers to closed questions were analyzed using descriptive statistics and visual displays. The answers to open questions were analyzed using both qualitative coding and descriptive statistics: In a first step, the first author and a municipality researcher independently familiarized with the data and coded the answers from a subset of 15 respondents in Atlas.ti 8 using the constant comparison method (Boeije, 2002). In a second step, those codes and the emerging codebook were discussed and revised. The first author then used the resulting codebook to code the answers from all respondents in a third step and validated ambiguous quotes with the municipality researcher. Later additions to the codebook were made in agreement between the two coders.

The coded answers were used to generate a code-respondent table that showed for each respondent (as rows) if a specific code (as columns) had been applied to the respondent’s answer. Note that this table was based on a code-document table generated in Atlas.ti 8 where answers from each respondent were stored in different documents. Consider a hypothetical example: The answer from respondent K elaborating on a nudge “I found it an eye-catcher” may be assigned the code “salient.” The cell value belonging to respondent K and the “salient” code would then be 1. It would be 0 if the “salient” code had not been applied. Counting the number of respondents that had provided answers related to the “salient” code and comparing it to the total number of respondents would then allow us to examine the prevalence of the code. With the “salient” code being applied to the answers of 15 respondents (i.e., the sum of the values in the “salient” column), the prevalence of this code would be 15/142 = 11%. The prevalence of different codes was compared for all codes belonging to the same measure.

Note that prevalence is a feature of an individual code and that prevalence provides no information on the overall distribution of codes across respondents. As an example, if the codes “salient” and “dull” both were to have a prevalence of 50% this would not imply that half of the respondents provided answers related to the “salient” code and the other half answers related to the “dull” code. In fact, the same half could have provided answers to which both codes were applied. However, our code-respondent table allowed us to investigate the co-occurrence of codes (i.e., whether codes tended to be assigned to the answers from the same or different respondents).

Results

In terms of self-reported compliance, the majority of respondents indicated to always or nearly always wear their lanyard (Figure 3). At the same time, some respondents (15%) reported to never wear their lanyard. Inspecting the prevalence of the codes concerning the lanyard policy, we found that more than one third found the lanyard policy “fine” (Table 2), which in some cases included positive judgements such as “I find it nice.” More than one fourth of the respondents expressed incomprehension concerning the policy. This was mainly because they believed that they could recognize all employees without the lanyard, or that the requirement to scan one’s badge to get access to the office area of the department was sufficient protection. In contrast to that, a group of similar size did express comprehension for the policy and its underlying motivation of being recognizable. Nearly one fifth of the respondents expressed a negative judgment of the policy, for instance they found wearing the lanyard “not pleasant.” 15% of the respondents found that visibly wearing the lanyard in any way, rather than around one’s neck, would suffice. Finally, roughly one out of ten respondents stated that the policy put employees of the department in a separate position compared to their peers from other departments who were not required to wear any identifiable objects.

Figure 3. Answer distributions for self-reported compliance (left) and paternalism ratings of the nudges (right).

Table 2. Codes related to the lanyard policy and the nudges with a minimum prevalence of 10%.

Prevalence	Code	Quotes (translated)
Lanyard policy
35% (50)	Fine	“fine,” “good,” “I find it nice”
27% (39)	Incomprehension	“I don’t find it necessary,” “We already occupy a locked area where it is difficult to enter”
25% (36)	Comprehension	“I have comprehension for it,” “good for recognizability”
18% (26)	Negative	“terrible,” “not pleasant,” “going too far”
15% (22)	Visibly suffices	“Wearing it well visibly seems okay to me as well”
11% (15)	Separate position	“It’s strange that it only applies to us,” “It’s in the way of connecting with other branches”
Nudges
37% (53)	Positive/fine	“fine,” “playful,” “clear,” “fine, good stimulation”
29% (41)	Negative	“irritating,” “a bit too much,” “annoying”
18% (25)	Incomprehension	“unnecessary,” “I think a message about it is already enough”
16% (23)	Paternalistic	“patronizing,” “authoritarian,” “pedantic”
10% (14)	Mirror positive	“I find the mirrors useful though”
Explanation for the paternalism ratings
16% (23)	Incomprehension	“I don’t need a sticker to be reminded to wear it”
14% (20)	Negative	“Too much of a ‘hassle’,” “The message was conveyed too often and to prominently,” “Language was not nice”
13% (19)	Childish	“childish,” “elementary-school-like,” “We are professionals. Treat me accordingly”
11% (16)	Paternalistic	“patronizing,” “authoritarian,” “pedantic”
11% (16)	Unjust treatment	“Seems like you don’t dare to talk to each other anymore,” “Seem like that employees of […] can’t think for themselves anymore”

Turning to the head-start and the norm-awareness nudge, a large majority of respondents remembered to have seen them (84% and 82% respectively). Importantly, the answers from the open-ended questions did not differ much between the two nudges and respondents often copied their answers or referred to answers given concerning the other nudge. We therefore did not differentiate between the two nudges in the analysis.

More than one third of the respondents found the nudges fine or positive. This code has a double label because “fine” (Dutch “prima”) can be interpreted as “satisfactory” as well as “good” and we could, in many cases, not differentiate what specific meaning was intended. Close to one third of the respondents expressed negative judgements of the nudges, sometimes quite strongly. Nearly one fifth showed little or no comprehension for the nudges, for instance stating that they did not need a reminder, or that being informed once concerning the policy suffices. The nudges were found some sort of paternalistic by 18% of the respondents. Note that these answers likely were provided before respondents read the question that asked them to rate the paternalism of the nudges. Inspecting the answer distributions for those ratings, most respondents found the nudges paternalistic to some extent at minimum (Figure 3). Asked to explain their paternalistic ratings, respondents provided answers related to the codes of incomprehension, negative judgment, and paternalistic aspects (Table 2). In addition, more than 10% found the nudges paternalistic because they perceived them as childish and reflecting an unjust treatment of employees.

We report the results for the remaining measures in less detail because fewer respondents answered the related questions. In addition, we judged the results to overlap in part with the results reported above or to confirm earlier research findings, thereby providing little additional insight. For instance, answers about reasons for negative reactions overlapped to a large extent with what many respondents thought of the lanyard policy and the nudges (e.g., many respondents finding the nudges paternalistic using this to explain the negative reactions). In addition, these answers often seemed to be speculative (e.g., many respondents indicated that they were speculating about their peers’ motives). Concerning the counting instances, the answers generally reflected a discomfort of being a research subject, thereby confirming earlier research findings discussed elsewhere (Jones & Whitehead, 2018). 12% of the respondents indicated that they assumed or knew that the purpose of the counting instances was to count employees wearing the lanyard. The most prevalent suggestion for improving compliance was made by 23% of the respondents and was to generate a better understanding for the lanyard policy. Tables with the full results for these measures can be found in the Supplementary material.

Inspecting the co-occurrence of codes, we found that the evaluative judgment of the lanyard policy corresponded with the judgment of the nudges. Respondents who judged the lanyard policy as fine or positively also tended to judge the nudges positively as shown in the Venn diagram in Figure 4. Both codes together were assigned to 27 respondents, which translates to 19% of all respondents. If both codes had been assigned to respondents randomly at the same base rate, one would expect 19 (13%) respondents with both codes assigned together. Moreover, there was little overlap between positive and negative judgements, implying that they form different clusters. In fact, only 19 respondents provided answers to which both positive codes (concerning either the lanyard policy or the nudges) and negative codes (concerning either the lanyard policy or the nudges) were assigned. This translates to 13% of all respondents. Again, if codes had been assigned to respondents randomly at the same base rate, one would expect 28 (20%) respondents to be assigned both positive and negative codes.

Figure 4. Co-occurrence of codes related to the nudges and the lanyard policy.

Discussion

The results show that after the experiment unsupportive attitudes concerning the lanyard policy prevailed among a substantial subgroup of employees. The tendency of employees to judge both the lanyard policy and the nudges negatively seems to imply that negative reactions did not stem from the nudges alone but plausibly from a combination of thinking negatively about both. This reasoning, however, only applies to a subgroup of employees as the most prevalent codes concerning the lanyard policy (and the nudges) were positive or at least neutral.

General discussion

This study aimed to investigate the effectiveness of the head-start nudge and the norm-awareness nudge in increasing compliance with a security policy requiring employees to wear a lanyard with their employee badge attached to it. We found insufficient support for the effectiveness of the nudges (hypotheses 1 and 2). Unfortunately, it is common that behavior change interventions have no effect (Osman et al., 2020). It is therefore important to investigate factors that likely influence effectiveness. In doing so, a survey among the department’s employees revealed that more respondents judged the nudges and the policy positively rather than negatively. Yet, the nudges were judged negatively by a substantial part of the employees and some found them unnecessary or paternalistic. Despite the lower prevalence of negative judgements, the discussion will elaborate more on those because in our opinion they lead to interesting and relevant insights.

We hypothesize that employees holding attitudes that were unsupportive of the lanyard policy limited the effectiveness of the nudges, particularly the norm-awareness nudge. It has been argued that unsupportive attitudes can cause nudges to be ineffective because of the choice preserving nature of nudges (de Wijk et al., 2016; Sunstein, 2017). Individuals then follow their attitudes rather than the nudge. Interviews before the experiment and the survey both show that employees did not believe in the policy’s effectiveness or necessity (code “incomprehension”). The nudges aimed to change attitudes in a subtle way, mostly by raising awareness for social norms. Following norm activation theory (Schwartz, 1977; Yazdanmehr & Wang, 2016) however, the injunctive social norm may not have led to attitude change and associated compliant behavior because defense mechanisms allowed employees to neutralize the obligation to wear the lanyard. Specifically, it may be difficult to comply with the policy when there is a lack of personal consequences (e.g., sanctions, incentives) and the effectiveness or necessity of the lanyard policy were not comprehended. From that perspective, it follows that the nudges cannot be concluded to be ineffective per se but that they were ineffective in the specific context of this study. It remains an open question if the nudges had been more effective if they had been backed up by explanation about the lanyard policy as suggested by some employees, or if more rationality enabling nudges had been employed as sometimes suggested in the literature (John et al., 2009; Tor, 2020).

Similarly, we can only speculate about how the outcomes of the experiment might have differed if the messenger nudge had been executed. The department head stressing his commitment to the lanyard policy, especially if accompanied with explanation for why the policy was needed and how it was assumed to contribute to information security, might have given the lanyard policy and the nudges more credibility and authority, thereby potentially increasing effectiveness of the nudges.

We hypothesize that the negative reactions following the nudges were the result of autonomy issues experienced by some employees (Weaver, 2014). In fact, the employed nudges may have been less choice preserving than what is typically assumed for nudges (Thaler & Sunstein, 2003, 2008) as complying with the policy was the only acceptable response. From this perspective, the nudges were reminders of a mandatory policy and hence shoves rather than subtle features of a choice context. Consistent with this, some of the employees found the nudges paternalistic. The negative reactions as well as not complying can accordingly be interpreted as attempts to restore one’s perceived autonomy whether to wear the lanyard or not (Brehm, 1966; Brehm & Brehm, 1981). This effect was plausibly reinforced by some employees associating the counting instances with the lanyard policy and feeling uncomfortable about being observed. Note however that nudges were found to be effective in other regulated contexts where nudges also might be considered shoves (e.g., Hallsworth et al., 2017; Rogers & Feller, 2018; Wu & Paluck, in press).

It is plausible that some negative reactions could have been prevented if there had been an opportunity for employees to voice their concerns about the lanyard policy in a constructive manner instead (e.g., by stating a contact person on the nudges) as it is also indicated by the survey results: Some employees felt being treated like children (code “childish”) and some employees felt a lack of open communication (code “unjust treatment”). This points to the importance of interactional justice (Colquitt et al., 2005) in organizational settings for nudges to work.

It is not consistent that Dutch public servants who in general tend to embrace the application of behavioral insights, reacted negatively when they themselves were the subject of the application. In the Netherlands, behavioral insights are popular and have been embraced by various government institutions such as the Dutch Scientific Council for Government Policy (Wetenschappelijke Raad voor het Regeringsbeleid, 2014). Somewhat inconsistent with that, negative perceptions of the nudges were of high prevalence among the employees (codes “negative,” “incomprehension,” “paternalistic”). This inconsistency can be intra-personal and inter-personal: The former implies that public servants endorse applying behavioral insights but dislike being the subject of that application and the latter suggests that there may be a divide between public servants who generally approve the application and those who reject it.

Like all research conducted, this study is not without limitations. First, the pre-post design limits the potential to draw any causal conclusions. In the given context, a more rigorous randomized control trial was not possible. Moreover, the context of this research with its specific mix of preexisting preferences concerning the target behavior limits generalizability claims. However, in adopting a case study perspective, this study yields valuable hypotheses to be tested in future research. For instance, that nudges perceived as childish will lead to feelings of reactance and noncompliance.

Another limitation is the delay of 14 weeks between the end of the intervention period and the start of the survey. Although approximately four out of five respondents indicated to remember the nudges, recalling immediate and original reactions to the nudges might have been difficult due to decaying or distorted memories. As a result, answers might have been affected by talks that employees had with other employees during the delay or knowledge concerning nudging they acquired in the meantime.

Another important limitation was that to conduct this research, some employees needed to be informed about it (e.g., the department head). In fact, 12% of the employees indicated to associate the counting instances with the lanyard policy which means that some additional employees had inferred the purpose of the counting instances from the behavior of the counters. We find it unlikely though that this led to socially desirable behavior (i.e., wearing the lanyard) because there were no incentives or sanctions conditioned on the employees’ behavior and because it was unpredictable for employees when counting instances would take place. In fact, we observed that the employees who needed to be informed did not always wear their lanyard either. Note that relatively more supervisors than employees needed to be informed about the experiment and that this might explain the larger increases in compliance among that group.

A strength of this study is its reliance on both quantitative and qualitative methods to evaluate the nudges beyond their effectiveness. The effectiveness measure and the survey represent measurements on different levels and from different perspectives: the former reflects an objective group characteristic and the latter subjective beliefs and judgements of individuals. One should keep in mind however, that individual judgements are conceptually limited when explaining group characteristics without investigating translational processes (e.g., interactions related to the lanyard between employees, differential treatment effects). Nevertheless, the survey enabled us to evaluate the context more thoroughly investigating limiting conditions for nudges to work.

We conclude with practical implications for field experiments involving nudges and suggestions for future research. Concerning the first, we advise researchers as well as practitioners to survey and reflect on the target group’s attitudes and preferences before the start of an experiment testing nudges and we advise to inform the target group as much as possible about the nature of the planned experiment to anticipate potential negative reactions. In the case of unsupportive attitudes researchers may consider rationality enabling nudges or other behavior change techniques. For future research, we suggest more research into nudging within organizations which, as this research exemplifies, are a challenging environment for nudging interventions. It is also of interest how nudging in such contexts affects or hurts relational concepts such as trust and organizational citizenship behavior.

Acknowledgements

We thank the municipality of Rotterdam for this research opportunity. We thank Suzanne van den Berge, Bianca Stam, Lianne Raven, Roos Middelkoop and all anonymous counters for their support in the preparation and execution of this study. In addition, we thank Thomas Hulst, Joran Jongerling, and Bianca Stam for their support with the analysis and Inge Merkelbach for providing feedback for the manuscript.

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Notes

1 Velleman^®, Gavere, Belgium; CNT2.