Promoting academic honesty: a Bayesian causal analysis of an integrity pilot campaign

ABSTRACT

We examine the effect of an integrity pilot campaign on undergraduates' behavior. As with many costly small-scale experiments and pilot programs, our statistical inference has to rely on small sample size. To tackle this issue, we perform a Bayesian retrospective power analysis. In our setup, a lecturer intentionally makes mistakes that favors students' grades, who decide whether to disclose them or not. We find evidence that at least in the short term, the pilot campaign has a positive impact on the students' disclosure probability.

KEYWORDS:

• Academic integrity
• Bayesian inference
• honesty
• statistical power

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Different effect sizes refer to the values of the chains obtained in the Bayesian estimation, which implies that they are consistent with the observed data.

2 This means a direct observation in a natural setting, where researchers intervene to have some control over the observed events (Shaughnessy and Zechmeister 2012).

3 Notice that the potential number of observations is 96 (4 students × 2 groups × 12 quizzes). Nonetheless, three students (two from the control group and one from the treatment group) did not answer the survey, which was highly encouraged to answer but not compulsory. For this reason, they could not be taken into account for the final sample size.

4 To design the interventions, a literature review was carried on, searching for which kind of actions, activities or experiences elicited acting honestly. The references in Table 1 point out to the article that used as a basis for designing the corresponding interventions.

5 The reason why we do not estimate a binomial model (e.g., probit or logit) is because we are interested on calculating the marginal effects for the representative student, rather than performing prediction. The marginal effects from the normal model akin those of the nonlinear models for the representative individual. Accordingly, and for the sake of parsimony, we pick the normal model. Table 8 lends evidence in regard of how similar are the estimates, regardless of the model.

6 In fact, we have a scaled version of the C-index. The original scale for the calculations ranges from −4 to 4 (in steps of 1) and our questions range from 0 to 9 (also in steps of 1).

7 $\mathcal{N}$ stands for a normal distribution, and $\mathcal{I}\mathcal{G}$ for a inverse gamma distribution.

8 The Gibbs sampler is a Markov chain Monte Carlo (MCMC) algorithm that consists in sampling from the conditional posterior distributions of the parameters recursively in order to obtain the marginal posterior distribution for each parameter. See Supplementary material the Appendix, Section VI.C, for more detail on the Gibbs sampler for our model.

9 The usefulness of such a power analysis is conditional on the fact that the simulated data imitates the actual data (Kruschke 2014). However, this is an implicit assumption one makes when modeling in a parametric framework: that the real data generating process (DGP) is the one assumed.

10 Since the response variable is binary, we simulate the latent variable according to Equation (1(1) $y_{it}={\mathit{x}}_{it}^{\prime }\text{β}+b_i+{ϵ}_{it},$(1) ) and draw the response variable from a Bernoulli distribution by means of a logistic link function. In particular, $y_{it}\sim \mathrm{B}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{o}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{i}\left({\theta }_{it}\right)$ such that $\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{i}\mathrm{t}\left({\theta }_{it}\right)=y_{it}^{\ast }=\log \left(\frac{{\theta }_{it}}{1-{\theta }_{it}}\right)={\mathit{x}}_{it}^{\prime }\text{β}+b_i+{ϵ}_{it}$. As mentioned before, assuming for estimation of a normal DGP instead of a logistic one provides consistent estimates and has no particular drawbacks if one is interested in the marginal effects for the representative agent.

11 The main difference from the Bayesian longitudinal linear model and the other ones, is that it is the only model controlling for unobserved heterogeneity. In every specification we have used the full set of controls.