Pre-registration as behaviour: developing an evidence-based intervention specification to increase pre-registration uptake by researchers using the Behaviour Change Wheel

Figures & data

Figure 1. The Behaviour Change Wheel.

Figure 2. The COM-B model of behaviour change.

Table 1. Means and standard deviations for barriers and enablers to pre-registration item responses

Items	n	M	SD
Psychological capability
I am confident I know what pre-registration is (Q12)	101	90.37	15.83
I am confident I know at least one website which offers a pre-registration service (Q13)	99	93.43	19.70
I am confident I know how to select an appropriate pre-registration template (Q14)	105	66.65	30.81
I am confident I know how to specify a study’s hypotheses in advance (Q15)	102	86.35	14.24
I am confident I know how to specify a study’s predictions in advance (Q16)	103	82.45	17.26
I am confident I know how to specify a study’s design in advance (Q17)	104	88.33	13.40
I am confident I know how to specify a study’s inference criteria in advance (Q18)	105	66.34	28.53
I am confident I know how to specify a study’s data stopping rule in advance (Q19)	105	69.18	34.29
I am confident I know how to specify a study’s data inclusion criteria in advance (Q20)	103	87.31	16.03
I am confident I know how to specify a study’s data exclusion criteria in advance (Q21)	102	87.50	15.25
I am confident I know how to pre-register a study (Q22)	105	78.13	27.71
I am confident I know how pre-registration aims to improve research rigour (Q24)	101	92.27	10.90
I am confident I know how pre-registration aims to improve research transparency (Q25)	103	93.72	9.97
I am confident I know how pre-registration aims to improve research credibility (Q26)	103	91.80	12.26
Social opportunity
I am influenced by someone junior to me who encourages pre-registration (Q28)	105	16.87	28.57
I am influenced by someone junior to me who discourages pre-registration (Q29R)	101	1.69	4.18
I am influenced by someone in a similar role to me who encourages pre-registration (Q30)	105	54.70	37.46
I am influenced by someone in a similar role to me who discourages pre-registration (Q31R)	101	4.84	11.58
I am influenced by someone senior to me who encourages pre-registration (Q32)	105	58.06	38.54
I am influenced by someone senior to me who discourages pre-registration (Q33R)	102	14.08	23.73
Physical opportunity
I have the time to pre-register (Q35)	105	67.90	31.96
I have access to the training I would need to pre-register (Q36)	105	58.02	35.14
Reflective motivation
I am incentivised by at least one funder to pre-register (e.g., increased likelihood of funding) (Q37)	105	22.30	31.33
I am incentivised by at least one publisher to pre-register (e.g., increased likelihood of publication) (Q38)	105	35.78	36.47
I am incentivised by my university to pre-register (e.g., increased likelihood of career progression) (Q39)	103	16.23	22.86
I believe pre-registration could stifle my creativity (Q41R)	103	15.17	22.91
I believe pre-registration could make me vulnerable to being scooped (Q42R)	104	18.00	23.61
I believe pre-registration could hamper my ability to make unexpected discoveries (Q43R)	105	19.20	28.26
I believe pre-registration could reduce my chances of being published (Q44R)	105	14.57	24.04
I believe pre-registration could lead to embarrassment, because if a mistake were made, it would become public (Q45R)	102	26.08	27.92
I believe pre-registration could improve research rigour (Q47)	101	87.19	18.60
I believe pre-registration could improve research transparency (Q48)	102	89.95	15.98
I believe pre-registration could improve research credibility (Q49)	102	86.30	20.11

Note. Slider-scales were used to assess barriers and enablers to pre-registration items (0% = “complete and utter disagreement” and 100% = “complete and utter agreement” with the item statement). “R” indicates reverse-scored items.

Table 2. Correlations between barriers and enablers to pre-registration item responses, and past pre-registration behaviour

Items	n	rs
I am confident I know how to pre-register a study (psychological capability) (Q22)	102	0.53***
I believe pre-registration could improve research transparency (reflective motivation) (Q48)	99	0.43***
I am confident I know what pre-registration is (psychological capability) (Q12)	98	0.39***
I believe pre-registration could improve research rigour (reflective motivation) (Q47)	98	0.36***
I believe pre-registration could improve research credibility (reflective motivation) (Q49)	99	0.33***
I am confident I know how to specify a study’s data inclusion criteria in advance (psychological capability) (Q20)	100	0.23*

Note. *p < .05. ***p < .001.

Table 3. Reflexive thematic analysis themes from comment box responses

Themes	Career stage			Total	%
	ECR	MCR	LCR
Psychological capability
Pre-registration requires greater knowledge (barrier)	7	2	0	9	19%
Pre-registration requires determination (enabler)	4	4	0	8	17%
Pre-registration does not prevent exploration (enabler)	4	1	1	6	13%
Pre-registration requires greater skill (barrier)	4	0	0	4	9%
Social opportunity
Pre-registration is encouraged by others (enabler)	5	5	0	10	21%
Pre-registration is discouraged by others (barrier)	2	4	0	6	13%
Physical opportunity
Pre-registration costs time (barrier)	1	5	0	6	13%
Pre-registration saves time (enabler)	2	1	0	3	6%
Pre-registration templates are unsuitable (barrier)	2	1	0	3	6%
Reflective motivation
Pre-registration is not incentivised (barrier)	6	4	1	11	23%
Pre-registration is desirable, despite its potential to reveal mistakes or be embarrassing (enabler)	5	2	0	7	15%
Pre-registration is ineffective (barrier)	3	2	2	7	15%
Pre-registration contributes to enhanced research outcomes (enabler)	5	1	0	6	13%
Pre-registration is incentivised (enabler)	3	1	0	4	9%
Pre-registration remains desirable, despite its potential to reduce publication opportunities (enabler)	2	1	0	3	6%

Note. Percentages express the proportion of respondents who commented on a specific theme, to those who commented (n = 47).

Table 4. Intervention specification

Participants	Researchers employed by an academic institution who collect or analyse data to report research findings.
Design	Pre-post intervention study.
COM-B	Psychological capability; social opportunity; physical opportunity; reflective motivation.
Intervention functions	Education; persuasion; incentivisation; training; modelling.
Policy categories	Communication/marketing; guidelines; fiscal measures; service provision.
Behaviour change techniques	Instruction on how to perform a behaviour (BCT 4.1); information about social and environmental consequences (BCT 5.3); demonstration of the behaviour (BCT 6.1); prompts/cues (BCT 7.1); credible source (BCT 9.1); material reward (BCT 10.2).
Implementation options	Researcher (BCIO:010083); relatedness between person source and the target population (BCIO:010094); expertise of person source (BCIO:010120); face to face (BCIO:011003); at a distance (BCIO:011004); email (BCIO:011025); website (BCIO:011027); university facility (BCIO:026028).
Intervention summary	The intervention will be delivered within an academic institution (university facility) across five stages, with a 3-month gap between each stage to assess the impact of each stage. The intervention will be fully delivered by month 12 but will continue for as long as it is beneficial. Stage 1 (S1) assesses baseline levels of pre-registration. Stages 2–5 (S2-S5) aim to increase pre-registration uptake using four complementary incremental interventions (thus making this a complex intervention; Craig et al., 2008). Emails will be sent to researchers from their various Heads of Department (credible source; relatedness) at the beginning of each stage from S2 onwards to introduce each intervention, as well as to educate (increased psychological capability) them about pre-registration and persuade them to pre-register (e.g., sharing information about the potential for pre-registration to enhance their research as well as contribute to enhanced research outcomes; information about social and environmental consequences; increased reflective motivation).
Stage 1	S1 aims to assess baseline levels of pre-registration. To assess levels, a question will be added to the institutional ethics application form most researchers need to complete prior to commencing their research: “Have you pre-registered this study? If yes, please provide a link”.
Stage 2	S2 aims to increase pre-registration uptake by providing researchers with access to an institutional (credible source) pre-registration website (service provision). Information about the social and environmental consequences of pre-registration will be provided to educate (increased psychological capability) and persuade (increased reflective motivation) researchers to pre-register. Furthermore, instruction on how to pre-register to three incremental standards will be provided (instruction on how to perform a behaviour; increased psychological capability): basic (differentiation between exploratory or confirmatory research), intermediate (basic plus specification of methods), and advanced (intermediate plus specification of analysis strategy; see, Section 3.2.). Thus, researchers may match their pre-registration knowledge and skill, as well as their available time, to an appropriate template (mitigating against insufficient psychological capability and physical opportunity).
Stage 3	S3 aims to increase pre-registration uptake by offering researchers an opportunity to meet face to face or online (at a distance) with a departmental pre-registration champion (credible source; expertise; researcher) to receive training (service provision) on how to pre-register their research (instruction on how to perform a behaviour; increased psychological capability) as well as to demonstrate how studies can be pre-registered (modelling; demonstration of the behaviour; increased social opportunity).
Stage 4	S4 aims to increase pre-registration uptake by updating institutional career progression criteria (guidelines) available online (website) so that researchers are incentivised (fiscal measures; material reward; increased reflective motivation) to pre-register their research and for being a departmental pre-registration champion (see S3).
Stage 5	S5 aims to increase pre-registration uptake by updating the S1 ethics application form question (website) to: “Pre-registering your study could improve the credibility of your findings. To find out more, click here [link to S2 website]. Have you pre-registered this study? If yes, please provide a link” (communication/marketing; education; information about social and environmental consequences; prompts/cues; increased psychological capability).
Evaluation	This intervention will be primarily assessed using the S1 ethics application form question, providing a measure of behaviour (whether studies were pre-registered). However, S2 will be further assessed by recording the specific ways in which the website has been accessed; S3 by recording the number of researchers meeting with each departmental pre-registration champion, as well as the overall total number of meetings held; and S4 the number of career progression applications that include evidence of pre-registration as well as other open research practices (to assess the potential for positive side-effects). Rather than assessing S5, an additional sentence will be added to the updated question in S4: “If no, please select from one of the following options to state your reason for not doing so”. This additional question will be used to refine the intervention during the months following its full delivery.

Note. This intervention specification was produced using BCW (Michie et al., 2014). Italicised words or phrases indicate COM-B components, intervention functions, policy categories, behaviour change techniques, or implementation options.

Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: The new medical research council guidance. BMJ (Clinical Research Ed.), 337:a1655. https://doi.org/10.1136/bmj.a1655

 Google Scholar

Michie, S., Atkins, L., & West, R. (2014). The Behaviour Change Wheel: A guide to designing interventions. Silverback Publishing.

 Google Scholar

Data availability statement

The data described in this article are openly available in the Open Science Framework at https://osf.io/dfcem/.

Data availability statement

All data and materials for this project are available: https://doi.org/10.17605/osf.io/dfcem