Realist review

Figures & data

Table 1. Common terms and definitions in a realist review.

Term	Definition
Realist review	A type of literature review that seeks to explain why programmes work the way they do, who they work for and in what circumstances they work. It is characterised by its theory led approach and its generative understanding of causation.
Theory-led	A realist review is described as theory-led because it will look at programme outcomes to develop theories to explain the patterns, then consult the literature and stakeholders to test the theories. They will also use formal, well-established theories (like self-determination theory) to inform their theory development.
Generative Causation	The idea that forces which cause something to happen are triggered in particular contexts and may not always be observable
Context	The influential force that determines whether a mechanism is activated or not. There are many different types of context (e.g. social, economic, psychological); and they can operate at different levels (e.g. personal, or institutional)
Mechanism	The real but invisible forces that make the programmes work (or not). A mechanism is not a particular component of the programme but rather, they are the reactions the participants have to the resources the programme offers (e.g. increased confidence [reaction] as a result of a friendly coach [resource])
Outcome	The intended or unintended consequences of a programme.
CMO configuration (CMOc)	Stands for context-mechanism-outcome configuration and is a model to remind researchers of the realist way of explaining things: e.g. this particular context, generates this mechanism, which produces this outcome. It can be used at the beginning of the research project to help make predictions of how a programme might work in certain circumstances and/or at the end as a framework to structure the analysis
Retroductive logic	A way of making sense of things by identifying hidden causal forces to explain why observable patterns in behaviour/events come about. Unlike abduction, deduction and induction that explain things by using observations, retroduction seeks to find explanations by examining things that are unseen (e.g. love, fear, empathy).
Abduction	Taking an observation or set of observations and making logical assumptions to explain them. Could be considered a form of ‘reading between the lines’
Deduction	Creating ideas on how behaviour/events might work and testing these ideas in the real world to see if they are true. Could be considered a top-down approach to reasoning
Induction	Making sense of things by observing behaviour or events. Could be considered a bottom-up approach to reasoning
Initial Rough Programme Theory (IRPT)	In a realist review these are ideas created at the beginning of a research project that attempt to explain how and why the programme(s) of interest is thought to work by describing the context, mechanism, and outcomes at play
Refined Programme Theory (RPT)	In a realist review, these are the ideas that have been produced after the initial rough programme theories have been tested against the literature and in consultation with stakeholders. In this review process, initial rough programme theories may have been proven wrong or some parts may have been well-supported and the initial rough programme theories are adjusted accordingly to produce refined programme theory.
Middle Range Theory (MRT)	Theories about the context(s) and mechanism(s) that bring about particular programme outcome(s). MRTs are more general than programme theory which means they can be used to guide the implementation of a similar programme in a different setting.
Substantive Theory	Theories that have been well-established in a particular field that help to explain certain behaviours or outcomes. Examples of substantive theory in the sports and exercise psychology domain would be Need Achievement theory; Attribution theory; Deliberate Practise theory.

Table 2. Differences between a traditional systematic review and a realist review.

Stages	Traditional systematic review	Realist review
1. Question	Identify the review question – usually with the aim of assessing the effectiveness of well-defined interventions.	Identify the review question – usually with the aim of identifying programme theory around what works, for whom and in what circumstances
2. Search	Extensive and comprehensive search for primary studies within bibliographic databases using strings of relevant terms found in title/abstracts/keywords to identify suitable papers. Inclusion/exclusion criteria pre-specified ahead of the literature search.	Search for relevant evidence, including empirical evidence found out with the usual academic databases e.g. policy papers, grey literature etc. Less emphasis placed on ‘key words’ to guide search, more focus on identifying context, mechanisms and outcomes to build theory. Inclusion/exclusion criteria refined in light of the emerging data and as programme theories are refined.
	Usually a single, exhaustive literature search performed at the beginning of the review	Complementary searches performed iteratively throughout the review
3. Appraisal	Studies are appraised using a predefined and validated critical appraisal checklist. Relevance to research question and methodological rigour are main considerations	Studies are appraised using the researcher’s judgement to supplement formal checklists. Data is included based on its relevance, rigour, and richness to make a judgement on whether it is ‘fit for purpose’
4. Extraction	Data is extracted from all primary studies using a standardised template or matrix	An eclectic and iterative approach to data extraction. Bespoke templates are created and piloted by the research team to extract different data from different studies
5. Synthesis	Data is synthesised with the aim of finding effect size and confidence interval. Transferable themes from qualitative studies may also be included	Data is synthesised to refine programme theory looking to create context, mechanism, and outcome configurations to determine what works for whom, how and under what circumstances.
6. Recommendations	Recommendations are made on the basis of whether the results are definitive or whether further research is needed.	Recommendations are made with programme implementers in mind referring to particular aspects of the programme implementation that need to be considered (e.g. pitfalls to avoid, timing of certain components).
7. Dissemination	Findings are disseminated and evaluated based on the extent to which a change is brought about.	Findings are disseminated and evaluated based on the extent to which the existing programme is changed based on the elements of programme theory identified by the review.

Note: Adapted from Pawson et al. (2004, p. 14) and Booth et al. (2018, p. 150).

Table 3. Six stages of realist review.

Stages of realist review
1	Identifying the review question Map the research area Informal literature search Concept mining Develop initial rough programme theories informed by substantive theory Consult stakeholders Build hypothetical model of key initial rough programme theories to test in the literature
2	Searching for primary studies Database searches Berry Picking Citation tracking Snowballing Kinship & sibling studies Grey literature Contact authors
3	Quality Appraisal Bespoke screening and appraisal tools Consider rigour and relevance
4	Extracting the data Bespoke extraction templates Look for rich description and thick detail
5	Synthesising the data Juxtaposing Adjudicating Reconciling Consolidating Situating
6	Disseminating the findings Consult with stakeholders for widest impact Recommendations based on middle range theories around which particular aspects of programme implementation need to be considered

Note: Adapted from Pawson et al. (2004, p. iv); Pawson (2006a, p.103) and with reference to Booth et al. (2020).

Table 4. Four explanatory paths to refine the purpose of a realist review.

Explanatory path	Review focus	Hypothetical example
Reviewing for programme theory integrity	Examines how a programme fairs as it passes through various stages of implementation. Did it stay on course? Did it get diverted or blocked? This review identifies potential weak spots or stumbling blocks in programme implementation.	A programme was introduced whereby coaches gave dedicated time and tailored support to young athletes with low self-esteem. However, in school districts with endemic under resourcing and poor staff retention there was a failure to recruit sufficient coaches.
Reviewing to adjudicate between rival programme theories	Examines the evidence and judges which, out of several potential theories, helps further an understanding of how the programme works.	In a young athlete coaching programme, the theory around ‘enhanced self-worth’ was compared to a theory focusing on ‘peer pressure’. Collected evidence suggested that coaches who enhanced athletes self-worth improved participation and performance whereas peer pressure was less of a factor.
Reviewing the same theory in comparative settings	Examines how a programme theory fairs in different settings	Research evaluated a young athlete coaching programme which was theorised to improve self-worth by comparing how it faired in a youth correction facility as opposed to a public-school setting.
Reviewing official expectations against actual practice	Compares the ‘official’ intervention theory to what happens when it is implemented in ‘real life’.	A young athlete coaching programme was developed by Sports U.K. Their policy paper claimed that, in providing funding for additional specialised coaches, adolescents who have traditionally not involved themselves in sports will be given the one-to-one attention they need to encourage participation. However, in reality, the funding was used to upskill existing coaches and not add additional coaches. This meant that there were too few coaches to provide the one-to-one support the programme theory was based on.

Note: Adapted from Pawson et al. (2004, p. 15 and pp. 25–26).

Table 5. Activities involved in the process of synthesising data from diverse sources.

Activity	Definition
Juxtaposition	For example, where evidence about high anxiety levels in one paper leads to insights about behaviour change outcomes in another article
Reconciling	Developing possible explanations to explain different programme outcomes in comparable settings
Adjudication	Determining the methodological strengths and weaknesses of pieces of evidence
Consolidation	When a number of sources suggest agreement about mechanisms and outcomes, leading to rich causal explanations
Situating	Developing possible explanations to explain different programme outcomes in particular settings

Note: Adapted from Brennan et al. (2014) and Pearson et al. (2012).

Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004). Realist synthesis: An introduction (pp. 1–55). Manchester: ESRC Research Methods Programme, University of Manchester.

 Google Scholar

Booth, A., Wright, J., & Briscoe, S. (2018). Scoping and searching to support realist approaches. In N. Emmel, J. Greenhalgh, A. Manzano, M. Monaghan, & S. Dalkin (Eds.), Doing realist research (pp. 147–165). Sage.

 Google Scholar

Pawson, R. (2006a). Evidence-based policy: A realist perspective. Sage.

 Google Scholar

Booth, A., Briscoe, S., & Wright, J. M. (2020). The “realist search”: A systematic scoping review of current practice and reporting. Research Synthesis Methods, 11(1), 14–35. https://doi.org/https://doi.org/10.1002/jrsm.1386

 PubMed Web of Science ®Google Scholar

Brennan, N., Bryce, M., Pearson, M., Wong, G., Cooper, C., & Archer, J. (2014). Understanding how appraisal of doctors produces its effects: A realist review protocol. BMJ Open, 4(6), e005466–e005466. https://doi.org/https://doi.org/10.1136/bmjopen-2014-005466

 PubMed Web of Science ®Google Scholar

Pearson, M., Chilton, R., Woods, H. B., Wyatt, K., Ford, T., Abraham, C., & Anderson, R. (2012). Implementing health promotion in schools: Protocol for a realist systematic review of research and experience in the United Kingdom (UK). Systematic Reviews, 1(1), 1–7. https://doi.org/https://doi.org/10.1186/2046-4053-1-48

 PubMedGoogle Scholar

Data availability statement

The authors confirm that the data supporting the findings of this study are available within the article [and/or] its supplementary materials.