Evaluating outcome-based payment programmes: challenges for evidence-based policy

ABSTRACT

We review the state of evaluation within outcome-based commissioning in the United Kingdom. This is the first review to include empirical evaluations of both PbR and SIB programmes. We find a paucity of evaluation and that the quality of evaluations is not high. Moreover, studies tend to conflate the outcomes-based commissioning mechanism with the intervention or services that are funded, and are unable to assess the contribution of these separate elements to impact. Our review also highlights the challenges faced by evaluators in measuring social outcomes. We suggest ways to address these challenges.

KEYWORDS:

• Outcome-based commissioning
• payment by results
• social impact bonds
• evaluation
• rapid evidence assessment

JEL CLASSIFICATIONS:

• C93
• O35

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. Policy instruments are techniques through which the State attempts to achieve its goals and that carry specific concepts of the politics/society relationship (Linder and Peters 1990; Lascoumes and Le Galès 2007).

2. http://www.socialfinance.org.uk/database/.

3. http://www.mmuperu.co.uk/assets/uploads/files/Review_of_PbR_and_SIB_evaluation_evidence_METHODS_AND_PAPERS.pdf .

4. Shortly after the review was completed the final impact evaluation for the Peterborough SIB was published (Anders and Dorsett 2017). It is not part of the review.

5. https://data.gov.uk/sib_knowledge_box/node/183.

6. Shortly after the review was completed the final impact evaluation for the Peterborough SIB was published (Anders and Dorsett 2017). It is referred to later in this paper, but was not part of the review.

7. Detail on these findings is included in Albertson et al. (2018).