Based on outcomes? Challenges and (missed) opportunities of measuring social outcomes in outcome-based contracting

References

• Albertson, K., C. Fox, C. O’Leary, and G. Painter. 2020. “Towards a Theoretical Framework for Social Impact Bonds.” Nonprofit Policy Forum 11(2):1–10. doi: 10.1515/npf-2019-0056.
   Web of Science ®Google Scholar
• Arena, M., I. Bengo, M. Calderini, and V. Chiodo. 2016. “Social Impact Bonds: Blockbuster or Flash in a Pan?” International Journal of Public Administration 39(12):927–39. doi: 10.1080/01900692.2015.1057852.
   Web of Science ®Google Scholar
• Battye, F. 2015. “Payment by Results in the UK: Progress to Date and Future Directions for Evaluation.” Evaluation 21(2):189–203. doi: 10.1177/1356389015577464.
   Web of Science ®Google Scholar
• Battye, F., and J. Sunderland. 2011. Thinking About … Payment by Results. London: GHK.
   Google Scholar
• Bengo, I., M. Arena, G. Azzone, and M. Calderini. 2016. "Indicators and Metrics for Social Business: A Review of Current Approaches." Journal of Social Entrepreneurship 7(1):1–24.
   Web of Science ®Google Scholar
• Berndt, C., and M. Wirth. 2018. “Market, Metrics, Morals: The Social Impact Bond as an Emerging Social Policy Instrument.” Geoforum 90(January):27–35. doi: 10.1016/j.geoforum.2018.01.019.
   Google Scholar
• Boyle, D. 2011. “The Pitfalls and Perils of Payment by Results.” Local Economy: The Journal of the Local Economy Policy Unit 26(8):627–34. doi: 10.1177/0269094211422203.
   Google Scholar
• Broccardo, E., M. Mazzuca, and M. L. Frigotto. 2020. “Social Impact Bonds: The Evolution of Research and a Review of the Academic Literature.” Corporate Social Responsibility and Environmental Management 27(3):1316–32. doi: 10.1002/csr.1886.
   Web of Science ®Google Scholar
• Buckingham, H. 2009. “Competition and Contracts in the Voluntary Sector: Exploring the Implications for Homelessness Service Providers in Southampton.” Policy & Politics 37(2):235–54. doi: 10.1332/030557309X441045.
   Web of Science ®Google Scholar
• Buckingham, H. 2012. “Capturing Diversity: A Typology of Third Sector Organisations’ Responses to Contracting Based on Empirical Evidence from Homelessness Services.” Journal of Social Policy 41(3):569–89. doi: 10.1017/S0047279412000086.
   Web of Science ®Google Scholar
• Carè, R., and R. De Lisa. 2019. “Social Impact Bonds for a Sustainable Welfare State: The Role of Enabling Factors.” Sustainability (Switzerland) 11(10):2884. doi: 10.3390/su11102884.
   Web of Science ®Google Scholar
• Carter, E. 2021. “More than Marketised? Exploring the Governance and Accountability Mechanisms at Play in Social Impact Bonds.” Journal of Economic Policy Reform 24(1):78–94. doi: 10.1080/17487870.2019.1575736.
   Web of Science ®Google Scholar
• Carter, E., and A. Whitworth. 2015. “Creaming and Parking in Quasi-Marketised Welfare-to-Work Schemes: Designed out of or Designed in to the UK Work Programme?” Journal of Social Policy 44(2):277–96. doi: 10.1017/S0047279414000841.
   PubMed Web of Science ®Google Scholar
• Chen, S., and R. Harrison. 2020. "Beyond Profit vs. Purpose: Transactional-Relational Practices in Impact Investing." Journal of Business Venturing Insights 14(2020):e00182:1–8. doi: 10.1016/j.jbvi.2020.e00182.
   Google Scholar
• Clist, P. 2019. “Payment by Results in International Development: Evidence from the First Decade.” Development Policy Review 37(6):719–34. doi: 10.1111/dpr.12405.
   Web of Science ®Google Scholar
• Commissioning Better Outcomes and Social Outcomes Fund. 2019. “In-Depth Review: Mental Health and Employment Partnership (MHEP) Social Impact Bond”, 1–21. https://www.tnlcommunityfund.org.uk/media/insights/documents/CBO_In-Depth_Reviews_MHEP_Visit_1_FINAL.pdf?mtime=20191120173907&focal=none
   Google Scholar
• Considine, M., S. O’Sullivan, and P. Nguyen. 2018. “The Policymaker’s Dilemma: The Risks and Benefits of a ‘Black Box’ Approach to Commissioning Active Labour Market Programmes.” Social Policy & Administration 52(1):229–51. doi: 10.1111/spol.12309.
   Web of Science ®Google Scholar
• Dayson, C., A. Fraser, and T. Lowe. 2020. “A Comparative Analysis of Social Impact Bond and Conventional Financing Approaches to Health Service Commissioning in England: The Case of Social Prescribing.” Journal of Comparative Policy Analysis: Research and Practice 22(2):153–69. doi: 10.1080/13876988.2019.1643614.
   Web of Science ®Google Scholar
• De Pieri, B., and S. Teasdale. 2021. “Radical Futures? Exploring the Policy Relevance of Social Innovation.” Social Enterprise Journal 17(1):94–110. doi: 10.1108/SEJ-11-2020-0111.
   Web of Science ®Google Scholar
• Dey, C., and J. Gibbon. 2018. “New Development: Private Finance over Public Good? Questioning the Value of Impact Bonds.” Public Money & Management 38(5):375–8. doi: 10.1080/09540962.2018.1477676.
   Web of Science ®Google Scholar
• Dixon, R. 2021. “Performance Management in Social Impact Bonds: How an Outcomes-Based Approach Shapes Hybrid Partnerships.” International Journal of Public Sector Management 34(3):356–76. doi: 10.1108/IJPSM-03-2020-0070.
   Web of Science ®Google Scholar
• Dufour, B. 2019. “Social Impact Measurement: What Can Impact Investment Practices and the Policy Evaluation Paradigm Learn from Each Other?” Research in International Business and Finance 47:18–30. doi: 10.1016/j.ribaf.2018.02.003.
   Web of Science ®Google Scholar
• Edmiston, D., and Alex. Nicholls. 2018. “Social Impact Bonds: The Role of Private Capital in Outcome-Based Commissioning.” Journal of Social Policy 47(1):57–76. doi: 10.1017/S0047279417000125.
   Web of Science ®Google Scholar
• Fischer, R. L., and F. G. C. Richter. 2017. “SROI in the Pay for Success Context: Are They at Odds?” Evaluation and Program Planning 64:105–9. doi: 10.1016/j.evalprogplan.2016.11.012.
   PubMed Web of Science ®Google Scholar
• FitzGerald, C., E. Carter, R. Dixon, and M. Airoldi. 2019. “Walking the Contractual Tightrope: A Transaction Cost Economics Perspective on Social Impact Bonds.” Public Money & Management 39(7):458–67. doi: 10.1080/09540962.2019.1583889.
   Web of Science ®Google Scholar
• FitzGerald, C., A. Fraser, and J. Kimmitt. 2020. “Tackling the Big Questions in Social Impact Bond Research through Comparative Approaches.” Journal of Comparative Policy Analysis: Research and Practice 22(2):85–99. doi: 10.1080/13876988.2020.1726177.
   Web of Science ®Google Scholar
• Fox, C., and K. Albertson. 2011. “Payment by Results and Social Impact Bonds in the Criminal Justice Sector: New Challenges for the Concept of Evidence-Based Policy?” Criminology & Criminal Justice 11(5):395–413. doi: 10.1177/1748895811415580.
   Web of Science ®Google Scholar
• Fox, C., and K. Albertson. 2012. “Is Payment by Results the Most Efficient Way to Address the Challenges Faced by the Criminal Justice Sector?” Probation Journal 59(4):355–73. doi: 10.1177/0264550512458473.
   Google Scholar
• Fox, C., and S. Morris. 2021. “Evaluating Outcome-Based Payment Programmes: Challenges for Evidence-Based Policy.” Journal of Economic Policy Reform 24(1):61–77. doi: 10.1080/17487870.2019.1575217.
   Web of Science ®Google Scholar
• Fraser, A., S. Tan, A. Boaz, and N. Mays. 2020. “Backing What Works? Social Impact Bonds and Evidence-Informed Policy and Practice.” Public Money & Management 40(3):195–204. doi: 10.1080/09540962.2020.1714303.
   Web of Science ®Google Scholar
• Fraser, A., S. Tan, M. Lagarde, and N. Mays. 2018. “Narratives of Promise, Narratives of Caution: A Review of the Literature on Social Impact Bonds.” Social Policy & Administration 52(1):4–28. doi: 10.1111/spol.12260.
   Web of Science ®Google Scholar
• Gadenne, V., M. Maglicic, D. Nolan, H. Wright, and J. Frerichs. 2020. “Individual Placement Support: A Social Impact Bond Model.” The Behavioural Insights Team. https://www.tnlcommunityfund.org.uk/media/insights/documents/Social-Impact-Bond-IPS-Research-Report-Final.pdf?mtime=20200717173854&focal=none
   Google Scholar
• Gehman, J., V. L. Glaser, K. M. Eisenhardt, D. Gioia, A. Langley, and K. G. Corley. 2018. “Finding Theory–Method Fit: A Comparison of Three Qualitative Approaches to Theory Building.” Journal of Management Inquiry 27(3):284–300. doi: 10.1177/1056492617706029.
   Web of Science ®Google Scholar
• Gioia, D. A., K. G. Corley, and A. L. Hamilton. 2013. “Seeking Qualitative Rigor in Inductive Research: Notes on the Gioia Methodology.” Organizational Research Methods 16(1):15–31. doi: 10.1177/1094428112452151.
   Web of Science ®Google Scholar
• Gosling, H. 2016. “‘All This Is about Is Money and Making Sure That Heads Are on Beds’: Perceptions of Payment by Results in a Therapeutic Community.” Probation Journal 63(2):144–52. doi: 10.1177/0264550516637241.
   Google Scholar
• Hargreaves, C., P. Hodgson, J. N. Mohamed, and A. Nunn. 2019. “Contingent Coping? Renegotiating ‘Fast’ Disciplinary Social Policy at Street Level: Implementing the UK Troubled Families Programme.” Critical Social Policy 39(2):289–308. doi: 10.1177/0261018318780094.
   Web of Science ®Google Scholar
• Heath, A. W. 1997. “The Proposal in Qualitative Research.” The Qualitative Report 3(1):147–51. doi: 10.46743/2160-3715/1997.2026.
   Google Scholar
• Hevenstone, D., and M. von Bergen. 2020. “Public–Private Partnerships in Social Impact Bonds: Facilitating Competition or Hindering Transparency?” Public Money and Management. 30(3):205–12.
   Web of Science ®Google Scholar
• Hill, J. G. 2013. “The Marketization of Employment Services and the British Work Programme.” Competition & Change 17(2):197–207. doi: 10.1179/1024529413Z.00000000033.
   Google Scholar
• Howard, G., and Z. White. 2020. “Does Payment by Results Work? Lessons from a Multi-Country Wash Programme.” Journal of Water, Sanitation and Hygiene for Development 10(4):716–23. doi: 10.2166/washdev.2020.039.
   Web of Science ®Google Scholar
• Huckfield, L. 2020. “The Mythology of the Social Impact Bond. A Critical Assessment from a Concerned Observer.” Historical Social Research 45(3):161–83.
   Web of Science ®Google Scholar
• Jackson, E. T. 2013. “Evaluating Social Impact Bonds: Questions, Challenges, Innovations, and Possibilities in Measuring Outcomes in Impact Investing.” Community Development 44(5):608–16. doi: 10.1080/15575330.2013.854258.
   Google Scholar
• Jamieson, D., R. Wilson, M. Martin, T. Lowe, J. Kimmitt, J. Gibbon, and M. French. 2020. “Data for Outcome Payments or Information for Care? A Sociotechnical Analysis of the Management Information System in the Implementation of a Social Impact Bond.” Public Money & Management 40(3):213–24. doi: 10.1080/09540962.2020.1714306.
   Web of Science ®Google Scholar
• Joy, M., and J. Shields. 2013. “Social Impact Bonds: The Next Phase of Third Sector Marketization?” Canadian Journal of Nonprofit and Social Economy Research 4(2):39–55. doi: 10.22230/cjnser.2013v4n2a148.
   Google Scholar
• Joy, M., and J. Shields. 2018. "Austerity In The Making: Reconfiguring Social Policy Through Social Impact Bonds." Policy & Politics 46(4):681–95. doi: 10.1332/030557318X15200933925397.
   Web of Science ®Google Scholar
• Klimavičiūtė, L., V. Chiodo, B. De Pieri, and V. Gineikytė. 2021. Study on the Benefits of Using Social Outcome Contracting in the Provision of Social Services and Interventions—A Cross-Country Comparative Assessment of Evolving Good Practice in Cross-Sectoral Partnerships for Public Value Creation. Brussels: European Commission. https://echttp://europa.eu/social/main.jsp?catId=738&langId=it&pubId=8384&furtherPubs=yes.
   Google Scholar
• Loeffler, E., and T. Bovaird. 2020. “Assessing the Impact of Co-Production on Pathways to Outcomes in Public Services: The Case of Policing and Criminal Justice.” International Public Management Journal 23(2):205–23. doi: 10.1080/10967494.2019.1668895.
   Web of Science ®Google Scholar
• Lowe, T., and R. Wilson. 2017. “Playing the Game of Outcomes-Based Performance Management. Is Gamesmanship Inevitable? Evidence from Theory and Practice.” Social Policy & Administration 51(7):981–1001. doi: 10.1111/spol.12205.
   Web of Science ®Google Scholar
• Maier, F., G. P. Barbetta, and F. Godina. 2018. “Paradoxes of Social Impact Bonds.” Social Policy & Administration 52(7):1332–53. doi: 10.1111/spol.12343.
   Web of Science ®Google Scholar
• Maier, F., and M. Meyer. 2017. “Social Impact Bonds and the Perils of Aligned Interests.” Administrative Sciences 7(3):24. doi: 10.3390/admsci7030024.
   Web of Science ®Google Scholar
• Maynard, A., A. Street, and R. Hunter. 2011. “Using ‘Payment by Results’ to Fund the Treatment of Dependent Drug Users-Proceed with Care!.” Addiction (Abingdon, England) 106(10):1725–9. doi: 10.1111/j.1360-0443.2011.03503.x.
   PubMed Web of Science ®Google Scholar
• McHugh, N., S. Sinclair, M. Roy, L. Huckfield, and C. Donaldson. 2013. “Social Impact Bonds: A Wolf in Sheep’s Clothing?” Journal of Poverty and Social Justice 21(3):247–57. doi: 10.1332/204674313X13812372137921.
   Google Scholar
• Nazari Chamaki, F., G. P. Jenkins, and M. Hashemi. 2019. “Social Impact Bonds: Implementation, Evaluation, and Monitoring.” International Journal of Public Administration 42(4):289–97. doi: 10.1080/01900692.2018.1433206.
   Web of Science ®Google Scholar
• O’Flynn, P., and C. Barnett. 2017. “Evaluation and Impact Investing: A Review of Methodologies to Assess Social Impact.” IDS Evidence Reports no. 222, 1–31.
   Google Scholar
• Rania, F., A. Trotta, R. Carè, M. C. Migliazza, and A. Kabli. 2020. “Social Uncertainty Evaluation of Social Impact Bonds: A Model and Practical Application.” Sustainability (Switzerland) 12(9):3854–34. doi: 10.3390/su12093854.
   Web of Science ®Google Scholar
• Reeder, N., A. Colantonio, J. Loder, and G. Rocyn Jones. 2015. “Measuring Impact in Impact Investing: An Analysis of the Predominant Strength That Is Also Its Greatest Weakness.” Journal of Sustainable Finance and Investment 5(3):136–54. doi: 10.1080/20430795.2015.1063977.
   Google Scholar
• Rees, J., A. Whitworth, and E. Carter. 2014. “Support for All in the UK Work Programme? Differential Payments, Same Old Problem.” Social Policy & Administration 48(2):221–39. doi: 10.1111/spol.12058.
   PubMed Web of Science ®Google Scholar
• Ronicle, J., N. Stanworth, E. Hickman, and T. Fox. 2014. “Social Impact Bonds: The State of Play: Full Report.” Commissioning Better Outcomes Evaluation. Big Lottery Fund. https://golab.bsg.ox.ac.uk/documents/Social_Impact_Bonds_-_the_State_of_Play_-_Full_Report_Ronicle_et_al_BLF.pdf
   Google Scholar
• Roy, M. J., B. Macaulay, C. Donaldson, S. Teasdale, R. Baker, S. Kerr, and M. Mazzei. 2018. “Two False Positives Do Not Make a Right: Setting the Bar of Social Enterprise Research Even Higher through Avoiding the Straw Man Fallacy.” Social Science & Medicine 217(September):42–4. doi: 10.1016/j.socscimed.2018.09.058.
   PubMedGoogle Scholar
• Roy, M. J., N. McHugh, and S. Sinclair. 2017. “Social Impact Bonds—Evidence-Based Policy or Ideology?” Pp. 263–75 in Handbook of Social Policy Evaluation, edited by B. Greve, Cheltenham UK: Edward Elgar Publishing.
   Google Scholar
• Scherer, J., and L. Schenk. 2012. “Advancing Social Impact Measurement to Build an Asset Class: The Appeal of Social Impact Bonds.” Community Investments 24(1):10–4.
   Google Scholar
• Sinclair, S., N. McHugh, L. Huckfield, M. Roy, and C. Donaldson. 2014. “Social Impact Bonds: Shifting the Boundaries of Citizenship.” Social Policy Review 26: Analysis and Debate in Social Policy 2014(June):119–36.
   Google Scholar
• Sinclair, S., N. McHugh, and M. J. Roy. 2021. “Social Innovation, Financialisation and Commodification: A Critique of Social Impact Bonds.” Journal of Economic Policy Reform 24(1):11–27. doi: 10.1080/17487870.2019.1571415.
   Web of Science ®Google Scholar
• Strauss, A., and J. Corbin. 1990. Basics of Qualitative Research. Sage Publications.
   Google Scholar
• Tan, S., A. Fraser, N. McHugh, and M. E. Warner. 2021. “Widening Perspectives on Social Impact Bonds.” Journal of Economic Policy Reform 24(1):1–10. doi: 10.1080/17487870.2019.1568249.
   Web of Science ®Google Scholar
• Torre, M., A. Trotta, H. Chiappini, and A. Rizzello. 2019. “Business Models for Sustainable Finance: The Case Study of Social Impact Bonds.” Sustainability 11(7):1887. doi: 10.3390/su11071887.
   Web of Science ®Google Scholar
• Tse, A. E., and M. E. Warner. 2020a. “A Policy Outcomes Comparison: Does SIB Market Discipline Narrow Social Rights?” Journal of Comparative Policy Analysis: Research and Practice 22(2):134–52. doi: 10.1080/13876988.2019.1609789.
   Web of Science ®Google Scholar
• Tse, A. E., and M. E. Warner. 2020b. “The Razor’s Edge: Social Impact Bonds and the Financialization of Early Childhood Services.” Journal of Urban Affairs 42(6):816–32. doi: 10.1080/07352166.2018.1465347.
   Web of Science ®Google Scholar
• VanLandingham, G., and T. Silloway. 2016. “Bridging the Gap between Evidence and Policy Makers: A Case Study of the Pew-MacArthur Results First Initiative.” Public Administration Review 76(4):542–6. doi: 10.1111/puar.12603.
   Web of Science ®Google Scholar
• Van Maanen, J. 1979. “Reclaiming Qualitative Methods for Organizational Research: A Preface.” Administrative Science Quarterly 24(4):520–26. doi: 10.2307/2392358.
   Web of Science ®Google Scholar
• Verrinder, N. B., K. Zwane, S. Vaca, and D. Nixon. 2018. “Evaluative Tools in Impact Investing: Three Case Studies On The Use of Theories of Change.” African Evaluation Journal 6(2):1–9. doi: 10.4102/aej.v6i2.340.
   Google Scholar
• Wang, H., R. M. Sohail Jafar, and B. Fei. 2018. “Research on an Evaluation Model of Social Impact Bonds to Reduce the Short-Term Achievements Tendency.” 2018 15th International Conference on Service Systems and Service Management (ICSSSM), pp. 1–5, Hangzhou, China. doi: 10.1109/ICSSSM.2018.8511010.
   Google Scholar
• Warner, M. E. 2013. “Private Finance for Public Goods: Social Impact Bonds.” Journal of Economic Policy Reform 16(4):303–19. doi: 10.1080/17487870.2013.835727.
   Web of Science ®Google Scholar
• Wilkins, P., J. Phillimore, and D. Gilchrist. 2016. “Public Sector Collaboration: Are We Doing It Well And Could We Do It Better.?” Australian Journal of Public Administration 75(3):318–30. doi: 10.1111/1467-8500.12183.
   Web of Science ®Google Scholar
• Yin, R. K. 2003. “Designing Case Studies.”Qualitative Research Methods 5(14):359–86.
   Google Scholar
• Yin, R. K. 2003. Case Study Research: Design and Methods. 3rd ed. Thousand Oaks, CA: Sage Publications.
   Google Scholar