Impact evaluation with process tracing: explaining causal processes in an EU-interreg sustainable tourism intervention

References

• Agovino, M., Casaccia, M., Garofalo, A., & Marchesano, K. (2017). Tourism and disability in Italy. Limits and opportunities. Tourism Management Perspectives, 23, 58–67. https://doi.org/10.1016/j.tmp.2017.05.001
   Web of Science ®Google Scholar
• Airey, D. (2015). Developments in understanding tourism policy. Tourism Review, 70(4), 246–258. https://doi.org/10.1108/TR-08-2014-0052
   Web of Science ®Google Scholar
• Ashley, C., & Mitchell, J. (2008). Doing the right thing approximately not the wrong thing precisely: Challenges of monitoring impacts of pro-poor interventions in tourism value chains. Overseas Development Institute London.
   Google Scholar
• Baggio, R. (2020). The science of complexity in the tourism domain: A perspective article. Tourism Review, 75(1), 16–19. https://doi.org/10.1108/TR-04-2019-0115
   Web of Science ®Google Scholar
• Beach, D. (2016). It’s all about mechanisms–What process-tracing case studies should be tracing. New Political Economy, 21(5), 463–472. https://doi.org/10.1080/13563467.2015.1134466
   Web of Science ®Google Scholar
• Beach, D., & Pedersen, R. B. (2016). Causal case study methods: Foundations and guidelines for comparing, matching, and tracing. University of Michigan Press.
   Google Scholar
• Beach, D., & Pedersen, R. (2018). Process-Tracing Methods. University of Michigan Press.
   Google Scholar
• Befani, B., & Mayne, J. (2014). Process tracing and contribution analysis: A combined approach to generative causal inference for impact evaluation. IDS Bulletin, 45(6), 17–36. https://doi.org/10.1111/1759-5436.12110
   Web of Science ®Google Scholar
• Befani, B., & Stedman-Bryce, G. (2017). Process tracing and Bayesian updating for impact evaluation. Evaluation, 23(1), 42–60. https://doi.org/10.1177/1356389016654584
   Web of Science ®Google Scholar
• Bennett, A. (2010). Process tracing and causal inference. In H. Brady & D. Collier (Eds.), Rethinking social inquiry: Diverse tools, shared standards (2nd ed.). Rowman & Littlefield Publishers.
   Google Scholar
• Bennett, A., & Checkel, J. T. (2015). Process tracing. Cambridge University Press.
   Google Scholar
• Bray, J., Gray, M., & T Hart, P. (2019). Evaluation and learning from failure and success: An ANZSOG research paper for the Australian public service review panel.
   Google Scholar
• Busetti, S. (2023). Causality is good for practice: Policy design and reverse engineering. Policy Sciences, 56(2), 419–438. https://doi.org/10.1007/s11077-023-09493-7
   Web of Science ®Google Scholar
• Busetti, S., & Dente, B. (2017). Using process tracing to evaluate unique events: The case of EXPO Milano 2015. Evaluation, 23(3), 256–273. https://doi.org/10.1177/1356389017716738
   Web of Science ®Google Scholar
• Byrne, D. (2013). Evaluating complex social interventions in a complex world. Evaluation, 19(3), 217–228. https://doi.org/10.1177/1356389013495617
   Web of Science ®Google Scholar
• Camacho Garland, G., & Beach, D. (2023). Theorizing how interventions work in evaluation: Process-tracing methods and theorizing process theories of change. Evaluation, 29(4), 390–409. https://doi.org/10.1177/13563890231201876
   Web of Science ®Google Scholar
• Cartwright, N., & Hardie, J. (2012). Evidence-based policy: A practical guide to doing it better. Oxford University Press.
   Google Scholar
• Chen, S., & Henry, I. (2020). Assessing Olympic legacy claims: Evaluating explanations of causal mechanisms and policy outcomes. Evaluation, 26(3), 275–295. https://doi.org/10.1177/1356389019836675
   Web of Science ®Google Scholar
• Clarke, G. M., Conti, S., Wolters, A. T., & Steventon, A. (2019). Evaluating the impact of healthcare interventions using routine data. BMJ (Clinical Research ed.), 365, l2239. https://doi.org/10.1136/bmj.l2239
   PubMedGoogle Scholar
• Collier, D. (2011). Understanding process tracing. PS: Political Science & Politics, 44(4), 823–830. https://doi.org/10.1017/S1049096511001429
   Web of Science ®Google Scholar
• Deaton, A., & Cartwright, N. (2018). Understanding and misunderstanding randomized controlled trials. Social Science & Medicine, 210, 2–21. https://doi.org/10.1016/j.socscimed.2017.12.005
   PubMed Web of Science ®Google Scholar
• Delahais, T., & Toulemonde, J. (2017). Making rigorous causal claims in a real-life context: Has research contributed to sustainable forest management? Evaluation, 23(4), 370–388. https://doi.org/10.1177/1356389017733211
   Web of Science ®Google Scholar
• Dredge, D., & Jamal, T. (2015). Progress in tourism planning and policy: A post-structural perspective on knowledge production. Tourism Management, 51, 285–297. https://doi.org/10.1016/j.tourman.2015.06.002
   Web of Science ®Google Scholar
• Dredge, D., & Jenkins, J. (Eds.) (2011). Stories of practice: Tourism policy and planning. Ashgate Publishing, Ltd.
   Google Scholar
• Eckardt, C., Font, X., & Kimbu, A. (2019). Realistic evaluation as a volunteer tourism supply chain methodology. Journal of Sustainable Tourism, 28(5), 647–662. https://doi.org/10.1080/09669582.2019.1696350
   Web of Science ®Google Scholar
• Elsenbroich, C., & Badham, J. (2023). Negotiating a future that is not like the past. International Journal of Social Research Methodology, 26(2), 207–213. https://doi.org/10.1080/13645579.2022.2137935
   Web of Science ®Google Scholar
• European Commission. (2013). Evaluation guidance EVALSED sourcebook: Method and Techniques.
   Google Scholar
• Falleti, T. G., & Lynch, J. F. (2009). Context and causal mechanisms in political analysis. Comparative Political Studies, 42(9), 1143–1166. https://doi.org/10.1177/0010414009331724
   Web of Science ®Google Scholar
• HM Treasury. (2020). The Magenta Book - Central Government guidance on evaluation. UK Government. www.gov.uk/official-documents.
   Google Scholar
• Kay, A., & Baker, P. (2015). What can causal process tracing offer to policy studies? A review of the literature. Policy Studies Journal, 43(1), 1–21. https://doi.org/10.1111/psj.12092
   Web of Science ®Google Scholar
• Machamer, P., Darden, L., & Craver, C. F. (2000). Thinking about mechanisms. Philosophy of Science, 67(1), 1–25. https://doi.org/10.1086/392759
   Web of Science ®Google Scholar
• Mellon, V., & Bramwell, B. (2018). The temporal evolution of tourism institutions. Annals of Tourism Research, 69, 42–52. https://doi.org/10.1016/j.annals.2017.12.008
   Web of Science ®Google Scholar
• Montano, L. J., Font, X., Elsenbroich, C., & Ribeiro, M. A. (2023). Co-learning through participatory evaluation: An example using theory of change in a large-scale EU-funded tourism intervention. Journal of Sustainable Tourism, 1–20. https://doi.org/10.1080/09669582.2023.2227781
   Web of Science ®Google Scholar
• Mori, I. (2021). NAMA facility – 2nd Interim evaluation and learning, final report (Issue February).
   Google Scholar
• Moumoutzis, K., & Zartaloudis, S. (2016). Europeanization mechanisms and process tracing: A template for empirical research. Journal of Common Market Studies, 54(2), 337–352.
   Web of Science ®Google Scholar
• OECD. (2020). Rebuilding tourism for the future: COVID-19 policy responses and recovery. OECD. Paris: OECD Publishing. https://www.oecd.org/coronavirus/policy-responses/rebuilding-tourism-for-the-future-%20covid-19-policy-responses-and-recovery-bced9859/
   Google Scholar
• O’Sullivan, D., Pickernell, D., & Senyard, J. (2009). Public sector evaluation of festivals and special events. Journal of Policy Research in Tourism, Leisure and Events, 1(1), 19–36. https://doi.org/10.1080/19407960802703482
   Google Scholar
• Pawson, R., & Tilley, N. (1997). Realistic evaluation. Sage.
   Google Scholar
• Phi, G. T., Whitford, M., & Reid, S. (2018). What’s in the black box? Evaluating anti-poverty tourism interventions utilizing theory of change. Current Issues in Tourism, 21(17), 1930–1945. https://doi.org/10.1080/13683500.2016.1232703
   Web of Science ®Google Scholar
• Regeer, B. J., de Wildt-Liesveld, R., van Mierlo, B., & Bunders, J. F. (2016). Exploring ways to reconcile accountability and learning in the evaluation of niche experiments. Evaluation, 22(1), 6–28. https://doi.org/10.1177/1356389015623659
   Web of Science ®Google Scholar
• Ricks, J. I., & Liu, A. H. (2018). Process-tracing research designs: A practical guide. PS - Political Science and Politics, 51(4), 842–846.
   Web of Science ®Google Scholar
• Rothgang, M., & Lageman, B. (2021). The unused potential of process tracing as evaluation approach: The case of cluster policy evaluation. Evaluation, 27(4), 527–543. https://doi.org/10.1177/13563890211041676
   Web of Science ®Google Scholar
• Schimmelfennig, F. (2015). Efficient process tracing. Analyzing the causal mechanisms of European integration. In A. Bennett and J. T. Checkel (Eds.), Process tracing in the social sciences: From metaphor to analytic tool (pp. 98–125). Cambridge University Press.
   Google Scholar
• Schmitt, J., & Beach, D. (2015). The contribution of process tracing to theory-based evaluations of complex aid instruments. Evaluation, 21(4), 429–447. https://doi.org/10.1177/1356389015607739
   Web of Science ®Google Scholar
• Stern, E., Stame, N., Mayne, J., Forss, K., Davies, R., & Befani, B. (2012). Broadening the sharing the of designs and benefits of trade methods for impact evaluations, Working Paper 38. Department for International Development, February 2011, 1–127.
   Google Scholar
• Stevenson, N., Airey, D., & Miller, G. (2009). Complexity theory and tourism policy research. International Journal of Tourism Policy, 2(3), 206–220. https://doi.org/10.1504/IJTP.2009.024553
   Google Scholar
• Tamm, H., & Duursma, A. (2023). Combat, commitment, and the termination of Africa’s mutual interventions. European Journal of International Relations, 29(1), 3–28. https://doi.org/10.1177/13540661221112612
   Web of Science ®Google Scholar
• Taplin, J., Dredge, D., & Scherrer, P. (2014). Monitoring and evaluating volunteer tourism: A review and analytical framework. Journal of Sustainable Tourism, 22(6), 874–897. https://doi.org/10.1080/09669582.2013.871022
   Web of Science ®Google Scholar
• Trampusch, C., & Palier, B. (2016). Between X and Y: How process tracing contributes to opening the Black Box of causality, New political economy, 21(5), 437–454.
   Google Scholar
• Twining-Ward, L., Messerli, H., Sharma, A., & Villascusa Cerezo, J. M. (2018). Tourism theory of change. Tourism for development knowledge series. World Bank. https://openknowledge.worldbank.org/handle/10986/35459
   Google Scholar
• Vanclay, F. (2015). The potential application of qualitative evaluation methods in European regional development: Reflections on the use of Performance Story Reporting in Australian natural resource management. Regional Studies, 49(8), 1326–1339. https://doi.org/10.1080/00343404.2013.837998
   Web of Science ®Google Scholar
• Van der Knaap, P. (2016). Responsive evaluation and performance management: Overcoming the downsides of policy objectives and performance indicators. Evaluation, 12(3), 278–293. https://doi.org/10.1177/1356389006069135
   Google Scholar
• Wadeson, A., Monzani, B., & Aston, T. (2020). Process tracing as a practical evaluation method: comparative learning from six evaluations. Monitoring and Evaluation News. https://mande.co.uk/2020/media-3/unpublished-paper/process-tracing-as-a-practical-evaluation-method-comparative-learning-from-six-evaluations/?utm_source=dlvr.it&utm_medium=twitter
   Google Scholar
• Wauters, B., & Beach, D. (2018). Process tracing and congruence analysis to support theory-based impact evaluation. Evaluation, 24(3), 284–305. https://doi.org/10.1177/1356389018786081
   Web of Science ®Google Scholar
• Weiss, C. H. (1998). Evaluation methods for studying programs and policies. Pearson College Division.
   Google Scholar
• Zaks, S. (2017). Relationships among rivals (RAR): A framework for analyzing contending hypotheses in process-tracing. Political Analysis, 25(3), 344–362. https://doi.org/10.1017/pan.2017.12
   Google Scholar