Challenges related to the analytical process in realist evaluation and latest developments on the use of NVivo from a realist perspective

References

• Adams, A., Sedalia, S., McNab, S., & Sarker, M. (2016). Lessons learned in using realist evaluation to assess maternal and newborn health programming in rural Bangladesh. Health Policy and Planning, 31(2), 267–275.
   PubMed Web of Science ®Google Scholar
• Alvarado, N., Honey, S., Greenhalgh, J., Pearman, A., Dowding, D., Cope, A., … Kotze, A. (2017). Eliciting context-mechanism-outcome configurations: Experiences from a realist evaluation investigating the impact of robotic surgery on teamwork in the operating theatre. Evaluation, 23(4), 444–462.
   Web of Science ®Google Scholar
• Astbury, B. (2013). Some reflections on Pawson’s Science of evaluation: A realist manifesto. Evaluation, 19(4), 383–401.
   Web of Science ®Google Scholar
• Astbury, B. (2016). From Evaluation Theory to Tests of Evaluation Theory? In R. Stockmann & W. Meyer (Eds.), The future of evaluation - Global trends, new challenges, shared perspectives (pp. 309–325). London, UK: Palgrave Macmillan.
   Google Scholar
• Astbury, B., & Leeuw, F. L. (2010). Unpacking black boxes: Mechanisms and theory building in evaluation. American Journal of Evaluation, 31(3), 363–381.
   Web of Science ®Google Scholar
• Banks, J., Wye, L., Hall, N., Rooney, J., Walter, F. M., Hamilton, W., … Rubin, G. (2017). The researchers’ role in knowledge translation: A realist evaluation of the development and implementation of diagnostic pathways for cancer in two United Kingdom localities. Health Research Policy and Systems, 15(1), 103.
   PubMedGoogle Scholar
• Bergeron, D. A., Talbot, L. R., & Gaboury, I. (in press). Intersectoral oral health promotion interventions for schoolchildren living in remote rural Andean communities: A realist evaluation. Global Health Promotion. Manuscript accepted for publication.
   Web of Science ®Google Scholar
• Blaise, P., Marchal, B., Lefèvre, P., & Kegels, G. (2010). Au-delà des méthodes expérimentales: L’approche réaliste en évaluation. In L. Potvin, M.-J. Moquet, & C. Jones (Eds.), Réduire les inégalités sociales en santé (pp. 285–296). St-Denis, FR: INPES.
   Google Scholar
• Byng, R., Norman, I., & Redfern, S. (2005). Using realistic evaluation to evaluate a practice-level intervention to improve primary healthcare for patients with long-term mental illness. Evaluation, 11(1), 69–93.
   Google Scholar
• Caffrey, L., Wyatt, D., Fudge, N., Mattingley, H., Williamson, C., & McKevitt, C. (2016). Gender equity programmes in academic medicine: A realist evaluation approach to Athena SWAN processes. BMJ Open, 6(9), e012090.
   PubMed Web of Science ®Google Scholar
• Chouinard, M.-C., Hudon, C., Dubois, M.-F., Roberge, P., Loignon, C., Tchouaket, É., … Sasseville, M. (2013). Case management and self-management support for frequent users with chronic disease in primary care: A pragmatic randomized controlled trial. BMC Health Services Research, 13(1), 1.
   PubMedGoogle Scholar
• Coldwell, M. (2019). Reconsidering context: Six underlying features of context to improve learning from evaluation. Evaluation, 25(1), 99–117.
   Web of Science ®Google Scholar
• Dalkin, S. M., Forster, N., Hodgson, P., Lhussier, M., & Carr, S. (2015, December 21). Using NVivo to enhance transparency in a realist evaluation. Retrieved from http://www.qsrinternational.com/nvivo/nvivo-community/the-nvivo-blog/how-researchers-use-nvivo-to-enhance-transparency
   Google Scholar
• Dalkin, S. M., Greenhalgh, J., Jones, D., Cunningham, B., & Lhussier, M. (2015). What’s in a mechanism? Development of a key concept in realist evaluation. Implementation Science, 10(1), 49.
   PubMed Web of Science ®Google Scholar
• de Souza, D. E. (2013). Elaborating the context-mechanism-outcome configuration (CMOc) in realist evaluation: A critical realist perspective. Evaluation, 19(2), 141–154.
   Web of Science ®Google Scholar
• Dickinson, H. (2006). The evaluation of health and social care partnerships: An analysis of approaches and synthesis for the future. Health & Social Care in the Community, 14(5), 375–383.
   PubMed Web of Science ®Google Scholar
• Doi, L., Jepson, R., & Cheyne, H. (2015). A realist evaluation of an antenatal programme to change drinking behaviour of pregnant women. Midwifery, 31(10), 965–972.
   PubMed Web of Science ®Google Scholar
• Doi, L., Jepson, R., Hardie, S., & Harris, F. (2017). Realist evaluation of an enhanced health visiting programme. PloS One, 12(7), e0180569.
   PubMed Web of Science ®Google Scholar
• Eastwood, J. G., Jalaludin, B. B., & Kemp, L. A. (2014). Realist explanatory theory building method for social epidemiology: A protocol for a mixed method multilevel study of neighbourhood context and postnatal depression. SpringerPlus, 3(1), 12.
   PubMedGoogle Scholar
• Goicolea, I., Vives-Cases, C., Hurtig, A.-K., Marchal, B., Briones-Vozmediano, E., Otero-García, L., … San Sebastian, M. (2015). Mechanisms that trigger a good health-care response to intimate partner violence in Spain. Combining realist evaluation and qualitative comparative analysis approaches. PloS One, 10(8), e0135167.
   PubMed Web of Science ®Google Scholar
• Greenhalgh, T., Pawson, R., Wong, G., Westhorp, G., Greenhalgh, J., Manzano, A., & Jagosh, J. (2017a). Developing realist programme theories. The RAMESES II Project.
   Google Scholar
• Greenhalgh, T., Pawson, R., Wong, G., Westhorp, G., Greenhalgh, J., Manzano, A., & Jagosh, J. (2017b). Realist evaluation, realist synthesis, realist research – What’s in a name? The RAMESES II Project.
   Google Scholar
• Greenhalgh, T., Pawson, R., Wong, G., Westhorp, G., Greenhalgh, J., Manzano, A., & Jagosh, J. (2017c). The realist interview. The RAMESES II Project.
   Google Scholar
• Greenhalgh, T., Pawson, R., Wong, G., Westhorp, G., Greenhalgh, J., Manzano, A., & Jagosh, J. (2017d). “Theory” in realist evaluation. The RAMESES II Project.
   Google Scholar
• Guichard, A., & Ridde, V. (2010). Une grille d’analyse des actions pour lutter contre les inégalités sociales de santé. In L. Potvin, M.-J. Moquet, & C. Jones (Eds.), Réduire les inégalités sociales en santé (pp. 297–312). St-Denis, FR: INPES.
   Google Scholar
• Hamilton-Smith, N., & Hopkins, M. (1998). Theory-driven interviewing: From theory into practice. Bulletin of Sociological Methodology/Bulletin De Méthodologie Sociologique, 60(1), 80–105.
   Google Scholar
• Houghton, C., Murphy, K., Meehan, B., Thomas, J., Brooker, D., & Casey, D. (2017). From screening to synthesis: Using nvivo to enhance transparency in qualitative evidence synthesis. Journal of Clinical Nursing, 26, 873–881.
   PubMed Web of Science ®Google Scholar
• Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288.
   PubMed Web of Science ®Google Scholar
• Hutchison, A. J., Johnston, L. H., & Breckon, J. D. (2010). Using QSR‐NVivo to facilitate the development of a grounded theory project: An account of a worked example. International Journal of Social Research Methodology, 13(4), 283–302.
   Web of Science ®Google Scholar
• Jackson, S. F., & Kolla, G. (2012). A new realistic evaluation analysis method linked coding of context, mechanism, and outcome relationships. American Journal of Evaluation, 33(3), 339–349.
   Web of Science ®Google Scholar
• Jeffries, M., Phipps, D. L., Howard, R. L., Avery, A. J., Rodgers, S., & Ashcroft, D. M. (2017). Understanding the implementation and adoption of a technological intervention to improve medication safety in primary care: A realist evaluation. BMC Health Services Research, 17(1), 196.
   PubMedGoogle Scholar
• Lacouture, A., Breton, E., Guichard, A., & Ridde, V. (2015). The concept of mechanism from a realist approach: A scoping review to facilitate its operationalization in public health program evaluation. Implementation Science, 10(1), 153.
   PubMedGoogle Scholar
• Linsley, P., Howard, D., & Owen, S. (2015). The construction of context-mechanisms-outcomes in realistic evaluation. Nurse Researcher, 22(3), 28–34.
   PubMed Web of Science ®Google Scholar
• Maluka, S., Kamuzora, P., SanSebastián, M., Byskov, J., Ndawi, B., Olsen, O. E., & Hurtig, A.-K. (2011). Implementing accountability for reasonableness framework at district level in Tanzania: A realist evaluation. Implementation Science, 6(11), 1–15.
   PubMedGoogle Scholar
• Manzano, A. (2016). The craft of interviewing in realist evaluation. Evaluation, 22(3), 342–360.
   Web of Science ®Google Scholar
• Marchal, B., Van Belle, S., Van Olmen, J., Hoerée, T., & Kegels, G. (2012). Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation, 18(2), 192–212.
   Web of Science ®Google Scholar
• Maxwell, J. A. (2012). A realist approach for qualitative research. New York, NY: Sage.
   Google Scholar
• Maxwell, J. A., & Miller, B. A. (2008). Categorizing and connecting strategies in qualitative data analysis. In P. Leavy & S. Hesse-Biber (Eds.), Handbook of emergent methods (pp. 461–477). New York, NY: Guilford Press.
   Google Scholar
• McEvoy, P., & Richards, D. (2006). A critical realist rationale for using a combination of quantitative and qualitative methods. Journal of Research in Nursing, 11(1), 66–78.
   Google Scholar
• McGaughey, J., O’halloran, P., Porter, S., Trinder, J., & Blackwood, B. (2017). Early warning systems and rapid response to the deteriorating patient in hospital: A realist evaluation. Journal of Advanced Nursing, 73(12), 3119–3132.
   Web of Science ®Google Scholar
• Nielsen, K., & Miraglia, M. (2017). What works for whom in which circumstances? On the need to move beyond the ‘what works?’ Question in organizational intervention research. Human Relations, 70(1), 40–62.
   Web of Science ®Google Scholar
• Oliver, D. P. (2011). Rigor in qualitative research. Research on Aging, 33(4), 359–360.
   Web of Science ®Google Scholar
• Onwuegbuzie, A. J., Leech, N. L., & Collins, K. M. (2012). Qualitative analysis techniques for the review of the literature. The Qualitative Report, 17(28), 1–28.
   Google Scholar
• Paulus, T., Woods, M., Atkins, D. P., & Macklin, R. (2017). The discourse of QDAS: Reporting practices of ATLAS. ti and NVivo users with implications for best practices. International Journal of Social Research Methodology, 20(1), 35–47.
   Web of Science ®Google Scholar
• Pawson, R. (1996). Theorizing the interview. British Journal of Sociology, 47, 295–314.
   Web of Science ®Google Scholar
• Pawson, R. (2013). The science of evaluation: A realist manifesto. London, UK: Sage.
   Google Scholar
• Pawson, R. (2015). The ersatz realism of critical realism: A reply to Porter. Evaluation, 22(1), 49–57.
   Web of Science ®Google Scholar
• Pawson, R. (2018). Realist Memorabilia. In N. Emmel, J. Greenhalgh, A. Manzano, M. Monaghan, & S. Dalkin (Eds.), Doing realist evaluation, synthesis and research (pp. 204–220). London: Sage.
   Google Scholar
• Pawson, R., & Manzano-Santaella, A. (2012). A realist diagnostic workshop. Evaluation, 18(2), 176–191.
   Web of Science ®Google Scholar
• Pawson, R., & Sridharan, S. (2010). Theory-driven evaluation of public health programmes. In A. Killoran & M. P. Kelly (Eds.), Evidence-based public health: Effectiveness and efficiency (pp. 43–61). Oxford, UK: Oxford University Press.
   Google Scholar
• Pawson, R., & Tilley, N. (1997). Realistic evaluation. London, UK: Sage.
   Google Scholar
• Reddy, S., Orpin, V., Herring, S., Mackie‐Schneider, S., & Struber, J. (2017). Use of clinical guidelines in remote Australia: A realist evaluation. Journal of Evaluation in Clinical Practice, 21(6), 1114–1120.
   PubMed Web of Science ®Google Scholar
• Ridde, V., & Haddad, S. (2013). Pragmatisme et réalisme pour l’évaluation des interventions de santé publique. Revue d’epidemiologie et de sante publique, 61, S95–S106.
   PubMed Web of Science ®Google Scholar
• Ridde, V., Robert, E., Guichard, A., Blaise, P., & Van Olmen, J. (2012). L’approche REALIST à l’épreuve du réel de l’évaluation des programmes. The Canadian Journal of Program Evaluation, 26(3), 37–59.
   Google Scholar
• Robert, É., & Ridde, V. (2013). L’approche réaliste pour l’évaluation de programmes et la revue systématique: De la théorie à la pratique. Mesure et évaluation en éducation, 36(3), 79–108.
   Google Scholar
• Robins, C. S., & Eisen, K. (2017). Strategies for the effective use of NVivo in a large-scale study: Qualitative analysis and the repeal of don’t ask, don’t tell. Qualitative Inquiry, 23(10), 768–778.
   Web of Science ®Google Scholar
• Rycroft-Malone, J., Burton, C. R., Wilkinson, J., Harvey, G., McCormack, B., Baker, R., … Thompson, C. (2016). Collective action for implementation: A realist evaluation of organisational collaboration in healthcare. Implementation Science, 11(1), 1.
   PubMedGoogle Scholar
• Salter, K. L., & Kothari, A. (2014). Using realist evaluation to open the black box of knowledge translation: A state-of-the-art review. Implementation Science, 9(1), 115.
   PubMedGoogle Scholar
• Shankardass, K., Renahy, E., Muntaner, C., & O’Campo, P. (2014). Strengthening the implementation of health in all policies: A methodology for realist explanatory case studies. Health Policy and Planning, 30(4), 462–473.
   PubMed Web of Science ®Google Scholar
• Staley, K., Buckland, S. A., Hayes, H., & Tarpey, M. (2014). ‘The missing links’: Understanding how context and mechanism influence the impact of public involvement in research. Health Expectations, 17(6), 755–764.
   PubMed Web of Science ®Google Scholar
• Tan, H. T. R., & Harvey, G. (2015). Unpacking the black box: A realist evaluation of performance management for social services. Public Management Review, 18(10), 1456–1478.
   Web of Science ®Google Scholar
• Tracy, S. J. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry, 16(10), 837–851.
   Web of Science ®Google Scholar
• Van Belle, S., Marchal, B., Dubourg, D., & Kegels, G. (2010). How to develop a theory-driven evaluation design? Lessons learned from an adolescent sexual and reproductive health programme in West Africa. BMC Public Health, 10(1), 1.
   PubMedGoogle Scholar
• Van Belle, S., Rifkin, S., & Marchal, B. (2017). The challenge of complexity in evaluating health policies and programs: The case of women’s participatory groups to improve antenatal outcomes. BMC Health Services Research, 17(1), 687.
   PubMedGoogle Scholar
• Westhorp, G. (2018). Understanding mechanisms in realist evaluation and research. In N. Emmel, J. Greenhalgh, A. Manzano, M. Monaghan, & S. Dalkin (Eds.), Doing realist evaluation, synthesis and research (pp. 42–57). London: Sage.
   Google Scholar
• Westhorp, G., Prins, E., Kusters, C., Hultink, M., Guijt, I., & Brouwers, J. (2011). Realist evaluation: An overview. Report from an Expert Seminar with Dr. Gill Westhorp. Wageningen: Wageningen University & Research Centre for Development Innovation.
   Google Scholar
• Wong, G., Westhorp, G., Manzano, A., Greenhalgh, J., Jagosh, J., & Greenhalgh, T. (2016). RAMESES II reporting standards for realist evaluations. BMC Medicine, 14(1), 96.
   PubMed Web of Science ®Google Scholar
• Woodhead, C., Collins, H., Lomas, R., & Raine, R. (2017). Co‐located welfare advice in general practice: A realist qualitative study. Health & Social Care in the Community, 25, 1794–1804.
   PubMed Web of Science ®Google Scholar
• Woods, M., Macklin, R., & Lewis, G. K. (2016). Researcher reflexivity: Exploring the impacts of CAQDAS use. International Journal of Social Research Methodology, 19(4), 385–403.
   Web of Science ®Google Scholar