Qualitative Outcomes in CME/CPD: Exploring Non-Linear Contexts and Lived Experiences in Patient-Directed Interventions

ABSTRACT

Qualitative analysis is often used to gather insights about learning, behavioural and practice change. Given the rich detail that qualitative data delivers, we are puzzled at the relative absence of qualitative approaches to outcomes assessment in the field of CME/CPD, especially as patient-directed education becomes increasingly tethered or adjunctive to CME/CPD programmes as a way to directly engage patients in disease self-management and improve health outcomes. Education outcomes for both clinicians and patients are contextualised by norms, motivations, and values that shape how learners interact with education activities and materials. These properties are linked to and shape the mechanisms that drive education outcomes but are rarely the focus of assessments that are often rooted in quantitative, positivist frameworks. In order to illustrate the role that qualitative methodologies can play in outcomes assessment, we describe insights from three education programmes designed to improve the health of patients with specific conditions and outline a range of qualitative methodologies appropriate for outcomes evaluation.

KEYWORDS:

• Patient education
• qualitative outcomes
• interviews
• perspectives
• tethered education

Introduction

In healthcare research, qualitative analysis is often used to gather insights about the role that workplace culture, stakeholder perspectives, and clinical settings play in learning, behavioural and practice change. Given the rich detail that qualitative data delivers, we are puzzled at the relative absence of qualitative approaches to outcomes analysis in the field of continuing medical education/continuing professional development (CME/CPD). When articles in industry journals refer to qualitative data, they typically point to learner responses to open-ended questions embedded in surveys. While it is true that such data are routinely collected by many CME/CPD provider organisations, these responses represent a tiny drop in the potential of the qualitative ocean. The CME/CPD field has shifted considerably towards specifying instruments and scales to accurately measure education outcomes and their likely effects on clinical practice and patient behaviours. This shift is timely and welcome. However, the environments in which clinicians practice and patients navigate health and health care, are complex[1]. Education outcomes for these groups are contextualised by norms, motivations, and values that shape how learners interact with education activities and materials. These properties are linked to and shape the mechanisms that drive education outcomes but are rarely the focus of assessments that are often rooted in quantitative, positivist frameworks. In order to illustrate the role that qualitative methodologies can play in outcomes assessment, we describe insights from three education programs designed to improve the health of patients with specific conditions and outline a range of qualitative methodologies appropriate for outcomes evaluation.

The Programs

Between 2016 and 2018 PlatformQ Health Education (PQHE) and Thistle Editorial, LLC partnered to evaluate educational outcomes for three patient education programmes on diabetic retinopathy (DR), B-cell non-Hodgkin lymphoma, and pseudobulbar affect (PBA). Patient-directed education is increasingly tethered or adjunctive to CME/CPD programs as a strategy to directly engage patients and improve patient health [2,3]. Patient-directed education is a vital component of disease self-management for chronic conditions to build the knowledge and skills patients need to modify their behaviours, make informed decisions about their care, and more effectively interact with healthcare providers (HCPs). Current evidence suggests that education on self-care strategies can help to reduce uncertainty for patients, foster a sense of control over the experience of living with cancer and other chronic conditions, and buttress self-efficacy [4–7]. In models of behaviour change (e.g. the Transtheoretical Stages of Change Theory, self-efficacy is measured quantitatively as a predictor of future behaviour following exposure to education [8–10]. In the context of health and illness, self-efficacy theories posit that learning, action, and behaviour change are also influenced by self-beliefs, values, and motivations derived from interactions between personal, environmental or situational and behavioural determinants[11].

While many CME/CPD providers are embracing patient-directed education, there is as yet little published research on the potential to stimulate qualitative changes in patient knowledge and self-efficacy. To this end, PQHE designed online education activities that could be engaged with live or on-demand to: 1) support self-efficacy for patients living with B-cell NHL; 2) provide resources to support patients in the recognition and self-management of PBA symptoms.; and 3) empower patients to be their own advocates for optimal ophthalmological care (DR) [12–14]. We evaluated the impact of these activities on patient knowledge and self-efficacy via a mixed-methods approach to data collection and analysis. Following survey-based quantitative outcomes assessments, we interviewed education participants to explore contextual factors that may be important to knowledge uptake and behaviour change.

Which Qualitative Approach Do We Use?

We typically evaluate clinician and patient education outcomes using semi-structured interviews that engender deep conversations with education participants and encourage reflection and “summative judgement” about the experience of education participation[15]. For these three activities, we interviewed a small sample of learners (n = 21; 12; 10, respectively) 6–10 weeks following education participation. We conducted interviews by phone that were guided by semi-structured questions designed to explore a range of topics relevant to the education and to patients’ experience of living with DR, B-cell NHL, or PBA. Interviews were audio-recorded via cloud-based, web-conferencing software, transcribed verbatim and imported into NVivo (QSR International), a software package designed to support systematic analysis of unstructured data. We used a process of constant comparison to structure analysis of participant responses to questions both across interviews and within interviews[16]. Although there are many approaches to coding, constant comparison includes three main components: (1) data immersion and familiarisation, (2) descriptive data coding and (3) thematic coding[17]. For all studies, we initially coded transcript content into descriptive categories that broadly followed the structure and focus of the interview categories concerning the education activity impact on self-reported knowledge and behaviour. Following descriptive coding, a second round of coding identified themes across the dataset until thematic saturation was achieved.

Results

Education as Self-Efficacy Reinforcement: B-Cell NHL

Many patients that self-select to participate in education already have experiential knowledge about living with their disease (or caring for others with the disease). Participants are often emphatic about the importance of engaging in self-care practices as a normalised part of their everyday experience to maintain health and well-being and participate in education to deepen their learning. Following participation in the B-cell NHL programme , participants demonstrated sustained knowledge about practical self-care behaviours and were able to describe key messages presented in the education. For instance, they emphasised the importance of intentionally caring for themselves via functional behaviours (e.g. practising good nutrition or being physically active within their personal limits).

Qualitative evaluation of this education identified how participants contextualised the self-care strategies they described. For instance, participants told us how important it was for them to be acutely aware of their bodies and checking for changes (e.g., you have to listen to your body and your body is going to tell you that things are a little bit different now) and that they needed education to evolve with them as the disease progressed and their bodies changed over time. Participants told us about the importance of being prepared to engage in self-care practices, and how vital access to education is to support such preparation. The tangible self-care behaviours that participant described as part of being prepared included asking questions about which tests they ought to have at diagnosis, building relationships with oncologists and finding second opinions if necessary, and actively connecting with others living with B-cell NHL. Participants highlighted how the patient vignettes included in the education format helped them affirm their connection to the B-cell NHL community and learn “from people that actually are going through it”, “how they experienced it, what they decided to do and where they are today.”

Through interviews, we gleaned insights into how education touches the lived experience of patients and the very granular ways in which they think about integrating learning into their daily lives. We used these insights to explore the meaning and enactment of self-efficacy in the context of cancer and to engage critically with the implications for education of how people interpret self-efficacy. The participants in this study were very clear that the experience of living with B-cell NHL differs for people by many factors, including subtype, and that education should help patients make tailored adjustments and modifications to daily activities in response to changes in the ways that disease and treatment-specific signs and symptoms show up in particular ways in their bodies. The use of vignettes authenticated participants’ existing self-care strategies by reflecting variations in the lived experience of others. This personalised window into the experience of disease could be interpreted as a positive “vicarious experience”, which is one of the elements necessary to build and sustain self-efficacy[10].

Education as Validation: PBA

Participants in this study of PBA education were stroke survivors or caregivers of stroke survivors. They collectively painted a dismal picture of low physician awareness about PBA in primary care and neurology. PBA symptoms represented a “new normal” for participants, which was characterised by “lack of control” in functionality and social status and disrupted established patterns of communication between family and friends. Prior to education exposure, many participants talked about adapting to PBA, describing themselves as “hustling through”, “hanging in there”, or just “getting through” episodes of socially inappropriate laughing or crying. They managed their PBA symptoms by “accepting” symptoms as part of their stroke experience or intuitively adopting strategies such as changing position or distraction. Following education, participants were able to identify a range of new strategies that they were willing to try as ways to manage their PBA symptoms, describe practical self-care strategies and tips to share with PBA patients and their caregivers to help them manage symptoms, and outline a range of questions that people should ask their physicians when they are diagnosed with PBA. We asked participants if there were elements of the education that were especially effective. Participants emphasised that educational content affirmed their experience of the social and emotional burden of PBA symptoms, validated their experience of this stigmatised condition by emphasising its neurological versus psychiatric aetiology, and provided hope and reassurance that they were not alone in their experience of PBA.

Education as Empowerment: Diabetic Retinopathy

In DR program interviews we explicitly asked “before” and “after” education questions to explore the potential for qualitative changes in knowledge and self-efficacy as a precursor to behaviour change. Following education exposure participants were better able to identify risk factors for DR and a wider range of diabetes self-management and prevention strategies such as blood glucose monitoring, regular eye exams, and medications. Participants emphasised that the education reinforced the importance of certain aspects of diabetes self-care (e.g., glucose monitoring) and they expressed a heightened awareness about the need to more assiduously monitor and control diabetes. Notably, before education, few participants identified their HCPs as sources of verbal or written information about DR unless there was “bad news” to report. They acknowledged the importance of asking questions and of being actively engaged in the management of their own eye health, but most assumed that providers “knew what they were doing” and did not question their care. After education, all participants were able to list specific questions they would like to ask their primary diabetes care and eye health providers and said they felt empowered to do so. In this evaluation, we learned that online education can empower patients with longstanding diabetes and established DR to more proactively engage and communicate with HCPs. Although participants acknowledged the importance of active engagement in the management of diabetes eye health before the education, active engagement was an exception rather than a rule. After education, participants were able to identify questions they would ask their diabetes and eye health HCPs and emphatically endorsed the importance of internet-based education as an empowering part of their ongoing approach to diabetes self-management.

Discussion

Outcomes evaluation using qualitative data could be considerably expanded within CME/CPD. Text- and image-based qualitative data provide insight into the explanations that individuals themselves give for behaviours and actions, as well as the perspectives, values and beliefs that underpin their behaviours. Qualitative data are best used to explore, in-depth, the context of education interventions and the perspectives of multiple stakeholders who are likely to be affected by an intervention. We use qualitative methodologies where possible because we believe that this approach helps to identify what really matters to patients and providers, detect obstacles to changing behaviour, and explain why behaviour change or performance improvement does or does not occur. Qualitative approaches to outcomes evaluation generate unique insights not only about the role of education in building patient knowledge and awareness about their respective conditions but also about the process factors that support education as a tool for behaviour change. In the studies we describe here, education reinforced existing self-efficacy behaviours (B-cell NHL), validated the lived experience of disease (PBA), and built confidence to support active engagement in communicating with HCPs (DR).

Although we typically use interviews in our work, there are four main methods for collecting qualitative data: interviews, focus groups, observations, and document analysis. Structured (predetermined questions and sequences) or semi-structured (supported by topic guides, but looser in format and with more of a conversational flow) interviews provide a space in which education participants can share their individual perspectives on, or experiences of a situation, process or event. This approach adds insight to quantitative findings and flexibly allows for follow-up questions to probe participant responses in more depth. Focus group techniques are valuable in consensus-building contexts such as roundtable discussions. In addition to face-to-face or online focus groups (e.g., via email, message boards, chat rooms, or Zoom), nominal group technique or Delphi method can be used to brainstorm around issues in a more structured, open-ended way via a mix of phone, teleconference and email. The benefits of online and telephone interviews/focus groups, versus in-person approaches, include cost savings, wide geographical representation, ease of scheduling, and anonymity, which often means that participants are less inhibited in answering questions. Observational research involves real-time immersion in a natural setting (e.g., a clinic) and allows the researcher to capture the richness of everyday clinical practice. This approach can be valuable in developing an understanding of, and appreciation for, the specific setting of an educational or quality improvement intervention, and in identifying site-specific issues that could pose barriers or be enablers to learning and implementation of new practices. Observational research can also be used to provide rapid, real-time process feedback. Finally, any type of document can be used as a qualitative source for analysis, such as diaries, practice logs, policy documents, posters, film, or discussion transcripts from social media forums.

There are also many analytic strategies for textual and visual data that are supported by software programs that store, systematically retrieve, and support coding. A grounded theory approach uses a method of constant comparison to explore the data without prejudice in an open-ended way. Content analysis counts the frequency of pattern occurrences within the data to identify the strength of participant perspectives on a topic or theme. Narrative analysis involves looking at one dataset as a whole to identify the stories that participants tell (content) and the way they tell the stories (framing). This approach takes advantage of the way stories function across most cultures to help us categorise and make sense of our experiences.

Conclusion

Qualitative approaches to outcomes evaluation are based on an assumption that human behaviour is not only cognitive, but is framed by social and cultural context, and is influenced by beliefs, attitudes and values. The addition of qualitative approaches to outcomes evaluation allows us to ask “how” and “why” questions and investigate the processes through which learning occurs. These characteristics make qualitative methodology an especially good fit for education interventions designed to foster changes in clinician or patient behaviours because they recognise the complexity of change and lend themselves to in-depth exploration of the context and lived experiences of intervention[18]. Quantitative-only outcomes assessment reinforces an approach to learning as an orderly input-output model rather than the rather more messy, non-linear but ultimately fertile experience that qualitative research reveals.

Acknowledgements

The authors are grateful to participants of the three education programs described in this manuscript for sharing their experiences.

Disclosure Statement

The authors have nothing to disclose.

Additional information

Funding

These educational activities in this manuscript were supported by educational grants from the following companies: 1) B Cell NHL (Pharmacyclics, LLC https://www.pharmacyclics.com/, an Abbvie Company; Janssen Biotech, Inc., administered by Janssen Scientific Affairs, LLC; Genentech), 2) PBA (Avanir Pharmaceuticals https://www.avanir.com/), and 3) Diabetic Retinopathy (Genentech).