Can Outcome Research Respect the Integrity of Chaplaincy? A Review of Outcome Studies

Abstract

In recent years, some within chaplaincy have advocated for a stronger focus on outcomes, including outcome research, whereas others in the field have questioned an outcome-oriented perspective. In this article, existing outcome studies are reviewed in relation to the ongoing discussion about a process- or outcome-oriented approach to chaplaincy. A central question emerges from this discussion: how can outcome research be designed that respects the integrity of the profession of chaplaincy? A literature search in MEDLINE/Pubmed produced twenty-two chaplaincy outcome studies that met the inclusion criteria. A review of these studies shows that thus far most have focused on secondary chaplaincy outcomes (e.g., satisfaction) using quantitative designs. To respect the integrity of chaplaincy, it is recommended that future studies should also focus on characteristic chaplaincy outcomes, use mixed methods designs, and articulate more clearly how their chosen outcomes, outcome measures, and interventions relate to the work of chaplaincy.

Introduction

Recent years have seen the growth of research in the profession of chaplaincy. Specifically, the importance of research into the outcomes of chaplaincy care has been emphasized by some chaplaincy leaders and practicing chaplains. O'Connor and Meakes (1998) and Peery (2012), for example, argue that outcome research is important for quality development within the profession because chaplains need to know if the care they deliver results in the outcomes they hope to achieve. From another perspective, Handzo, Cobb, Holmes, Kelly, and Sinclair (2014) argue for outcome research because, in the context of modern evidence-based healthcare, the “funding of interventions and care providers is increasingly evaluated against the data for the efficacy of the intervention” (p. 43). They emphasize that chaplains are not exempt from this imperative and are required to prove their added value through outcome research. In Europe, the European Network of Health Care Chaplaincy issued the Salzburg Statement, calling for high quality research into healthcare chaplaincy outcomes in order to provide best spiritual care (2014). In a secularized country such as The Netherlands, the Dutch Association of Spiritual Caregivers prioritizes outcome research to expand the legitimacy and basis for funding of the profession (Vereniging voor Geestelijk VerZorgers, 2014, p. 8). Finally, in two studies in the United States and the Netherlands, chaplains describe outcome research as a top research priority for the coming years (Damen, Delaney, & Fitchett, 2018; Damen, Schuhmann, Lensvelt-Mulders, & Leget, 2019).

Yet, outcomes and chaplaincy care are not obvious partners. Historically, the profession has valued personal competence, presence, and relationship building (inspired by, e.g., Sigmund Freud, William James, Carl Rogers and, in the Netherlands, Andries Baart) over methods and outcomes (Gleason, 1998; Kruizinga, 2017). From this perspective, chaplaincy is often characterized as process- rather than outcome-oriented, and chaplains are mostly familiar with thinking about process instead of outcome (Cadge, Calle & Dillinger, 2011; Lyndes et al., 2012). This began to change at the turn of the 21st century, when some chaplains started advocating for a stronger focus on outcomes (e.g., see VandeCreek & Lucas, 2001; Vandenhoeck, 2007 ). Following Gleason (1998), Peery refers to a paradigm shift within chaplaincy, calling outcome-oriented chaplaincy the “operant paradigm for professional chaplaincy in the twenty-first century” (2012, p. 346). Some chaplains support this shift, as the recent calls within the literature demonstrate. Others, more cautious, suggest a critical approach toward outcomes (Jorna, 2005; Nolan, 2013, 2015), pointing out that “the question for chaplains is whether adapting to the new reality will be at the cost of distorting what we value and believe to be important about spiritual care” (Nolan, 2015, p. 94).

This article will discuss the role of outcome research in chaplaincy. Our central question is: How can outcome research be designed that respects the integrity of the profession of chaplaincy? To answer this question, we first start with a review of existing chaplaincy outcome studies in order to not only discuss the question theoretically but also look at empirical examples. We restrict ourselves to quantitative and mixed methods outcome studies. These studies are closer to the peak of the evidence-based pyramid (the level of cohort studies and higher), using the types of evidence considered to be the most reliable in the healthcare research paradigm (Straus, Glasziou, Richardson, & Haynes, 2019).

Second, to get insight into how these studies relate to chaplaincy values, we discuss the arguments about process- or outcome-oriented chaplaincy that influence chaplains' outlook on outcome research. Third, based on the arguments for or against outcomes in chaplaincy, we identify four questions that arise when designing chaplaincy outcome research and then critically discuss how to address these questions in order to respect the integrity of the profession. Finally, we turn back to the existing studies to examine how they deal with these questions and what we can learn for future outcome research. In the conclusion, we answer our central question.

Review of outcome studies: Method and results

Method

Eligible studies were identified by searching the MEDLINE/PubMed database following the PRISMA checklist for systematic reviews (Moher, Liberati, Tetzlaff, Altman, and PRISMA Group, 2009). The final search was run on April 24, 2018 with the following keywords: (Chaplain* OR pastor* OR “spiritual care*”) AND (outcome* OR effect*). No restrictions were made regarding the publication date. To supplement the search, review articles of chaplaincy research were scanned for relevant studies (Fitchett, 2017; Galek, Flannelly, Jankowski, & Handzo, 2011; Jankowski, Handzo, & Flannelly, 2011; Kalish, 2012; Lichter, 2013; Morgan, 2015; Mowat, 2008; Pesut, Sinclair, Fitchett, Greig, & Koss, 2016; Proserpio, Piccinelli, & Clerici, 2011; Steinhauser et al., 2017). Included articles were (a) peer-reviewed, (b) published in English, and (c) discussed outcomes for patients and/or relatives in healthcare. The flow chart of the search is presented in Figure 1.

Figure 1. Literature search.

Results

Twenty-two articles reporting chaplaincy outcome research were identified. The articles are presented in Table 1. When we look at the chosen outcomes for the studies, we see that satisfaction with (chaplain) care is the outcome that is used most frequently. Other frequent outcomes include quality of life, anxiety, depression, religious coping, and spiritual well-being. Most studies focus on patient-reported outcomes from a wide variety of validated questionnaires; only four studies focus on other types of outcomes, such as care received at the end of life, medical costs, length of stay, and care visits (Balboni et al., 2010; 2011; Iler, Obenshain, & Camac, 2001; Rabow, Dibble, Pantilat, & McPhee, 2004). Many of the studies examine the impact of chaplain care by itself. Several of the studies examine the effects of chaplain care as a component of a multi-disciplinary intervention (Balboni et al., 2007, 2010, 2011; Daaleman, Williams, Hamilton, & Zimmerman, 2008; Rabow et al., 2004; Rummans et al., 2006; Sun et al., 2016; Williams, Meltzer, Arora, Chung, & Curlin, 2011). Most studies have a quantitative design; three studies add a qualitative component, resulting in mixed methods designs (Berning et al., 2016; Daaleman et al., 2008; Steinhauser et al., 2016).

Table 1. Outcome studies.

First author, year, journal	Research design	Sample	Intervention	Outcome	Outcome measure	Results
Balboni et al., 2007, Journal of Clinical Oncology	Cross-sectional survey	Patients with advanced cancer (N = 230)	Patient-reported level of spiritual support from the medical team (including a chaplain) and/or religious community at study entry	Quality of life	McGill Quality of Life questionnaire	Higher spiritual support is positively associated with patient quality of life
Balboni et al., 2010, Journal of Clinical Oncology	Prospective cohort study	Patients with advanced cancer (N = 343)	Patient-reported level of spiritual support from the medical team (including a chaplain) at study entry	Care received at end of life Quality of life	Hospice care at end of life Aggressive care at end of life (care in an intensive care unit, ventilation, or resuscitation) McGill Quality of Life questionnaire	Spiritual support is associated with greater hospice utilization and, among high religious copers, less aggressive care at end of life. It is also associated with better patient quality of life near death
Balboni et al., 2011, Cancer	Prospective cohort study	Patients with advanced cancer (N = 339)	Patient-reported level of spiritual support from the medical team (including a chaplain) at study entry	Medical costs in the final week of life	Medical costs in the final week of life	Patients who report that their spiritual needs are insufficiently supported have increased medical costs in the final week of life
Bay, Beckman, Trippi, Gunderman, & Terry, 2008, Journal of Religion and Health	Randomized controlled trial	Coronary artery bypass graft patients (N = 166)	Five chaplain visits to the intervention group, none to the control group. Visits were somewhat standardized	Anxiety Depression Hope Religious coping	Hospital Anxiety and Depression Scale Herth Hope Index Brief Religious Coping Scale Religious Problem Solving Scale	Chaplain visits are associated with increased positive religious coping and decreased negative religious coping
Berning et al., 2016, Annals of the American Thoracic Society	Quasi-experimental Mixed methods	Mechanically ventilated patients in the ICU (N = 50, of which N= 25 for quantitative study and N = 26 for qualitative study)	One visit of a chaplain using a picture guide for spiritual care	Anxiety	100-mm visual analog scale Semistructured interviews	Chaplain visits using the picture guide are associated with a reduction in anxiety
Daaleman, Williams, Hamilton, & Zimmerman, 2008, Medical Care	Cross-sectional survey Mixed methods	Family members of decedent residents from residential care/assisted living facilities and nursing homes (N = 284)	Spiritual support from multiple sources including chaplains	Satisfaction with overall care for decedent	Impression of overall care (4 very good - 1 poor) for decedents Semistructured interviews	Patients who received spiritual support were perceived by family members to have had better overall care
Donahue, 2017, Journal of Palliative Medicine	Cross-sectional survey	Parents of formerly hospitalized children (N = 74)	One or more visits of a hospital chaplain	Satisfaction with hospital care	Investigator-developed survey	Chaplains positively influence satisfaction with hospital care
Iler, Obenshain, & Camac, 2001, Chaplaincy Today	Randomized controlled trial	Patients with COPD (N = 49)	On average 4.2 visits by a chaplain, none to the control group	Anxiety Length of stay Satisfaction with hospital care	Beck Anxiety Inventory Length of stay Patient rating of satisfaction	Chaplain visits are associated with a decrease in anxiety and length of stay, and a higher satisfaction with care
Johnson et al., 2014, Critical Care Medicine	Cross-sectional survey	Family members of patients who died in the ICU (N = 118)	One or more visits of a chaplain or chaplaincy student reporting their activities	Family members' satisfaction with ICU care and decision-making	Family Satisfaction in the ICU questionnaire with an additional question about spiritual care	Chaplain care in general and care regarding decision-making specifically, is associated with increased satisfaction with care
Kestenbaum et al., 2017, Journal of Pain and Symptom Management	Pre-post test	Patients with advanced cancer (N = 31)	Three chaplain visits using the Spiritual Assessment and Intervention Model (‘‘Spiritual AIM’’)	Cancer-related symptoms Religious coping Spiritual well-being Spiritual, existential and social distress Depression Anxiety	Edmonton Symptom Assessment Scale Steinhauser's Are you at peace? Brief Religious Coping Scale Functional Assessment of Chronic Illness Therapy - Spiritual Mini-Mental Adjustment to Cancer Patient Dignity Inventory Center for Epidemiological Studies Depression Scale Spielberger State Anxiety Inventory	Chaplain visits are associated with significant increases on the FACIT-Sp-12 Faith subscale, the MiniMAC Fighting Spirit subscale, and Mini-MAC Adaptive Coping factor. Two trends were observed, i.e., an increase in Positive religious coping on the Brief RCOPE and an increase in Fatalism (a subscale of the Mini-MAC)
Kevern, & Hill, 2015, Primary Health Care Research and Development	Pre-post test	Patients in Primary Healthcare (N = 246)	One or more consultations with a chaplain	Mental well-being	Warwick and Edinburgh Mental Wellbeing Scale	Chaplain consultations are associated with increased mental well-being
Marin et al., 2015, Journal of Healthcare Chaplaincy	Cross-sectional survey	Hospital patients (N = 8978)	Any chaplain visit (no/yes) from patient EMR	Satisfaction with hospital care	Hospital Consumer Assessment of Healthcare Providers and Systems and Press Ganey surveys	Chaplain visits are associated with higher patient satisfaction
Purvis, Crowe, Wright, & Teague, 2018, Journal of Religion and Health	Cross-sectional survey	Hospital patients (N = 93)	One or more visits of a student chaplain	Satisfaction with chaplain and hospital care	Investigator-developed survey	The majority of patients appreciated student chaplain visits and these encounters may have positively influenced their overall hospital experience
Piderman et al., 2017, Psychooncology	Pre-post test two follow-ups	Patients with neurologic illnesses and their support persons (N = 32)	Chaplain-led spiritual legacy intervention Hear My Voice	Spiritual well -being Religious coping Quality of life	Functional Assessment of Chronic Illness Therapy- Spiritual Brief Religious Coping Scale Linear Analog Scales of Assessment	Improvement on several aspects of spiritual well-being, spiritual coping and quality of life at first follow up and favorable changes in peacefulness and positive religious coping at both time points
Rabow, Dibble, Pantilat, & McPhee, 2004, Archives of Internal Medicine	Randomized controlled trial	Palliative care outpatients (N= 90)	Multi-faceted 1-year program delivered by an interdisciplinary palliative medicine team including a chaplain, that visited at least once, standard care for the control group	Physical functioning and symptoms Anxiety Depression Spiritual Well-being Quality of life Healthcare satisfaction	Rapid Disability Rating Scale – 2 University of California, San Diego Shortness of Breath Questionnaire Brief Pain Inventory 6 sleep items from the Medical Outcomes Study Profile of Mood States Center for Epidemiological Studies Depression Scale Spiritual Well-being Scale Quality of Life Scale – Cancer Version Group Health Association of America Consumer Satisfaction Survey	Improved patient outcomes in dyspnea, sleep quality, anxiety and spiritual well-being, and less primary care and urgent care visits
Rummans et al., 2006, Journal of Clinical Oncology	Randomized Controlled Trial	Radiation therapy patients (N = 103)	Eight-session structured multidisciplinary intervention (including a chaplain), standard care for the control group	Quality of life Physical symptoms Mood state Spiritual well-being	The Spitzer Quality of Life Uniscale Linear Analog Scales of Assessment Symptom Distress Scale Profile of Mood States - Short Form Functional Assessment of Chronic Illness Therapy–Spiritual	Quality of life maintained or improved in the intervention group, control participants experienced a significant decrease in their quality of life
Sharma et al., 2016, Journal of Healthcare Chaplaincy	Cross-sectional survey	Hospital patients (N = 16478)	Any chaplain visit (no/yes) from patient EMR. Chaplains documention of their interventions on a list of 59 interventions	Satisfaction with hospital care	Hospital Consumer Assessment of Healthcare Providers and Systems and Press Ganey surveys	Chaplain visits in general but especially religious/spiritual interventions are associated with higher patient satisfaction
Steinhauser et al., 2016, Palliative and Supportive Care	Pre-post test two follow-ups Mixed methods	Caregivers of patients with advanced cancer or amyotrophic lateral sclerosis (ALS) (N = 31)	Three chaplain-led Caregiver Outlook sessions via telephone focusing on (1) a relationship review, (2) forgiveness, and (3) legacy	Quality of life Spiritual well-being Mood state Depression Grief Religious coping Caregiver burden	QUAL–E (family) Functional Assessment of Chronic Illness Therapy - Spiritual Brief Profile of Mood States Center for Epidemiology Studies Depression Scale Prolonged Grief Scale Brief Religious Coping Scale Caregiver Reaction Assessment Qualitative evaluation	Quantitative outcomes were stable over time
Sun et al., 2016, Psycho-oncology	Quasi-experimental	Lung cancer patients (N = 475) and their family caregivers (N = 354)	Patients and FCGs in the intervention group were presented at inter- disciplinary care meetings and received four educational sessions that included one session focused on spiritual well-being	Spiritual well-being Quality of life	Functional Assessment of Chronic Illness Therapy-Spiritual City of Hope Quality of Life Tool	Patients of the intervention group report significant changes on the Meaning/Peace subscale. Religiously affiliated patients report better scores in the Faith subscale and items on finding strength and comfort in faith and spiritual beliefs compared to nonaffiliated patients. Nonaffiliated patients report better scores for feeling a sense of harmony within oneself
VandeCreek, 2004, Journal of Pastoral Care and Counseling	Cross sectional survey	Hospital patients (N = 1440)	One or more chaplain visits	Satisfaction with chaplain and hospital care	Patient Satisfaction Instrument-Chaplaincy	Patients are generally satisfied with chaplain care, in some hospitals more than others. Satisfaction with chaplain care is associated with greater satisfaction with the overall hospital stay
Wall et al., 2007, Critical Care Medicine	Cross-sectional survey	Family members of patients who died in the ICU or within 24 hrs of ICU discharge (N = 356)	One or more chaplain visits in 24 hours before death	Satisfaction with chaplain and hospital care	Family Satisfaction in the Intensive Care Unit	Family members are more satisfied with spiritual care if the chaplain is involved in the last 24 hours of the patient’s life. In addition, there is a strong association between satisfaction with spiritual care and satisfaction with the total ICU experience
Williams, Meltzer, Arora, Chung, & Curlin, 2011, Journal of General Internal Medicine	Pre-post test	Hospital patients (N = 3141)	Of the group who received spiritual care, 61% was given by a chaplain and 39% by the medical team or others	Satisfaction with hospital care	Inpatient Interview Survey Discharge Interview Survey including questions from the Picker-Commonwealth patient satisfaction survey	Spiritual support is associated with increased satisfaction with hospital care

When we look at the findings, nine studies find a higher satisfaction with hospital or overall care of patients and relatives who were visited by a chaplain (Daaleman et al., 2008; Donohue et al., 2017; Iler et al., 2001; Johnson et al., 2014; Marin et al., 2015; Sharma et al., 2016; Vandecreek, 2004; Wall et al., 2007; Williams et al., 2011). Three studies find a high satisfaction with chaplain care (Purvis, Crowe, Wright, & Teague, 2018; Vandecreek, 2004; Wall et al., 2007). Four studies find a positive relationship between chaplaincy care and better patient quality of life (Balboni et al., 2007, 2010; Piderman et al., 2017; Rummans et al., 2006) and four studies show an increase in spiritual well-being (Kestenbaum et al., 2017; Piderman et al., 2017; Rabow et al., 2004; Sun et al., 2016). Three studies document a decrease in anxiety by patients visited by a chaplain (Berning et al., 2016; Iler et al., 2001; Rabow et al., 2004). Two studies find an increase in positive religious coping (Bay, Beckman, Trippi, Gunderman, & Terry, 2008; Piderman et al., 2017). One study finds a decrease in medical costs at the end of life (Balboni et al., 2011); one study a shorter length of stay (Iler et al., 2001); one study better mental well-being (Kevern & Hill, 2015); and one study a decrease in dyspnea, primary and urgent care visits, and an increase in sleep quality (Rabow et al., 2004).

It is clear from the articles that the studies are first attempts to research outcomes of chaplaincy care. Most study samples are relatively small; it is often unclear what the chaplain did that brought about the effect, and, in the case of the multidisciplinary interventions, whether the effect could be attributed to the chaplain’s intervention. The results should therefore be understood as initial indications of outcomes of chaplaincy care.

Outcomes and process in chaplaincy: Incompatible views or two sides of a coin?

The studies previously discussed are viewed with skepticism by some within chaplaincy. Understanding these concerns about outcome research will give insight into how research can be designed that respects the integrity of chaplaincy. We therefore continue with a critical analysis of the central arguments put forward in the discussion about chaplaincy as a process- or outcome-oriented profession.

A first key issue presented by chaplains critical of adopting an outcome-oriented approach is the characterization of chaplain activity as “presence.” Nolan writes:

For some, the idea that chaplains are “just there” encapsulates the essence of chaplaincy. Being “just there” means chaplains are those healthcare professionals who have time for people; those who humanize a healthcare system that is driven by the adrenaline of managerialist expediency and the testosterone of medical technology (2015, p. 95).

Being there is understood as forging a caring relationship with the other without the confinement of protocols, a focus on problem-solving, or any predetermined agenda (Baart, 2002; Madison, 1998). Elements within the intervention of presence emphasize being instead of doing, person-centeredness (leaving the agenda in the hands of those being served), and formation of a healing relationship as a means of providing effective spiritual care (Baart, 2002; Bouwer, 2003; Handzo et al., 2014; Jorna, 2005; Madison, 1998; Nolan, 2015). In summary, a central objection to adopting an outcome-oriented approach is that the focus on relationship will shift toward a focus on identifying and solving problems to attain a desired outcome (Baart, 2002; Bouwer, 2003; Jorna, 2005; Nolan, 2015).

Outcome-oriented chaplaincy, rooted in the work of Lucas and colleagues, comprises a cyclical model based on assessing another's needs, hopes, and resources; determining appropriate interventions; and measuring outcomes (Lucas, 2001). Chaplains also refer to this model as the diagnostic or medical model. Those who object to this model say that it places the patient in a pathological schema in which something needs to be fixed. It invites stigmatization and dehumanization of the patient because the patient is perceived as an object instead of a subject (Bouwer, 2003; Nolan, 2015). Moreover, the model encourages standardization, a development that runs counter to the centrality of the relationship in chaplaincy practice, in which it matters who performs the intervention and in which the unique other in his/her context is the focus of the chaplain’s attention. This especially holds true for outcome research, in which standardization is required for the predictability and replicability of the results (Nolan, 2015). Standardization could furthermore mitigate variance in practice, restrict professional decision-making, and reduce whole person care by employing predefined interventions for localized needs (Nolan, 2015). Finally, it has been observed that some chaplains fear the model instrumentalizes the spiritual that actually transcends us (humanity) (see Gleason, 1998, p. 10; Smit, 2015).

Chaplains who advocate for an outcome-oriented approach counter that all chaplains, including process-oriented chaplains, carry out assessments and perform specific interventions directed at outcomes because they interpret the other's situation (diagnose) and use this information in their response (intervene) to cultivate a relationship (effect outcome) (Bouwer, 2003). They argue that in focusing solely on process, chaplains may not sufficiently attend to their interpretations, responses, and outcomes, forgoing the conscious reflection that increases accountability (Bos, 2003; Bouwer, 2003; Peery, 2012).

Furthermore, pitfalls of the outcome-oriented model can be avoided by developing narrative assessments that provide space for unique individual context and a transcendent reality; such assessments can be carried out in dialog with the patient (Bos, 2003; Bouwer, 2003). Mackor (2007) proposes to stop talking about standardization in general and start differentiating between various standards. The outcome-oriented model should also not be understood as a goal in itself but as a means to better understand the patient (Bos, 2003). A final advantage is that speaking about assessments, interventions, and outcomes aligns with the language spoken in healthcare and therefore promotes integration of chaplaincy into the healthcare system, demonstrating chaplains’ added value to clinical colleagues (Bouwer, 2003; Handzo et al., 2014; Lucas, 2001; Peery, 2012). The emphasis on evidence-based chaplaincy (Fitchett, White, & Lyndes, 2018) may create related concerns for some chaplains. Frequently, these concerns are based on a misunderstanding of evidence-based practice that includes care informed by the best available research but uses clinical judgment to apply that care in light of patient/client unique values (APA Presidential Task Force on Evidence-Based Practice, 2006).

In order to arrive at a nuanced understanding of what it means to respect the integrity of chaplaincy in outcome research, it seems important to take the discussion a step further beyond either-or positions. Models can be found for this in the domain of psychotherapy—where a similar discussion around outcomes is taking place—that support the idea that the focus on process and the focus on outcome “should not really be considered as separate, but rather as two sides of a coin” (Ardito & Rabellino, 2011, p. 2). An understanding of the connection between processes and outcomes in chaplaincy can be advanced by the view that chaplaincy practice—like other professions—is directed toward and guided by an aspiration to certain “visions of the good” (MacIntyre, 2007; Schumann & Damen, 2018; Sennett, 2009; Taylor, 1989). MacIntyre (2007) argues: “every activity, every enquiry, every practice aims at some good” (p. 173).

Likewise, chaplains aim at effecting some change for the better in their practice. Within chaplaincy, goods are the ends that, however implicit, chaplains strive towards, and these goods constitute what they see as changes for the better. We may understand outcomes in terms of these goods. Chaplaincy practices have a good outcome when they contribute to chaplaincy goods and a bad outcome when they do not. As a result, outcomes, informed by visions of the good inherent in chaplaincy, necessarily resonate in chaplaincy practices.

In this view, processes and outcomes are intrinsically connected in chaplaincy; processes are directed toward outcomes and outcomes guide processes. This also applies to a process-oriented approach to chaplaincy. We might say that chaplains who use a process-oriented approach work with an “agenda” consisting of visions of the good to which they hope to contribute. This is not the same as having an agenda of what they wish to achieve in a particular encounter. Thinking about the outcomes of chaplaincy is then related to the question of what are visions of the goods that guide chaplaincy practices.

In order to evaluate the contribution that chaplains make to chaplaincy goods in certain situations, we need to translate the concept of chaplaincy goods into concrete standards applicable to actual practices. This is where outcome research is important. It involves systematically thinking about the question of how to translate visions of the good into manageable outcomes that allow for empirical exploration. In the case of chaplaincy this is obviously not a straightforward question, bearing in mind the issues raised in the discussion about process- and outcome-oriented chaplaincy. Furthermore, chaplaincy goods involve concepts like spirituality and transcendence, which are notoriously difficult to define, let alone explore empirically. Finally, there is minimal consensus within the profession about what are essential chaplaincy goods and relevant chaplaincy outcomes (Handzo et al., 2014). None of these are reasons to give up on outcome research into chaplaincy. These uncertainties do, however, necessitate that researchers be precise, thoughtful, and transparent in their choices when translating chaplaincy goods into outcomes that can be measured empirically. To clarify these requirements for precision, thoughtfulness, and transparency in outcome research, in the next section we take a closer look at several of the choices that researchers face when designing outcome research in chaplaincy.

Designing outcome studies in chaplaincy: Four questions

Based on the arguments in the discussion regarding process- or outcome-oriented chaplaincy, we have identified four questions that come up when translating chaplaincy goods into a specific study design: (a) What is the audience and context of our study? (b) What outcomes do we choose? (c) How do we assess these outcomes? (d) Do we choose to study outcomes in the form of effects of standardized interventions? The answer to (a) will influence the answers to the other three questions.

What is the audience and context of our study?

The first question we need to answer when designing a chaplaincy outcome study concerns the audience of our study: whom do we address with our study? In doing empirical chaplaincy research, we give an account of chaplaincy practices. Giving an account involves addressing an audience, the people to whom we give the account. This audience may just consist of oneself when the motive for doing research is the desire to answer the question: do I deliver good care as a chaplain? When we publish our research, however, we have a larger audience than just ourselves in mind. In chaplaincy research, an obvious audience that we may address is other chaplains. If we also have a strategic objective, then we need to ask ourselves whom we want to convince of the value of healthcare chaplaincy through this research and what this implies for choices in designing an outcome study.

The choice of the study audience indicates the inevitable political aspect of outcome research: the choices we make when designing an outcome study have a bearing on its status as evidence of the value of chaplaincy. In designing outcome research, we cannot avoid issues of power: who has power in deciding what counts as good healthcare, as appropriate outcomes, and as convincing research. For instance, at present, research in healthcare follows the paradigm of evidence-based medicine, valuing quantitative research in the form of randomized controlled trials (RCTs), systematic reviews, and meta-analyses as the highest levels of evidence (The Council for Public Health and Society, 2017). This research follows a positivist natural science paradigm. Researchers get to know reality by deducing general laws from empirical observations in standardized experiments. With its emphasis on individual processes of meaning-making, chaplaincy has a strong connection with the constructivist paradigm, according to which facts cannot be separated from interpretation and meaning making (Gergen, 2001).

The previous section showed that, in line with this difference in paradigms, research following the positivist paradigm is often seen as incompatible with chaplaincy values. Taking the political aspect of outcome research into account, however, it may be advisable—to be accepted in certain contexts—to follow the dominant paradigm in healthcare outcome research when designing outcome studies in chaplaincy (Leget, 2017).

Yet, more generally, we may argue that to respect the integrity of chaplaincy we need different narratives regarding the value of chaplaincy, not only in order to have this value recognized by different audiences, but also in order to enrich our accounts of chaplaincy. Carl Rogers (1963) distinguishes three types of knowing, all of which are required for the scientific understanding of practice: objective knowing, involving the measurement of operationalized constructs; subjective knowing; and interpersonal knowing (Balkin, 2014). In this view, different forms of research add to a better understanding of chaplaincy practice.

What outcomes do we choose?

The second choice we need to make when designing an outcome study in healthcare chaplaincy concerns the outcome itself. Do we want to study characteristic chaplaincy outcomes that represent intrinsic chaplaincy goods, or do we want to study secondary outcomes of chaplaincy care (e.g., clinical or physiological outcomes)?

With regard to the first choice, we have already pointed out that there seems to be no shared understanding among chaplains of what constitute characteristic chaplaincy outcomes. With regard to the second option, when we want to prove the value of chaplaincy in healthcare, restricting ourselves to characteristic chaplaincy outcomes may limit the evidential power of our studies. Still, secondary outcomes of chaplaincy are often regarded with suspicion by chaplains, in particular when these involve hard outcome measures such as cost reduction: “The particular outcomes the system now values, especially cost reduction, are also considered by many chaplains as antithetical to how the profession has seen itself, because they violate what chaplains perceive as their focus on the human spirit as opposed to any alignment with health care being perceived as business-related” (Handzo et al., 2014, p. 46).

It might be helpful to notice that an outcome represents a normative understanding of what is seen as important and desirable in a certain context. In particular, outcomes that are generally understood to be relevant healthcare outcomes represent specific understandings of what is considered good in healthcare. For instance, clinical outcomes represent the value that good healthcare cures people; the outcome of cost reduction represents the value that in good healthcare, resources are used efficiently.

Consequently, the question that needs to be addressed with respect to the choice of outcome is: how does the understanding of good healthcare represented by a chosen outcome relate to chaplains’ visions of good healthcare? Addressing this question may prevent us from evaluating chaplaincy using values and visions that are not consistent with chaplaincy’s own values and visions. However, it may also prevent us from too easily writing off outcomes of cost reduction or efficiency as purely “business-related” and therefore antithetical to chaplaincy values. For example, efficient use of resources may also be understood from the perspective of justice: efficiency provides good care to as many people as possible. Therefore, at the level of choosing an outcome, respecting the integrity of chaplaincy requires that we explicate the connection between the chosen outcome and chaplains’ visions of good healthcare.

How do we assess these outcomes?

Given a certain choice of outcome, the next choice we face when designing an outcome study in healthcare chaplaincy is how to assess the outcome. Assessments of outcomes are attempts to capture change that has taken place during a process. What we consider an appropriate method to assess outcomes depends on our (explicit or implicit) understanding of change. For instance, when the focus is on easily observable changes (e.g., behavioral change or symptom reduction), it is more obvious that such change might be measured quantitatively than when we focus on complex changes in, for example, experience, relationships, or meaning making.

Chaplains generally consider themselves to be dealing with extremely complex and idiosyncratic change processes, the richness and depth of which is easily lost, especially when assessed quantitatively with standardized outcome measures. However, outcome studies into chaplaincy do not necessarily need to capture the whole complexity of change that occurs in chaplaincy processes. To respect the integrity of the profession does require an explication of the kind of change we are attempting to capture and our considerations regarding the choice of particular methods and instruments.

Do we choose to establish outcomes in the form of effects of standardized interventions?

As highlighted earlier, a main objection to outcome research is that it requires standardization to make outcomes replicable and predictable (Nolan, 2015). According to this view, standardization runs counter to important chaplaincy values. The rejection of standardized interventions can leave chaplain researchers with no description of the substance of their intervention. In this situation some have said the substance of the chaplains’ intervention is “a black box” whose contents cannot be observed. While this type of research informs us about the effect of a visit by a chaplain, it does not clarify what brought about the effect. In light of our previous argument about the importance of various research types for creating a rich account of the value of chaplaincy, simply ruling out research that looks for replicable and predictable outcomes seems too hasty a conclusion.

Another possibility is to look at chaplaincy practice as consisting of several elements, some of which are appropriate for standardized research and others that are not. This requires recognition that any standardized intervention does not represent the whole of chaplaincy practice. Therefore, in respecting the integrity of chaplaincy, it is important that we explicate how the standardized intervention that we choose to research is related to chaplaincy practices more generally.

RCTs probably constitute the most obvious type of standardized outcome research. The use of RCTs is a subject of discussion not only in chaplaincy, but also in other types of counseling where the issue is whether or not the use of RCTs is at odds with a focus on the individual person and the uniqueness of each encounter. Scholl, Ray, and Brady-Amoon (2014) argue that RCTs (and, more generally, quantitative methods in outcome research) should not be written off as inevitably dehumanizing. They describe several possibilities for designing RCTs such that they do not conflict with respect for the individual person. For example, a standardized intervention might include work with the (individual) life story. They also discuss the limitations of a restriction to qualitative methods alone and advocate the use of mixed methods as the approach to research that is “the most thorough approach to research and is also the most theoretically aligned with humanistic principles” (Scholl, Ray, & Brady-Amoon, 2014, p. 233). Following these ideas, when we decide to use standardized interventions in our chaplaincy study, we may, for instance, consider how additional qualitative research might add to an understanding of the relation between intervention and outcome in descriptive terms in addition to analysis of quantitative data. If possible, we could include this in our design.

Reviewing existing outcome studies: Audiences, outcomes, outcome measures, and interventions

We will now turn back to the existing chaplaincy outcome studies to take a closer look at the choices that were explicitly or implicitly made in relation to the four questions just discussed. With respect to the first question—what is the audience and context of the study—we see that the studies address roughly two different audiences: clinicians and chaplains. Approximately three-quarters of the studies were published in medical journals and one-quarter in journals on chaplaincy or religion. There is little variation in study designs; the vast majority of the studies have a quantitative research design regardless of audience. Only three studies add qualitative methods, resulting in mixed methods designs (Berning et al., 2016; Daaleman et al., 2008; Steinhauser et al., 2016). We argued that for respecting the integrity of chaplaincy we need different narratives from different forms of research for a good understanding of chaplaincy practices. Following from the existing studies, extra attention could be paid to mixed methods designs.

Applying the second question—what outcomes do we choose—we saw before that the outcome of satisfaction with (chaplain) care was chosen most frequently. Other outcomes included quality of life, anxiety, depression, spiritual well-being, and religious coping. When we look at the chosen outcomes specifically in relation to characteristic chaplaincy goods, the majority do not appear to be outcomes exclusive to chaplaincy; many could also be considered relevant to the work of other healthcare professionals. For example, focusing on patient experience (satisfaction) or well-being is crucial in good healthcare. It is by no means incompatible with chaplaincy values but also not unique to the profession.

The outcomes that seem specific to chaplaincy, for example, spiritual well-being, religious coping, and hope, have been studied less frequently. We argued that respecting the integrity of chaplaincy in choosing an outcome requires that we explicate the connection between the chosen outcome and chaplains’ visions of good healthcare. This explanation is missing in most studies. Only a few studies give a brief description in broad terms, explaining, for example, that chaplains attend to spiritual/religious needs and psycho-emotional distress and, therefore, spiritual/religious and psycho-emotional measures are a suitable choice.

Answering the third question—how do we assess these outcomes—we see that the chosen outcomes were predominantly assessed quantitatively, using a wide variety of patient-reported questionnaires. We argued that in choosing a method to assess an outcome, respecting the integrity of chaplaincy requires explicating both what kind of change we attempt to capture and our rationale for selecting particular methods and instruments. The most frequently researched outcome, satisfaction, is an outcome that can only be assessed in hindsight and therefore does not necessarily give us information about what kind of change is associated with chaplain care.

The other outcomes in the studies referred to more complex concepts like quality of life, anxiety, or depression. Change for these outcomes was assessed with questionnaires developed in contexts broader than chaplaincy. In the studies that focus on apparent characteristic chaplaincy outcomes such as spiritual well-being, religious coping, and hope, the most often used questionnaires are the Functional Assessment of Chronic Illness Therapy-Spiritual (FACIT-SP) and the Brief Religious Coping Scale (Brief RCOPE), followed by the Religious Problem Solving Scale, the McBride Spirituality Assessment, Steinhauser's “Are you at peace?,” the Spiritual Well-being Scale and the Herth Hope Index. It is outside of the scope of this article to discuss if those questionnaires are valid measures for chaplaincy outcomes; it would be interesting for further exploration. None of the studies explicate their rationale for using these measures in relation to chaplaincy goods. Note that there is a questionnaire specifically developed for chaplaincy: the Patient Reported Outcome Measure (PROM) by Snowden and Telfer (2017). We are looking forward to further results from this questionnaire.

When we review the final question—do we choose to establish outcomes in the form of effects of standardized interventions—we see that most studies worked with a “black box” design. These studies focus on chaplaincy care as usual, without examining specific predefined interventions. It therefore remains unclear which actions of the chaplain brought about a certain effect, a problem compounded in studies in which chaplains contribute to multidisciplinary interventions.

Some studies chose to document the activities of the chaplain (Johnson et al., 2014; Sharma et al., 2016), giving some information about what could have brought about the effect. Several studies examined a predefined structured intervention, in which it was clear what the chaplain was going to do, despite strong resistance within the field to standardizing interventions. Interestingly, however, the studied interventions;Spiritual AIM (Kestenbaum et al., 2017), Hear My Voice (Piderman et al., 2017) and the Caregiver Outlook (Steinhauser et al., 2016)—offered a framework or semistructured protocol while also keeping the content of conversation highly individualized, since the interventions are intended to enable clients to relate personal stories. We argued that respecting the integrity of chaplaincy requires explicating how the standardized intervention is related to chaplaincy practice more generally. All three studies clearly explicate how the interventions relate to chaplaincy practice, pointing out that the interventions are embedded in the profession's focus on meaning-making, life review, relational well-being, and sense of loss and hope.

Conclusion

In this article we first explored the question: how can outcome research be designed that respects the integrity of the profession of chaplaincy? We started with a literature search of outcome studies to situate these studies in the ongoing discussion about process- or outcome-oriented chaplaincy. Twenty-two articles reporting chaplaincy outcomes were identified that met inclusion criteria.

Second, we reviewed the discussion about chaplaincy as a process- or outcome-oriented profession to get a better understanding of the resistance to outcome research. We stated that chaplaincy practice is ultimately guided by certain visions of the good that can also be understood as outcomes of chaplaincy. We concluded that chaplaincy is necessarily outcome-oriented because it is aimed at supporting some change for the better towards these goods. Hence, processes and outcomes are connected; processes are directed toward outcomes and outcomes guide processes.

Third, we developed four questions from our understanding of the resistance to outcome research, questions that are relevant when relating chaplaincy outcomes to a specific study design: (a) what is the audience and context of the study? (b) what outcomes do we choose? (c) how do we assess these outcomes? (d) do we choose to establish outcomes in the form of effects of standardized interventions? We concluded that respecting the integrity of chaplaincy in study design means: first, we need various narratives in order to get a rich account of the value of chaplaincy that can be recognized by different audiences. Quantitative as well as qualitative types of research can provide this evidence. Second, when choosing an outcome for our study, we have to explicate the connection between the chosen outcome and chaplains’ visions of good healthcare. Third, we need to specify what kind of change we attempt to demonstrate in our study and explain our rationale for using particular methods and instruments. Finally, because no standardized intervention represents the whole of chaplaincy practice, we need to explicate how the researched intervention is related to chaplaincy practice more generally.

We then reviewed existing outcome studies with the help of these considerations. We found that the studies were aimed at two different audiences. Most studies were quantitative, but some added qualitative methods, resulting in mixed methods designs. Only a few of the outcomes studied could be described as outcomes unique to chaplaincy, for example, spiritual well-being, religious coping, and hope. Other outcomes used, such as satisfaction with care, quality of life, anxiety, and depression are also relevant for other professions. The outcome measures used were developed and validated for broader contexts than just chaplaincy. Most studies did not explicate how their outcomes and outcome measures related to chaplaincy practices specifically. Most of the research was “black-box” research; that is, the studies examined chaplain care but remained unclear what the chaplain did that brought about the effect. A few studies focused on more standardized interventions which followed a protocol but invited a highly individualized response. These studies did explicate the relation between the standardized interventions and practices within chaplaincy.

To respect the integrity of chaplaincy, it is recommended that future outcome studies more often use mixed methods designs in order to give a broader account of the variance in chaplaincy practices and to continue reaching different audiences. Furthermore, future studies should focus more explicitly on characteristic chaplaincy outcomes than existing studies, which thus far mainly look at general outcomes of chaplaincy care. As this requires well-founded ideas about what are understood as distinctive features of chaplaincy, we want to point out the importance of having a thorough and continuing dialog among chaplains and chaplaincy researchers about what are characteristic chaplaincy goods and their related outcomes. Finally, more studies could be conducted that focus on opening the “black box” of chaplaincy care. This is an area where the growing body of chaplain case studies may be helpful (Fitchett & Nolan, 2015, 2018).

Because of the brief history of chaplaincy research and the minimal resources available, most studies so far have been opportunistic in that chaplaincy researchers have taken advantage of available opportunities. We hope that with growing resources the time may be ripe for more intentionally developing chaplaincy outcome research. This would include explicating in research designs how the studied outcomes, outcome measures, and interventions relate to the chaplaincy profession.

Acknowledgment

We appreciate the assistance of Marcia Nelson in preparing the manuscript for publication.