Research evidence on psychotherapist skills and methods: Foreword and afterword

Abstract

This article serves as both the foreword and the afterword to the special section of Psychotherapy Research devoted to research reviews of psychotherapist skills and methods: it introduces the interorganizational Task Force that guided the reviews and then features its conclusions. We begin by operationally defining therapist skills and methods and then contrasting them with other components of psychotherapy. Next, we consider the typical assessment of skills and methods and how they are linked with outcomes (immediate in-session, intermediate, and distal) in the research literature. We summarize the strength of the research evidence on the skills and methods reviewed in the 8 articles in this special section and in the companion special issue in Psychotherapy. We end with diversity considerations, research limitations, and the formal conclusions of the interorganizational Task Force on Psychotherapy Skills and Methods that Work.

Keywords:

• psychotherapy
• psychotherapy process
• psychotherapy outcome
• psychotherapy skills
• psychotherapy methods
• meta-analysis
• therapist-client interactions
• psychotherapy research

Clinical significance statement: Skills and methods are fundamental components of psychotherapy, alongside the treatment approach, client and therapist factors, and the therapeutic relationship. We provide research evidence for the effectiveness of 27 therapist skills and methods in terms of immediate in-session, intermediate post-session, and distal end-of treatment outcome.

A warm welcome to this special section of Psychotherapy Research dedicated to research on psychotherapy skills and methods. This section, and a parallel special issue of Psychotherapy, features invited research reviews on 27 skills and methods based on a mammoth collaboration among dozens of scientist-practitioners and four professional organizations dedicated to improving psychotherapy. Our goal is to provide research and clinical evidence on what we know about the effectiveness of therapist skills and methods.

Psychotherapists must do something in their sessions—listen, reflect, question, encourage, challenge, interpret, guide, instruct. It would thus be helpful for psychotherapists across theoretical orientations to have evidence about when, how, under what conditions, and with whom to use these various skills and methods to guide their practice and training.

Whereas there is considerable research evidence for the overall effectiveness of psychotherapy (Wampold & Imel, 2015) and for the association between a strong therapeutic relationship and positive outcome (Norcross & Lambert, 2019), there is, surprisingly, less empirical support for the effectiveness of therapist skills and methods. We strive to redress that deficiency by focusing on skills and methods that can be used across theoretical perspectives.

This article serves as both the introduction (foreword) and the conclusion (afterword) of the special section. The section is comprised of this article and research reviews on eight therapist skills and methods: empathic reflections (Elliott et al., this issue), Socratic questioning (Overholser & Beale, this issue), facilitating dyadic synchrony (Atzil-Slonim et al., this issue), routine outcome monitoring (Barkham et al.), strength-based methods (Fluckiger et al., this issue), meditation, mindfulness, and acceptance methods (Goldberg et al., this issue), emotional regulation (Iwakabe et al., this issue), and behavioral activation (Cuijpers et al., this issue). We define and contextualize therapist skills and methods, describe how skills and methods are assessed and linked to outcomes (immediate in-session, intermediate, and distal). We outline the purposes and processes of the interorganizational Task Force that guided this work, and summarize the associations between skills/methods and outcomes, highlighting diversity considerations and research limitations. Finally, we present the formal conclusions of the Task Force of Psychotherapy Skills and Methods that Work.

Conceptualizing and Contextualizing Therapist Skills and Methods

Psychotherapist skills (e.g., effective use of questions) and methods (e.g., effective use of role induction) matter, although in a complicated and nuanced interaction with therapist effects, patient contributions, the therapeutic relationship, and evolving responsiveness. As frequently lamented, what contributes to psychotherapy effectiveness “is messy,” but the common denominator and sine qua non of all perspectives on what accounts for psychotherapy outcome is that the therapist does something—uses skills and methods. There is no psychotherapy without therapist behavior; it is consequential and essential to the enterprise.

The effects of the hundreds of therapist skills and methods on psychotherapy outcome is admittedly complicated. Skills and methods are not used in isolation, but rather build on what has gone before. For example, some therapists try to establish a working relationship and have clients explore before they move to insight and action, while others may start with behavioral activation and then move to exploration (Hill, 2020). Further, a variety of skills and methods can work at any given time, although all might lead to slightly different outcomes.

These skills and methods exert immediate impacts on session process and also interact with other skills and methods as well as extra-therapy experiences to influence distal outcomes. Further, skills and methods are impacted by and impact other components of psychotherapy process (and life outside of psychotherapy) such that it is difficult to tease out unique impacts.

The relative contribution of therapist behaviors can be disentangled when we construct research paradigms and analytic methods for capturing the complexity of psychotherapy. Such research can direct practitioners in what to offer their patients and faculty in how to train their students. The major limitation is that such research is more complex and strains our current analytic abilities.

We conceptualize the overt behaviors of psychotherapists as occurring at four nested levels of abstraction (see Table I). At the highest level are large, multifaceted brand-name treatment approaches (packages or theoretical orientations), such as psychodynamic, cognitive, and experiential, which involve a multitude of skills and methods often applied by therapists in idiosyncratic ways. Although these overall approaches share some common factors (e.g., the therapeutic relationship, a credible rationale, Wampold & Imel, 2015), they also have some skills and methods that differentiate them (Blagys & Hilsenroth, 2000, 2002).

Table I. Four levels of overt therapist behavior.

Level of Abstracton	Therapist Behavior	Examples
High	Treatment packages/theoretical orientations	Psychodynamic, cognitive-behavior, experiential, systemic
High-Medium	Change principles/processes	Consciousness raising, commitment, counterconditioning
Medium	Methods	Behavioral activation, chair work, paradoxical intervention
Low	Skills	Questions, empathic reflections, affirmations, metaphors

At a second level of abstraction, between global theories and specific methods, are change principles or processes (e.g., raising awareness and counterconditioning) that cut across treatment packages. It is at this intermediate level that meaningful points of convergence are probably found among psychotherapy systems—such as enhancing the patient’s awareness of the problem or defenses and teaching them skills that counter or inhibit the problem (Castonguay et al., 2019; Goldfried, 1980; Prochaska & Norcross, 2018).

At the next level of abstraction are psychotherapy methods, such as chair work and teaching mindfulness. These have also been characterized as techniques, but we avoid that term here because it has also been used to characterize entire treatment packages. Although methods were typically developed within a certain theoretical framework (e.g., relaxation within behavior therapy), they can be used pragmatically by therapists of all theoretical orientations to address specific client concerns; for instance, a cognitive-behavioral therapist might use an experiential exercise to arouse emotions.

At the lowest level of abstraction are relatively discrete therapist or helping skills (sometimes also called verbal response modes), such as empathic reflections and immediacy, used by therapists to respond to or direct client behavior in the immediate moment in sessions. These skills are readily taught, as is evident from the empirical evidence for their effectiveness (Knox & Hill, 2021). These skills are sometimes packaged into methods, but often they are used more loosely to accomplish specific goals/intentions for how the therapist wants the client to respond in the moment (e.g., express feeling, practice new behaviors).

Hence, in a complementary and hierarchical fashion, skills are nested within methods, which are in turn embedded within change principles, which are embedded within treatment approaches. Therapists behave at any given moment in terms of their overall treatment approach, their principles about what promotes change, their methods, and their moment-to-moment skills.

Assessing Skills and Methods

The most frequent approaches for assessing psychotherapist skills and methods are: (a) estimates by trained judges of verbal response modes (and other skills) used within sessions; (b) quantitative ratings by therapists and clients of in-session behaviors via post-session or post-treatment report; (c) qualitative assessments of skills and methods; and (d) open-ended interviews of participants about the skills/methods.

Codings by Trained Judges of Verbal Response Modes

Starting soon after the invention of recording machines, coding systems were developed for counting therapist verbal response modes. A notable early example was Snyder’s (1945) system for categorizing therapist and client behaviors into verbal response modes (VRMs), with more recent versions having been developed by Hill (1978, 2020) and Stiles (1979). These systems typically rely on trained judges coding VRMs in grammatical sentences from transcripts or videos of psychotherapy sessions. Some consistency has been found among category systems, indicating that a few VRMs can be reliably identified (Elliott et al., 1987).

An advantage of this approach is that the VRMs can be identified in sessions from watching videotapes or reading transcripts. Another advantage is that we know which skills “actually” transpired in sessions, as opposed to the distortions of subjective recall or self-report. A disadvantage is the reliance on overt behaviors rather than inner reactions. In addition, although it seems relatively straightforward to code therapist behaviors into a few discrete categories, many skills fall into gray areas that prove difficult to judge. For example, “Are you sure?” is grammatically a closed question, but might serve as a reflection or as a challenge. Moreover, each skill can be divided into multiple subcategories, which may exert differing effects. For example, questions can be used to gather facts, to explore thoughts or feelings, to probe for insight, or to encourage action ideas.

Compounding the complexity is that each therapist employs these skills in an individualized, probably idiosyncratic manner. Some therapists ask closed questions in an inviting way whereas others ask them in a clipped interview-like manner. These individual differences and the complexity of psychotherapy make it difficult to collapse skills into a few categories. Having many response categories with only a few instances in each, however, causes difficulties in data analyses. Further, process coding is time- and labor-intensive (e.g., multiple judges are needed given relatively low agreement levels among judges), which leads to challenges in gathering sufficient data for analyses.

Retrospective Reports of In-Session Behaviors

The frequency of therapist skills and methods have also been assessed through post-session reports (e.g., Comparative Psychotherapy Process Scale, Hilsenroth et al., 2005; Multitheoretical List of Therapeutic Interventions, McCarthy & Barber, 2009; Helping Skills Measure, Hill & Kellems, 2002; Psychotherapy Q-Set, Jones & Pulos, 1993). On these measures, clients, therapists, or trained judges estimate how often or how well therapists used the skills within a given session or treatment.

An advantage of post-session measurement is the ease of data collection, given that administration requires only a few minutes after a session or treatment. The patient’s lived experience of psychotherapy is quickly captured in terms of how they felt and responded to therapist behaviors. In addition, the results can easily be correlated with other post-session measures (e.g., of the therapeutic relationship) and with treatment outcome (e.g., changes in symptomatology), using sophisticated statistical methods such as multilevel modeling, structural equation modeling, and moderation/mediation analyses (Crits-Christoph & Gibbons, 2021).

Unfortunately, these retrospective ratings are often impressionistic summaries (halo effects) about what transpired rather than documentable records of what actually occurred. Some clients provide completely positive or negative ratings on post-session measures because they liked or disliked their therapist. Our own experience as psychotherapists completing these measures is that they prove valuable in gauging the overall response, but that it is incredibly challenging to estimate how much or how well skills were used given that memory is notoriously biased. They do not provide valid identification of specific skills nor do they allow for linking with consequences (e.g., assessing the effects of specific transference interpretations in the moment).

Qualitative Assessments of Skills and Methods

Trained judges can observe sessions and conduct qualitative analyses of skills/methods. For example, using a large team of trained judges, Hill and colleagues (2022) identified all therapist challenges in the initial, middle, and final sessions of one case. For each identified challenge, the researchers consensually assessed the type, manner of delivery, quality, and intent of the antecedents, challenge event, and consequences; they then aggregated across events to summarize when and why challenges were effective.

Advantages of this approach are that researchers can focus on one skill/method at a time within a given case and include multiple perspectives on the research team. Disadvantages are the time-intensive nature of the process and the likelihood of clinical and theoretical biases.

Open-Ended Interviews of Participants

Clients and clinicians can be interviewed about therapist skills or methods. For example, Knox and associates (1997) interviewed clients about their memories of salient therapist self-disclosures. These interviews are then typically analyzed utilizing rigorous qualitative approaches, such as critical-constructivist grounded theory (Levitt, 2021), descriptive-interpretive qualitative research (Elliott & Timulak, 2021), or consensual qualitative research (CQR; Hill & Knox, 2021).

An advantage of open-ended interviews is that therapists and clients are the ones who identify and locate the skill or method, so we can be sure it registered with them. By contrast, even if the method appeared in a session transcript or in a video to an external observer, there is no guarantee that the client heard or absorbed it. Disadvantages are that interviews are often conducted long after the sessions were conducted; perspectives are thus altered by what occurred later in the session or after the treatment. It also often proves difficult to link the event described in the post-session interview with transcripts of what happened overtly in the session. Some participants also are unable to know and articulate what they were experiencing at the time and the process of talking about the events frequently alters perceptions of events (reactivity).

Overall Considerations

The different methods of assessing therapist skills and methods likely yield quite different results, although few studies have tested this question empirically. In one study, a disturbingly low correlation was found between items about skills on judge-rated post-session measures of therapists’ skills, on the one hand, and judges’ coding of actual skills (VRMs) used within sessions, on the other (Heaton et al., 1995). These findings suggest that disparate results might emanate from differing measures and perspectives, raising concerns about validity and generalizability. If researchers want to determine the effectiveness of particular skills/methods, as in the present research reviews, we need a valid identification of the actual occurrence of the skill or method within the session.

One major unresolved question is whether to measure simply the presence of a therapist skill or method or to measure its quality or competence. Assessing presence is relatively easy (although we still do not obtain high levels of agreement), but assessing intensity, quality, or competence requires considerable clinical judgement and is prone to subjectivity.

Assessing Outcomes of Skills and Methods

A basic question, of course, is what we mean by “works” when we say psychotherapy skills and methods that work. Psychotherapy outcomes are, fundamentally, of three sorts: (a) proximal or immediate in-session outcomes (e.g., client’s immediate response to a specific skill or method); (b) intermediate post-session or between sessions (e.g., client’s appraisal of or satisfaction with an entire session or amount of homework completed); and (c) distal end-of-treatment outcomes (e.g., changes in a client’s symptoms or interpersonal functioning). Most practitioners and researchers would agree that all three types of outcomes are crucial and should be assessed (in some fashion), but they rarely are all assessed in a given research study. When they are, they frequently yield disparate effects (see Hill et al., 1988). Perspectives change over time given intervening events, so that distal end-of-therapy results are often different from immediate in-session outcomes. And obviously, the three types of outcomes build on each other: an accumulation of immediate effects is associated with session-by-session effectiveness, which lead to more distal outcomes (although not typically in a direct or linear way).

Proximal or Immediate In-Session Outcomes

Immediate outcomes (or impacts) involve the client response to a therapist skill/method as it occurs in a given session. Immediate outcomes have most often been assessed by having trained judges code or rate some client observable behavior. For example, Town and associates (2012) studied clients’ affect following therapists’ use of confrontation, clarification, support, questions, self-disclosure, and information. For another example, Prass and colleagues (2021) chose the immediate outcome of changes in client collaboration following therapist use of advice. An advantage of this approach is that consistent criteria and consistent units of time (e.g., the first two client sentences, a three-minute window after the skill occurs) can be used by judges across clients.

A second method has been to have clients or therapists observe a recording of the session and rate the helpfulness of each therapist speaking turn. An advantage of this method is that the perceptions of the participants (the “insiders”) can be obtained; a disadvantage is that ratings are retrospective and thus susceptible to the influence of what came after the event in the session (e.g., if there was a rupture later, the patient would probably rate the event as less helpful).

A methodological choice arises in determining the optimal unit to assess the immediate consequences. One can look for the client response in the next sentence, the next speaking turn, the next three-minute window, or the remainder of the session. Of course, not all responses are exhibited immediately and vary across clients and situations. Yet, for both practice and research, we need to connect the patient response to the specific skill or method—to determine what “works.” If we use the remainder of the session as the unit, then the client could well be responding to something other than the designated skill or method. To be completely accurate, we would need access to the client’s inner experience, and even that would change as they reflected on the skill/method over time.

The prime advantage of assessing immediate outcome is that a temporal link can be made between the skill/method and the outcome, allowing us to determine which therapist behaviors are associated with which client responses. The disadvantage is that it is time-intensive and costly to assess immediate outcome, and the direct link is confounded by all the events that have occurred previously in the treatment (e.g., the outcome of a challenge depends on its timing). In addition, outcomes might be delayed rather than immediate (clients might initially not express a reaction but later say something), or outcomes could change over time (e.g., a therapist challenge might initially hurt, but the client might later realize its value).

Intermediate Outcomes

Intermediate outcomes are assessed through ratings provided after sessions, between sessions, or before the next session. These outcomes can be assessed by clients, therapists, or trained judges using standardized, psychometrically sound questionnaires (e.g., Session Rating Scale, Miller et al., 2004; Working Alliance Inventory, Horvath & Greenberg, 1989, Session Evaluation Scale, Hill & Kellems, 2002), and are evaluative and indicate overall session satisfaction. Between-session measures include behavioral recordings of homework. Session-to-session outcomes include client ratings on the Outcome Rating Scale (Miller & Duncan, 2003) or the Outcome Questionnaire (Lambert et al., 1996) at the beginning (or end) of sessions.

An advantage of using intermediate measures is their relative ease of administration and minimal intrusion into the therapy process. Practitioners also typically profit from obtaining session-level feedback from their clients, such as in routine outcome monitoring (ROM). A disadvantage is the inability for researchers to directly link specific skills or methods with intermediate outcomes.

Distal Post-Treatment Outcomes

Post-treatment outcomes are typically assessed through clients completing standardized, psychometrically sound self-report measures (e.g., Beck Depression Inventory, Beck et al., 1961; Complementary Measure of Psychotherapy Outcome, Chui et al., 2021) before beginning psychotherapy and then again at its completion, and sometimes before or after every few sessions. In the vast majority of published randomized clinical trials (RCTs) comparing the relative efficacy of two or more treatment packages, the only measures of outcome involve distal outcomes. Similar to measures of intermediate outcomes, the relative ease of administration and minimal intrusion into the therapy process are offset by the inability to directly link specific skills/methods to outcomes.

Overall Considerations

Much has been written about assessing outcomes (e.g., Lutz et al., 2021; Ogles, 2013), so we will not cover the entire terrain here. Rather, our aim is to identify outcomes that can be linked directly with specific therapist skills and methods. In that regard, immediate outcomes have at least a temporal association with the skills/methods, whereas intermediate and distal outcomes cannot be directly linked with specific skills/methods, although they are more easily gathered and used in outcome research. Returning to our refrain of “it’s messy” and adding “it depends,” all psychotherapists recognize that the impressive effects of psychotherapy are not all immediate, not all measurable, and not at all invariant. We hope that more effective ways will be developed to assess such varied outcomes.

Linking Skills and Methods to Outcomes

Associating therapist skills and methods with psychotherapy outcomes is, simultaneously, the reward of process-outcome research and the bane of every psychotherapy investigator. How can we establish, with reasonable certainty, the causal link between what the psychotherapist does and patient improvement? Or how can we disconfirm plausible rival explanations that account for patient improvement above and beyond what the therapist does?

Here, we concentrate on the most frequent approaches for linking therapist skills/methods to outcome: (a) correlations between the frequency of skills/methods with post-session or post-treatment outcome, (b) sequential analyses linking the skill/method with outcome, (c) qualitative analyses linking skills/methods with outcomes, (d) task analyses linking the skills/methods and outcome (or relatedly called event or episode analyses), and (e) conversation analyses.

Correlations Between Frequency of Skills/Methods and Outcomes

Much of the early process-outcome research involved linking the frequency of a discrete therapist skill with session or treatment outcome. Thus, researchers often correlated the proportion of the therapist skills/methods (divided by the total number of skills/methods to control for verbal activity) to post-session or post-treatment outcomes.

For example, Hill and colleagues (1988) correlated the proportion of therapist VRMs in sessions with client outcome. In terms of session outcome, they found that when therapists gave more interpretations and less information, clients rated the depth (quality) of sessions high. In contrast, the more confrontations therapists gave, the lower therapists rated the depth of sessions. In terms of distal outcome, Hill and colleagues found that the more open questions and paraphrases therapists provided, the more clients decreased in anxiety. In contrast, the more paraphrases and the fewer approval and closed questions therapists gave, the more the client improved in self-concept.

An advance on associating simple frequency or proportion of the skill/method with outcome is multi-level modeling. Here, researchers can separate the effects of sessions, clients, and therapists (Crits-Christoph & Gibbons, 2021). Multi-level modeling thus removes variance associated with these effects, but still is problematic in that frequency is related to intermediate or distal outcome, revealing little about immediate client response.

Stiles (1988; Stiles et al., 1998) has argued that it makes little sense to correlate frequency of skills/methods with outcome given that more is not necessarily better. One good interpretation is obviously better than 10 poor interpretations. In fact, therapists might give more interpretations when clients are not responding to those already given. Or it may be that a moderate amount is necessary, and that amount might differ across skills and clients.

Likewise, the quality or competency with which the skill/method was rendered obviously makes a difference. Unfortunately, the field has not yet developed effective methods for routinely assessing quality. In addition, what clients absorb from the skills is often different from what the therapist intended (e.g., the therapist might intend to encourage the client to explore feelings but the client explores cognitions).

Sequential Analyses Linking the Skill/Method with Immediate Outcome

Researchers can directly link therapist behavior with the immediate in-session outcome using sequential analyses, under the assumption that there is a temporal link between the two. Typically, this requires having one set of trained judges code the therapist skill/method and another set code the immediate client response. Again using the Hill et al. (1988) study as an example, a sequential analysis linked the judge-coded VRMs in the therapist speaking turns with the peak client experiencing levels as rated by trained judges in the subsequent speaking turn. Therapist self-disclosure was linked with the highest subsequent client experiencing, whereas confrontation was linked with the lowest subsequent client experiencing.

Valuable additions to this method are to control for antecedent behavior when assessing the outcome of a skill/method and nesting events within clients and therapists. For example, when studying therapist advice, researchers controlled for antecedent client collaboration when examining subsequent collaboration and nested events within therapists (Prass et al., 2021).

As noted above, an advantage of sequential analysis is that it gets closer to a direct outcome link given the temporal nature. The disadvantage is that it does not address concerns that client responses do not occur at consistent intervals, in that some are delayed or evolve. Further, clients have different reactions to therapist behaviors, and sequential analysis does not rule out or control for the impact of other variables beyond the skill or method.

A variation of sequential analysis is to link the judged skill/method with participants’ judgements of outcome (most often helpfulness ratings) based on post-session reviews (here considered immediate delayed outcome). Thus, researchers can assess the perceived helpfulness of specific skills or methods. In the study cited above (Hill et al., 1988), therapists and clients watched the videotapes of session immediately afterwards and rated the helpfulness of each therapist speaking turn. Results indicated that clients rated therapist self-disclosures as the most helpful, whereas they rated open questions, closed questions, and direct guidance as least helpful. By contrast, therapists rated interpretations as most helpful and self-disclosures as least helpful.

Advantages of this methodology are that skills can be linked with direct evaluations. Disadvantages are that video reviews are time-consuming and intrusive into the treatment process. Finally, post-session evaluations tend to render different results than in-session experiences (Hill et al., 1994).

Qualitative Approaches to Linking Skills/Methods and Outcomes

A more recent approach has been to have a team of experienced judges view the skill/method within the context of the entire session and case and then consensually determine the consequences (outcomes) of skills or methods. In the study of therapist challenges described above using consensual qualitative research-cases (Hill & Knox, 2021), for instance, judges not only identified when challenges occurred but also examined their antecedents and consequences (Hill et al., 2022). Thus, researchers relied on clinical judgment to determine the outcomes for each challenge given the context of the case. Results indicated that, in the presence of a strong therapeutic alliance when the client was less defended, a clear and empathic therapist challenge led the client to become more open and reflective. By contrast, when the client was defensive and the therapist challenge was of lower quality, the client became even more defended and closed. Sophisticated qualitative research can thus generate the elusive when-then guidance that practitioners crave.

Advantages of this approach are that multiple perspectives among judges can be included and considered, hopefully leading to more valid conclusions. In addition, researchers can use their clinical judgment and consider the dynamics of the entire case. Disadvantages are the time-intensive nature of the process, the inevitable problems in establishing causality, and the likelihood of raters’ biases.

Task Analysis Linking Skills/Methods with Outcome (Event/Episode Analyses)

Task analysis examines outcomes after sequences of psychotherapist and patient behavior (Greenberg, 2007; Pascual-Leone & Greenberg, 2009). In the discovery stage, researchers identify the theorized steps of some method (e.g., rupture repair), watch multiple events to see how the process unfolds in psychotherapy, and then merge those observational results into a model of the change process. In the validation stage, researchers test the model by judges rating the client and therapist behaviors in each step using standardized measures that fit the description of that stage (e.g., client experiencing for a step of exploration, resolution for the final step) on a new sample of clients and therapists. The model is iteratively modified based on the results. By comparing replicated results across successful and unsuccessful cases, researchers can determine what probably leads to successful outcomes across cases. The usual problems with establishing conclusive causality between the therapy method and patient outcome persist, but the temporal linkages are consistent across multiple patients.

Conversation Analysis

Conversation analysts (CA; e.g., Hepburn & Potter, 2021) study both verbal and non-verbal conduct in social interactions, including psychotherapy. CA yields a detailed description not only of the sequential steps of a method but also the surrounding interactions between clinician and client, given that the same statement said in different ways can produce profoundly varying meanings and impacts (Wachtel, 2011). CA allows researchers to consider interactional nuances and, further, separates competing causal processes, thus offering reasonable approximations to causal sequences. Elliott and colleagues (this issue) provide an excellent example of how CA can be applied to investigate empathic reflections.

Overall Considerations

Alas, studies using different research approaches for linking therapist skills/methods with outcome frequently do not yield consistent results. As illustrated in the Hill et al. (1988) study referred to earlier, for example, different results were obtained from (a) correlating judgments of verbal response modes with immediate judgments of subsequent client behavior, (b) comparing the average client-rated and therapist-rated helpfulness of the therapist skills, and (c) correlating the occurrence of the skills with post-session or post-treatment outcomes. Depending on the research approach, the most effective therapist skill was either interpretation, open question, paraphrase, or self-disclosure, whereas the least effective therapist skill was either confrontation, approval, closed question, self-disclosure, or direct guidance. It’s messy and it depends indeed!

Qualitative methods, task analyses, and conversation analysis may be more clinically useful for studying process-outcome relations given that contextual variables can be included in the models, but they prove more intensive and typically involve just a few clients. Similarly, multi-level modeling can account for therapist, client, session, and event effects, but fails to consider the clinical and contextual variables.

Researchers currently struggle to take the complexity of psychotherapy and the sequencing of skills/methods into account. To do so requires that we consider the antecedents of skills/methods (e.g., in what circumstances are they helpful or hindering), therapist and client variables (e.g., personality, attachment style, diagnosis, motivation, social support), stage of treatment, as well as an assortment of immediate and distal outcomes. The impact of skills/methods cannot be examined in isolation because they occur in the context of what has gone before and what transpires after, as well as what transpires outside of therapy.

As should now be apparent, we strongly assert that the association between therapist skills/methods and outcome has not received adequate attention in the research literature. In part, this neglect is undoubtedly due to the time-intensive nature of the phenomenon. In part, it is also attributable to the dominant funding mechanisms and research paradigms that favor comparing the relative efficacy of brand-name treatment packages for specific diagnoses. As is frequently said in evidence-based practice, “the absence of evidence is not evidence of absence.”

The Interorganizational Task Force

The interorganizational task force was commissioned to identify, compile, and disseminate the research evidence on psychotherapist skills and methods used across theoretical orientations. This Task Force was initiated by the Society for the Advancement of Psychotherapy (APA Division of Psychotherapy) and co-sponsored by the Society of Counseling Psychology (APA Division of Counseling Psychology), the Society for Psychotherapy Research, and the Society for the Exploration of Psychotherapy Integration.

Each co-sponsoring organization nominated one or two individuals for a Steering Committee. We invited additional people with expertise in psychotherapy research, ensuring that we obtained gender, cultural, theoretical, and international representation on a Steering Committee. In addition to ourselves as Co-Chairs, the members of the Steering Committee were (in alphabetical order):

Louis G. Castonguay, PhD, Pennsylvania State University, USA

Melanie M. Domenech Rodriguez, PhD, Utah State University, USA

Barry A. Farber, PhD, Columbia University, USA

Christoph Flückiger, PhD, University of Kassel, Switzerland (representing SPR)

Myrna L. Friedlander, PhD, University at Albany/State University of New York, USA

Beatriz (Betty) Gomez, PhD, Aigle Foundation, Argentina (representing SEPI)

Matthew J. Miller, PhD, Loyola University, USA (representing SCP)

Bernhard Strauss, PhD, University of Jena, Germany

Patricia T. Spangler, PhD, Uniformed Services University, and Henry M. Jackson Foundation for the Advancement of Military Medicine, Inc. USA (representing SAP)

Sigal Zilcha-Mano, PhD, University of Haifa, Israel

The first challenge for the Task Force was to determine which skills and methods to include. We struggled to differentiate skills and methods from total treatment packages (e.g., psychodynamic approaches) on the one hand, and the therapeutic relationship (e.g., warmth, alliance), the person of the therapist (e.g., authoritarian), and the person of the client (e.g., motivated, depressed) on the other hand. We regret that we could not include more skills and methods. We especially miss chapters on therapist challenges, development of mentalization, working with cultural diversity, and exposure methods. The research evidence on four relational skills and methods, originally included in Psychotherapy Relationships That Work (Norcross & Lambert, 2019), was briefly summarized in a single article (Farber et al., in press).

Contributors compiled the available research evidence about the association between therapist skills and methods and immediate (within-session), intermediate (post-session or between session), and distal (post-treatment) outcomes. The evidence is based on actual psychotherapy (rather than analogue) conducted with adults in an individual format (thus excluding group, couples, child/adolescent, and family therapy). When available, the research was reviewed for what works and with whom under what conditions (moderators).

The chapters were peer-reviewed by the two editors and subsequently underwent at least three revisions. When the chapters were finalized, at least three psychotherapy experts from the 12-person Steering Committee reviewed and rated the evidentiary strength of each skill and method. In this way, we reached an expert consensus on which skills and methods work, which do not, and which have not yet been sufficiently researched. That consensus informed and fueled the Task Force’s conclusions (presented later in this article).

We developed detailed guidelines for the contributors to ensure standard section headings and similar chapter structures. Through special arrangement with the book publisher (Oxford University Press), authors truncated their chapters into journal articles, which are published in this special section of Psychotherapy Research and a companion special issue of Psychotherapy. The guidelines, in condensed fashion, are presented below.

• Introduction: Introduce the method or skill in a couple of reader-friendly paragraphs.

• Definitions and Clinical Description: Define in theoretically neutral language the skill or method being reviewed. Identify similar constructs from diverse theoretical traditions. Operationalize the method or skills and describe how it is actually rendered in session.

• Assessment: Review how the skill/method and outcomes are typically assessed.

• Clinical Example: Provide one concrete example of the skill/method.

• Previous Reviews: Offer a quick synopsis of the findings of any previous reviews on the topic.

• Research Review: Systematically compile all available empirical studies of the skill/method in individual psychotherapy written in the English language (and other languages, if possible). Each included study must link the skill/method to some kind of outcome (immediate, intermediate, distal), and can include case studies, qualitative studies, process studies, controlled trials, and quantitative studies, but must exclude analogue studies or surveys. The research review should include the links between the skill/method and: (a) immediate, in-session outcomes, (b) intermediate (post-session, between sessions) outcomes, and (c) distal, treatment-end outcomes. If possible, conduct an original meta-analysis of all available studies in each section. If it is not possible to conduct a meta-analysis because there were not enough outcome studies, conduct a box-score analysis of the results in each study. Box scores reflect the judgment of the authors based on the evidence of the outcomes. By a process of designating criteria and discussion among themselves, authors determined whether the association between the skill/method and each outcome in a given study was positive (+), neutral (0,  = ), or negative (-).

• Diversity Considerations: Describe the results related to diversity (e.g., gender, race/ethnicity, sexual orientation, socioeconomic status) and their intersection in the research evidence of the link between the skill/method and outcome.

• Limitations of the Research: Point to the major limitations of the research conducted to date, as well as the implications for future research.

• Training Implications: Explicate the take-home points of your research review for clinical educators and supervisors.

• Therapeutic Practices: Highlight the practice implications from the research, primarily in terms of the therapist’s contribution.

Research Evidence for the Effectiveness of the Skills/Methods

Several contributors provided quantitative meta-analyses on the association of their therapy method to outcome. These quantitative meta-analyses were typically for distal outcomes, mostly of methods (e.g., role induction, collaborative assessment methods, routine outcome monitoring, strength-based methods, meditation/mindfulness/acceptance, behavioral activation, emotion regulation).

Far more challenging was locating empirical studies involving immediate in-session outcomes, especially of the methods, and then determining how to review these few studies. Given the paucity of studies, especially for immediate outcomes, contributors sometimes used box scores to summarize the evidence (e.g., questions, paradoxical interventions, metaphors). In instances where quantitative syntheses were not appropriate (e.g., self-disclosure, empathic reflections, silences, chairwork), contributors summarized some or all of the evidence qualitatively.

At least three members of the Task Force’s Steering Committee independently judged the research evidence for the immediate, intermediate, and distal outcomes for each skill or method as demonstrably effective, probably effective, neutral or mixed evidence, not effective, or had insufficient research to judge. They based their judgments on six criteria (similar to those used for psychotherapy relationships that work; Norcross & Lambert, 2019): number of empirical studies, consistency of empirical results, independence of supportive studies, magnitude of association between the skill/method and outcome, evidence for a causal link between the skill/method and outcome, and ecological or external validity of the findings.

The Task Force co-chairs/co-editors then aggregated and reviewed the Task Force’s judgments of the research evidence (18 on immediate outcomes, 8 on intermediate outcome, and 21 on distal outcomes). The majority judgment was honored in all instances. In cases of divided judgments, the co-chairs again reviewed the criteria, discussed the research evidence, and arrived at a consensus judgment.

Table II summarizes the research results on the associations between the 27 therapy skills/methods and patient outcomes. The table also provides the Steering Committee’s consensus judgments on the typical effectiveness of these psychotherapy skills and methods.

Table II. Summary of associations between psychotherapist skills/methods and psychotherapy outcomes.

Skill/Method	Type of Outcome	# Studies	# Clients/Patients	Type of Analysis	Results of Research Review	Consensus on Research Evidence
Affirmation/Validation (skill)	Distal	64	3,528	Meta-analysis	g = 0.28–0.36	Demonstrably Effective
Self-disclosure (skill)	Immediate	5	99	Qualitative meta-analysis	Improved relationship (64%)	Insufficient Research
					Gained insight (46%)
					Better mental health (45%)
					Helpful (37%)
Immediacy (skill)	Immediate	15	78	Qualitative meta-analysis	Improved relationship (40%)	Insufficient Research
					Opened up (40%)
					Not helpful (39%)
Rupture Repair (method)	Immediate	5	21	Task analysis	Acknowledged & explored rupture	Insufficient Research
		3	10	Task analysis	Renegotiated goal
	Distal	11	1,318	Meta-analysis	d = 0.51–0.62	Probably Effective
	Training	6	407	Meta-analysis	d = 0.22
Questions (skill)	Immediate	10	328	Box score	5+, 5=, 6 -	Neutral/Mixed
Socratic Questioning (method)	Immediate	6	39	Box score	6+, 0 = , 0-	Insufficient Research
	Distal	1	55	Box score	2+, 0 = , 0-	Insufficient Research
Empathic Reflections (skill)	Immediate					Neutral/Mixed
	In-Session	24	1,217	Meta-analysis	d = 0.05
		4	44	Qual: Narrative	Increased self-understanding
		21	(1,152 egs)	Conver. analysis	Positive if therapist responds to empathic opportunity, reflects client's stance, and makes a correction
	Delayed	3	72	Meta-analysis	d = 0.04
	Distal	22	1,734	Meta-analysis	d = 0.04	Neutral/Mixed
Metaphors (skill)	Immediate					Insufficient Research
	In-Session	5	9	Box score	5+, 0=, 0-
	Delayed	5	17	Box score	4+, 0=, 3-
	Distal	5	80	Box score	3+, 5=, 0-	Insufficient Research
Interpretations (skill)	Immediate	6	88	Box score	10+. 6=, 4-	Neutral/Mixed
	In-Session
	Intermediate	4	288	Box score	8+, 5=, 3-	Neutral/Mixed
	Distal	12	866	Box score	9+, 19=, 3-	Neutral/Mixed
Paradoxical Interventions (method)	Intermediate	12	35	Box score	12+, 0=, 1-	Probably Effective
	Distal					Demonstrably Effective
	Pre-post	17	422	Meta-analysis	d = 1.1
	Compared to non-PI	17	471	Meta-analysis	d = 0.49
Advice, Suggestions, Recommendations (skill)	Immediate					Neutral/Mixed
	In-Session	5	141	Box score	2+, 14=, 4-
	Delayed	3	22	Box score	0+, 10=, 1-
	Intermediate	2	123	Box score	2+, 0=. 0-	Insufficient Research
Homework (method)	Immediate In-Session	9	322	Box score	3+, 0=, 6-, WBS = .26	Neutral/Mixed
	Intermediate	16	982	Box Score	10+, 2=, 4-, WBS = .53	Probably Effective
	Distal (previous review)	14	477	Meta-analysis	d = 0.53	Demonstrably Effective
Silences (method)	In-dialogue silences Immediate	17	276	Integrative Meta-analysis	Therapist allowing productive pauses was associated with more client emotional attunement, new symbolization, meaning making, lowered emotional distress, and increased collaboration. Therapist permitting obstructive pauses was associated with client withdrawal, relational rifts, greater emotional distress, and poorer alliance.	Probably Effective
	Extended silences Immediate	16	33	Qualitative meta-analysis	Therapist support as well understanding and identifying client experiences during silences was associated with increases in client insight/self-understanding, capacity for relationships, containment of emotions, and autonomy.	Probably Effective
Dyadic Synchrony (method)	Intermediate	8	323	Box score	WBS = .257	Neutral/Mixed
	Movement	6	259	Box score	WBS = .214
	Physiological	2	64	Box score	WBS = .479
	Distal	14	786	Box score	WBS = .343	Neutral/Mixed
	Movement	10	667	Box score	WBS = .389
	Vocal	2	116	Box score	WBS = -.222
	Physiological	1	60	Box score	WBS = .500
	Linguistic	1	7	Box score	WBS = 0
Role Induction (method)	Immediate					Probably Effective
	In-session	8	651	Meta-analysis	d = 0.64
	Delayed	3	151	Box score	3+, 0=, 0-
	Distal					Probably Effective
	Dropout	15	1,307	Meta-analysis	OR = 1.64
	Mid-Treatment	5	288	Meta-analysis	d = 0.26
	End-Treatment	8	662	Meta-analysis	d = 0.33
Collaborative Assessment Methods (method)	Distal					Probably Effective
	Process	7	320	Meta-analysis	d = 0.59
	Symptoms	7	332	Meta-analysis	d = 0.19
	Growth	5	264	Meta-analysis	d = 0.42
	Cases	6	6	Box score	6+, 0=, 0-
Routine Outcome Monitoring (method)	Distal (All clients)
	End-treatment	58	21,699	Meta-analysis	d = 0.15	Demonstrably Effective
	Follow-up
	Dropout	4	1,027	Meta-analysis	d = 0.18
	Deterioration	39	14,369	Meta-analysis	OR = 1.19
	Distal (Not-on-track)	26	10,413	Meta-analysis	OR = 1.16
	End-treatment	27	15,146	Meta-analysis	d = 0.17
Strength-based Methods (method, treatment)	Immediate	8	416	Box score	6+, 2=, 0-	Probably Effective
	Distal					Demonstrably Effective
	Disorder-specific
	Previous combined	10–51	max 6,139	Meta-analysis	g = 0.07–0.94
	Current bona fide	9	729	Meta-analysis	g = 0.18
	Other Outcomes
	Previous combined	10–51	Max 6,139	Meta-analysis	g = 0.02–0.92
	Current bona fide	9	729	Meta-analysis	g = 0.16
Emotional Regulation (method, treatment)	Immediate	8	84	Box score	6+, 2=, 0-	Probably Effective
	Intermediate	2	14	Box score	0+, 3=, 0-	Insufficient Research
	Distal	28	1,034	Meta-analysis	g = 0.82	Demonstrably Effective
Chairwork (method, treatment)	Immediate					Neutral/Mixed
	Btwn groups	4	295	Meta-analysis	d = 0.10
	Btwn groups	4	98	Meta-analysis	d = 0.90
	Distal					Probably Effective
	Pre-post single sess	7	326	Meta-analysis	d = 1.73
	Btwn grps, post-sess	3	105	Meta-analysis	d = −0.02
	Btwn grps, post-tx	5	238	Meta-analysis	d = 0.40
		4	100	Task analysis	Step 1: activate problem state
					Step 2: explore inner reaction
					Step 3: enact core concern
					Step 4: express emotion
					Step 5: change perspective
					Step 6: negotiate engagement
Dream Work and Nightmare Treatment (method, treatment)	Dream work CEDM
	Intermediate	8	539	Meta-analysis	g = 0.67–0.70	Demonstrably Effective
	Nightmare Tx
	IRT/ERRT					Demonstrably Effective
	Distal
	Previous	13	511	Meta-analysis	g = .59 to .93
	Current	6	293	Box score	24+, 1=, 0-	Insufficient Research
	NDR
	Immediate	1	11	Box score
	Distal	2	40	Box score	4+, 2=, 0-	Insufficient Research
Meditation, Mindfulness, Acceptance (methods, treatment)	Immediate					Insufficient Research
	In-session	3	176	Box score	WBS = .06
		1	9	Box score	WBS = 1.0
	Distal					Demonstrably Effective
	Previous reviews	28	3,009	Meta-analysis	g = 0.40–0.95
	(pre-post)
	MMA vs no tx	89	5,748	Meta-analysis	g = 0.55
	MMA vs bona fide
	active control	42	3,810	Meta-analysis	g = 0.23
	MMA vs evidence-based tx	28	2,806	Meta-analysis	g = −0.004
Behavioral Activation (method, treatment)	Distal	22	819	Meta-analysis	g = 0.85	Demonstrably Effective (for depression)
Cognitive Restructuring (method, treatment)	Intermediate and Distal	4	353	Meta-analysis	d = 0.85
	Intermediate	4	550	Box score	5+, 0=, 0-	Demonstrably Effective (for depression)
	Distal	2	261	Box score	3+, 0=, 0-	Demonstrably Effective (for depression)

Note. Skill = discrete verbal response mode; Method = a collection of skills oriented toward a specific aim which is implemented within an overall treatment approach; Treatment (tx) = entire treatment package focusing on and inseparable from the methods.

For Type of Outcome: Immediate includes both Immediate In-Session outcomes (typically judged by trained raters) and Immediate Delayed (ratings of in-session process typically made after sessions by therapists, clients, and trained judges); Intermediate = outcomes rated at post-session or between sessions; Distal = end-of-treatment outcome.

Tx: treatment; egs: excerpts; Qual.: Qualitative; Previous: previous published review.

For Type of Analysis: Box scores: + is a positive score, = is a neutral score, - is a negative score. WBS = weighted box score (between −1.00 to −0.50 is considered negative; between −0.49 to +0.49 is considered neutral, and between +0.50 to +1.00 is considered positive); Conver. analysis = conversational analysis.

Under Dream Work and Nightmare Tx, CEDM = cognitive-experiential dream model, IRT = imagery rehearsal therapy, ERRT = exposure, relaxation, and rescripting therapy, CDR = nightmare deconstruction and reprocessing therapy.

Consensus on Research Evidence: the consensus judgment of the Task Force Steering Committee about the strength of the research evidence for the effectiveness of each type of outcome for each skill/method: demonstrably effective, probably effective, neutral or mixed evidence, not effective, insufficient research to judge.

Research Evidence for the Effectiveness of Eight Skills

For immediate outcomes, there was a sufficient amount of research to make judgments about four skills. Questions, empathic reflections, interpretations, advice/suggestions/recommendations were all judged to be neutral/mixed in terms of effectiveness. For intermediate outcomes, there was a sufficient amount of research evidence to make judgments about one skill. Interpretations were judged to be neutral/mixed in terms of effectiveness. For distal outcomes, there was a sufficient amount of research evidence to make judgments about three skills. Affirmation/validation was judged to be demonstrably effective; empathic reflections and interpretations were judged to be neutral/mixed.

In sum, we had evidence that, on average, therapist affirmation/validation was demonstrably effective in terms of distal outcomes. The only skill (interpretations) that was tested across all types of outcomes was judged as having neutral/mixed evidence for all three. The remaining research evidence on the effectiveness of the other six discrete therapy skills was decidedly mixed or neutral.

Research Evidence for the Effectiveness of 19 Methods

For immediate outcomes, there was a sufficient amount of research to make judgments about seven methods. Both in-dialogue and extended silences, role induction, strength-based methods, and emotion regulation were judged to be probably effective; homework and chairwork were judged to be neutral/mixed in terms of effectiveness.

For intermediate outcomes, there was a sufficient amount of evidence to make judgments about five methods. Cognitive experiential dream work and cognitive restructuring were judged to be demonstrably effective, paradoxical interventions and homework were judged as probably effective, dyadic synchrony was judged as neutral/mixed.

For distal outcomes, there was a sufficient amount of evidence to make judgments about 14 methods. Nine (homework, paradoxical interventions, routine outcome monitoring, strength-based methods, emotion regulation, imagery rehearsal therapy for nightmares and exposure/relaxation/rescripting therapy for nightmares, meditation/mindfulness/acceptance, behavioral activation, cognitive restructuring) were judged to be demonstrably effective. Four (rupture repair, role induction, collaborative assessment methods, chairwork) were judged to be probably effective, and one (dyadic synchrony) was judged to be neutral/mixed across both of its two types of outcomes.

In short, there was evidence for probable or demonstrable effectiveness for 15 of the 19 methods (all but dyadic synchrony; Socratic questioning; nightmare deconstruction and reprocessing therapy for nightmares). In fact, six therapy methods evidenced probable or demonstrable effectiveness across two types of outcomes (paradoxical interventions, role induction, strength-based methods, emotion regulation, homework, cognitive restructuring).

Research Evidence on Type of Outcomes

The strength of the research evidence systematically varied as a function of the type of therapy outcome, which itself was confounded with the type of statistical analysis. Of the 47 determinations rendered by the Steering Committee on therapy skills and methods (see Table II), 12 were of demonstrable effectiveness, 11 of probable effectiveness, 11 of neutral/mixed effectiveness, and 13 of insufficient research. Interestingly, 10 of the 12 judgments of demonstrable effectiveness were based completely on quantitative meta-analyses conducted on distal, end-of-treatment outcomes and one used a quantitative meta-analysis on intermediate outcomes. On the other end of the spectrum, no therapy skill or method was determined to be demonstrably effective for immediate outcomes; these were rarely analyzed by quantitative meta-analyses. As noted previously, it is easier to study distal outcomes (which are typically based on patient self-report of symptom changes) than immediate in-session outcomes (which typically require labor-intensive coding of microprocesses by trained judges).

Although there was minimal research on the connection between skills/methods and immediate outcomes, there was more research on the relation between skills/methods and intermediate and distal (treatment) outcomes. Research on the latter two types of outcomes, as noted earlier, suffer from difficulties in connecting skills/methods given the lack of a temporal connection, the lack of contextual specificity, and the multitude of confounding variables.

In the absence of research evidence on the immediate in-session outcomes of therapist skills and methods, practitioners are frequently left to extrapolate from the global results of RCTs and effectiveness studies using those skills and methods as part of a larger treatment package. Those interested in the effectiveness of, say, mindfulness and meditation methods (Goldberg et al., this volume) for instance, will look at the effectiveness of entire mindfulness treatments or a course of behavioral activation to indirectly glean some information on the usefulness of the constituent methods. Such research evidence complements the specific and superior evidence provided on the methods themselves.

Although obviously useful, such outcome research on entire treatment packages suffers from numerous limitations. First, those research results do not determine which components of the treatment package prove efficacious. Was it the particular skill/method, the therapeutic relationship, the patient, the expectation, or the responsiveness? Second, clinical trials focus primarily on the distal, end of treatment outcomes, which are insensitive for identifying the impacts of skills or methods offered months ago. More researchers are, however, examining changes on a session-by-session basis to examine the trajectory of change over time.

Diversity Considerations

Like most of psychological science (Henrich et al., 2010), research on skills/methods continues to emanate primarily from WEIRD (Western/White, educated, industrialized, rich, and democratic) societies and is thus limited to a WEIRD socio-historical-cultural context. The current evidence base largely excludes other ways of understanding the constructs themselves and the relations between them.

Minimal research has been conducted on the effectiveness of therapy skills and methods with culturally diverse patients and practitioners. Only three of the research reviews offered evidence for the effects of client cultural diversity on treatment outcome (and again, these results were based on few studies). Levitt and Morrill (in press) cited a study in which male therapists and clients more often than female therapists and clients used silence as a “shield” or boundary between each other in therapy (Hill et al., 2003). Swift et al. (in press) found that role induction was generally more effective in samples that had more non-Hispanic White clients than diverse racial and ethnic minorities. Ezawa and Hollon (in press) reported a study in which White therapists delivered less cognitive restructuring to Black than White patients even though cognitive restructuring has been found to be equally effective for Black and White patients (Ezawa & Stuck, 2022a). They also cited a study in which therapists viewed cognitive restructuring as less therapeutic for Black than White patients (Ezawa & Stuck, 2022b).

Similar outcomes were reported across cultural groups for empathic reflections, between-session homework, and dreamwork. Elliott and colleagues (this issue) noted that similar results have been found for motivational interviewing across sociocultural dimensions for empathic reflections, although most of the research was conducted with Black and Hispanic clients. For between-session homework, Ryum et al. (in press) found that the impact of client characteristics was mostly negligible, but added that most research has been conducted with Western clients. Spangler and Sim (in press) reported that cognitive-experiential dreamwork produced similar benefit for university clients in both the US and Taiwan.

For the other 16 skills and methods reviewed, there was no empirical evidence for the impact of patient diversity on treatment outcomes. Nevertheless, the authors all thought that diversity makes a difference in the outcomes of their respective skills and methods.

Where might the field look more sensitively for the effects of cultural diversity? First, researchers could follow the lead of research on cultural adaptations and other means of personalizing psychotherapy (Cohen et al., 2021; Norcross & Wampold, 2019) and adapt the skill or method to salient cultural identities of patients and compare the results to those obtained with generic implementations. They could, second, avoid simply looking at race/ethnicity, sexual orientation, or country of origin as moderators of distal outcome, as these are proxy measures that do not directly address culture and likely obscure cultural effects (Betancourt & Lopez, 1993). Third, they could proactively test theoretically derived cultural differences in psychotherapy. For example, given that Asian cultures are theorized to be more collectivist and less individualistically oriented than Western cultures (Duan et al., 2012; Sue & Sue, 2022), researchers could test whether individualistically-oriented patients respond differently to therapist’s suggestions to strive to achieve for oneself rather than to honor the family. Fourth and final, they could examine the conceptual equivalence (Brislin, 1993) of psychotherapy phenomena and the ways in which these phenomena do or do not manifest in an equivalent manner in differing societal and cultural contexts.

Research Limitations

The authors of these research reviews all bemoaned the flaws and inadequacies of the research in their respective literature reviews. Such limitations can guide future researchers as well as caution practitioners and policymakers from reaching premature conclusions about “what works” and “what does not.” Here we present several compelling caveats in interpreting the results of the preceding chapters and of the entirety of the Task Force.

Low Power

Unfortunately, for most of the skills and methods reviewed in this task force, there were shockingly few studies. Furthermore, even for those skills/methods that had a sufficient number of studies, different definitions, measures, and analytic methods were often employed across studies, making it questionable whether it was valid to summarize across studies. The source of strength in psychotherapy research—its diversity in designs, outcomes, and measures—also proved the root of the problem. In statistical terms, most of the meta-analyses and research syntheses suffered from low power.

Absence of Moderators

The summary numbers in Table II apply mostly to the average or typical client. There were scattered reports in the chapters of moderators, but these were rarely replicated across studies. Combining all clients and disorders obscures what works when with whom. There are occasions when every skill or method used by some therapists proves helpful and other occasions when harmful. Practitioners need to know more than what works on average; practitioners need to know for which clients and which situations each skill/method is effective. We need, in short, tests of moderation, particularly with populations rarely studied.

Complexity of Psychotherapy Not Captured

Psychotherapy is a dynamic and complex process, experienced differently by each participant in varying contexts. Some of the therapy skills (particularly the response modes) might have received consensual judgments as neutral/mixed or insufficient research because they were considered out of context. The paucity of context in the research reviews might also explain certain adverse effects, for instance, the ineffective use of the skills in a given moment.

To capture the individual nature of this complex phenomenon, researchers would need to conduct intensive case studies, with input from therapists, clients, significant others, and external observers, using measures unique to the individuals involved, taking into account historical, contextual, and cultural factors. Instead, most of the research reviewed focused on the frequency of a particular therapist skill/method in relation to distal outcomes, using only patient self-report measures, and disregarding the context and nuance of how psychotherapy operates, and neglecting the evidence that clients often conceal, distort, and even lie about their therapeutic experiences (Farber et al., 2019).

We can all recount personal experiences as clinicians and clients when empathic reflections, interpretations, and the rest of the skills and methods covered in this book had profound positive or negative effects. Yet, the research literature disappoints by not reflecting these experiences. We hope that in the future, we refine our ability to study these phenomena.

Patterns and Sequences Not Revealed

Many of the skills/methods were researched as if they occur in isolation, rather than in sequence or in interactions with other variables. An interpretation, for instance, is often preceded by multiple exchanges of exploration (e.g., questions) and asking the client about their understanding before a tentative interpretation is offered. The therapist then follows with questions about accuracy, and the interpretation is co-constructed between the therapist and client (i.e., reshaped, reformulated, and repeated in subsequent sessions). It is probably the sequencing and repetition of interpretations rather than the single interpretation that is helpful.

Cannot Claim Causality

To assert causality between a skill/method and an outcome, researchers usually need to conduct experimental studies (e.g., manipulating whether metaphors are used or not used). But (and this is a huge “but”) experimentally manipulating many of these therapist skills/methods changes the very nature of the psychotherapy itself. For example, the metaphors provided by therapists probably need to be tailored to the individual patient and provided at the optimal moment when the patient is “ready.” Response to a metaphor undoubtedly depends on the timing and the manner in which it is presented (e.g., empathically and tentatively versus as “Truth”). With few prominent exceptions in this volume (such as role induction and routine outcome monitoring), the research did not afford confident causal connections between the respective skill or method and the treatment outcome.

Cannot Reliably Separate Therapist and Client Contributions

Most skills and methods are products of the interaction between the therapist and client at a particular time and place. Hence, these research syntheses have not, and perhaps cannot, reliably separate the therapist’s contribution from the client’s contribution to the process.

Much like parenting, the influences are bidirectional, but we expect therapists to “lead” and “shape.” It is challenging to disentangle who contributed what, but we focused on the psychotherapist’s contribution because of the need to determine what therapists do to lead and shape and what trainers can do to improve their effectiveness.

Results Depend on Type of Outcome

Immediate in-session outcomes allow for perhaps the best research evidence on the effectiveness of a particular skill or method given the temporal link between what the therapist did and how the client responded. But even here, caution is needed because we know clinically that outcomes vary when assessed at different points. For example, a client might have an immediate negative reaction to a therapist’s advice, but later reconsider it and decide to implement it and find the advice to be helpful. Intermediate and distal outcomes are most suspect because of the difficulty in clearly linking the skill/method to the intermediate (e.g., the association between the skill/method and working alliance is tenuous given that many factors together lead to developing, maintaining, and repairing the working alliance) and the distal outcomes (e.g., the association between a particular question and symptom change cannot be established given the distance in time and the interaction with many other variables).

The research evidence looks more “solid” for distal outcomes than immediate and intermediate outcomes because it is easier to collect and quantitatively summarize the former studies given that they are often conducted with large numbers of participants using self-report data and sophisticated meta-analyses. Table II shows that most of the methods determined to be demonstrably effective were reviewed by meta-analyses on distal outcomes, whereas no therapy skill or method was determined to be demonstrably effective for immediate outcomes and only two for intermediate outcomes. Research on immediate in-session outcomes require time-intensive coding of psychotherapy sessions, fewer participants are involved, and often qualitative analyses are used, making it difficult to aggregate results across studies.

A related caveat is that researchers sometimes chose non-bona fide comparisons to test the effects of their methods. In other words, they compared a genuine psychotherapy method (e.g., a strength-based method) to a non-psychotherapy method (e.g., a discussion between strangers). Of course, it is easier to find positive effects in such a comparison than comparing the skill/method to another bona fide skill/method.

Task Force Conclusions

In bringing the work of this interorganizational, multi-year Task Force to an end, we are reminded that much has been done and done well, but more must be done before we can be confident about the effectiveness of therapy skills and methods, especially in terms of immediate in-session and intermediate outcomes.

The Steering Committee has drawn the following conclusions from the Task Force on Psychotherapy Skills and Methods That Work. These statements do not constitute the formal policies of any organization nor are they considered practice or treatment standards. Rather, they represent current scientific knowledge and the consensual judgment of the Steering Committee.

• Mental health professionals regularly use multiple skills and methods to facilitate client behavior and personality change.

• Skills/methods are largely transdiagnostic (used across patient diagnoses or disorders) and pantheoretical (employed across diverse theoretical orientations).

• Efforts to promulgate best practices without attending to the effectiveness of their constituent skills/methods are clinically dubious and potentially misleading.

• Understanding psychotherapy effectiveness requires attention to impacts on patients at four temporal intervals: immediate in-session; intermediate (end of session of between sessions); distal (end of treatment), and follow-up (post-treatment).

• Skills and methods are not offered by therapists at random, but rather are used in response to perceived client needs (in that moment, in a session, or as part of an overarching treatment plan).

• Therapists vary in how they present the skills/methods. For example, therapists who provide positive regard do not do so in the same style, intensity, or awareness of patient needs.

• The amount of outcome variance accounted for by therapist skills and methods (relative to other factors) cannot be determined because of insufficient research evidence.

• For distal (end-of-treatment) outcomes, nine skills/methods (affirmation/validation, paradoxical interventions, homework, routine outcome monitoring, strength-based methods, emotional regulation, meditation/mindfulness/acceptance, behavioral activation, and cognitive restructuring) were judged as demonstrably effective and five (rupture repair, role induction, collaborative assessment methods, chairwork, and imagery rehearsal therapy for nightmares) were judged as probably effective.

• For intermediate (post-session or mid-treatment), cognitive experiential dream work and cognitive restructuring were judged as demonstrably effective; paradoxical interventions and homework were judged as probably effective.

• For immediate in-session outcomes, five methods (in-dialogue and extended silences, role induction, strength-based methods, and emotion regulation) were judged as probably effective.

• These research-supported or evidence-based skills and methods complement and extend evidence-based psychotherapy relationships and treatment packages.

• Four therapy skills and methods lack sufficient research evidence for determining their effectiveness on any of the outcomes at this time (self-disclosure, immediacy, Socratic questioning, metaphors). There is a need for empirical research on these skills/methods, especially assessing specific outcomes that therapists intend to facilitate when using them.

• Most of these research results address average findings for typical patients, thus masking individual variability and obscuring the clinical reality that these skills and methods prove sometimes helpful and sometimes harmful for particular patients and circumstances.

• There is also an urgent need for extending the scope of this research base beyond WEIRD societies and cultures. Failure to do so will increase the irrelevance of this work.

• A determination or classification of any psychotherapeutic skill, method, and treatment package as effective must necessarily refer to the temporal interval (e.g., immediate, intermediate, distal), the type of outcome (e.g., symptom reduction, wellbeing), and the comparator group (e.g., within vs between groups, bona fide psychotherapy comparisons vs. non-clinical controls).

• Research is urgently needed on immediate in-session outcomes to establish at least a temporal link between the skill/method and the client response. Promising approaches include task analyses and conversation analyses. Such research is admittedly laborious, but such investigations will bring us closer to clinical reality.

• The implementation of any skill and method must be responsive to the cultural identities, singular situations, and clinical needs of the individual client at specific moments in therapy.

• Future researchers should prioritize the “what works for whom under which conditions” imperative that guides psychotherapy.

• Additional research is required on training processes to ensure students competently provide these skills and methods.

Disclosure Statement

The authors are receiving royalties for the book upon which this article is based.