7,848
Views
3
CrossRef citations to date
0
Altmetric
EMPIRICAL PAPERS

Learning from clients: A qualitative investigation of psychotherapists’ reactions to negative verbal feedbackFootnote*

ORCID Icon, , , , &
Pages 545-559 | Received 26 Jan 2016, Accepted 28 Sep 2016, Published online: 09 Nov 2016

Abstract

Objective: To explore how therapists experience, react to, and learn from negative feedback from their clients. Method: Eighteen experienced therapists’ written descriptions of episodes where they had received negative verbal feedback from clients were analyzed according to the Consensual Qualitative Research methodology. Results: Receiving feedback was experienced as challenging, but educational. Learning was manifested in different ways: (a) Immediately Applied Learning—therapists improved the following therapy process by changing their behavior with the client, (b) Retrospectively Applied Learning—therapists made changes in their way of working with subsequent clients, and (c) Non-Applied Learning—new ideas generated by the experience had not been translated into behavior. We compared cases describing these manifestations of learning and found differences in the nature of the feedback and how therapists understood, reacted, and responded to it. Conclusions: The therapists benefitted from obtaining and being open to specific feedback from their clients, regulating their own emotional reactions, accommodating dissatisfied clients, and considering how they themselves contributed to negative therapy processes.

Resumo

Objetivo: Explorar como os terapeutas experimentam, reagem e aprendem com o feedback negativo de seus clientes. Método: Dezoito descrições escritas de terapeutas experientes de episódios nos quais receberam feedbacks verbais negativos de clientes foram analisadas de acordo com a metodologia da Pesquisa Qualitativa Consensual. Resultados: Receber feedback foi experimentado como desafiador, mas educacional. A aprendizagem se manifestou de diferentes maneiras: Aprendizagem Aplicada Imediatamente - os terapeutas melhoraram o processo terapêutico seguinte, mudando seu comportamento com o cliente, (b) Aprendizagem Aplicada Retrospectivamente - os terapeutas fizeram mudanças em sua maneira de trabalhar com os clientes, e (c) Aprendizagem não aplicada - novas ideias geradas pela experiência não foram traduzidas em comportamentos. Comparamos casos descrevendo essas manifestações de aprendizagem e encontramos diferenças na natureza do feedback e no modo como os terapeutas entenderam, reagiram e responderam a isso. Conclusões: Os terapeutas se beneficiaram de obter e estar aberto ao feedback específico de seus clientes, regulando suas próprias reações emocionais, acomodando clientes insatisfeitos, e considerando como eles mesmos contribuíram para os processos negativos de terapia.

Zusammenfassung

Ziel: Zu erforschen, wie Therapeuten negative Rückmeldungen von ihren Klienten erleben, darauf reagieren und daraus lernen. Methode: Achtzehn schriftliche Beschreibungen von erfahrenen Therapeuten zu Episoden, in denen sie negative verbale Rückmeldungen von Klienten erhielten, wurden gemäß der Methode der Consensual Qualitative Research Methodik analysiert. Ergebnisse: Feedback zu erhalten wurde als herausfordernd, aber lehrreich erlebt. Das Lernen hat sich auf verschiedene Arten manifestiert: (a) Sofort angewandtes Lernen - Therapeuten verbesserten den folgenden Therapieprozess durch eine Veränderung ihres Verhaltens mit dem Klienten, (b) Retrospektiv angewandtes Lernen - Therapeuten änderten ihre Arbeitsweise mit nachfolgenden Klienten und (c) nicht-angewandtes Lernen - neue Ideen, die durch die Erfahrung generiert wurden, wurden nicht in Verhalten umgesetzt. Wir verglichen Fälle, die diese Manifestationen des Lernens beschreiben, und fanden Unterschiede in der Art der Rückmeldung und wie Therapeuten sie verstanden, auf sie reagierten und darauf antworteten. Schlussfolgerungen: Die Therapeuten profitierten, wenn sie spezifische Rückmeldungen von ihren Klienten erhielten, dafür offenblieben, ihre eigenen emotionalen Reaktionen regulierten, unzufriedenen Klienten entgegenkamen und darüber nachdachten, wie sie selbst zu negativen Therapieprozessen beigetragen haben.

摘要

目的:探索治療師如何經驗到、回應和從案主的負向回饋中有所學習。方法:採用共識質性研究方法學,分析十八位有經驗的治療師以書面描述他們曾收到的案主負向口語回饋的情境。結果:收到回饋的經驗雖具挑戰性,但是也具教育性。有不同面向的學習:(a)立即性應用學習:治療師在下一次晤談即透過改變自己對待案主的行為而有所改善;(b)回顧性應用學習:治療師改變與後續其他案主工作的方式;(c)未能應用學習:此經驗讓治療師產生新的想法,但是還未能轉化到行為層面。我們比較描述這些學習面向的不同研究參與者,發現回饋的性質和治療師如何理解、反應和回應此回饋是有所差異。結論:治療師藉由收到並對案主具體的回饋抱持開放的態度、調節自己的情緒反應、配合不滿意的案主並檢討自己對此負向治療歷程應負的責任,而由此經驗中受益。

Obiettivo: esplorare come i terapeuti percepiscono il feedback negativo dei loro pazienti, come vi reagiscono e cosa ne apprendono. Metodo: sono state analizzate diciotto descrizioni scritte di episodi in cui terapeuti esperti hanno ricevuto feedback verbali negativi da parte dei pazienti, secondo la metodologia di ricerca qualitativa consensuale. Risultati: ricevere feedback è stato vissuto come una sfida, ma educativa. L'apprendimento si è manifestato in diversi modi: (a) Apprendimento Immediatamente Applicato - i terapeuti hanno migliorato successivamente il processo terapeutico cambiando il loro comportamento con il paziente, (b) Apprendimento Applicato in Retrospettiva – i terapeuti hanno apportato cambiamenti nel loro modo di lavorare con i pazienti successivi e (c ) Apprendimento non Applicato – le nuove idee prodotte dall'esperienza non sono state tradotte in comportamenti. Abbiamo confrontato i casi che descrivevano queste manifestazioni di apprendimento e abbiamo riscontrato differenze nella natura del feedback e nel modo in cui i terapeuti hanno compreso, reagito e risposto ad esso. Conclusioni: i terapeuti hanno tratto beneficio dal ricevere e dall'essere aperti al feedback specifico dei loro pazienti, regolando le proprie reazioni emotive, accogliendo i pazienti insoddisfatti e considerando come essi stessi potessero aver contribuito ai processi terapeutici negativi.

The relationship between psychotherapists’ experience and expertise is not clear. In general, therapists have a sense of continually evolving as a result of their work with clients and in particular, information that they gain from clients’ reactions to their actions (Ronnestad & Skovholt, Citation2003, Citation2012), but evidence for differences in actual outcomes between more and less experienced therapists is mixed (Beutler et al., Citation2004; Tracey, Wampold, Lichtenberg, & Goodyear, Citation2014). In the first longitudinal study of its kind, Goldberg et al. (Citation2016) analyzed 170 therapists’ outcomes with more than 5500 clients over the average of almost 5 years. At a group level these therapists’ effect sizes were found to decrease slightly as they became more experienced, although almost 40% of the sample did improve their results over time. Discussing their results, the authors commented that the quality of experience might be more important for learning than the quantity and that therapists’ deliberate efforts, such as practicing skills based on performance feedback, might facilitate their professional development. They observed however that conditions necessary to do so are typically not present in most practice settings. When and how do therapists learn from experience, and in particular, from corrective or negative feedback from their clients?

Feedback is defined as “a response to an action that shapes or adjusts that action in subsequent performance” (Claiborn & Goodyear, Citation2005, p. 209). The present investigation focuses on verbally expressed negative feedback that prompt a change in the therapist, that is, that he or she learns from. This is a broader conceptualization of feedback than what has typically been the case within the patient-focused research paradigm (Howard, Moras, Brill, Martinovich, & Lutz, Citation1996), where client feedback often is used synonymous to information obtained through regularly administered measures of clients’ treatment responses (Routine Outcome Monitoring [ROM]; see, for instance, Lambert, Citation2007). Although researchers are beginning to explore the potential of these interventions as learning tools for therapists (Miller, Hubble, Chow, & Seidel, Citation2015), they represent a relatively new development in psychotherapy and it is likely that also less formal feedback from clients facilitate therapists’ professional development. Psychodynamic models offer valuable perspectives on situations where negative verbal feedback from clients typically occurs, such as impasses, ruptures, or breaches in the therapeutic alliance (see, for instance, Hill & Knox, Citation2009). Here, however, the emphasis is different: How therapists’ behavior is affected by the negative feedback rather than why the client is dissatisfied (e.g., transference, defense mechanisms), why the therapist reacts as he or she does (e.g., countertransference), or how the therapist can respond in a way that helps the client (e.g., repair ruptures, provide insight or corrective experiences).

While it is unclear why not all therapists improve their ability to help clients as they become more experienced, some obstacles to learning from experience are suggested in the literature. A first barrier is that it can be hard for therapists to obtain clear and unambiguous feedback about how they are doing. Many clients find it difficult to express dissatisfaction and instead choose to conceal negative feelings about therapy or the therapist (Blanchard & Farber, Citation2015; Farber, Citation2003). Therapists are frequently not aware of what their clients leave unsaid (Hill, Thompson, Cogar, & Denman, Citation1993; Hill, Thompson, & Corbett, Citation1992; Regan & Hill, Citation1992). As a result, therapists may have restricted access to the kind of feedback that according to feedback theory (Sapyta, Riemer, & Bickman, Citation2005) is most likely to motivate them to change: Immediately and frequently delivered after a given behavior and containing new and specific information about that behavior.

A second barrier to learning from feedback is the therapists’ covert reactions to negative appraisal, or “ the enormous difficulty that human beings, even highly trained therapists, have in dealing with interpersonal conflict in which they are participants” (Binder & Strupp, Citation1997, p. 123). Therapists have disclosed feelings of guilt, anxiety, incompetence, confusion, and irritation when confronted with their clients’ dissatisfaction in several qualitative studies (Coutinho, Ribeiro, Hill, & Safran, Citation2011; Hill, Nutt-Williams, Heaton, Thompson, & Rhodes, Citation1996; Hill et al., Citation2003; Moltu, Binder, & Nielsen, Citation2010). Negative emotional reactions may tax the therapists’ attention resources and thus, make it difficult to respond effectively (Kluger & DeNisi, Citation1996). Furthermore, the risk of losing face and damaging one’s self-image prevent people from seeking negative feedback (Anseel, Beatty, Shen, Lievens, & Sackett, Citation2015), and also therapists have been shown to be prone to self-assessment bias (Walfish, McAlister, O’Donnell, & Lambert, Citation2012) and self-serving attributions (Murdock, Edwards, & Murdock, Citation2010). Such cognitive biases may make therapists less open to, and thus less likely to benefit from, negative feedback (Macdonald & Mellor-Clark, Citation2014).

A third barrier to learning from experience is that simply receiving and being open to negative feedback may not be enough to learn from it. Miller et al. (Citation2015) suggested that the process of translating insight gained from feedback into actual behavior changes requires considerable effort. They recommended that therapists engage in “deliberate practice,” that is, “setting aside time for reflecting on one’s performance, receiving guidance on how to improve specific aspects of therapeutic practice, considering any feedback received, identifying errors, and developing, rehearsing, executing, and evaluating a plan for improvement” (Miller et al., Citation2015, p. 453; see also Tracey et al., Citation2014). In support of this model, Chow et al. (Citation2015) found more effective therapists spent more time than less effective ones in solitary practice aimed at improving their skills.

Interacting with dissatisfied clients can clearly be challenging. The current study was designed to investigate how therapists experience, react to, and learn from negative, verbally expressed client feedback. Because of the explorative nature of the research questions, we adopted the qualitative methodology of Consensual Qualitative Research (CQR; Hill, Citation2012; Hill et al., Citation2005). Its main benefits are the integration of multiple perspectives to yield a more complex and less biased understanding of the data through rigorous and replicable analytic steps.

Method

Participants

Therapists

The participants were 18 therapists at a Norwegian mental health hospital. Demographic data was available for 16 of the therapists. The group consisted of 10 women and 6 men, with ages ranging from 28 to 64 years old (M = 47.4). Seven were clinical psychologists, six psychiatrists, and three other health care professionals. Highly experienced therapists dominated the sample. Five participants (31.3%) had worked as a therapist for more than 20 years, 10 (62.5%) for 10–20 years, and 1 (6.3%) for less than 5 years.

All 18 therapists provided information about theoretical orientation and attitudes toward clients’ feedback. On a 4-point Likert scale (1 = low, 4 = high), they rated themselves as being strongly influenced by psychodynamic models (M = 3.56, SD = 0.70), followed by cognitive/behavioral models (M = 2.67, SD = 0.78) and humanistic/existential models (M = 2.61, SD = 0.49). Also on a 4-point Likert scale (1 = low, 4 = high), participants indicated that they felt confident as therapists (M = 3.1, SD = 0.47) and were highly (M = 3.3, SD = 0.49) concerned with their clients’ perceptions of them and their way of working. Nine therapists (50%) reported receiving process feedback (i.e., meta-communications from clients about treatment) daily, 5 (27.8%) weekly, and the remaining 4 less frequently. Eight therapists did not use ROM in their work; the remaining had worked with the Partners for Change Outcome Management System (PCOMS; Bertolini & Miller, Citation2012; Duncan, Citation2012) for a period ranging from 2 months to 15 years. PCOMS utilizes regularly administered self-report questionnaires to track clients’ progress in therapy as well as their experiences of the therapeutic alliance. As it is designed to facilitate communication of negative feedback and thus relates to the research questions, a comparison of therapists working with ROM versus those not doing so is presented in the results section.

Researchers

Following CQR guidelines (Hill, Citation2012), four researchers or judges conducted the investigation, and two auditors overlooked the process and gave feedback about the judges’ conclusions throughout the process. Team members were selected to ensure a diversity of opinions and viewpoints. The principal investigator was a female, eclectically oriented clinical psychologist who at the time of the investigation worked on a doctoral thesis on ROM. The second judge was a female employee at a service user competence center, with experience being a therapy client and professional interest in examining psychotherapy processes from the client’s point of view. The third judge was the male head of research department at the hospital mental health center where the investigation took place, and the fourth was a male, psychodynamically oriented clinical group psychologist. All judges had some prior experience conducting qualitative research. The auditors were a male professor in psychology and a female professor in psychiatry, both skilled in qualitative research methodology.

To increase awareness of factors that might influence the understanding of the data so that these might be set these aside or “bracketed” throughout the process, the team of judges reflected upon, discussed, and recorded their expectations (i.e., anticipated findings) and biases (i.e., personal issues that might make it difficult to respond objectively to the data) early in the investigation. A synthesis of the team’s biases and expectations is presented here. All viewed client dissatisfaction as inevitable and highly important to acknowledge and work through in therapy. We expected negative feedback to elicit difficult feelings in the therapists that would make it challenging to think clearly and respond effectively, and speculated that they would be more likely to learn from the feedback if they attributed it to themselves rather than to the client. Although not mutually exclusive viewpoints, the first and second judges both tended to take client feedback at face value, with a bias toward understanding it as a result of some mistake made by the therapist, whereas the fourth judge leaned toward interpreting client dissatisfaction as transference or defense mechanisms and thus thinking that it reflected the client more than the therapist.

Questionnaire

The Negative Client Feedback Questionnaire was developed for this investigation in the following manner: First, the principal investigator conducted two face-to-face interviews with therapists to gain a preliminary understanding of the topic. Then, a pilot questionnaire was developed based on the interviews as well as our reading of relevant research literature and our own experiences as therapists and/or therapy clients. Finally, to test the questionnaire, it was administered to three therapists, and their answers as well as feedback from the external and internal auditors helped further refined the questions. Data from the pilot studies was not included in the analysis.

In the final questionnaire, therapists were asked to identify one specific episode when they had received negative feedback from a client. The experience was then explored in detail through several open-ended questions following a structure similar to that used in several other CQR investigations (Coutinho et al., Citation2011; Rhodes, Hill, Thompson, & Elliott, Citation1994): (a) background of the event (i.e., “Describe briefly the clients’ reasons for seeking therapy”), (b) description of the event and immediate context (i.e., “What was the feedback, and how was it communicated?”), (c) the participant’s thoughts, feelings, and actions during the event (i.e., “What did you think in that moment? How did you feel?”), (d) how the event evolved (i.e., “How did the client react to your response ?”), (e) how the participant understands the event (i.e., “What are your thoughts about situation today?”), and (f) consequences of the event (i.e., “Has the experience had any consequences for you as a therapist and your work with new clients; if so, what?”).

Procedures for Data Collection

Recruiting

Sixty therapists working at outpatient departments at a hospital mental health center in Mid-Norway were invited to participate. To promote participation, the principal investigator visited each of four teams in person and gave a presentation about the topic of negative client feedback. The invitation and information about the investigation was repeated in an e-mail that was sent to each of the 60 prospective participants immediately following the presentation. In total, 20 therapists (33%) returned the questionnaire. Two respondents did not describe a specific example and were excluded from the analysis, yielding a total of 18 cases in the final sample.

Administration

The questionnaire was administered via a web-based survey program (www.questback.com). E-mails with personal links to the survey were sent to each of the 60 invited therapists. These personal links were deleted following completion of the survey, and all demographic information (age, profession, and years of experience) was separated from the qualitative data by the administrator, so that no identifying information was contained in the raw data material that was made accessible to the research team. Consent to participation was given by completing the survey. The project was registered with the Norwegian Data Protection Authorities.

Working as a CQR Team

All judges had previously attended university-level courses in qualitative research. To familiarize ourselves with the methodology of CQR, we read and discussed Hill’s (Citation2012) book Consensual qualitative research: A practical resource for investigating social science phenomena, and made a plan for the analysis prior to conducting the investigation. Following CQR guidelines, in each step in the analysis, judges individually analyzed a previously agreed upon subset of cases or domains before working with the team to compare, discuss, adjust, and reach consensus. The process was then shortcut by splitting the team into pairs of judges, with the principal investigator serving on each pair, and sharing the remaining cases/domains between them. We counteracted power differences in the team by encouraging open expressions of differences of opinions and discussing differences in an accepting manner until the team reached consensus.

Data Analysis

The three major steps in a CQR data analysis are as follows: (a) segment raw data into domains, (b) formulate core ideas case for case, and (c) cross-analyze across all cases to identify similarities and formulate categories (see Hill, Citation2012).

Coding of domains and core ideas, and audit

Starting with a preliminary domains list based on the topics covered in the questionnaire, we sorted or coded the raw data (text from the questionnaire) into main thematic areas while continually modifying the domains list to fit the data. Core ideas (i.e., formulations of the content of interview data in clear and concise wording and incorporating relevant context; Hill, Citation2012) were then developed by carefully reading each case, parsing the interview data into smaller units according to content, and agreeing upon wording of the core idea. The resulting core ideas, with corresponding raw data and organized by domains case by case, were given to the auditors. Their feedback was discussed in the team, resulting in adjustments when deemed appropriate.

Cross-analysis, audit, and frequency calculations

Working with domain for domain and across all cases, core ideas that were similar in content were grouped together. Each of these groups or categories was given a label that reflected the content. Categories as well domains were repeatedly modified by frequently going back to the raw text to make sure that the categories represented the data, until a stable list of categories emerged. The list of domains and categories were audited and adjustments were made accordingly, resulting in a final three-level structure of the data: Domains consisting of main categories consisting of sub-categories. Frequency labels were assigned to each of the categories according to how many cases it applied to. Following Hill’s (Citation2012) recommendation, categories were labeled general if present in all or all but one of the cases, typical if present in the range from half of the cases and up to the cutoff for the general, and variant if present less than half of the cases.

Development and comparison of subgroups

Following cross-analysis, we examined the category list with the research question (“When do therapists learn from negative feedback?”) in mind. Two categories (titled “Repair” and “Meets new clients differently”) were considered to be particularly relevant as they represented different manifestations or applications of learning. Accordingly, we sorted cases into two groups based on these categories, leaving remaining cases in a third group.

For each of the three groups of cases, categories were assigned frequency labels according to the same rules as those that guided the frequency calculations for the entire sample (i.e., the category was considered general if present in all or all but one of the cases within that group, typical if present in the range from half of the cases up to the cutoff for general within that group, and variant if present in less than half of the cases within that group; in addition, a category was given the frequency label none if not present in any cases within a given group). When comparing groups of cases, we followed the recommendations made by Hill (Citation2012). Categories were considered more or less frequent in a given group if differing by two frequency categories from one or both of the other two groups (i.e., general vs. variant, typical vs. none). A comparison of cases described by ROM versus non-ROM users was done following the same procedure.

Results

The 18 texts that formed the basis for our analysis varied in length, ranging from 739 to 3172 words. We formulated 23–58 core ideas per case, resulting in 715 for the total sample. The results are presented in a three-level structure: Domains (thematic areas) consisting of main categories (sets of ideas with similar content) consisting of subcategories (descriptions of that content) (see ).

Table I. Therapists’ reactions to negative feedback: domains and categories with frequency labels in total and within different manifestations of learning.

The structure of domains and categories as distributed across all cases is presented first, followed by in-depth description and comparison of the three different manifestations of learning that we identified in the body of cases, and finally a comparison of cases described by therapists working with ROM versus those not doing so.

Presentation of Domains and Categories

Domain I. Background

The main category “Client’s presenting problems” refers to the therapists’ description of the clients’ reasons for seeking therapy. The subcategories “Relational difficulties,” “Complex/comorbid psychopathology,” and “Depression” were typical, whereas “Traumatized,” “Anxiety” and “Personality disorders” were less frequent. Looking back at the “Therapeutic process prior to feedback,” therapists typically indicated that there had been early “Indications of client dissatisfaction,” but it was also typical that “Indications of client satisfaction.” As a variant, “Therapist made attempts to resolve problems” before receiving the negative feedback.

Domain II. Negative feedback

As evident in the main category “Content of feedback,” clients were typically “Dissatisfied with something specific,” although “Global dissatisfaction” was also typical. Clients typically “Did not want to continue working with therapist,” and a variant communicated that “Therapy did not help.” The “Communication of feedback” was typically “Face to face” and variantly “With anger” and “In writing and/or through others.” In none of the cases was the feedback obtained through ROM.

Domain III. Therapist reactions

The main category “Therapist covert reactions” refer to therapists’ description of what went on inside them immediately after receiving the feedback. They typically “Experienced situation as challenging,” “Experienced situation as unusual,” had “Negative feelings toward client,” and “Attributed dissatisfaction to client.” Variant subcategories were “Negative feelings toward self,” “Thought feedback was important,” “Felt surprise,” and “Self-critical thoughts.” As for “Therapist actions,” what therapists did when presented with the feedback, the typical response was to “Invite to further dialogue” (i.e., make room to discuss the feedback more thoroughly) and several subcategories occurred as variants: “Reflect after session,” “Give in” (i.e., try to do as the client requested), “Flexibly accommodate,” “Express understanding and support,” “Explain” (including interpretation), “Apologize,” and “Stay calm” (i.e., make efforts to regulate their own emotions).

Domain IV. Consequences of feedback

Typical in the main category “Consequence for client and therapy” was that “Client dissatisfaction persisted” in the time following the feedback situation, while variant subcategories were “Therapy discontinued,” “Client seemed relieved,” and “Repair” (i.e., improved therapy process). The latter subcategory forms the base for the group Immediately Applied Learning and will be discussed in detail in the next section. A second main category, “Therapist understands the event differently now,” refers to the new ideas that therapists generally expressed when discussing the situation in retrospect. They typically conveyed “Self-acceptance and understanding” of their own reactions and “Understanding of own contribution to event” (including mistakes they had made), and variantly “Understanding of relational aspects of event” (i.e., how the therapist’s and the client’s unique contributions interacted), “Satisfaction with own management of situation,” and “Understanding of how client psychopathology influenced event.” The “Consequences for therapist” were typically “Ideas about what to do differently” and variantly “Changed feelings about self and therapy” both in positive and negative direction, “Reminder of the importance of the client’s perspective,” and “Meets new clients diffently.” The latter category forms the base of the group Retrospectively Applied Learning that is discussed in the next section.

Characteristics of Cases Describing Learning Manifested in Three Ways

Three manifestations of learning emerged from our analysis: Immediately Applied Learning (therapists changed their behavior in ways that improved the following process with the present client; occurred in six cases), Retrospectively Applied Learning (therapists described changes in his or her way of meeting new clients; occurred in eight cases) and Non-Applied Learning (therapists did not describe behavior changes that benefitted the present or future clients; occurred six cases). We understand these as different manifestations of learning that exist on a continuum rather than being mutually exclusive. In two cases, behavior changes that benefitted both the present and future clients were described. These were included in both the Immediately and Retrospectively Applied groups of cases. Characteristics of each manifestations of learning (i.e., categories that are general or typical within that group of cases and/or more or less frequent in that group compared to the other two) are presented in the following (see also ).

Figure 1. Summary of main findings.

Figure 1. Summary of main findings.

Immediately applied learning

Cases were included when therapists described changes in their own behavior as a result of client’s feedback that were maintained and helped improve the following therapy with that client (i.e., subcategory “Repair”). Their responses to the feedback incorporated a here-and-now, mutual learning—the client’s feedback was processed in the dyad, and both the therapist and the client learned something that helped them relate differently to another in the future.

Background

Clients were generally described in terms of complex and comorbid psychopathology as well as relational difficulties. These were evident both in the therapy room and with others and included intimacy issues, feelings of inferiority, or difficulties adapting to social norms. Typical also was personality disorders. Much like with the other two manifestations of learning, therapists generally suspected that their clients were dissatisfied with treatment based on lack of progress, disagreement over goals or approach, or problematic client behavior such as withdrawing or acting aggressive, critical, or demanding. However, like therapists describing Retrospectively Applied Learning, they also generally highlighted indications that their clients were satisfied with treatment prior to the feedback, balancing positive and negative aspects (e.g., “He was skeptical, somewhat condescending in the first session [ … ] I felt like he increasingly came to respect me, and that I challenged him just enough to raise interest and hope”).

Negative feedback

The feedback generally concerned something specific that the therapist had or had not done. Examples include wanting medical treatment, not wanting to talk about specific topics that the therapist introduced, being outraged by something the therapist said, or not feeling understood. This was communicated face to face, typically with anger (e.g., “The client reacts strongly and becomes, as I see it, increasingly angry and upset”).

Therapist reactions

The most frequent of the “Covert reactions” was feeling that the feedback was important (e.g., “I experienced the feedback as honest. She expressed something we’d both probably felt for some time”) and challenging. Therapists were typically surprised by the feedback and reported negative emotions toward the client (e.g., “I was irritated with her “unreasonable” need for help, wish to be cared for despite her intellectual resources”). They typically also felt negatively toward themselves, but few had self-critical thoughts. Instead, they generally attributed the client’s dissatisfaction to the client, often referring to transference or defense mechanisms (e.g., “Q: How did you understand the feedback? A: Passive aggression”).

In terms of “Therapist actions,” therapists generally accommodated client’s request, often after some negotiation or trying out different strategies, for example, by accepting the client’s request not to talk about certain topics, yet insisting that the client came back for another session before discontinuing therapy. They typically also described making conscious efforts to stay calm, for example, by “taking a step back” and reflect.

Consequences of feedback

The clients generally responded positively to the therapists’ efforts to repair, for instance, accepting of the therapist’s apology or displaying positive emotions (e.g., “The client seemed relieved, and the atmosphere in the room alleviated”). The relationship between therapist and client was described as improved as a result of the therapists’ changed behavior (e.g., “Rather than being stuck in a closed system characterized by defense positions and anxiety, the client and I together managed to create the foundation of a new way of being together”). Many also reported that the client benefitted more from therapy in the following therapy.

Looking back, the therapists generally expressed satisfaction with how they had acted during the situation (e.g., “I think the boundaries that I tried to set for the client helped him in the long run, even though it was uncomfortable for me to encounter his reaction then and there”), and typically also conveyed acceptance and understanding of their own reactions. Only infrequently had the feedback had an influence on the therapists beyond the “here-and-now”-learning that led the therapists to change their behavior within that particular therapy.

Retrospectively applied learning

As indicated by the subcategory “Meets new clients differently,” lessons learned through the feedback were manifested in therapists’ behavior changes with future clients therapists in this group of cases.

Background

Clients were generally described as having relational difficulties and were typically traumatized (e.g., “Client is referred with a history of severe neglect and foster care placement where she suffered sexual abuse before puberty”). Generally, there were indications of client being satisfied with the therapy prior to the feedback, although therapists also typically referred to early signs of client dissatisfaction.

Negative feedback

The content of the feedback was generally something specific the therapist had done or had failed to do, and these clients typically also said that they wanted to quit working with the therapist. This was typically communicated face to face (e.g., “The feedback was given directly, ‘when you say things like that I can’t talk to you anymore’”), but rarely in an agitated or angry fashion, in contrast to what was the case with Immediately Applied Learning.

Therapist reactions

No general categories emerged that described how therapists immediately felt inside when confronted with the negative feedback, although therapists typically experienced the situation as challenging and unusual and experienced negative feelings, such as shame or guilt, toward themselves (e.g., “I felt guilty about having, as he indicated, made him worse, felt completely helpless as all my attempts to talk about what happened between client and therapist ‘stranded’”). Similarly, there were no general “Therapist actions.” Therapists typically described giving in to the client’s request and frequently going to some extent to try to fulfill the client’s wishes, such as changing therapeutic style or behaving in ways that felt foreign to the therapist. Other strategies were sporadically referred to.

Consequences of feedback

The clients’ dissatisfaction typically persisted, in most cases for an extended period of time, amounting to a continued struggle to improve the therapeutic relationship (e.g., “I think the client saw how I tried to change my therapeutic style, but that it did not quite work. Think she felt respected [ … ] but that she still felt frustrated. She conveyed feeling that I cared for her, but that I was unable to give her what she needed.”

In retrospect, therapists had typically gained a more nuanced and complex understanding of their own behavior, both how they had contributed to the client’s dissatisfaction and the events that followed (e.g., “The feedback as I see it today probably referred to the treatment being a bit too ‘superficial’, and that I hadn’t succeeded in touching upon emotions that were important to the client”), and why they reacted as they did (e.g., “I felt too warmly for the client. Perhaps got a little too eager to help”).

In retrospect, the negative feedback had inspired various changes or adjustments in therapists’ behavior toward new clients. The changes related to each therapist’s understanding of why the client was dissatisfied in that specific instance; however, a common theme across all cases was attempting to bridge the gap between therapists’ and clients’ perspectives. For instance, behavior changes were aimed toward clarifying roles and responsibilities (e.g., “I now request more of the clients that I meet, therapy needs to be their (our) project, not something that I perform on someone”), changing routines to be able to be more present with clients (e.g., “I’ve gained more respect for [the importance of early alliance work] by making enough time and eliminating unnecessary stress before and after the first session with a client”), ascertaining that the therapist’s interventions were well received (e.g., “I’ve become even more attentive toward making sure that my interpretations are palpable to the client and that it’s understandable and acceptable for the client that I do what I do”), and meta-communicating more openly about the therapeutic relationship (e.g., “I’ve also become less afraid to explore what happens between me and the client so that it’s hopefully easier to talk about the negative experiences in therapy”).

Non-applied learning

Inspection of the cases where therapists did not describe any behavior changes following the feedback revealed that also in these cases, therapists described the feedback as influential and educational, most notably by generating new ideas.

Background

Clients were generally described as being depressed in addition to having complex psychopathologies and typically also relational difficulties. Therapists generally reported that there were early indications of client dissatisfaction. In contrast to the two applied learning case groups however, few therapists reported any indications of their clients also being satisfied prior to the feedback situation.

Negative feedback

Clients typically expressed a global dissatisfaction with the therapist, often referred to as “bad chemistry” or general, nonspecific dislike of the therapist. Most clients communicated that they wanted to quit working with the therapist (e.g., “She ended the contact, said that ‘this does not suit me’”). The negative feedback was typically communicated indirectly, in letters, e-mails, or through others such as the client’s family members or the client’s physician.

Therapist reactions

The situation was generally experienced as unusual to the therapist (e.g., “I haven’t had similar experiences with other clients that I can think of”), but few experienced it as challenging, and they rarely reported negative feelings. Their most typical reaction was to invite to further dialogue (e.g., “I offered to continue our contact and suggested that we talk about the things that didn’t work between us in order to set a new course together”). Few had made efforts to accommodate the client, and none apologized.

Consequences of feedback

The client’s dissatisfaction typically persisted, and most clients terminated therapy shortly after the feedback situation (e.g., “The client quit and I haven’t seen her since”). Like in Retrospectively Applied Learning, therapists typically reported a change in how they understood their own contribution to the situation from the time it occurred to the present, both in terms of how their own actions influenced the event, and in acceptance and understanding of how they had reacted. They typically reported that the feedback changed how they felt about themselves and their work (e.g., “I think my professional self-esteem suffered a blow”). The feedback had typically given them a reminder of the importance of the client’s perspective (e.g., “I think it illustrates the importance of getting the client’s reactions to me as therapists, and any differences in theory of change, “on the table,” talk about it”) and new ideas about what to do differently if they were to find themselves in a similar situation in the future (e.g., “I would focus more on where the client is at. Not assume that the client intuitively see that it’s wise to let one’s wishes, dreams and needs guide one in life”). In contrast to with the other two groups of cases, however, this new insight was not described as having translated into changed behavior with new clients.

Comparison of Cases Reported by Therapists Using Versus Not Using ROM

To examine if working with ROM influenced how therapists understood and responded to the negative feedback, we compared the 10 cases reported by ROM users to the eight cases reported by therapists who did not work with ROM. Two therapists made references to feedback obtained through PCOMS (Bertolini & Miller, Citation2012; Duncan, Citation2012) in their descriptions of the background to the feedback (e.g., “The feedback on PCOMS was [ … ] within the ‘green area’”). In none of the cases did ROM feature in the description of the actual feedback event, nor in the consequences of the event. Cases described by ROM vs. non-ROM users were found to differ only in the subcategory “Indications of client satisfaction,” with therapists working with ROM generally and those not working with ROM only variantly mentioning signs of clients being satisfied with treatment prior to the feedback. The category “Indications of client dissatisfaction” was typical within both categories. ROM users generally reported cases that were classified as Immediately Applied Learning, typically cases that were classified as Retrospectively Applied Learning, and variantly cases that were classified as Non-Applied Learning.

Discussion

The 18 therapists’ narratives provide rich descriptions of their reactions to negative feedback from clients. All described the feedback as educational, but learning was manifested in different ways. In the cases we categorized as Immediately Applied Learning, therapists used the insight that the feedback gave them to adjust their own behavior with that client, leading to improved therapy processes and outcomes. In Retrospectively Applied Learning cases, the feedback inspired changes in therapists’ way of working with new clients, specifically behaviors aimed toward bridging the gap between clients’ and therapists’ diverging perspectives. Finally, in the Non-Applied Learning cases, new insight and ideas that had been generated by the feedback had not led therapists to change their behavior. When comparing cases describing these manifestations of learning, we found consistent differences both the nature of the feedback and the therapists’ reactions to it.

The Nature of the Feedback

Our results support feedback theory (Sapyta et al., Citation2005) in emphasizing the value of direct, specific, and promptly delivered feedback. Therapists who had been given face-to-face, unambiguous feedback about something they had done or failed to do were more often able to use this new information in ways that benefitted the current and/or future clients. This contrasted with the global, nonspecific, and indirectly communicated dissatisfaction that was described in the Non-Applied Learning cases. Therapists’ experiences in these latter cases bear similarities to those of therapists in a qualitative study of the impact of premature termination (Piselli, Halgin, & MacEwan, Citation2011): Several general and vague ideas about what they could have done differently, yet a global sense of insecurity about how to prevent similar situations from happening again due to the lack of unambiguous, specific information about what went wrong.

The quality of the therapeutic relationship may help explain why the clients’ feedback was specific and direct in some cases and not in others. In accordance with the finding that therapists tend to view the therapeutic alliance in slightly less positive terms than clients do (Tryon, Blackwell, & Hammel, Citation2007), most participants in our study mentioned problematic aspects of the therapeutic alliance prior to the feedback. However, in the cases where therapists had applied what they had learned with clients, they also mentioned indications of client satisfaction. The overall description of the early alliance was thus more nuanced and more positive in Immediately and Retrospectively Applied Learning than in Non-Applied Learning, where the feedback was non-specific and indirectly communicated. Consistent with this, qualitative investigations of clients’ experiences of misunderstandings in therapy (Rhodes et al., Citation1994) and premature termination (Knox et al., Citation2011) found that poorer therapeutic relationships made it more difficult for clients to talk to their therapists about their dissatisfaction.

Only in the Immediately Applied Learning cases did the therapists generally perceive the clients as agitated or angry during the feedback situation. We speculate that the clients’ anger may have helped convey to the therapists the importance of the feedback and motivated their efforts to resolve the situation, as these cases are characterized by therapists immediately feeling that the feedback was important and trying out a variety of repair strategies with the client, in contrast to the other two expressions of learning

The Therapists’ Reactions

In Immediately Applied Learning cases, therapists understood the client’s dissatisfaction as a result of his or her pathology. This is in accordance with the central idea in dynamic theory that the client’s interpersonal problems are acted out in the therapeutic relationship and pose a challenge to alliance formation but also possibilities for therapeutic change (Hill & Knox, Citation2009; Horvath & Bedi, Citation2002). Other lines of research however give reason to caution against therapists’ blaming the client rather than assuming responsibility when therapy fails. Sapyta et al. (Citation2005) suggested that therapists are less likely to change ineffective behavior when they use external attribution as a way of reducing the cognitive dissonance that results from negative feedback, and Murdock et al. (Citation2010) demonstrated that therapists’ external attribution of premature termination had elements of self-image preserving biases. Others (Henry, Schacht, & Strupp, Citation1990; von der Lippe, Monsen, Ronnestad, & Eilertsen, Citation2008) have documented increasingly more problematic interactions and poor therapy outcomes when therapists respond to client hostility with hostility of their own. Nissen-Lie and colleagues found that therapists who more often experienced professional self-doubt had better outcomes with clients (Nissen-Lie, Monsen, Ulleberg, & Ronnestad, Citation2013) and formed better alliances (Nissen-Lie, Monsen, & Ronnestad, Citation2010), and Chow et al. (Citation2015) reported that the most effective therapists were more often surprised by their clients’ feedback, which the authors interpreted as indicative of an open attitude toward feedback. Finally, de Jong, van Sluis, Nugter, Heiser, and Spinhoven (Citation2012) found that clients of therapists with an “internal feedback propensity” (preference for relying upon their own judgments rather than feedback from others) had slower rates of change, although “external feedback propensity’ did not moderate the effect of feedback on outcome in this study.

Contrary to our expectations given these research findings, even while engaging in external attribution and having negative emotional reactions therapists in Immediately Applied Learning responded in ways that contributed positively to the therapeutic relationship. What they described doing was similar to the process of repairing alliance ruptures (i.e., tensions or breakdowns in the collaboration between therapist and client) described by Safran and Muran (Citation2000): Inviting the client to examine the feedback together and repairing the rupture through flexible negotiation and accommodation, without neither loosing nor stubbornly defending their own stance. Our analysis did not yield a clear explanation of how they managed to do this; however, it is possible that external attribution may have been less personally challenging, and the accompanying other-directed negative emotions easier to regulate, than the guilt and shame that dominated in Retrospectively Applied Learning. Thus, external attribution may have actually helped therapists remain open enough to the feedback to make it possible to respond effectively to it in these cases. In support of this, therapists typically reported being surprised by the feedback and immediately thinking that it was important, indicating perhaps the openness to feedback without being derailed by it that has been described as one of the characteristics of master therapists (Jennings & Skovholt, Citation1999).

In cases where Retrospectively Applied Learning was described, a different pattern of reactions dominated. Therapists reported immediately feeling shame, guilt, or other negative emotions directed toward themselves. Perhaps motivated by this, their most frequent behavioral response was doing exactly as the client requested, for instance, by trying to change their therapeutic style. The therapists’ giving in may have prevented many of these clients from terminating therapy. Nevertheless, problematic aspects of the strategy are suggested by the typical persistence of the clients’ dissatisfaction following the feedback in these cases. We note similarities between what these therapists did, and what Nissen-Lie et al. (Citation2015) conceptualized as non-constructive coping mechanisms (among other behaviors, acting out by postponing the work of therapy or making changes to the therapeutic contract) and found to be associated with less client change.

Looking back, very few therapists in the Retrospectively Applied Learning cases were satisfied with how they managed the situation. Lingering self-negative feelings may have increased their motivation to change something in their way of working to prevent similar situations from happening. What they experienced was perhaps one in the “series of humiliations” that therapists in Ronnestad and Skovholt’s (Citation2012) large-scale investigation of therapist development went through when confronted with their own fallibility. Another finding from this study is the importance of continuous reflection as a prerequisite for learning; similarly, in our investigation, therapists had arrived at a more nuanced understanding of their own contributions to the situation in retrospect, suggesting that they had spent some time reflecting on what had happened. The behavior change that followed might have been informed by the specific negative feedback they had received as suggested in the feedback model by Sapyta et al. (Citation2005). Presumably, the changes they had made in their way of meeting new clients helped them form better alliances with new clients, although the design of this study does not provide any actual outcome data.

It is interesting to note that while almost all therapists in our investigation cited psychodynamic theory as their most important influence in their work, only in the cases describing Immediately Applied Learning did they seem to apply the model to the situation: Conceptualizing the feedback as revealing something of the client’s dynamic, and providing corrective emotional experiences and insight for the clients through exploration of the situation and repair of the impasse. As discussed above, we suggest that emotion regulation differentiated this group from the other two. Emotion regulation is central in the psychodynamic concept of countertransference management, and consistent with our findings, successful managing of countertransference has been found to be associated with better therapy outcomes (Hayes, Gelso, & Hummel, Citation2011). While it is unclear why some therapists were able to handle their emotions or countertransference reactions better than others, one possibility is in the nature of the feedback: In Immediately Applied Learning, clients expressed their dissatisfaction with anger, which may have triggered irritation rather than guilt or shame in their therapists. In contrast, clients in the Retrospectively Applied Learning cases typically wanted to discontinue working with the therapist, which may have indicated a more severe message from the client that the therapist was at fault. Therapist and/or client factors may also help explain the difference between groups. This will be discussed in the next section.

The Clients’ and Therapists’ Contributions

In this investigation, we have consciously chosen not to include the information that the therapists gave about themselves in the analysis (with the exception of use of ROM, see below). The reason for this is the problems of representativeness and generalizability that are inherent in our design, with each therapist describing just one negative feedback event that may or may not have been typical for that particular therapist. Similarly, we hesitate to speculate about whether or not characteristics of the clients explain our results, as the descriptions of the clients’ behavior and psychopathology are filtered through the therapists’ retrospective narrative, with no information from the clients themselves or other parties. Nevertheless, our results raise some interesting questions about what each of the two parties brought into the interaction. As discussed in Schröder, Orlinsky, Rønnestad, and Willutzki’s (Citation2015) summary of various lines of research on therapist difficulties, there are at least two sides to any story of negative therapy processes: The therapist’s personality and ability to deal with problems as well as the client’s interpersonal style and problems.

With regards to the therapists’ contribution, we found that most of the ROM users described cases where the feedback had led to either Immediately or Retrospectively Applied Learning; therapists not working with ROM dominated the cases that were categorized as Non-Applied Learning. This raises the possibility that ROM users elicit, understand, and/or respond to negative feedback differently. Our study was however not designed to explore this issue and does not provide unambiguous information regarding how ROM influences therapists’ attitudes toward client feedback. In comparison of the cases described by ROM vs. non-ROM users, only the subcategory of the quality of the relationship prior to the feedback event differed between the two groups, with ROM-users generally and non-ROM users only variantly mentioning indications of a positive alliance. ROM never featured in the descriptions of the feedback events nor its consequences, although mentioned by a few therapists when discussing the background to the events. One possible interpretation is that the feedback obtained through ROM is of a different character, perhaps more tied to specific aspects of the therapy process and thus, less salient and memorable, than the events that therapists chose to describe when responding to this investigation.

Our knowledge of client characteristics that may have influenced the events described is limited to the therapists’ (possibly biased) descriptions of their clients’ psychopathology. In Immediately Applied Learning cases, clients were generally described as having relational difficulties and personality disorders, and the therapists responded in accordance with the dynamic model by attributing the feedback to the client and handling it by flexible negotiation. In the Retrospectively Applied Learning cases, clients were typically described as traumatized. We speculate that therapists’ compassion and empathy, when faced with victims of trauma, may have made them more likely to turn frustration with the negative feedback toward themselves, and to give in to clients’ requests. Finally, in the Non-Applied Learning cases, clients were generally depressed, raising the possibility that their choices no to give direct feedback and drop out of treatment may have been caused by the hopelessness and lack of initiative that is symptomatic of depression.

Limitations

The study design, with therapist reported, written, retrospective accounts, limits the conclusions that can be drawn from this study. The descriptions of the negative feedback situations were filtered through the therapists’ perceptions, without any client data to fill in gaps or correct possible misrepresentations of the clients or the situations. What the participants remembered and chose to describe may have been influenced by their wish to share a certain narrative and their knowledge of the events that followed, and other, potentially valuable information was possibly under-communicated or lost. For instance, the description of the pre-feedback relationship as more problematic in the Non-Applied Learning group than in the other two groups may be the result the therapists’ cognitive and affective processes such as motivation to explain why therapy had failed. Possibly, the written format of our investigation accentuated memory biases, as the absence of an interviewer that challenges the participant’s history or asks in-depth explorative questions might make it less likely for new and unexpected information to come to light (Kvale & Brinkmann, Citation2009). On the other hand, the written format may have made social face-serving concerns less influential and thus facilitated sharing of personally difficult material.

More cases and richer descriptions of each case would have increased confidence our in the results. Two of the groups contain six cases each, one short of Hill’s (Citation2012) recommendations for subsample size when comparing groups. Furthermore, each participant described just one instance of negative feedback of their own choosing. Presumably these events were selected because they stood out in some way to the therapists, and we do not assume that they are representative for each particular therapist and also not for therapists in general. Rather, we understand them as different expressions of learning processes that therapists may experience with different clients or even within the same therapy. Our aim was to explore and work toward understanding an area where there is little prior knowledge.

Finally, the researchers’ expectations and biases may have influenced our findings. We did, however, take some precautions in selecting a research team that represented different background and perspectives, record and continually refer back to our biases and expectations, and critically question our own interpretations during the analysis. The effect of bracketing biases and expectations is illustrated by the unexpectedness of some of our findings, such as three different manifestations of learning and the external attribution in the Immediately Applied Learning cases.

Implications for Practice

While the qualitative design limits the ability to generalize from our results, therapists may find aspects of them transferrable to their own practices. Our results highlight the importance of obtaining clear, unambiguous feedback about the clients’ negative reactions to therapy. Therapists should be aware that clients may hesitate to express dissatisfaction, and work toward creating a safe therapeutic environment where meta-communicating about the therapeutic relationship is possible. One of the ROM systems that are available may be found to be of help, for instance, one that includes alliance feedback such as the PCOMS (Bertolini & Miller, Citation2012; Duncan, Citation2012) or the OQ system (Lambert, Citation2004).

We found that negative feedback elicited difficult feelings that even these experienced therapists had difficulties coping with. Therapists would be well advised to pay attention to their own emotional reactions, and regulate rather than act out difficult feelings when they arise. It may be especially challenging to cope constructively with self-directed negative feelings. We would suggest that openness around this seemingly normal phenomenon would be helpful to reduce shame and guilt, perhaps especially for new therapists.

Furthermore, when therapists experience difficult therapy processes or premature termination, it might be constructive for their own professional development to consider what their own contributions to the interaction may have been, think about what they could have done differently, and try this out with new clients. Based on our results, we would suggest that programs aimed at training novice therapists focus both on how to obtain, receive, and respond to negative feedback, and how to use information from clients to improve one’s skills.

Implications for Research

We believe that this investigation provides a good first step toward understanding how therapists react to negative feedback. It also raises some interesting questions that could be investigated further, such as: How do therapists translate new insights into new skill sets? When do therapists maintain and when do they lose the balance between being open to and assuming responsibility for negative feedback on the one hand, and having an understanding of the client’s contribution to negative processes on the other hand? How much of negative processes can be attributed to different sources such as the client, the therapist, and the unique interaction between the two? It would be interesting to see some of these questions investigated with different research methodologies. Associating therapists’ experiences with client outcome data would be of particular interest, as it is unclear whether or not the therapists’ experiences of repairing and learning in our study translated into actual client benefits. The clients’ perspectives on giving feedback to therapists should not be neglected. Finally, we suggest that incorporating situational and contextual factors in the study of therapist variability may shed further light on when therapists work efficiently and when not.

Acknowledgements

We are grateful to the 18 therapists who participated in this research for sharing their experiences of these frequently personally challenging therapy processes.

Additional information

Funding

This work was supported by the Liaison Committee between the Central Norway Regional Health Authority (RHA) and the Norwegian University of Science and Technology (NTNU) – Samarbeidsorganet HMN-NTNU [grant number 46056812 (653010 INM)].

Notes

* This research was conducted at St. Olavs University Hospital, Tiller, Norway.

References

  • Anseel, F., Beatty, A. S., Shen, W., Lievens, F., & Sackett, P. R. (2015). How are we doing after 30 years? A meta-analytic review of the antecedents and outcomes of feedback-seeking behavior. Journal of Management, 41(1), 318–348. doi: 10.1177/0149206313484521
  • Bertolini, B., & Miller, S. D. (2012). ICCE manuals on feedback-informed treatment (Vols 1–6). Chicago, IL: ICCE Press.
  • Beutler, L., Malik, M., Alimohamed, S., Harwood, T., Talebi, H., & Noble, S. (2004). Therapist variables. In M. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed., pp. 226–306). New York, NY: John Wiley & Sons.
  • Binder, J. L., & Strupp, H. H. (1997). “Negative process”: A recurrently discovered and underestimated facet of therapeutic process and outcome in the individual psychotherapy of adults. Clinical Psychology: Science and Practice, 4(2), 121–139.
  • Blanchard, M., & Farber, B. A. (2015). Lying in psychotherapy: Why and what clients don’t tell their therapist about therapy and their relationship. Counselling Psychology Quarterly, 29(1), 90–112. doi: 10.1080/09515070.2015.1085365
  • Chow, D. L., Miller, S. D., Seidel, J. A., Kane, R. T., Thornton, J. A., & Andrews, W. P. (2015). The role of deliberate practice in the development of highly effective psychotherapists. Psychotherapy, 52(3), 337–345. doi: 10.1037/pst0000015
  • Claiborn, C. D., & Goodyear, R. K. (2005). Feedback in psychotherapy. Journal of Clinical Psychology, 61(2), 209–217. doi: 10.1002/jclp.20112
  • Coutinho, J., Ribeiro, E., Hill, C., & Safran, J. (2011). Therapists’ and clients’ experiences of alliance ruptures: A qualitative study. Psychotherapy Research, 21(5), 525–540. doi: 10.1080/10503307.2011.587469
  • Duncan, B. L. (2012). The partners for change outcome management system (PCOMS): The heart and soul of change project. Canadian Psychology/Psychologie Canadienne, 53(2), 93–104. doi: 10.1037/a0027762
  • Farber, B. A. (2003). Patient self-disclosure: A review of the research. Journal of Clinical Psychology, 59(5), 589–600. doi: 10.1002/jclp.10161
  • Goldberg, S. B., Rousmaniere, T., Miller, S. D., Whipple, J., Nielsen, S. L., Hoyt, W. T., … Wampold, B. E. (2016). Do psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a clinical setting. Journal of Counseling Psychology, 63(1), 1–11. doi: 10.1037/cou0000131
  • Hayes, J. A., Gelso, C. J., & Hummel, A. M. (2011). Managing countertransference. Psychotherapy, 48(1), 88–97. doi: 10.1037/a0022182
  • Henry, W. P., Schacht, T. E., & Strupp, H. H. (1990). Patient and therapist introject, interpersonal process, and differential psychotherapy outcome. Journal of Consulting and Clinical Psychology, 56(8), 768–774. doi: 10.1037/0022-006X.58.6.768
  • Hill, C. E. (2012). Consensual qualitative research: A practical resource for investigating social science phenomena. Washington, DC: American Psychological Association.
  • Hill, C. E., Kellems, I. S., Kolchakian, M. R., Wonnell, T. L., Davis, T. L., & Nakayama, E. Y. (2003). The therapist experience of being the target of hostile versus suspected-unasserted client anger: Factors associated with resolution. Psychotherapy Research, 13(4), 475–491. doi: 10.1093/ptr/kpg040
  • Hill, C. E., & Knox, S. (2009). Processing the therapeutic relationship. Psychotherapy Research, 19(1), 13–29. doi:10.1080/10503300802621206
  • Hill, C. E., Knox, S., Thompson, B. J., Williams, E. N., Hess, S. A., & Ladany, N. (2005). Consensual qualitative research: An update. Journal of Counseling Psychology, 52(2), 196–205. doi: 10.1037/0022-0167.52.2.196
  • Hill, C. E., Nutt-Williams, E., Heaton, K. J., Thompson, B. J., & Rhodes, R. H. (1996). Therapist retrospective recall of impasses in long-term psychotherapy: A qualitative analysis. Journal of Counseling Psychology, 43(2), 207–217. doi: 10.1037/0022-0167.43.2.207
  • Hill, C. E., Thompson, B., & Corbett, M. (1992). The impact of therapist ability to perceive displayed and hidden client reactions on immediate outcome in first sessions of brief therapy. Psychotherapy Research, 2(2), 143–155. doi: 10.1080/10503309212331332914
  • Hill, C. E., Thompson, B. J., Cogar, M. C., & Denman, D. W. (1993). Beneath the surface of long-term therapy: Therapist and client report of their own and each other’s covert processes. Journal of Counseling Psychology, 40(3), 278–287. doi: 10.1037/0022-0167.40.3.278
  • Horvath, A., & Bedi, R. (2002). The alliance. In C. Norcross (Ed.), Psychotherapy relationships that work: Therapist contributions and responsiveness to patients (pp. 37–69). London: Oxford University Press.
  • Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996). Evaluation of psychotherapy: Efficacy, effectiveness, and patient progress. American Psychologist, 51(10), 1059–1064. doi: 10.1037/0003-066X.51.10.1059
  • Jennings, L., & Skovholt, T. M. (1999). The cognitive, emotional, and relational characteristics of master therapists. Journal of Counseling Psychology, 46(1), 3–11. doi: 10.1037/0022-0167.46.1.3
  • de Jong, K., van Sluis, P., Nugter, M. A., Heiser, W. J., & Spinhoven, P. (2012). Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychotherapy Research, 22(4), 464–474. doi: 10.1080/10503307.2012.673023
  • Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254–284. doi: 10.1037/0033-2909.119.2.254
  • Knox, S., Adrians, N., Everson, E., Hess, S., Hill, C., & Crook-Lyon, R. (2011). Clients’ perspectives on therapy termination. Psychotherapy Research, 21(2), 154–167. doi:10.1080/10503307.2010.534509
  • Kvale, S., & Brinkmann, S. (2009). Interviews: Learning the craft of qualitative research interviewing. London: Sage.
  • Lambert, M. J. (2004). Administration and scoring manual for the OQ-45.2 (outcome questionnaire). Salt Lake City, UT: OQ Measures, LLC.
  • Lambert, M. J. (2007). Presidential address: What we have learned from a decade of research aimed at improving psychotherapy outcome in routine care. Psychotherapy Research, 17(1), 1–14. doi: 10.1080/10503300601032506
  • Macdonald, J., & Mellor-Clark, J. (2014). Correcting psychotherapists’ blindsidedness: Formal feedback as a means of overcoming the natural limitations of therapists. Clinical Psychology & Psychotherapy, 22(3), 249–257. doi: 10.1002/cpp.1887
  • Miller, S. D., Hubble, M. A., Chow, D., & Seidel, J. (2015). Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy, 52(4), 449–457. doi: 10.1037/pst0000031
  • Moltu, C., Binder, P.-E., & Nielsen, G. H. (2010). Commitment under pressure: Experienced therapists’ inner work during difficult therapeutic impasses. Psychotherapy Research, 20(3), 309–320. doi: 10.1080/10503300903470610
  • Murdock, N. L., Edwards, C., & Murdock, T. B. (2010). Therapists’ attributions for client premature termination: Are they self-serving? Psychotherapy: Theory, Research, Practice, Training, 47(2), 221–234. doi: 10.1037/a0019786
  • Nissen-Lie, H. A., Monsen, J. T., & Ronnestad, M. H. (2010). Therapist predictors of early patient-rated working alliance: A multilevel approach. Psychotherapy Research, 20(6), 627–646. doi: 10.1080/10503307.2010.497633
  • Nissen-Lie, H. A., Monsen, J. T., Ulleberg, P., & Ronnestad, M. H. (2013). Psychotherapists’ self-reports of their interpersonal functioning and difficulties in practice as predictors of patient outcome. Psychotherapy Research, 23(1), 86–104. doi: 10.1080/10503307.2012.735775
  • Nissen-Lie, H. A., Rønnestad, M. H., Høglend, P. A., Havik, O. E., Solbakken, O. A., Stiles, T. C., & Monsen, J. T. (2015). Love yourself as a person, doubt yourself as a therapist? Clinical Psychology & Psychotherapy. Advance online publication. doi:10.1002/cpp.1977
  • Piselli, A., Halgin, R. P., & MacEwan, G. H. (2011). What went wrong? Therapists’ reflections on their role in premature termination. Psychotherapy Research, 21(4), 400–415. doi: 10.1080/10503307.2011.573819
  • Regan, A. M., & Hill, C. E. (1992). Investigation of what clients and counselors do not say in brief therapy. Journal of Counseling Psychology, 39(2), 168–174. doi: 10.1037/0022-0167.39.2.168
  • Rhodes, R. H., Hill, C. E., Thompson, B. J., & Elliott, R. (1994). Client retrospective recall of resolved and unresolved misunderstanding events. Journal of Counseling Psychology, 41(4), 473–483. doi: 10.1037/0022-0167.41.4.473
  • Ronnestad, M. H., & Skovholt, T. M. (2003). The journey of the counselor and therapist: Research findings and perspectives on professional development. Journal of Career Development, 30(1), 5–44.
  • Ronnestad, M. H., & Skovholt, T. M. (2012). The developing practitioner: Growth and stagnation of therapists and counselors. New York, NY: Routledge.
  • Safran, J. D., & Muran, J. C. (2000). Negotiating the therapeutic alliance: A relational treatment guide. New York, NY: Guilford Press.
  • Sapyta, J., Riemer, M., & Bickman, L. (2005). Feedback to clinicians: Theory, research, and practice. Journal of Clinical Psychology, 61(2), 145–153. doi: 10.1002/jclp.20107
  • Schröder, T., Orlinsky, D., Rønnestad, M. H., & Willutzki, U. (2015). Psychotherapeutic process from the psychotherapist’s perspective. In O. C. G. Gelo (Eds.), Psychotherapy research (pp. 351–365). Wien: Springer-Verlag.
  • Tracey, T. J., Wampold, B. E., Lichtenberg, J. W., & Goodyear, R. K. (2014). Expertise in psychotherapy: An elusive goal? American Psychologist, 69(3), 218–229. doi: 10.1037/a0035099
  • Tryon, G. S., Blackwell, S. C., & Hammel, E. F. (2007). A meta-analytic examination of client–therapist perspectives of the working alliance. Psychotherapy Research, 17(6), 629–642. doi: 10.1080/10503300701320611
  • von der Lippe, A. L., Monsen, J. T., Ronnestad, M. H., & Eiertsen, D. E. (2008). Treatment failure in psychotherapy: The pull of hostility. Psychotherapy Research, 18(4), 420–432. doi: 10.1080/10503300701810793
  • Walfish, S., McAlister, B., O'Donnell, P., & Lambert, M. J. (2012). An investigation of self-assessment bias in mental health providers. Psychological Reports, 110(2), 639–644. doi: 10.2466/02.07.17.PR0.110.2.639-644