Memorable Experiences in Therapeutic Assessment: Inviting the Patient's Perspective Following a Pretreatment Randomized Controlled Trial

ABSTRACT

Accumulating evidence documents the efficacy of Therapeutic Assessment (TA) in terms of symptom reduction and other outcomes, but only minimal data speak to the patient's perspective of what is memorable, or potentially important, about this intervention. In line with the humanistic and phenomenological philosophy of TA, we solicited patient input by asking personality disorder (PD) patients who participated in a recent randomized controlled trial (De Saeger et al., 2014) about their experiences. We report on 10 PD patients who were administered semistructured interviews designed to assess an in-depth perspective of undergoing TA. Our methodological approach can be described as phenomenological and integrative, approximating guidelines provided by the Consensual Qualitative Research paradigm (Hill, 2012). Four core content domains emerged from the transcribed and coded interview protocols: (a) relationship aspects, (b) new insight into personal dynamics, (c) sense of empowerment, and (d) validation of self. Novel experiences were mostly of a relational nature, and pertained to feeling of being treated like an equal and essential partner in a highly individualized venture. Research and clinical implications of these patient reports of TA participation are discussed.

Therapeutic Assessment (TA) is a semistructured method of collaborative psychological assessment, which, in addition to regular information gathering purposes, also explicitly aims for therapeutic impact (Finn & Tonsager, 1997). The TA assessor is a participating observer whose primary goal is to help the patient gain new information that could help him or her improve the patient's quality of life. Throughout TA, patient and assessor work collaboratively. The process commences by formulating individualized assessment questions, which subsequently orient the testing phase and the interactive summary and discussion session. Accordingly, assessment is client-focused rather than test-focused (Finn, 2007; Finn & Tonsager, 1997).

Accumulating evidence suggests that TA can be an effective intervention in various populations, although the specific outcomes appear to be contingent on several factors. For example, immediate symptomatic improvement, commensurate with or better than the impact of psychotherapy, was noted in student health center samples (Finn & Tonsager, 1992; Newman & Greenway, 1997), but not in patients with eating disorders or personality disorders (De Saeger et al., 2014; Peters, 2001). For these patients, however, favorable differences emerged on process variables like alliance ratings, satisfaction, and preparation for psychotherapy. Conversely, in a sample of patients with borderline personality disorder who received manual assisted cognitive therapy, somewhat greater clinical improvement was associated with the introduction of add-on TA, but, contrary to expectation, no superior treatment retention was found (Morey, Lowmaster, & Hopwood 2010). Some of the noted inconsistencies might be due to differences in patient characteristics (e.g., differences in diagnostic severity, age) or nature of referral (self-referred or other-referred), but these could also be due to differences in the operationalization of the model. For example, De Saeger et al. (2014) were the first to conduct a trial in adult patients executing the full model; that is, including performance-based testing (i.e., Rorschach Inkblot Method in that study) and assessment intervention sessions. Presumably, a better understanding of this complex pattern of outcomes would be furthered if we better understood the critical mechanisms of change during TA. Such research, however, requires highly powered statistical designs (e.g., dismantling studies with a large sample size), which are often hard to conduct in real-life clinical settings for both logistic and ethical reasons.

An alternative heuristic strategy, much in line with TA's underlying humanistic philosophy, is to enlist the patients' perspectives on what they deem memorable or novel about the TA experience. Fischer (2006), developer of the related collaborative model of assessment, has long emphasized the similarities in the philosophy of collaborative and therapeutic assessment and the method of qualitative psychological research: “It is my hope that considering the similarities of individualized–collaborative psychological assessment and qualitative psychological research will encourage us to regard data as being instances of persons in lived relations to environment, others, and self ” (p. 354). Indeed, in TA, patients are treated as collaborators throughout the assessment process, and to involve them in this way in the empirical research seems like a highly consistent extension of this general approach. Accordingly, we wanted to conduct an in-depth, qualitative examination of the patients' experience of TA to better understand what they remember and take away from it. Remarkably little is known from this perspective, although the research by Ward (2008) broke new ground by capturing the experiences of six assessee and six assessor experiences of significant events in psychological assessment feedback. Although these findings inform us about assessment feedback experiences generated in a regular information gathering assessment (IGA), they cannot speak to TA specifically.

Our methodological approach can be described as phenomenological and integrative (e.g., following suggestions by Kuckartz [2014], Creswell [2012], Elliott, Fischer, & Rennie [1999], and Malterud [2001]), but overall comes closest to the guidelines provided by consensual qualitative research (CQR; Hill, 2012; Hill, Thompson, & Williams, 1997). Its essential components are the use of (a) open-ended questions in semistructured data collection techniques; (b) several judges throughout the data analysis process to foster multiple perspectives; (c) consensus to arrive at judgments about the meaning of the data; (d) at least one auditor to check the work of the primary team of judges; and (e) domains, core ideas, and cross-analyses in the data analysis. We approximated adherence to these guidelines, but opted for a slightly modified approach. Specifically, instead of thickening the raw material into core ideas, we decided to stay very close to the wording of the participants' reports, and coded original text segments directly into domains and categories (akin to in vivo coding). We largely modeled the presentation of our results according to the principles of CQR, again relying heavily on verbatim reports to illustrate the emerging content.

In sum, among personality disorder (PD) patients who had participated in our randomized controlled trial (RCT) comparing TA with an alternative, highly structured pretreatment motivation package (goal-focused pretreatment intervention [GFPTI]), as described in De Saeger et al. (2014),¹ we conducted semistructured patient interviews to capture the patient's experience of TA with respect to three specific questions:

1.	What aspects of TA (if any) do patients report as memorable?
2.	What aspects of TA (if any) do patients deem new, different, or distinctive?
3.	What aspects of TA (if any) do patients report as negative about TA?

Patient reports on these questions might elucidate and explain the generally highly favorable process outcomes observed in the original RCT, as well as generate hypotheses about particularly effective ingredients of TA from the patient's perspective. These hypotheses in turn, could then be put to experimental tests in future studies (e.g., using a dismantling design), and, more generally, serve to tighten the conceptual underpinnings of TA.

Method

TA patients

Participants were drawn from the sample of a recent RCT in patients with PDs (N = 37) awaiting their treatment (De Saeger et al., 2014). This RCT ran from June 2010 to September 2012 at de Viersprong, a specialized clinic in the Netherlands for the assessment and evidence-based treatment of PDs.

The total sample consisted of 4 women and 6 men with a mean age of 47.3 years (SD = 11.0). The average time that elapsed between the end of the intervention and the qualitative study was 1.5 years (SD = 0.56). Following TA, 6 patients received outpatient treatment for 6 months, and 4 patients received inpatient clinical treatment. All but 1 of the participating patients had completed their subsequent treatments, and 1 patient was scheduled to terminate treatment the next week.

Of note, all participants received a primary clinical diagnosis of one or more PDs, which was confirmed in 5 out of 7 cases (71.4%) by the administration at intake of the Structural Clinical Interview for DSM Disorders, Axis II (SCID–II; First, Gibbon, Spitzer, Williams, & Benjamin, 1997, Weertman, Arntz, & Kerkhofs, 2000); 3 patients were not administered the SCID due to logistic problems at the institution. These latter patients did receive a clinical PD diagnosis; 2 patients were evaluated as meeting criteria for PD not otherwise specified (with borderline and narcissistic features, and with borderline, paranoid, and avoidant features, respectively), and 1 met criteria for avoidant PD. Of those who were administered the SCID–II, 3 patients met criteria for a Cluster B PD diagnosis, and two for a Cluster C diagnosis. The patients who did not fully satisfy the diagnostic threshold for PD met four PD criteria; 1 patient for obsessive–compulsive PD and the other for narcissistic PD.

This sample consisted of patients with an extensive treatment history; 9 out of 10 (90%) of the patients reported their problems as existing for more than 5 years; 9 had received psychotherapy before entering the RCT study (De Saeger et al., 2014), 4 at the inpatient level; and half of the sample was currently on medication. Six out of 10 patients rated the severity of their problems as high. In sum, this sample appeared highly similar in clinical characteristics to those included in the larger sample frame of the RCT.

The original RCT included a comparison between TA and a highly structured motivational pretreatment package (GFPTI). The TA intervention comprised four sessions, which together operationalized the full model of TA. More specifically, collaborative assessment question formulation was followed by administration of both self-report tests and performance-based tests, including the Minnesota Multiphasic Personality Inventory–2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) and Rorschach Inkblot Method (Exner, 2009). After the standardized test administration and coding phase, the experiential nonstandardized techniques were employed in the assessment intervention sessions (Finn, 2007). Finally, individualized interactive feedback was provided, as well as written feedback formatted as a personalized letter. For more detail on this study, we refer the reader to De Saeger et al. (2014).

Researchers

The research team consisted of five researchers with diversely pertinent backgrounds, including two graduate psychology students, a researcher and psychotherapist in training, a licensed psychotherapist and junior researcher, and a university professor. All but one were female, and all were White Europeans. Several researchers have emphasized the importance of researchers bracketing their expectations and owning their perspective prior to their qualitative inquiry (e.g., Creswell, 2012; Elliott et al., 1999; Morrow, 2005). Accordingly, each of the authors prepared a personal statement detailing his or her pertinent theoretical, methodological, and personal orientation to this research project. These statements are available on request.

Interview

Three researchers (J. H. Kamphuis, H. De Saeger, A. Bartak) developed the schedule for the semistructured in-depth patient interviews. As can be seen in the Appendix, the interview schedule solicited material according to three broadly introduced themes: (a) what patients deemed most memorable about TA (What stuck with you most about TA?), (b) novel experiences related to TA (Was there anything that surprised you about the TA sessions? Anything new?), (c) possible negative experiences related to TA (Were there any aspects of TA that you experienced as unpleasant?). Each of these open-ended entry questions could be followed up with (specifically suggested) helpful probes. As a preliminary “checking in,” the interviewer always posed a couple of general questions to reconnect, break the ice, and get a first spontaneous recall of the patient's experience of TA. Also, at several points during the interview, the interviewer checked in with participants to make sure they felt comfortable.

Procedures

Recruiting patients

As distant memories are presumably less reliable, we opted for a convenience sampling strategy that favored most recent cases. An additional consideration that guided this choice was that we did not want to intrude in the lives of patients who might have completed their treatments a long time ago. Data collection in qualitative studies usually continues until stability of content and themes occur (Miles & Huberman, 1994). Generally, it is hard to predict when this will emerge. We based the number of participants on studies of somewhat similar clinical themes and questions (see, e.g., Hill, 2012; Van der Heiden, Gebhardt, Willemsen, Nagelhout, & Dijkstra, 2013; Wittkampf et al., 2008). Based on these estimates, we invited the last 15 patients from the TA pretreatment intervention to take part in this qualitative study. These patients received an invitation letter, together with an information sheet and informed consent form. One week later they were contacted by phone (by H. De Saeger) who inquired about their participation, addressed any remaining questions, and when consent was given, arranged a convenient time to conduct the interview. Two patients refused participation and the remaining 3 patients did not respond to either the letter or voice mail messages. A final sample of 10 patients (67% response) completed the interview. Estimating stability is a complex issue, as one can never rule out that new content might emerge in a subsequent interview. Nevertheless, we felt reasonably confident about the representativeness of our sample, and concurred that the final two patient interviews had not yielded new content categories. Patients received no compensation for their participation, and participation was fully confidential.

Interviewing

The first author (H. De Saeger) conducted the interviews (N = 10) during July 2013. We reasoned that she would be in the best position to relate to the experiences of these patients, given (a) her clinical experience with this sometimes challenging group of patients (e.g., establishing rapport), and (b) familiarity with the interventions (both with respect to TA and subsequent treatment programs in the de Viersprong setting). Moreover, she is a licensed clinician with extensive experience in conducting unstructured and semistructured clinical interviews, but had never served as assessor or therapist for any of the patients in this sample. Two mock interviews (with the auditor, J. H. Kamphuis) were conducted and processed in detail, before we proceeded to administer the interview to the patients. The first two patient interviews were monitored by the (internal) auditor (J. H. Kamphuis), to make sure no leading questions were asked, and that the interviewer kept sufficient focus. No changes were made. Each interview was conducted and recorded from a secure connection using a Skype voice recorder and lasted approximately 30 min (range = 22–36 min). At the beginning of each interview, we reminded the patient of the procedure of audiotaping and confidentiality, and then started the recorder.

Training and transcribing

As part of their training, two graduate students studied two textbooks on qualitative data analysis (Kuckartz, 2014; Miles & Huberman, 1994) as well as selected qualitative research articles. They were also provided with several pertinent papers to familiarize themselves with TA, PD, and the original RCT (De Saeger et al., 2014). The training further consisted of discussing the literature in the primary research team, and deciding on a coding strategy. Subsequently, one afternoon was scheduled to familiarize the primary coders with the MAXQDA program, a qualitative data analysis software package (Verbi, Germany) we used for the transcription, coding, and analysis of the interviews. All identifying information was removed from the transcripts, and each received an anonymous code instead. The graduate students then transcribed the first two interviews into MAXQDA-11. Every transcript was cross-checked by the other transcriber.

Data analytic strategy

Consensus coding

The emphasis on the consensus process is one of the hallmarks of the CQR method. Weekly meetings of the primary coding team (H. De Saeger, A. Bartak, E.-E. Eder, T. Velthuis) were held to discuss coding into domains and categories. All members of the team were encouraged to share their thoughts and feelings regarding the optimal organization and labeling of content. These meetings, in large part by telephone or Skype, were dynamic and spirited, but took shape without major conflict. Critical views on coding decisions could be expressed in an atmosphere of mutual respect. Senior members did not claim expert status, but encouraged the student members to express their opinions freely. Returning to the original data frequently helped to resolve discrepancies and to verify results. Throughout this process, team members kept memos and produced a weekly report to document and constantly review the evolution of ideas and decisions.

Identification of and assignment to domains

A review of the relevant literature, our research questions, the interview guide, and data from the interviews of the first two cases provided the point of departure for identifying a “start list” of domains (Miles & Huberman, 1994), which was then continually modified throughout the consensus process to reflect the data more accurately. This “start list” was developed by J. H. Kamphuis, H. De Saeger, and A. Bartak, and consisted of (a) perception of self, (b) relationship aspects, (c) novel experience or key moment in TA, and (d) negative experiences related to TA. After several revisions, we finally identified the following domains: (a) relationship aspects, (b) new insight into personal dynamics, (c) sense of empowerment, (d) validation of self, (e) novel experiences (doubly coded; as such, and attributed with a content category), and finally (f) negative experiences related to TA. The primary team coded all relevant text segments and meaning units into one or more domains, first independently, and then by consensus, while keeping memos for guiding the next step of clustering ideas within each domain into categories.

Cross-analyses and development of categories

By comparing, contrasting, and clustering the responses within each domain across all cases, we arrived at coherent themes or categories of different levels of abstraction, which were later translated into our final coding system. We used a doubly layered consensus procedure: First, the two graduate students reached consensus in their cross-analysis, which served as input for the (second layer) discussion with the two senior members of the primary coding team to reach overall consensus. Throughout this process we kept revising the domains and categories (e.g., changed titles, collapsed or divided domains and categories, etc.) to ensure representativeness of the data. To illustrate this, the initial categories of profound listening and attention were collapsed into interest in details of personal history and context, which after discussion was relabeled to better convey its content to being heard from a personal perspective. All team members reviewed and discussed the results of the final cross-analysis to confirm consensual decisions until all were satisfied with the final product.

Internal audit

The auditor (J. H. Kamphuis) examined the degree to which the content matched the assigned domain, and provided feedback on the wording of the domains. The same process was followed in the cross-analysis phase. Feedback was then discussed with the primary coding team, and several revisions were made.

Feedback from external auditor

An external referent was invited to read and study four interview transcripts, and to appraise to what extent she judged the interview material and the domains and categories to match. Specifically, we invited a senior licensed psychiatrist from a Belgian psychiatric center providing care for patients with severe Axis I and Axis II conditions. She believed that the identified content themes provided an accurate characterization of the patient reports. In her judgment, one interview contained all themes, whereas the other interviews included only one to three clearly recognizable themes. Overall, she felt she was able to reconstruct how the findings related to the data and was impressed with how well patients were able to mentalize about the TA experience.

Results

Across the 10 interviews, 175 interview segments were coded. Of these, 145 (82.9%) applied to four main domains, which accordingly emerged as the main thematic content of the patients' responses to our queries: (a) relationship aspects, (b) new insight into personal dynamics, (c) sense of empowerment, and (d) validation of self. Twenty-two text segments related to novel experiences, which were doubly coded (both as general domain, and as answers to the separately probed for novel experiences). Following the CQR methodology (Hill et al., 2005), we assigned general, typical, and variant labels of our content. We considered domains to be general if they applied to all or all but one interview, typical if they applied to five to eight interviews, and variant if they applied to two to four interviews. Table 1 provides a summary of the frequency of the specific domains and categories. In what follows, we first describe these domains and categories as we defined them, and then illustrate each with one or more typical quotes from the interviews.

Table 1. Final domains and subcategories, organized into general, typical, and variant.

Domains	Categories	Type
Relationship aspects	Working together throughout the intervention	Variant
	Being heard from a personal perspective	General
	Explanation	Variant
	Being treated as an equal	Typical
	Perceived guidance	Variant
	Positive feeling about relationship with assessor	Typical
New insight into personal	Insight	Typical
dynamics	Deepening	Typical
	Focusing	Typical
	Organizing information	Variant
Sense of empowerment	Self-confirmation	Typical
	Motivation, stimulation	Variant
	Hope	Variant
	Getting started	Typical
Validation of self (VoS)	VoS: Letter	Typical
	VoS: In-session	Typical
Novel experiences		General
Negative experiences related to Therapeutic Assessment		Typical

Note. We defined categories to be general if they applied to all or all but one interview, typical if they applied to 5 to 8 interviews, and variant if they applied to 2 to 4 interviews.

Relationship aspects

Several aspects of the relationship with the TA assessor were reported by the patients as memorable. As can be seen from Table 1, overall, many segments were related to positive feelings about the therapeutic relationship (e.g., the therapist being described as warm, kind, pleasant; positive feelings about the relationship with assessor subcategory). Reports of collaboration, equality, and validation were evident, sometimes in notable contrast to previous experiences in their treatment history. The following excerpt serves to provide a global illustration of these issues:

Above all I was surprised by the luxury that opened up to me … the amount of attention that I was given then. And the fact that such an extensive and good report was written, [and] the number of tests that I did then. … It was actually heart-warming that this happened. I found that … [patient becomes emotional] … that was actually not acceptable, that so much attention was going out to me. (Male, 53)

Several typical subcategories emerged within this domain that deserve further exposition, including (a) being heard from a personal perspective, and (b) being treated as an equal.

Being heard from a personal perspective

Patients reported that the therapists also paid careful attention to aspects of their personal perspective, history, or context, which again had been rather lacking in previous therapeutic encounters.

And all the other [therapists] exclusively focused on or asked about the depressive symptoms, even though they could have gotten so much more out of all these questionnaires, like you do; you derived much more [from the questionnaires] than just the basics; and [as a patient], you notice this. (Female, 48)

They [the TA therapist] first pay attention to who you are, what your character is, what kind of person, and what kinds of things you have experienced, you know … At the previous [mental health] center, it was much more superficial. (Female, 43)

Being treated as an equal

In addition to emphasizing the collaborative nature, patients perceived the relationship as remarkably equal, with some noting this as different from previous experiences in mental health settings. Specific aspects that were mentioned include the use of “normal human language” (no jargon), and more akin to a dialogue than to unilateral information gathering. They described the intervention as a true “joined venture,” with both participants being essential to the outcome. The following vignettes might illustrate this best.

The [assessment] questions … that I was expected to phrase them myself, and that they needed my approval, word for word … To her [the TA therapist], it did not matter what the questions [turned out to be], as long as I was happy with them. (Male, 30)

The collaborative discussion of the test findings, and whether or not these fit according to me. And that when I did not recognize myself in a test finding, I did not get the feeling I was totally off or wrong. (Female, 37)

New insight into personal dynamics

Patients reflected on the fact that they had gained new, deeper, or more focused insights or new attitudes about personal dynamics. The following excerpts illustrate this.

The most important finding that I remember is that I do not experience distress from my behavior, but that [people in my] environment do. … I was so egoistic and egocentric that it did not cause me any trouble. (Male, 30, insight)

I learned there that there is a big difference in the way I act and the way I feel and how confusing that must be to others. (Male, 30, insight)

It was not … that emotion is wrong! You have to learn to deal with it. She [the TA therapist] took the time to listen to my emotions and helped me gain deeper insight into why I felt what I felt. (Female, 43, deepening)

Or, to better keep my focus. Especially because sometimes it makes a big difference whether you are working in therapy, and say, outside of therapy. Life goes on there. At the end I received this letter, about of all that we had done and achieved in the assessment. I could then return to this letter to read … to return to the focus. (Male, 52, focusing)

Sense of empowerment

Patients reported in various ways how they felt empowered by the TA experience, and became ready to get started with treatment. The following excerpts serve to illustrate the types of verbalizations we coded as such.

I became aware that I didn't have to isolate myself, that I was capable of taking personal responsibility, that I could face my problems. (Male, 30, self-confirmation)

I noticed later in [subsequent] therapy that I had more self-confidence compared to my group members, as if I was less busy searching for … [approval] (Male, 53, self-confirmation)

I experienced [in TA] … that I AM capable of sharing my personal opinion. (Male, 30, getting started)

Validation of self

TA patients frequently reported that it was important to them that the assessor provided them with good veridical, validating feedback in sessions and by means of test feedback or the narrative report (in TA usually formatted as a personal letter). The following excerpts serve to illustrate this aspect.

Yes, also validation, actually … recognition of what I felt … that I am not—not … a crybaby or overacting … that I, that I was actually allowed to feel what I felt. (Male, 52, validation of self: in session)

The therapist gave me a narrative that fit me completely. … Afterwards she even wrote it in a letter. I had no more questions about myself. I just had to look at the letter. (Female, 37, validation of self: letter)

The tests indicated things about both certain aspects of vulnerability, and also certain aspects of personal strengths. I actually rather liked it that this more complete picture became available to my therapist. (Female, 37, validation of self: letter)

Novel experiences in TA

Novel experiences were explicitly solicited according to the interview schedule (see Appendix). Moreover, spontaneously offered novel content was also coded as such, provided the patient specifically indicated these as novel; for example, “for the first time I ….” As previously noted, several patients reported aspects of the relationship with the TA therapist as new or unfamiliar. They particularly emphasized the equality in the relationship with the assessor as a novel experience. The true collaboration and the sustained empathy were also reported as surprising. The following excerpt illustrates this.

I was asked more about myself, about my personal experiences and … yes, how I, myself, really perceived things … in contrast to what I have experienced [in treatment] before sometimes … that when you give a sketch of your biography you become immediately labeled in one way or another. (Female, 37)

Negative experiences related to TA

Negative experiences were few, and mostly not related to TA proper, but concerned the frustration of being referred back to the waiting list (as opposed to subsequent treatment) after completing TA. Patients were ready to start treatment, and felt “opened up,” but unfortunately treatment groups were not always immediately open to new members. Also, one patient wondered “why all these tests [were needed].”

Discussion

The main objective of this study was to elucidate what patients deem memorable about TA, what it is that is novel about it to them, and what they experience as negative. As became evident from the interviews, our sample of PD patients frequently reported gaining important new insights into personal dynamics, as well as a sense of validation and empowerment. Moreover, they frequently reflected on relational aspects of TA, particularly that of being heard from a personal perspective, and being treated as an equal and essential partner in a highly individualized undertaking. These relational aspects were also the most frequently mentioned new experiences. Negative experiences related to TA were limited to logistical constraints of the original RCT: People reported being ready to take on therapy, but were frequently referred back to the waiting list.

Patients with personality pathology frequently exhibit strong ambivalence about change and treatment, are often attributed with limited capacity for introspection, and tend to hold extreme, as well as rigid, cognitions about self and others that burden their interpersonal relationships (Emmelkamp & Kamphuis, 2007). In this respect, it is especially noteworthy that our sample commented on new self-awareness, gained self-confidence, and the relationship with the assessor. Moreover, although this sample generally had extensive experiences with a diversity of mental health professionals (more than 80% had more than 5 years of therapy before coming to de Viersprong), TA still offered novel experiences, and particularly a novel relationship model to them. In line with literature on psychotherapy (Horvath, 2000; Lambert, Hansen, & Finch, 2001), the nature of the relationship with the therapist or assessor seems to be highly salient to the patients. Overwhelmingly, their reports paint the picture of an emotionally intense, collaborative relationship in which they felt attended to and taken seriously. Presumably, if these patients, often characterized by significant attachment problems, can be helped to develop a positive alliance in such a short time (i.e., four sessions), they might start their subsequent therapy with more optimistic outcome expectations. To be sure, the extent to which these factors are predictive of positive outcomes should be put to the test in, for example, the type of (replicated) single-case experimental designs put forward by Smith and others (e.g., Smith, 2012; Smith, Handler, & Nash, 2010), preferably with planned long-term follow-up moments in subsequent treatment.

Several points of convergence are notable between this study and the findings of the (to our knowledge) only other in-depth qualitative study on patient (and assessor) experiences (Ward, 2008). Although departing from a traditional IGA (Ward, 2008), patients in both studies regarded the specificity and personalized nature of analysis as significant, as well as the associated empowerment it generated. Another similarity was the sense of feeling personally and empathically understood and the appreciation for the sense of collaboration. Tentatively, it seems that the IGA assessees attributed this more to special qualities of the assessor, whereas TA assessees perceived it more as a relational phenomenon. Interestingly, in contrast to regular assessment, none of the TA participants commented on the processing of discordant or unwanted results. Because this theme did not show up in this analysis, processing of discordant results might be less of an issue for patients in TA. However, this finding might also be explained by differences in the domains of assessment (e.g., neuropsychological performance and scholastic achievement vs. personality and functioning).

Our findings are also reminiscent of the therapeutic mechanisms of self-verification/self-enhancement, and self-discovery/self-efficacy proposed by Finn and Tonsager (1997). The domain validation of self appears closely related to the mechanism of self-verification. Also, patients certainly reported experiences of emerging mastery (empowerment), which echoes the self-efficacy mechanisms proposed by Finn and Tonsager. Such reports are consistent with the pioneering research by Adler (2012), who observed increased agency preceding improvements in health in patients' narrative identity development across the course of psychotherapy. As such, they hearken back to much earlier theorizing by social cognitive researchers in psychopathology in general (most notably Bandura's self-efficacy research; Bandura, 1988). Likewise, the experiential theme of new insight into personal dynamics appears related to the mechanism of self-discovery. Finally, consistent with the proposed mechanism of self-enhancement, patients frequently reported on experiencing true collaboration on personal change in the domains they deemed important, which for many was considered novel to their treatment experience.

At a more general level, these findings underscore the complexity of the issue of what constitutes clinically significant treatment outcomes (Kazdin, 1999, 2001). Most extant outcome research defines treatment outcome rather narrowly in terms of acute symptoms and diagnostic status. In the original RCT, no significant symptom reduction was observed (De Saeger et al., 2014). However, this report strongly suggests that patients experienced TA as quite memorable, and consistent with the outcomes observed in the larger RCT, as positively affecting their treatment readiness and motivation. Such an analysis corresponds with the transtheoretical model of stages of (psychotherapeutic) change (Prochaska et al., 1994). According to this theory, patients can be in one of several stages of psychotherapeutic change: precontemplation, contemplation, preparation, action, and maintenance. Particularly relevant to TA in patients with PD might be the distinction between the precontemplation and the contemplation stages. As discussed previously in Finn and Kamphuis (2006), in the precontemplation stage, patients believe it is the environment or other people who need to do the changing, and they remain committed to their usual ways. In the contemplation stage, patients are aware of personal problems and are interested in whether these problems can be resolved and whether psychotherapy might be helpful. Of course, these conjectures are in need of empirical testing. A final recommendation for future research stems from a comment from our external reviewer on the integrity of our coding system. She suggested that it might be of interest to conduct another qualitative study into what TA assessors deem memorable or perhaps even transformative, much like the study by Ward (2008).

Several limitations of this study deserve comment. First and foremost, the time elapsed since the TA experience was considerable. We conducted this study after all patients (except one, who was terminating treatment that particular week) had completed their subsequent treatment for PD. As a result, on average, we asked patients about their TA experiences about 1.5 years after completion. The question can be raised of how patients' recollections might have been influenced by the passage of time and their subsequent therapy experience. Trying to mitigate these concerns, the interviewer repeatedly reminded patients to report on TA; whenever there was a doubt if the content of the answer applied to TA or rather to subsequent treatment experiences, the content was not coded. On the other hand, the argument can be made that if experiences stay on patients' minds even after such a long period of time, the intervention was indeed meaningful to them. All interviewees reported vividly about their TA experiences, a point that was also noted by the independent external referent. For example, the majority of patients spontaneously offered the name of their assessor, provided detailed in-session examples, and sometimes mentioned where they kept the letters they had received at the end of the TA intervention.

A second limitation is that these findings pertain to patients who were awaiting specialized psychotherapeutic treatment for PDs. Although this sample was drawn from a PD patient population, we hold that key clientele of TA frequently involves patients with the type of diffuse, intractable personal problems that involve complex personality dynamics. Whether different patient groups have similar experiences is ultimately an empirical question, worthy of further research.

Finally, although conducted with great care, it should be acknowledged that our methodological approach came close but did not fully adhere to the CQR guidelines. Future researchers are advised to more rigorously pursue specific research designs that include formal procedures fostering trustworthiness of the data representation (e.g., systematic sampling and member checking procedures). The very clear guidelines provided by Hill and colleagues (Hill, 2012; Hill et al., 2005) could serve such qualitative researchers well, or one might opt for systematic case study or grounded theory approaches.

In conclusion, we strongly concur with Elliott et al. (1999): “The aim of qualitative research is to understand and represent the experiences and actions of people as they encounter, engage and live through situations” (p. 216). We have been struck by how willing patients were to share their experiences with TA, and were highly gratified by what their perspective can yield beyond extant perceptions and ideas of practitioners, researchers, and theoreticians.

Acknowledgments

We would like to thank Tessa Velthuis for support with scoring the interviews, and Ann Berens for checking the integrity of our interview coding strategy.

1. General review of TA (approx. 10 minutes)

   Start questions: Can you tell me what has most stuck with you from the TA sessions? When you look back to the TA sessions, what was most important to you? What did TA bring you?

   • When participant mentions benefit, follow up with … in which way do you think the TA has been helpful?

   • When participant mentions several aspects, … what to you was the most important aspect (and follow through on this aspect first)

2. Novel experiences (approx. 5 minutes)

   Start question: Was there anything that surprised you about the TA sessions? Was it at all-in some aspects different from what you have experienced before in previous psychotherapy or mental health-related sessions?

   • If yes, how was that for you?

   • Helpful probes: Key moments (approx. 5 minutes). Probe question: Can you tell me anything about important moments during the TA sessions?

3. Negative experiences related to TA (approx. 5 minutes)

   Start question: Were there any aspects of TA that you experienced as unpleasant? If yes, can you say more?

Closing

Finally, is there anything else that we did not cover during the interview that you would like to share about TA? Then stop the recorder, and thank the respondent and take time to let him/her “blow off steam” (i.e., inquire how it was to do this interview; How do you think this went? How do you feel/ think about it?).