Causality in psychotherapy research: Towards evidential pluralism

ABSTRACT

Identifying causal relationships is at the heart of all scientific inquiry, and a means to evidence base practices and to guide policymaking. However, being aware of the complexities of interactions and relationships, scientists and academics are cautious about claiming causality. Researchers applying methods that deviate from the experimental design generally abstain from causal claims, reserving them for designs that adhere to the evidential ideals of empiricism (e.g., RCTs), motivated by the Humean conceptions of causality. Accordingly, results from other designs are ascribed lower explanatory power and scientific status. We discuss the relevance of also other perspectives of causality, such as dispositionalism and the power perspectives of various realist approaches, which emphasize intrinsic properties and contextual variations, as well as an inferentialist/epistemic approach that advocates causal explanations in terms of inferences and linguistic interaction. The discussion will be illustrated by the current situation within psychotherapy research and the APA Policy Statement on Evidence-Based Practice. The distinction between difference-making and causal production will be proposed as a possible means to evaluate the relevance of designs. We conclude that clarifying causal relationships is an ongoing process that requires the use of various designs and methods and advocate a stance of evidential pluralism.

Keywords:

• causality
• powers/dispositionalism
• inferentialism/epistemic causation
• evidence-based practice
• generalizability

Clinical or methodological significance of this article: We illustrate how various conceptualizations of causality allow the inclusion of a variety of research approaches needed to identify robust causal relationships in psychotherapy. This in turn serves to deepen our understanding of mechanisms of change and outcome, such as the effects of specific techniques, common factors, clinical theories, and the relationship between them.

The limitations of experimental designs such as Randomized Controlled Trials (RCTs) have been increasingly highlighted within the social sciences during the past few decades (e.g., Anjum et al., 2020; Carey & Stiles, 2016; Clarke et al., 2014; Deaton & Cartwright, 2018; Gillies, 2019; Krause & Lutz, 2009; Maxwell, 2004, 2021; Russo & Williamson, 2007; Wampold & Imel, 2015). Yet, the very same designs seem to persistently hold their status as ideal for identifying causal relationships, as reflected in for example the American Psychological Association (APA) Policy Statement on Evidence-Based Practice (APA, 2005, 2006), which states that “RCTs and their logical equivalents (efficacy research) are the standard for drawing causal inferences about the effects of interventions” (APA, 2006, p. 274). With reference to recent debates, we will discuss the status of causality claims in psychotherapy research, including the limitations of research designs conforming to the empiricist Humean regularity conception of causality. We will present alternative conceptions of causality, such as that of causal powers/dispositionalism, critical realism, and inferentialism/epistemic causality, and the methodological stance of evidential pluralism as a possible foundation for sound knowledge development within psychotherapy research.

Hume and Empiricism—Causation in Terms of Regularity, Counterfactuals, and Manipulability

Causality has been discussed in philosophy at least since Aristotle, who saw cause (aitios) as the driver of change in nature. As his views are traditionally understood, he saw causal principles as being both intrinsic and extrinsic, implying that no change would happen if either was missing. That is, all things have some intrinsic potential for change and a purpose (or telos) of becoming, which requires the coming together of internal and external types of causes, only one of which is external (Broadie, 2009). To Hume and later empiricists, no such drivers or purposiveness are constitutive of causation. Rather, Hume’s influential model of causality relies on observable regularities in the relations between discrete events. According to Hume, we cannot have any evidence or knowledge about potential causal mechanisms beyond observed regularities together with two other observable features: temporality (the cause must happen before the effect) and contiguity (the cause and effect must meet in time and space). A cause may thus be defined as

an object followed by another, and where all the objects, similar to the first, are followed by objects similar to the second. Or, in other words, where, if the first object had not been, the second never had existed. (Hume acc. to Lewis, 1973, p. 556)

Hume (1739/1978) recognized our tendency to form ideas of a necessary connection between these objects—a “causal sense”—but argued that this is not an observable feature and can therefore not be part of an empirically grounded concept of causality. The origin of our knowledge of necessary connections arises out of observed regularity or constant conjunction of certain events across many instances. All scientists can do is observe the repeated regularity of cause and effect and impose meaning. Hume’s deep skepticism of causes spawned three theories/ideas of causal identification: regularity (e.g., Psillos, 2009), counterfactual reasoning (e.g., Lewis, 1973), and manipulability (e.g., Woodward, 2003), which all three constitute the pillars of modern experimental designs. Regularity refers to the repetition of regular patterns, which may not, it has been argued, account for epiphenomena and spurious relationships, or asymmetry (i.e., smoking may cause cancer without cancer causing one to smoke). In his seminal paper on counterfactual reasoning, Lewis (1973) set out to get around the limitations of the regularity conception by picking up on another aspect of Hume’s definition, stating that:

We think of a cause as something that makes a difference, and the difference it makes must be a difference from what would have happened without it. Had it been absent, its effects – some of them at least, and usually all – would have been absent as well. (Lewis, 1973, p. 557)

Counterfactual reasoning thus emphasizes the unidirectional dependency of an effect (E) upon a cause (C). With manipulation-based accounts of causality, this dependency is demonstrated by intervening on the target variable, implying that C is identified as a cause if and only if E is changed by manipulating C (Woodward, 2003, 2009). These three theories have in each their ways nourished crucial sophistications of the experimental designs (e.g., Illari & Russo, 2014; Shadish et al., 2002).¹ Irrespective of their distinctions, they share some characteristics that reflect the scientific landscape of academia during the Enlightenment from the seventeenth century, often referred to in terms of positivism, which seems to have turned into a metaphor of the zeitgeist as a whole. According to Hacking (1983), “In the major West European languages ‘positive’ had overtones of reality, utility, certainty, precision … ” (p. 42), which we may say capture the essence of modernity and modern science, and the subsequent ideals of scientific method. While signaling optimism and forward movement that spawned myriads of methods and empirical research, the same ideals carried with them a mistrust of other aspects of knowledge production, such as untestable propositions, unobservable entities, causes, and deep explanation (Hacking, 1983). Correspondingly, claims of causality were linked to demonstrations of regularities, time precedence (the cause precedes the effect in time), and control over confounding variables, ideally free of context, values, or theoretical assumptions of causal mechanisms and purposiveness. While these characteristics reflect the heritage of empiricism and positivism, counterfactual reasoning and manipulability have also been related to hypothesizing, agency, and rationality (e.g., Woodward, 2009) that signal a departure from strict empiricism over to realism (e.g., Hacking, 1983) and critical rationalism (e.g., Popper, 1959). This reflects the sometimes blurred distinctions between the philosophical underpinnings of causal theories and, in turn, research methods and designs, a point we will get back to.

Humeanism has been highly influential—if not constitutive—of modern science, as mirrored in several acknowledged criteria for the evaluation of scientific quality, of which those of internal validity are deemed vital to ensure the control needed to identify causal relationships. Within psychotherapy research, along with other disciplines within the social sciences, experiments, such as RCTs, have come to represent the gold standard of rigorous knowledge production aimed at establishing causality.

The Evidence Crisis in Psychotherapy Research

Within psychotherapy research, we have witnessed a debate on the concept of evidence that flared up during the 1990s and early 2000 (e.g., Bohart, 2000; Elliott, 1998; Lampropoulos, 2000). Besides a growing disaffection for the RCT’s lack of ecological validity and the science-practice gap (cf. APA, 2006), a crucial driving force of the debate was the recurring failure to demonstrate comparative effects of various specific models used for specific diagnoses or effects of certain theory specific techniques. Rather than documentation of specificity, numerous studies, meta-analyses, and reviews identified factors that cut across the specific treatment models. Factors such as the working alliance between the therapist and the client, motivation, expectations, and various other aspects of the treatment processes consistently explained more of the outcome variance than the theory-specific models or techniques themselves (e.g., Lambert, 2013; Laska et al., 2014; Wampold & Imel, 2015; Wampold & Owen, 2021). The expectations of demonstrable, specific effects of methods or techniques, cautiously tested by clinical experiments, overall were not met. For therapy to work, it is not indifferent who the therapist is, nor the client’s particular circumstances, expectations, and wishes. Therapists’ way of tailoring the techniques to the particular client significantly impacts therapeutic processes and outcome, allowing change factors such as client motivation, engagement, and involvement to work (e.g., Bohart & Wade, 2013; Stiles, 2009b). Wampold (2001) replaced the “medical model” with the “contextual model” to be able to explain the somewhat counter-intuitive findings from the field. The main point is that contextual factors, such as the therapists and the way they apply the techniques, and the clients’ experiences, responses, and contributions (such as conceptualized and measured in terms of the working alliance) seemingly influence the therapeutic effect more than the theories and the techniques themselves.

The contextual model was only one (albeit strong and influential) manifestation of the voices against the gold standard of RCTs within psychotherapy research and the field of mental health (see also Carey & Stiles, 2016; Kazdin, 1998; Krause & Lutz, 2009; Norcross et al., 2013; Stiles, 2009b; Talley et al., 1994). Practical, ethical, as well as methodological considerations were claimed to restrict the utility and status of RCTs as providers of relevant knowledge about change. Objections typically centered on the separation of factors into “dependent” and “independent” variables. For example, in line with the arguments in favor of the contextual model, Krause and Lutz (2009) pointed out that one of the central premises of the RCTs, i.e., mutual causal independence of the independent variables, is a form of mathematical ideal that does not reflect the realities of psychotherapy. Factors continuously and inevitably influence each other; the effectiveness of a particular treatment will be influenced by the actual performance of the given therapist, who in turn will be influenced by the given client’s responsiveness, who in turn is influenced by the relational context of that therapist, and so forth (see also, Stiles, 2009b). Overall, a conclusion that may be drawn from the concerns raised by central scholars in the psychotherapy field during the past decades, is that, while the RCTs and their equivalents are useful for identifying relationships where these conditions make sense, they may not be optimal to explain the complex and total picture of processes and change in psychotherapy.

The debate on the status of RCTs and experimental designs culminated following the APA Division 12 Task Force on “Promotion and Dissemination of Psychological Procedures’. The Task Force identified 18 “empirically supported treatments’ for particular disorders (Chambless et al., 1996, 1998), which on the one hand instigated enthusiasm about the robust support of the overall effect of psychotherapy, while on the other hand sparked massive critique for its unilateral trust in RCTs as the proper source of evidence (APA, 2006). A new Task Force resulted in the APA’s (2005) Policy Statement on Evidence-Based Practice in Psychology, which reflected a marked turn in favor of multiple research designs, and emphasis on clinical utility. The weight had shifted from a sole emphasis on evidence-based methods to evidence-based practice, leading to the definition: “Evidence-based practice in psychology (EBPP) is the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences” (APA, 2005). Besides emphasizing psychotherapy as a contextually based activity, the report and policy statement explicitly highlighted the value of basing evidence on multiple research designs, applied for each their purposes, and not only RCTs. These included clinical observation, qualitative research, systematic case studies, single-case experimental designs, public health and ethnographic research, process–outcome studies, studies of interventions as these are delivered in naturalistic settings (effectiveness research), RCTs and their logical equivalents (efficacy research), and meta-analyses (APA, 2006, p. 274; Levant, 2005, pp. 7–8).

However, and of interest for our present purpose, causality was still exclusively associated with experiments, as already noted: “RCTs and their logical equivalents (efficacy research) are the standard for drawing causal inferences about the effects of interventions (context of scientific verification)” (APA, 2006, p. 274). Correspondingly, types of evidence were ranked according to their “contribution to conclusions about efficacy,” with “sophisticated empirical methodologies, including quasi-experiments and randomized controlled experiments or their logical equivalents” on top, followed by the clarification that among sophisticated empirical methodologies “randomized controlled experiments represent a more stringent way to evaluate treatment efficacy because they are the most effective way to rule out threats to internal validity in a single experiment” (APA, 2006, p. 275). Thus RCTs—and their equivalents—were still identified as most fit to establish causality, and the argument was based on the aim of ensuring internal validity, which equals causality within the experimental paradigm (Shadish et al., 2002). Moreover, the previous distinction between efficacy (fit for drawing causal inferences) on the one hand, and clinical utility/generalizability on the other, was maintained:

With respect to evaluating research on specific interventions, current APA policy identifies two widely accepted dimensions […] The first dimension is treatment efficacy, the systematic and scientific evaluation of whether a treatment works. The second dimension is clinical utility, the applicability, feasibility, and usefulness of the intervention in the local or specific setting where it is to be offered. This dimension also includes determination of the generalizability of an intervention whose efficacy has been established. (APA, 2006, pp. 274–275)

This distinction between the two dimensions of efficacy and utility/generalizability, implying that they are operating separately, is noteworthy, although common and reasonable from an experimental perspective. The distinction may be read as a strong recognition that ignoring a research design’s potential threat to ecological validity may perpetuate an unfortunate science-practice gap. Nevertheless, separating a particular design and utility/generalizability into two independent dimensions maintains an understanding of causality as solely associated to internal validity in line with the premises of experimental designs. Findings by other designs, however useful, are deemed inferior to explaining the effects of interventions, because the standard criteria for drawing causal inferences about the effect of interventions, and thereby the opportunities for predictions and control, are not met. As we will get back to, other conceptualizations of causality not restricting causality to internal validity as understood within the experimental tradition pave the way for the identification of causal processes that may be of higher utility.

Recurring Concerns—Improvements and Developments Within the Experimental Tradition—Moving on from Hume’s Regularity Conception

The arguments against the gold standard of RCTs are reflected in numerous publications also of recent date (e.g., Carey et al., 2017; Carey & Stiles, 2016; Laska et al., 2014; Philips & Falkenström, 2021). In their meta-analytic evaluation of recent meta-analyses of the efficacy of psychotherapy and pharmacotherapy for mental disorders in adults, Leichsenring et al. (2022) concluded that, after decades of research and thousands of RCTs demonstrating limited effect sizes, “[a] paradigm shift in research seems to be required to achieve further progress” (p. 141). Recurring concerns have been raised as to the RCT’s role as a guarantor of scientific trust within disciplines where the phenomena in question, by definition, include context in all its variants, spanning from mental states to external social structures. The claim of causality has been the subject of debate within as well as without the experimental research literature, and efforts are made to ensure the sophistication of designs aimed at identifying causal relationships. Experimental designs and mediation analyses have been put under scrutiny, and specific methodological steps are outlined to ensure proper causal inferences where requirements of RCTs cannot be met (such as randomization, and with immutable, unmanipulable factors; e.g., Antonakis et al., 2010; Bullock et al., 2010). Scholars have explicitly warned against avoiding claims of causality (e.g. by resorting to claims of correlations) that may camouflage implicit causal claims (Grosz et al., 2020) and hinder sound improvement of designs and methods (e.g., Antonakis et al., 2010; Bullock et al., 2010; Hèrnan, 2018; Höfler et al., 2021). Moreover, concerns have been raised about the unreflective checklist use of “causal criteria” (Höfler, 2005). Regarding psychotherapy research more specifically, see also, Crits-Christoph and Connolly Gibbons (2021) and Baldwin and Goldberg (2021) for reviews of recent sophistication of statistical methods to ensure causal inference in process-outcome research, and Kazdin (2022) for a recent detailed account of the potentials of RCTs, including an evaluation of what random assignment can and cannot do with respect to drawing causal inferences about effects of therapeutic interventions.

Concerns about the utility of RCTs and their equivalents have thus been raised both from within and without the empiricist tradition, nurturing solutions from both within and without, accordingly. Seemingly, however, the gold standard status of RCTs still thrives, which might be explained by its strong association with internal validity, which in turn equals causal identification within the experimental paradigm (e.g., Shadish et al., 2002). We will now turn to perspectives of realism where this premise for causal identification is challenged.

Causality Beyond Hume

Realist Causal Power Perspectives

The Humean skepticism to identifying underlying causes is challenged by various realist perspectives. Scientific realism is a philosophical perspective that is characterized by the premise that “the entities, states, and processes described by correct theories do exist” (Hacking, 1983, p. 21). As to causality, a realist perspective typically contrasts with an empiricist anti-cause/skeptic stance by its recognition of causal powers or intrinsic capacities/properties of entities. Picking up on Aristotle and the ancient tradition a relatively recent approach within analytic philosophy holds intrinsic properties—that is, causal powers or dispositions—as vital causal drivers (Bird et al., 2012; Cartwright, 1989; Groff & Greco, 2013; Jacobs, 2017; Marmodoro, 2010; Meincke, 2020). Allegedly, these dispositional properties can belong to a particular thing, individual, process, communities or social systems, thus spanning from the very concrete to higher-level, abstract entities. By recognizing an unobservable level of reality these theories shift the attention away from identifying observable law-like regularities to understanding the underlying dispositions and causal capacities that produce such regularities. A focal assertion is that powers or dispositions are dependent on actualizing factors, which within the various powers/dispositional perspectives are referred to as “mutual manifestation partners,” “support factors,” or “interactive variables” (e.g., Anjum & Mumford, 2018a, 2018b; Deaton & Cartwright, 2018; Mumford & Anjum, 2011). These may not be understood as mere conditions for the real cause to work. Rather, they are themselves part of the causal process, together with all the other factors that account for the outcome. Causality thus exists in a unique web of interacting dispositions working as manifestation partners towards the emerging results. Unlike with the regularity model, there is no temporal priority involved, rather a dynamic of symmetric, reciprocal relations between properties simultaneously interacting with each other. Hence, causation neither refers to pure observable contingencies nor some separate, underlying driver, but applies to the whole process of interacting dispositions or powers that altogether are responsible for some particular effect. This denotes a holistic take on causality where powers are a characteristic of a system rather than a particular object (Illari & Russo, 2014). Accordingly, causal powers/dispositionalism advocate process-oriented, complex understandings of the mechanisms of causality (e.g., Anjum & Mumford, 2018a, 2018b). By stating the irreducible existence of dispositional properties these perspectives call for a clarification of the ontology of causality. Thus these are ontological perspectives devoted to the fundamental question of what causation is rather than how to go about studying it. Critical realism, which we now turn to, bears on this ontology while at the same time including an epistemological aspect of relativism.

Critical Realism—Causality at the Level of Generative Mechanisms, Manifested in Open Systems. Critical realism is a philosophical perspective that was developed during the last part of the twentieth century, and which aimed to conceptualize “the independent reality of being […] in the face of the relativity of our knowledge” (Bhaskar, 1998, p. x), thus reconciling ontological realism, epistemological relativism, and judgmental reality. To Bhaskar (1998), “society is both the condition and outcome of human agency and human agency both reproduces and transforms society” (p. xvi). Claiming to bridge the classic dichotomy of positivism and hermeneutics, Bhaskar asserted that:

positivists are wrong to expect the social sciences to find constant conjunctions in the human world, for they are scarce enough in the natural; while hermeneutics are wrong to conclude from the absence of such conjunctions that the human sciences are radically unlike natural sciences. (Bhaskar, 1998, p. xv)

As opposed to what he referred to as “the epistemic fallacy” (i.e., “what is”, ontology, is reduced to “what we know”, epistemology), Bhaskar argued for the intransitive dimension of knowledge, which are the mechanisms/objects of knowledge that exist and operate before and independently of their discovery, beyond language and experience. These mechanisms may be actualized due to causal powers and abilities that are intrinsic to things. The “things” may be events, structures, intentions, reasons, and so on, that in virtue of their intrinsic structures possess powers and abilities that constitute “the ontological ties that bind the cause and effects together” (Harré & Madden, as cited by Bhaskar & Lawson, 1998, p. 8). Put simply, a piece of iron has got its intrinsic structure determining its potentials/abilities to influence the world, while a person’s intentions have other potentials. Powers and abilities are intrinsic tendencies that come into play as mechanisms if triggered, and the effects of these mechanisms are actualized (or not) depending on the play of countervailing mechanisms. Hence, a dynamic of intrinsic structures, powers, abilities/capacities, and mechanisms serves to generate or facilitate the phenomena that we experience, or do not experience, but which nevertheless is the causal effect (Bhaskar & Lawson, 1998).

While these generative mechanisms reflect the ontology of critical realism, the epistemological aspect of causality is reflected in the concept of “retroduction” (often also referred to as “inference to the best explanation” or “abductive reasoning”), which is the process by which we draw causal inferences or construct a theory of mechanism, such as “if it [the mechanism] existed and acted in the postulated manner, [it]could account for the phenomenon singled out for explanation” (Lawson, 1998, p. 156), which is then tested empirically. As this is a process drawing on researchers’ perspectives, various theories, and scientific approaches likely explain phenomena differently, although they, according to the ontology of critical realism, reflect the same intransitive objects (e.g., Richards, 2018). This will typically be the case within social sciences. We will get back to how this epistemological aspect of causal theory paves the way for conceptual, theoretical, and communicative generalizability, that is, other forms of generalizability than those based on definite concepts, random sampling, and the premise of universalism that mark the empiricist tradition.

Moreover, in “open systems,” such as the social world (as opposed to the “closed systems” of experimental conditions), the generative mechanisms are typically unsynchronized or out of phase with the actual effects, leaving an absence of one-to-one relationship between the (proposed) causal law and the particular manifestation of its effects (Bhaskar & Lawson, 1998). We shall shortly see how this asynchrony of causal mechanisms and effects may shed light on, or serve as a proper explanatory framework of, the relationship between processes and change in psychotherapy, including delayed and unconscious effects. In sum, from a critical realist perspective, causal mechanisms operate in a complex dynamic independently of our identification of them, often unsynchronized with our observations of their effects (if observed), and likely conceptualized by differing explanatory theories.

To sum up on the main points shared by critical realism and dispositionalism: Causal mechanisms may be explicit and identified—or implicit or not-yet articulated, but still crucial causal drivers. Causes (powers, dispositions) are tendencies that constitutively relate to manifestation partners, which implies a complex web of factors interacting with each other and that all form part of the causal picture. Moreover, causal or generative mechanisms may be unsynchronized with their manifest causal effects. These are premises that may serve to nuance explanations of the effects of psychotherapy.

For example, as clinicians we may recognize delayed effects and implicit, unconscious processes. Therapy may have effects that are not fully experienced or articulated until a while after the treatment ended and the client has been wandering about in the world, potentially affected this world in a slightly different way, in turn actualizing different responses, and gradually become more aware of the difference. After having acted and reflected upon these changes new insights may be integrated and the client may finally be able to articulate what has happened, at least parts of it. It would not be unreasonable to attribute some of this effect to what happened during therapy (e.g., the emotional climate, interventions used, responsively tailored to this particular client, etc.), although it might be difficult for the client to identify these factors and their interactions if asked during or right after therapy. The client may resort to formulations that are easy to explicate albeit not capturing the total picture or experience. Or they simply cannot find a proper answer although they have a clear (or vague) feeling that “this was good for me,” “something else,” “my therapist is good,” “totally needed.” Likewise, either as therapists or supervisors, we may experience that clients or former students come to us, revealing that “now, at last, I fully understand what we were talking about.” Change, whether in therapy or training, frequently happens unsynchronized with the actual interventions, and after interaction with a host of other drivers such as time, new actions, relations with others, own thinking, etc. Neither of these are the “real” causal drivers. Their effects are all dependent on the mutual facilitation of the web of other factors. Recognition of manifestation partners as part of causation itself is vital to a dispositionalist/powers conceptualization of causality and directs attention to the whole process of various constitutive factors, rather than establishing causality at the level of observable contingencies, where one factor has the temporal priority over the other.

Implications of a Powers/Dispositionalist View on Causality. An essential take-home message from powers/dispositionalist perspectives is that the identification of unique causal mechanisms, such as underlying structures and processes, is key to ensuring causal knowledge of a phenomenon. In this view, RCTs can indicate causal processes without that implying that they capture the essence of causation, as asserted by Kerry et al. (2012): “RCTs may be very good at displaying symptoms of causation, but they are not constitutive of causation.“ (p. 8; for a more complete reasoning of the philosophical rationale, see, e.g., Rocca & Anjum, 2020; Kerry et al., 2012; Lindstad, 2020). From a scientific perspective, we recognize a rather radical stance that reflects a focus on the ontology of causation, dealing with the fundamental structures of powers and dispositions. It is important to note, however, that albeit being radical as to causal ontology, proponents of these perspectives do not dismiss RCTs and experimental designs as forming vital part of the repertoire of designs needed to identify causal relationships (e.g., Deaton & Cartwright, 2018; Rocca & Anjum, 2020), a point we will get back to under the heading of difference-making vs. causal production. Nevertheless, the weight on the unique webs of powers and their manifestation partners/support factors—and an emphasis on causal tendencies as opposed to discrete events—favors in-depth studies of how these manifest in single instance cases, as “[t]his is where the real nature of causation is witnessed. The interaction between causal agents; subtractive and additive forces tending towards and away from an effect; causal powers being passed from one partner to another” (Kerry et al., 2012, p. 10).

Relatedly, concerns have been raised that the restrictive demands of experimental designs, aimed to ensure internal validity, hamper the identification of vital causal processes as these naturally unfold, thus limiting the usefulness of the results. For example, as argued by Cartwright (2014), findings by RCTs may be poor at predicting results in new settings, as you will not be sure that “enough individuals in the target population have the right arrangement of support factors“ (p., 318, our cursive). Likewise, Deaton and Cartwright (2018), in their critique of what they observed as an undue trust in RCTs in social sciences and medicine, raised concerns that the findings have limited predictive value as long as they are separated from support factors, which are responsible for the differing manifestations of effects across situations (see also Cartwright, 2010; Cartwright & Hardie, 2012; Rothwell, 2005, 2006). Here we recognize the priority of the unique, local before the general, universal that takes precedence within empiricism. The empiricist ideal of randomization and discrete entities (to control for confounding and to ensure representativeness) reflects a premise of universalism and determinism, most clearly illustrated by Hempel’s “covering law” model and the thesis of the “symmetry between explanation and prediction”: To explain a phenomenon is to demonstrate empirically that it is an instance of a general law, which in turn ensures prediction (Groff, 2011; Hempel, 1942).² Following the argumentation of powers/dispositional views, though, such predictions may be of limited value as, due to the experimental conditions themselves, the essential causal processes are not ensured. The procedures aimed at ensuring internal validity (and thus causality) may hamper the usefulness of results, that is, their external and ecological validity. The tension between internal and external validity is well known within experimental research (e.g., Shadish et al., 2002) and seems to be hard to solve within the experimental paradigm itself. We will get back to how theorizing as a means of generalization is recognized both within and without the experimental tradition.

Thus an overall message from the proponents of realist perspectives is that we cannot get to the core of causal explanations without studying phenomena as they appear in contexts that are constitutive of the phenomena themselves. In all, the powers/dispositionalist perspectives display a pervasively different view of the path through which knowledge on causal relationships develops than that of the empiricist tradition: In accordance with the Aristotelian view, this perspective holds that knowledge on the universal is developed through an inquiry into the particular, rather than the other way around.

Inferential and Epistemic Approaches to Causality—Causality at the Level of Human Mind and Linguistic Interaction

Facing the empiricist demands and the prominent status of the adjacent experimental designs, one stance to the question of causality within humanistic social sciences has been to lean on a Diltheyan distinction between explanations (Erklärung) and understanding (Verstehen) as two pervasively different modes of knowing (for a thorough discussion, see von Wright, 1971). While reserving explanations for experimental designs that provide knowledge about dependent (including causal) relationships between definite variables, understanding has been associated with qualitative, contextualized knowledge developed from language-based, interpretative, hermeneutic-oriented approaches. The argument goes that understanding captures the complex, intricate relationships between incidents and experiences that may not fit in with an approach whose aim is to establish observable, causal relationships between definite variables. In line with this, one would dismiss the claim of causality (e.g., Smedslund, 2015; Valsiner & Brinkmann, 2016), as punctuating a sequence of acts in the way proclaimed by the regularity model would hamper access to complex relationships, and thereby violate the phenomena themselves.

While this dismissive stance to claims of causality has long been embraced by various interpretative, post-structuralist, and postmodern perspectives, a more recent philosophical approach that covers various causal theories may serve to bridge the gap between causality and the broad neo-Kantian tradition (the Geisteswissenschaften) within humanistic social sciences. In these recently developed theories, causality is placed at the level of human mind in terms of inferences, reasoning, and social, linguistic interaction (Illari & Russo, 2014; Reiss, 2015; Williamson, 2013). One example is Reiss' (2012, 2015) inferentialism, which deals with the semantics of causal claims. More specifically, it locates the meaning of causal claims “in their inferential connections to claims about evidence for causal claims and cognitively or practically useful claims about explanation, prediction, and possible interventions.” (Reiss, 2012, p. 776). Another example is that of epistemic causality, where:

causality is a feature of the way we represent reality rather than a feature of agent independent reality itself; it is neither reducible to patterns of difference making nor to physical mechanisms. Our causal beliefs help us with our dealings with the world, since they allow us to predict, to explain and to control reality. We have these causal beliefs because of this inferential utility, not because there is some non-epistemic causal relation that is the object of those beliefs. (Williamson, 2013, p. 268)

Both inferentialism and an epistemic account of causality advocate views in line with contextualism (e.g., Kincaid, 2004; Reiss, 2012, 2015) implying that what counts as valid causal claims will vary according to the local community of researchers who are participating in the current discourse or interaction. There are no global standards to which all causal claims have to conform:

Standards are always local and contextual. They are local in that they apply first and foremost to specific episodes of scientific reasoning, which obtain in particular fields, periods and sometimes regions, and not to science as a whole. They are contextual in that they are relative to specific purposes of the inquiry, the questions asked, and the background knowledge that can be presumed. (Reiss, 2012, p. 772)

Interestingly, these perspectives might be seen as further developments of Hume’s (anti)causation theory as they elaborate on our tendency to make causal inferences. These theories are compatible with the epistemological aspect of critical realism (Bhaskar, 1998) and are based on the broad tradition of neo-Kantian approaches (Geisteswissenschaften), such as hermeneutics, phenomenology, Wittgensteinian language philosophy, and poststructuralism, which traditionally have not devoted themselves to causality.

Recent Causal Theories: Process, Context, and Theorizing

Although representing different philosophical traditions, the powers or dispositionalist perspectives, critical realism, and the recent inferentialism and epistemic causality share some fundamental arguments regarding causality: Causality is a processual and contextual phenomenon requiring an understanding of complex webs of interrelated factors or concepts that cannot be disentangled without changing the phenomenon in question. There is no global standard by which a causal claim may be justified, as the quality of justification will vary according to the local context, whether that context is a system, an organism, a social community, etc. Variation is crux, and the abstracted space of conceptualization and theorizing forms a vital part of causality, a point we will get back to further down.

A question that might be raised is whether recognition of the complex web of constitutive factors in reality requires any other research design than the experiment and its equivalents, as long as they include a necessary set of the presumed interacting factors. Obviously, all kinds of data and findings must be expressed, or operationalized, in some form that might compromise their ecological validity. No finding, however concise or contextually rich, is directly transferable to practice. The question is what form suits best, for which purposes? In the following, we will argue that no design or research method alone takes care of causality. All designs—in each their ways—put constraints on what may be identified. Hence, a valid approach to the intricate networks of human and social phenomena is to critically evaluate the capacities for causal explanations of all designs.

Causal Approaches: Difference-Making vs. Production

As witnessed by the various causal theories, causality means different things and may thus be established in different ways (see also, Cartwright, 2004). Here we may be reminded of the distinction between difference-making and production approaches to causality, which according to the “Russo-Williamson Thesis” (RWT) denotes different kinds/objects of evidence when deciding causality (Illari & Russo, 2014). While difference-making is associated with, for example, probability raising, counterfactuals, and manipulability, causal production typically deals with processes, mechanisms, and information. The first deals with whether causal relations exist or not, including the strength of the causal relationships, and the latter with how causal relations work. These may be viewed as complementary sources of evidence to build robust, nuanced knowledge on causal relations of a phenomenon. However, the differences between methods may be blurred, which in many respects suggest a continuum rather than two distinct groups.

While not going into detail on the various possible designs and methods to build knowledge on causal relations, we will line out just a few examples to get a picture of the possible repertoire of difference-making and production-oriented methods and how they may be combined, either within or between studies or research programmes. For example, RCTs are fit for picking out difference-making due to control conditions that allow inferences of the effects of therapeutic interventions. Nevertheless, due to the intricate interactions of theory-specific and common factors in therapy, we can hardly be sure about the specific causal mechanisms of an intervention. To cite Kazdin (2022): “[In] an RCT, we may have evidence that the intervention caused the change, but what facet of that intervention or why the change occurred can remain ambiguous.” (p. 9) Thus we may have established that the intervention worked (difference-making) without necessarily knowing how it worked (causal production). As pointed out by Shadish et al. (2002) in their extensive account of experimental and quasi-experimental designs for causal inference: Experimental designs are not fit for causal explanations, that is, the mechanisms reflecting how causal relations work. That must be dealt with by the use of other methods. Before we move over to production-oriented methods, however, we should notice variants of difference-making designs that may work well for drawing causal inferences despite the lack of the strict conditions of usual RCTs. Like discussed above, the experimental control of RCTs may ensure internal validity at the expense of external and ecological validity. However, pragmatic trials that allow more flexible use of manuals in response to patient responses than in the usual RCTs are, according to Kazdin (2022), an underused option to ease the transition of results from the experimental to the natural setting. Observational studies are other examples that may be fit for drawing causal inferences where the experimental control of RCTs, such as random assignment or manipulability, are not feasible for pragmatic or ethical reasons (e.g., Höfler et al., 2021; Kazdin, 2022). Examples of quantitative designs that are perhaps in the twilight between difference-making and production studies are variants of observational process studies, such as nonexperimental process-outcome studies (see Baldwin & Goldberg, 2021). One example is lagged multilevel models that examine how alliance at one session predicts outcome at the next, and vice-versa (e.g., Falkenström et al., 2013).

Designs fit for identifying causal production are perhaps most typically exemplified by various single-case designs (e.g., Kazdin, 2021) and qualitative methodology (e.g., Maxwell, 2004, 2021). Examples of research projects that reflect production approaches are those of Bohart et al. (2011) applicating the Jury Trial Model to evaluate the validity of descriptive and causal statements about psychotherapy process and outcome, and Elliott’s (2002) use of hermeneutic single-case efficacy design (HSCED), which by use of a combination of quantitative and qualitative methods illustrates a rigorous process of identifying causal mechanisms, and linking treatment processes to outcome. Mixed-methods designs allow the combination of different methods’ individual strengths in identifying causal relationships, which might serve to illuminate more of the complex web of causal drivers, such as RCTs in combination with in-depth qualitative analyses focused on identifying causal relationships. Relatedly, inquiries of so-called paradoxical outcomes, that is, cases where there is a marked difference between clients’ subjective reports on outcome and standardized outcome scores, have the potential to reveal other possible causal dispositions than what may be captured by use of predefined, categorical outcome measures (Stänicke & McLeod, 2021). Finally, an elaborate example of identifying sequences of change across studies is that of synthesizing studies by use of “ecological sentence synthesis,” as illustrated by Sandelowski and Leeman (2012). Here, findings from different studies are organized into thematic sentences by the following structure: “with this intervention,” “these outcomes occur,” and so on.

These examples and combinations are not new ideas, and are by no means exhaustive. The point here is to be aware of the particular causal focus that might be strengthened at all steps, either when using one single method/design, or when combining different methods within a mixed-methods design. In a similar vein, Rocca and Anjum (2020) have offered an overview of the advantages and disadvantages of various research methods in identifying intrinsic dispositions and their manifestation partners, demonstrating the necessity of combining the various methods and letting them play different roles in different stages during the research process. Moreover, from the perspective of critical realism as well as an inferential/epistemic approach to causality, one would emphasize the epistemological aspect as forming a crucial part of causal identification, whichever research design one is using for a specific analysis. Reiss (2012) for example argues:

Descriptions of specific applications of methods of causal inference such as experiments, randomised trials, regressions, applications of structural equations models, Bayes’ nets, expert judgements, meta-analyses and so on are all possible members of the inferential base for a specific causal claim. (Reiss, 2012, p. 770, italics here)

Thus, claims of causality may be based on various sources that are considered fit to the particular research questions and goals of the particular community of researchers. That is, the claims of credibility will vary, and “Causal claims are not normally established for their own sake but rather because they are considered useful for the attainment of certain purposes” (Reiss, 2012, p. 770).

Acknowledging the need for diversity may help making sense of seeming contra-intuitive empirical findings, such as overall scarce evidence of specific effects of particular methods and theory-specific interventions (e.g., Barkham & Lambert, 2021; Lambert, 2013; Wampold & Imel, 2015). How do techniques, theories and common factors (such as principles of change), the client and the therapist, operate together? Goldfried (1980, 2009) and Hatcher and Barends (2006) presented a straightforward model to illustrate the relationship between the various factors accounting for therapeutic change. According to this model specific techniques, common factors and clinical theories operate at different abstraction levels and must be understood as conceptually and empirically interwoven and reciprocally dependent. Rather than simply asking how much factors such as theory-specific techniques and common factors such as the working alliance contribute to change, these factors are perceived as conceptually dependent. There is no working alliance without theory-specific interventions (Bordin, 1979). Such an understanding is not conceivable by the experimentally produced findings alone, or by use of the experimental design itself. In whatever fashion we approach the complexities empirically no design on their own seem to help us understand the full picture of what works in psychotherapy. RCTs have been valuable to establish absolute effects of a treatment (i.e., better than controls), the effects of common factors, and also the general lack of comparative effects between different theoretical traditions and theory-specific interventions, to mention just a few examples. Thus, various difference-making designs may be used if the factors are practically identifiable and do not lose their “essence” by being operationalized in definite terms. Various non-experimental designs help examining the processes (observational process studies, narrative analyses, qualitative inquiries of sessions, etc.). Then again, we will need conceptual clarifications, theorizing, and theory-building studies (e.g., Stiles, 2009a, 2015) to make sense of the whole picture—such as conceptual relations between techniques, clinical theories and common factors, which is in line with a realist as well as an inferential/epistemic approach to causality.

Knowledge Transfer—Variation as the Gateway to Theorizing and Generalization

As we have seen, causality as understood by the approaches differing from Hume’s regularity conception all include causal inferences and the abstracted space of the mental world as a vital part of causality itself. We recall an emphasis on “retroduction” (inference to the best explanation) in critical realism, where causal explanations “must refer to something which, if true, would account for the observed pattern, and something which given background knowledge, could well be true” (Benton & Craib, 2011, p. 36). Likewise, theorizing—allowing abstractions that go beyond data, often a priori—is well recognized in most statistical models and analyses, as they are based on some presumed structure that guides hypothesizing, such as witnessed in models based on postpositivism/critical rationalism (Kuhn, 1970; Popper, 1959/2014, 1978) as well as in critical realism. Within inferentialism and epistemic causality, the causal identification operates in the conceptual or linguistic space. Thus, we recognize various types of causal explanations, based on different ontologies and epistemologies, but which all acknowledge the abstract reality as forming part of causality.

Abstractions inevitably imply variation and ambiguity. The more abstract, the more variants of a phenomenon may be included and captured by the concept or a theory. These various manifestations in turn allow identification of certain common features across the particular variants, serving to elaborate the meaning of the concept (e.g., Blumer, 1954). Herein lies the potential for generalizability. The ambiguity is crucial for the development of an adequate, comprehensive understanding that may be transferred to other, similar situations, either by elaborated conceptualizations or more or less developed theories. This dynamic of generalizability is widely recognized in the qualitative research literature and referred to by various terms such as “analytic generalization” (Kvale, 1996; Maxwell & Chmiel, 2014; see also Levitt, 2021) and “explanatory power” or “predictive ability” (Strauss & Corbin, 1998). Also proponents of the experimental tradition emphasize causal explanations as a means for knowledge transfer across variations in persons and settings. Referring to Bhaskar, Shadish et al. (2002) describes how knowledge of a complete causal system or of the total set of causally efficacious components “makes it easier to reproduce a given causal connection in a wide variety of forms and settings, including previously unexamined ones […] This is why causal explanation is often considered a primary goal of science” (p. 369).

Within psychotherapy research, Stiles (2009a, 2015) has elaborated on the dynamics of theorizing and empirical observations/analyses, illuminating different strategies for communication between theory and the empirical world, as well as different purposes of research (theory building, enriching, and fact gathering), which all may apply to quantitative as well as qualitative studies, and with each their implications for generalizability. Common to all these forms of generalizability is that they contrast with the mentioned principle of “symmetry of explanation and prediction,” where the applicability of findings is established in advance according to the premise of covering laws/universalism (Hempel, 1942). The various conceptions of knowledge transfer in terms of analytic generalization, explanatory power, conceptual development, enriching studies, and theory building all place the evaluation of applicability later on in the research process—either in the form of developed theories or elaborated concepts to be received by the relevant audience. The same line of reasoning may apply to causal explanations more specifically; variation by use of different methods—offering different viewpoints at different points in time of the research process—allows development of well-founded causal explanations, which is at the same time integral to theorizing. No one method is in a unique position to establish causality. We need empirical findings and rigorous theoretical arguments and discussions from various perspectives to complement the picture of causal relationships of a particular phenomenon. To ensure the inclusion of the variety of these knowledge sources, we will need to transcend the perspective of empiricism, not to exclude it, but to include approaches based in also other philosophical perspectives.

Evidential Pluralism—the Benefit of Variation

Speaking from the philosophy of pluralism one would assert that as the premises on which evaluations are based differ so fundamentally between the various systems of belief, value, and action (spanning from religion to reason to communication, etc.), any attempt at a consensus on truth—or valid evaluations of truth—is utopian (e.g., Rescher, 1993). Rather than aiming for ideal conditions of harmony, one would take a pragmatic position and recognize the unique viability of each system and arrangement—not because any of them are ideal, but because there is no alternative stance without undue conflicts and power struggles. Adopting the same approach to the methodological plurality within psychotherapy research, we would not only advocate the unique contribution of each design and method but refrain from any attempt at ascribing particular status to one specific design. Similar thoughts are advocated by Rocca and Anjum (2020), Höfler et al. (2021), and Deaton and Cartwright (2018) when they assert that the generalization of causal relationships hinge on prior knowledge in terms of various methods and theory within a cumulative program, which altogether help us understand not only “what works’—as established within experimental conditions—but “why things work” (see also Cartwright, 2004). Although not addressing causality itself, we recognize the same lines of reasoning by Carey et al. (2017) in their arguments for an increased repertoire of research methodologies to improve professional psychology practice, and by Smith et al. (2021) in their pluralistic perspective on research in psychotherapy. Moreover, similar ideas are advocated in the perspectives of methodological pluralism (Howard, 1983; Roth, 1987; Slife & Gantt, 1999), dialectical pluralism (Johnson, 2017; Johnson & Schoonenboom, 2016), and causal mosaic (Illari & Russo, 2014). Others have argued for a “realist synthesis’ approach (Emmel et al., 2018; Pawson, 2006), which is a perspective on evidence-based policy aimed at providing a viable alternative to standard systematic reviews. Thus, the examples are numerous, all recognizing the benefit of variation. Likewise, establishing causal relationships is to be conceived of as a processual endeavor involving a multitude of approaches to ensure the variation needed to develop knowledge of a phenomenon and its causal relationships.

Final Considerations

The distinctions between what may count as based in empiricism or variants of positivism and other philosophical approaches, such as realism, critical rationalism, and dispositionalism when it comes to research methods and designs may at times be blurred. For example, counterfactual reasoning (which is frequently tested by the use of experimental designs) may be understood as based on empiricism, but also realism and critical rationalism (due to the recognition of agency and hypothesizing/imagined conditions). Thus, the transition from one to the other philosophical perspective may be indefinite. As such, one might question the need for any discussion of the philosophical underpinnings at all as, naturally, there is not always a one-to-one relationship between research designs and underpinning philosophy. Indeed, and naturally, scientists go on developing proper designs without checking with the philosophical premises, leading to designs and methods that may have no clear philosophical grounding or may fit with several philosophies depending on what aspect of their method one pay attention to. Nevertheless, we would argue that an awareness of the philosophical underpinnings is helpful to make sure we do not unreflectively and habitually embrace some methods and designs while rejecting others that may be equally valuable, although leaning on fundamentally other premises. Moreover, increased awareness of the various philosophical conceptions of causation may inspire the developments of novel designs and methods fit to identify causal relations in new and creative ways. It is important to note that the powers and dispositions perspectives are ontological theories of causation, dealing with the fundamental question of what causation is, not necessarily how to go about studying it. As pointed out by Illari and Russo (2014), with some exceptions, less attention has been paid to the implications of powers/capacities/dispositions perspectives for evidence in scientific practice. Like we have seen, Cartwright (e.g., 2014; Deaton & Cartwright, 2018), Maxwell (2004, 2021), and proponents of dispositionalism (e.g., Kerry et al., 2012; Rocca & Anjum, 2020) have raised criticisms of RCTs in several publications over the last years. Developments of alternative methods and designs may, however, be an endeavor for the future.

For those who find philosophical subtleties to be of lesser use in developing proper designs and methods, an increased awareness of the distinctions between designs aimed at difference-making and causal production may still be a fruitful future path. Experiments—whether based in empiricism, realism, or critical rationalism—may be classified as designs devoted to difference-making, while designs fit for identifying causal production, that is, causal processes and mechanisms, are typically not based in empiricism, but realism and the various perspectives in the continental philosophy/neo-Kantian tradition. Being attentive to which purpose the various studies are meant to fulfill may increase our understanding of the implications and limitations of our causal claims, as well as sharpen our evaluation by use of relevant criteria.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Notes

1 We are not going into details on the various experimental designs and methods, but for the interested reader we recommend Illari and Russo (2014) “Causality: Philosophical theory meets scientific practice.” For further details on the range of experimental designs, we recommend Shadish et al. (2002) “Experimental and quasi-experimental designs for generalized causal inference,” and for comprehensive accounts and discussions of the various philosophies of causality, “The Oxford Handbook of Causation” (Beebee, Hitchcock, & Menzies, 2014).

2 We need to say that the fundamentality of laws of nature have been disputed and nuanced among scholars adhering to experimental theories of causation, although they may still stick to determinism (albeit softened) and a reductionist view of nature (e.g., Lewis, 1973; Psillos, 2009). Nevertheless, it is fair to say that the reported principles of knowledge production and generalizability still prevail within the broad experimental tradition as reflected in the ideal of randomization.