Implications of the TASI taxonomy for understanding inconsistent effects pertaining to free will beliefs

ABSTRACT

Whether people possess free will has been a long-lasting philosophical debate. Recent attention in social psychology has been given to the behavioral consequences of believing in free will. Research has demonstrated that manipulating free will beliefs has implications for many social behaviors. For example, free will belief manipulations have been associated with cheating, aggressiveness, and prejudice. Despite this work, some of these findings have failed to replicate. Testing theoretical predictions, such as whether believing in free will influences behavior, depends on theoretical, auxiliary, statistical, and inferential assumptions (TASI). In this paper we apply the TASI category of assumptions to the free will belief debate. In doing so we demonstrate why these assumptions should be considered when appraising the influence of free will belief manipulations on free will beliefs and behavior. To provide a nuanced view of free will beliefs, we believe researchers should pay careful attention to these critical assumptions.

KEYWORDS:

• Free will beliefs
• auxiliary assumptions
• theoretical assumptions
• statistical assumptions
• inferential assumptions
• social behavior

Introduction

The presence of free will has been debated extensively for centuries by philosophers. In a basic sense, free will can be understood as whether people have freedom of choice, or could have done otherwise (Nichols, 2004). On one side, some believe people are agents of their own behavior and can choose from alternatives. On the other, some posit behavior is directly or indirectly determined by the laws of nature, past experience, and external conditions. Positions, of course, vary within these extreme ends. For example, compatibilists believe that even if determinism is true, this does not preclude free will (Holton, 2009). Coverage of the many positions of this debate is beyond the scope of this paper, but readers are guided to coverage on this topic elsewhere (e.g., Fischer et al., 2009; Griffith, 2021).

Despite the subjective appeal of free will (Wegner, 2002) and the layperson belief in possessing it (Nahmias et al., 2005), scientists have somewhat converged on the idea that the role of free will may be limited (see Baumeister & Bargh, 2014). These views have recently reached the mainstream media, with the moral and ethical implications debated (e.g., Cave, 2016; Chivers, 2010; Griffin, 2016; Nahmias, 2011). Some have suggested that disclosing a lack of free will to the public could have detrimental societal consequences (e.g., Smilansky, 2002). People may, for example, act as they wish, simply ignoring rules and regulations (Cave, 2016; Shariff & Vohs, 2014). Disbelief in free will could demotivate people to exert self-control over urges, impulses, and thoughts (Baumeister et al., 2009). Conversely, accepting there is no free will could lead to greater empathy toward those living in poverty (Miles, 2013), and relieve the notion of a just world (Caruso, 2014). The idea that people are free to do otherwise is also important for retributive punishment in the legal system. People are held accountable for their behavior on the assumption that it was volitional and undertaken through free choice. But the idea that one should have done differently, which presupposes that one could have done so, is difficult to justify in the absence of free will.

The role of free will beliefs

Although the free will debate has largely occupied philosophers, social psychologists have recently entered the conversation. But rather than contributing to the discussion pertaining to the existence of free will, these psychologists have investigated the behavioral consequences of believing in free will. In the metaphysical sense, free will may not exist. Yet, people can hold beliefs about free will existence, and such beliefs may have behavioral consequences. This is akin to religiosity, as well as other psychological processes: a person’s belief in a deity may guide their moral principles and behavior, even if the deity may not be real. But we are not interested in the truthfulness of the deity or the truthfulness of free will. Rather, in taking an agnostic position, we are more concerned about how behavior is influenced by such beliefs.

Psychologists examining the consequences of free will beliefs have adopted empirical methods by manipulating free will beliefs and assessing behavioral outcomes. One of the first manipulation attempts came about from a pioneering paper by Vohs and Schooler (2008) comprising two experiments. In the first experiment, participants were asked to read an excerpt from Francis Crick’s The Astonishing Hypothesis (Crick, 1994). Participants in the experimental condition read a passage denouncing the notion of free will whereas those in the control condition read a passage not discussing free will. Participants were then asked to undertake a mental-arithmetic task that had been rigged to enable participants to access the answers. Participants were told the answers would not be revealed if they pressed a space bar and were encouraged to complete the task without cheating i.e., by pressing the space bar. They found that relative to the control condition, participants in the anti-free will condition demonstrated greater cheating behavior. In the second experiment, Vohs and Schooler applied a Velten-style technique (Velten, 1968) by providing participants with pro free will statements or anti-free will statements. Participants reading anti-free will statements cheated more than those in a control condition. Implications are that researchers can manipulate free will beliefs with behavioral consequences.

Following this study and using different types of manipulations, additional studies have examined the implications of free will beliefs. Some studies have found belief in free will leads to beneficial outcomes. For example, those with stronger free will beliefs have been shown to set more meaningful goals (Crescioni et al., 2016), have a truer sense of self (Seto & Hicks, 2016), and be more helpful (Baumeister et al., 2009) and gracious (MacKenzie et al., 2014). Weakening free will beliefs has led to aggressiveness (Baumeister et al., 2009), prejudice (Zhao et al., 2014), lower self-control (Rigoni et al., 2012), and reduced perceived autonomy (Alquist et al., 2013). This work implies stronger free will beliefs lead to prosocial behavior, but this is not always the case. Schrag et al. (2016) found stronger free will belief led to riskier decisions, and Krueger et al. (2014) showed those believing in free will demonstrated more punitiveness. Similarly, anti-free will manipulations have led to less judicial punishment and greater empathy toward offenders (Shariff et al., 2014), as well as reduced immoral behavior (Caspar & Vuillaume, 2017).

In addition to manipulation studies, research has examined the relationship between people’s preexisting free will beliefs and behavior. This work has shown stronger free will beliefs relate to greater self-efficacy (Crescioni et al., 2016) and perseverance (Li et al., 2018), and both academic (Feldman et al., 2016) and workplace (Stillman et al., 2010) performance. Moreover, a disbelief in free will has been associated with addictive behaviors (St Quinton et al., 2022; Vonasch et al., 2017) as well as meaninglessness and conformity (Moynihan et al., 2019).

Taken together, free will beliefs appear to have many influences on behavior. However, a challenge to this work, like in psychological research more generally (e.g., Open Science Collaboration, 2015), is the issue of replication. Indeed, some of the findings presented above have failed to replicate (e.g., Crone & Levy, 2019; Embley et al., 2015; Genschow & Vehlow, 2021; Genschow et al., 2021; Harms et al., 2017; Monroe & Ysidron, 2021; Monroe et al., 2016; Nadelhoffer et al., 2020). Embley et al. (2015) and Nadelhoffer et al. (2020) failed to replicate the findings of Vohs and Schooler (2008). Crone and Levy (2019) did not find a relationship between free will beliefs and moral behavior. Monroe et al. (2016) found no evidence that reduced free will influences judgments, punishment, and moral behavior. Genschow et al. (2021) found that successfully reducing free will beliefs in professional judges did not lead to more lenient sentences. Finally, Harms et al. (2017) found free will manipulations did not lead to greater altruistic behavior. Given these findings, the influence of free will beliefs is therefore unclear and warrants closer attention.

To summarize, recent correlational and experimental work examining the role of free will beliefs in human behavior has led to inconsistent results. Due to both the importance and relative infancy of this work, we believe it is timely to emphasize the importance of considering different assumptions when making theoretical predictions. In this paper we apply the taxonomy of assumptions devised by Trafimow (2019) to the free will belief debate. In doing so, we identify four categories of assumptions that must be considered to appraise the relevance of free will beliefs in human behavior. Considering these may help add to the accumulating evidence appraising the role of free will beliefs.

The TASI taxonomy

Trafimow (2019) suggested theory testing relies on four key categories of assumptions: theoretical (t), auxiliary (a), statistical (s), and inferential (i). We provide more detail on these below, but briefly; theoretical assumptions are the assumptions outlined in a theory or when making a theoretical prediction; auxiliary assumptions bridge the gap from the theory to the empirical hypothesis; statistical assumptions bridge the gap from the empirical hypothesis to the statistical hypothesis; and inferential assumptions bridge the gap from the statistical hypothesis to the inferential hypothesis. Empirical theory tests depend on theoretical, auxiliary, statistical, and inferential assumptions. We will now introduce theoretical and auxiliary assumptions and outline how this applies to the free will debate.

Theoretical and auxiliary assumptions

To test theoretical propositions, psychologists make empirical predictions. An example is that a person’s attitude will likely lead to them performing a behavior. Moving to free will beliefs, we may predict that people possessing stronger free will beliefs are more likely to be autonomous, successful, and goal driven. Or we may theorize that weaker free will belief cause empathy and compassion. These are theoretical assumptions. Predictions at the theoretical level therefore contain nonobservational terms such as attitude or free will belief. But when making empirical predictions, it is necessary to test hypotheses at the observational level. We therefore require a way to traverse the distance between nonobservational theoretical terms and observational empirical terms. This is where auxiliary assumptions are needed.

We therefore have two important, yet distinct categories of assumptions when making predictions: theoretical assumptions and auxiliary assumptions. But we cannot appraise the accuracy of the former without considering the latter. When a failed theoretical prediction is encountered, it may be that the theory is incorrect. An alternative explanation is not that the theory is wrong, but that one or more auxiliary assumptions is at fault (Duhem, 1954; Earp & Trafimow, 2015; Lakatos, 1976; Meehl, 1978; Quine, 1951; Trafimow, 2009, 2017). Or, when empirical predictions are successful, credit may be assigned to theory, auxiliary assumptions, or both (Trafimow, 2017). When making theoretical predictions, such as whether free will beliefs influence behavior, it is important therefore to carefully consider auxiliary assumptions, not just the theory (St Quinton et al., 2021).

Before identifying the auxiliary assumptions attached to the theoretical assumptions associated with free will predictions, let us first outline this relationship using a classical example from the psychological literature. The theory of planned behavior (Ajzen, 1991) suggests action is influenced by a person’s intention. To test this theoretical assumption, we first need to recognize that intention is, of course, a nonobservational term. And to understand whether intention influences behavior, we need to consider observational terms. At the observational level, a researcher may go about obtaining information about intention by asking participants to provide check marks on a questionnaire. To test the theoretical assumption that intention influences behavior, we must therefore ensure that these check marks accurately reflect the person’s intention toward the behavior. Hence, we have identified and accounted for an important auxiliary assumption. If the check marks do not accurately represent a person’s intention, this is not a fault at the theoretical level. To be clear, the theory suggests intention influences behavior, not that behavior is influenced by check marks measuring intention. An auxiliary assumption is therefore needed to traverse the gap between the nonobservational term in the theory (e.g., intention) and observational terms in empirical hypotheses (e.g., check marks on an intention measure).

Now we have clarified the importance of considering both theoretical and auxiliary assumptions, we subsequently apply these assumptions to the free will belief debate. Understanding the influence of free will beliefs requires considering: 1) whether free will beliefs are changeable and 2) whether free will belief change leads to behavioral change. And to answer these questions, auxiliary assumptions are needed to bridge the gap between unobservable free will beliefs and the observable manipulation; unobservable free will beliefs and the observable free will belief measures; and unobservable behavioral constructs used as the outcome variable and specific observable behaviors. These auxiliary assumptions will now be discussed.

Free will beliefs and manipulation

There are auxiliary assumptions attached to free will belief manipulations. That is, auxiliary assumptions are needed to bridge the gap between unobservable free will beliefs and observable manipulations that are hoped to influence them.

It is obviously important that a manipulation suitably influences free will beliefs. As we previously mentioned, different methods have been adopted to target beliefs including passages of text, vignettes, and Velten statements. However, some studies have found difficulty in modifying free will beliefs (e.g., Embley et al., 2015; Nadelhoffer et al., 2020). Of course, this could be because free will beliefs are not amendable to change. But there are other reasons why this may occur. And these explanations require the consideration of auxiliary assumptions.

First, the manipulation may target constructs outside of a belief in free will. For example, it has been suggested that instead of targeting a disbelief in free will, the Vohs and Schooler (2008) manipulation addressed a belief in fatalism (Miles, 2013). In addition to free will beliefs, manipulations may also modify affect, mood states, or determinism (Nadelhoffer et al., 2020), which may mask the intended manipulation effects (Genschow et al., 2021). If either of these occur, then studies have not suitably examined the effects of manipulations on free will beliefs. We therefore need to attach an auxiliary assumption here to bridge the gap between the observable manipulation and the unobservable free will belief.

Second, there may be problems with the content of the manipulation. Free will manipulations often involve passages of text or statements arguing for or against free will. However, participants may find the information unconvincing or implausible, which would have implications for the manipulation. For example, if a participant does not believe scientists’ claim that free will is an illusion, presenting the claim cannot be expected to change free will beliefs. Similarly, belief change would be less likely if participants perceive the information to be generic and not personally relevant. But this says nothing about the theoretical prediction concerning the role of free will beliefs on behavior. Instead, this represents an auxiliary assumption traversing the distance between the theoretical prediction and manipulation. It is interesting to note some studies have attempted to strengthen the manipulation by including follow-up statements associated with free will (e.g., Alquist et al., 2013; Katzir & Genschow, 2022). Specifically, after reading passages of text, participants are provided with statements pertaining to free will and asked to consider how they applied to their life, thinking of examples whilst doing so. Genschow et al. (2022) recently showed that manipulations following up with such statements are more effective at modifying free will beliefs than manipulations not doing so.

But even a carefully crafted manipulation does not guarantee free will belief change because, thirdly, manipulation efforts depend on participants paying adequate attention to the material. Obviously, beliefs are unlikely to change if the content within the manipulation is not acknowledged. A participant may give the manipulation requisite attention but may simply not understand the content. Thus, a fourth issue concerns the understanding of the manipulation. The language used in the Vohs and Schooler (2008) manipulation could be difficult to absorb for those unfamiliar with philosophical terms (Nahmias et al., 2005). Although these researchers found change in free will beliefs, the same passage of text used by Embley et al. (2015) did not. Like the foregoing examples, these are important considerations when the goal is to understand the behavioral consequences of free will beliefs; an accurate appraisal of the theoretical assumption relies heavily on these being considered. But, again, these issues say nothing about the theoretical prediction under investigation. We therefore need to attach auxiliary assumptions. Specifically, auxiliary assumptions are needed to bridge the gap between an observable manipulation and an unobservable free will belief.

To summarize, there are auxiliary assumptions needed to traverse the gap from free will beliefs to free will manipulations. Although modifying free will beliefs may require a strong manipulation (Schooler et al., 2014), failure to do so need not indicate that free will beliefs are unchangeable. Instead, it may be the case that one or more auxiliary assumptions attached to the manipulation are at fault. Consequently, this would not provide an accurate test of the theoretical assumption that free will beliefs influence behavior. Of course, manipulation checks can, to some extent, establish the success of a free will belief manipulation. But even if such a check supports that the manipulation influenced the construct it was supposed to influence, the manipulation may nevertheless also influence a competing construct. Unless trivial effects are found on the competing construct, the manipulation check does not convincingly demonstrate that the competing construct is not responsible for the effect. Moreover, successful manipulation checks may be due to demand effects. Typically, participants indicate whether they believe in free will after reading a passage of text arguing against its existence. Now, the manipulation check may correctly indicate a successful change in free will belief. Alternatively, participants may indicate reduced belief in free will for other reasons, such as perceived expectations.

Additional checks can also be undertaken to attend to other identified issues. For example, to check the manipulation was understood, Zhao et al. (2014) asked participants to identify the correct sentence representing the text. Similarly, Schrag et al. (2016) had participants specify their interpretation of the manipulation. To check sufficient attention was paid to the manipulation, Genschow and Vehlow (2021) asked participants to complete an attention question, and Nadelhoffer et al. (2020) removed participants failing an attention check. Although these checks can attend to some of the auxiliary assumptions related to the manipulation, there are auxiliary assumptions needed to bridge the gap between the observable free will measures and unobservable free will beliefs. These are discussed next.

Free will beliefs and measures

Acquiring information about people’s psychological beliefs typically occurs using questionnaires. As we previously illustrated in the theory of planned behavior example, attitude is assessed using check marks. But there is no guarantee that these check marks accurately represent attitude. This is also the case for free will beliefs. Participants are required to provide a check mark on several items purportedly representing free will. Yet, these items may not accurately represent free will beliefs. For example, the Free Will and Determinism scale (FAD; Rakos et al., 2008) treats free will and determinism as single dichotomous concepts. Therefore, items purported to measure belief in free will may be confounded with belief in determinism. Moreover, free will belief items in the typically adopted Free Will and Determinism-Plus scale (FAD-Plus; Paulhus & Carey, 2011) may indirectly measure aspects of control and moral evaluation due to focusing on responsibility (Monroe & Ysidron, 2021; Nadelhoffer et al., 2014). If measures of free will belief fail to measure it, or measure alternative constructs, then findings based on these measures fail to provide strong tests of theories concerning free will belief. An auxiliary assumption is therefore needed here. Specifically, an auxiliary assumption is needed to traverse the gap from an unobservable free will belief to an observable checkmark purporting to measure it. Without this auxiliary assumption, we cannot be sure that measures of free will beliefs are obtained.

It is also important that participants understand items addressing free will beliefs. Unlike asking about whether a person likes carrots, free will belief items can be difficult to comprehend. Not only may participants have difficulty understanding measures, but there is the possibility that participants fail to provide an honest account of their belief in free will. Why might this be the case? Well, some may believe endorsing free will is expected of them, despite having reservations. This expectation could come from culture, religion, or other societal influences. Response bias may also come from the perceived expectation of the researcher. For example, despite believing in free will, it may appear that items are presented in a way that this is not expected. This perceived expectation may have the opposite effect; those not endorsing free will may state otherwise to prevent satisfying the researcher. This will have implications for the conclusions made about free will beliefs. Although these issues will influence the validity of free will belief measures, they are auxiliary assumptions. Specifically, they are auxiliary assumptions traversing the gap between the psychological construct of free will and items purportedly measuring the construct.

Behavioral construct and measures

The foregoing auxiliary assumptions enable an understanding as to whether free will belief manipulations can engender changes in beliefs. But we also need to understand whether belief change can bring about behavior change. To do so, consideration needs to be given to the observational behavioral measures of nonobservational behavioral constructs. Auxiliary assumptions are therefore needed to traverse the distance from nonobservational constructs to specific observable behaviors or statements. Several behavioral outcomes have been used to examine the influence of free will beliefs. But there is no guarantee that the measures used validly index behaviors. There are different reasons why this may be so, the most obvious pertaining to the biases associated with self-report (Chan, 2009; Van de Mortel, 2008). To measure behavioral addictions, Vonasch et al. (2017) and St Quinton et al. (2022) asked participants to self-report their behavior. However, there is no guarantee that participants provided an accurate appraisal of their addictive behaviors. Participants may have been embarrassed or unable to recall their behavior. The issue here is clearly auxiliary to the question of whether free will beliefs theoretically influence behavior. Thus, here we have an auxiliary assumption. However, even objective measures need to consider auxiliary assumptions, as we shall now see.

Recall how Vohs and Schooler (2008) measured cheating behavior using the pressing of a space bar. Specifically, participants pressing the space bar did not see the answer and were therefore not classified as cheaters. However, as the authors acknowledged, the measure could have included passiveness because instead of attempting to cheat, failure to press the space bar could have been a consequence of a lack of attention. The measure could have therefore confounded cheating with inattentiveness, meaning the influence of free will beliefs on cheating behavior would have not been examined. As another example, Stillman et al. (2010) measured work performance by asking supervisors to provide reports on various aspects of performance. Specifically, employers were asked to rate employees’ work effort, reliability, consistency, social impact, and general assessment on a 1–7 scale. Now, is it certain that the employer provided accurate ratings of job performance for all 65 employees? We think this is unlikely. Amongst other reasons, it would require the employer to understand each item, know the employee well, and provide an honest appraisal of performance aspects. Failure to satisfy any of these would influence the accuracy of the behavioral measure. In these two examples we can see how auxiliary assumptions are needed to traverse the distance from the nonobservational behavioral constructs (cheating and work performance) to observable behaviors.

We previously mentioned how hypothetical scenarios are often used to examine the behavioral consequences of free will beliefs. For example, studies applied to anti-social and moral behavior necessitate participants first read or observe different scenarios and second decide on a punishment in the presence or absence of free will (e.g., Shariff et al., 2014). Of course, measures of hypothetical behavior may not align with real behavior. Participants may view hypothetical scenarios as an opportunity to act as one wishes, not how one would. Or, despite their good intentions, participants may simply not know how they would act outside of the scenario. The hypothetical behaviors also may lack personal relevance. Take, for example, the study conducted by Alquist et al. (2013) examining the influence of free will beliefs on conformity. The study adopted an artwork evaluation task where participants were asked to rate how much they liked some paintings. To assess conformity, participants also saw some bogus ratings from other participants prior to providing their own ratings. Confirming the prediction, they found that participants in the anti-free will condition mirrored the bogus ratings significantly more than participants in the free will condition, thus demonstrated greater conformity. Yet, it could be questioned how much attention a participant would really give to this measure. And the extent to which this measure of conformity applies to other prosocial behaviors could also be a concern. In each of these cases, the observational measure may not correspond with the nonobservational term, and the need for auxiliary assumptions is clear.

Finally, as with measures of free will beliefs, behavioral measures could be influenced by perceived expectations or the presence of a researcher. For example, participants could behave in a way they deem to be socially acceptable or that they perceive the researcher wants them to behave. Thus, auxiliary assumptions are needed here. Interestingly, some studies have attempted to account for demand characteristics (e.g., Baumeister et al., 2009; Schooler et al., 2014). For example, Baumeister et al. (2009) asked participants the degree to which they believed the researcher would want them to act in a helpful or aggressive way. Although the researchers found no differences between conditions, that is not to say honest accounts were provided. Thus, in addition to the auxiliary assumption attached to the behavioral measures, we may need to attach an additional auxiliary assumption here, too.

Summary

We have introduced theoretical (t) and auxiliary (a) assumptions. Specifically, theoretical assumptions are the assumptions outlined in a theory or when making a theoretical prediction; and auxiliary assumptions bridge the gap from the theory to the empirical hypothesis. Both are important when making empirical predictions, although auxiliary assumptions are rarely taken into consideration, at least not explicitly. This is perhaps due to the difficulty in untangling these assumptions from those at the theoretical level. We have suggested that an understanding of the effect of free will belief manipulations on behavior requires auxiliary assumptions to bridge the gap between unobservable free will beliefs and observable manipulations; unobservable free will beliefs and observable free will belief measures; and unobservable behavioral constructs and observable behaviors. Despite the importance of these, other assumptions require attention too. As part of the TASI taxonomy and to enable accurate predictions, we also need to consider statistical (s) and inferential (i) assumptions. We will now discuss statistical assumptions.

Statistical assumptions

Statistical assumptions traverse the gap from the empirical hypothesis to the statistical hypothesis. Tests of theoretical predictions typically involve studies comparing differences between experimental and control conditions with respect to the outcome variable. Differences in central distribution scores such as the mean, median, and mode would provide evidence either in favor or against the prediction. Thus, a successful experiment, depending on the manipulation and hypothesis generated, would be demonstrated if the experimental condition displayed significantly higher or lower scores relative to the control. The same line of thought is adopted in tests of free will beliefs. That is, following manipulation of free will beliefs in the experimental condition, comparisons are made on the behavioral outcome with respect to the control condition. The mean scores of the behavioral outcome are then used to establish whether the manipulation led to any observed differences in the hypothesized direction. For example, the Vohs and Schooler (2008) manipulation hypothesized that mean scores in cheating behavior would be greater in the anti-free will experimental condition than the control condition.

The statistics involved in establishing these differences depend on assumptions of normality. Normal distributions have two parameters: the mean and standard deviation. However, most distributions in research studies are skewed (Blanca et al., 2013; Ho & Yu, 2015). The family of normal distributions is a subset of the much larger family of skew normal distributions that has three parameters: location, scale, and shape. When the shape parameter is zero, the distribution is normal. For a normal distribution, the location and scale parameters correspond to the mean and standard deviation, respectively. This is not the case when the shape parameter does not equal zero, so the distribution is skew-normal. And in a skew-normal distribution, the mean and standard deviation do not correspond to the location and scale parameters, respectively.

This has implications for the evaluation of experimental manipulations. A difference in means may be in the same direction as a difference in locations. In such cases, one can evaluate the success of a manipulation more easily. But this is not always the case. A difference in means need not coincide with a difference in locations, and a difference in locations need not coincide with a difference in means. Additionally, a difference in means can be in the opposite direction of a difference in locations, thereby implying opposing substantive conclusions (Trafimow, 2019. Worse yet, seemingly trivial skewness can be sufficient to cause these effects. Therefore, comparing means between conditions may support the theory, but comparing locations in the two conditions may not. Conversely, comparing means between conditions may not support the theory, but comparing locations in the two conditions may do so.

Let us demonstrate using an example from the free will literature. Imagine we wish to understand the relationship between free will beliefs and anti-social behavior. We suspect that anti-social behavior comes from a lack of belief in free will. To achieve this, we develop a manipulation comprising passages of text claiming free will is an illusion. The experimental condition receives this treatment and the control condition receives text unrelated to free will. To support our hypothesis that reduced free will beliefs lead to anti-social behavior, we wish to increase mean behavioral scores in a positive direction in the anti-free will manipulation condition compared to the control condition. Let’s assume that after conducting the experiment, we successfully find anti-social behavior means differ, and in the hypothesized direction. Appearance suggests our prediction came true and the manipulation was successful; we observed greater mean behavioral scores in the experimental condition relative to the control condition. But when looking at locations, we find no difference between conditions. Rather than supporting that the manipulation shifted one distribution relative to the other, a more accurate statistical story might be that the manipulation merely changed the shape of one distribution relative to the other. To support the theory, the researcher ideally does need a distribution shift, not just a change in distribution shape. Therefore, in this example, an experiment that seemingly supports the theory does not.

In the foregoing example, we observed change in means but not locations. Yet had we only examined mean differences, we may have come to the wrong conclusion regarding the manipulation. This is not to say that one should always pay more attention to locations. Instead, location statistics should be considered if there is a necessity to obtain a shift in the location of one distribution relative to the other (as is usually true in tests of theories, though there may be applied contexts where a distribution shape is sufficient). To prevent making inaccurate conclusions, researchers should therefore give sufficient attention to the statistical assumptions used to bridge the gap from the empirical hypothesis to the statistical hypothesis.

Assumptions about distribution shapes are not the only statistical assumptions. Others include whether the data are continuous or discrete, the extent to which discrete responses such as on a 7-point scale can soundly be treated as continuous, linearity, whether transitivity can be assumed, and so on. The issue of statistical assumptions is no less complex than the issue of auxiliary assumptions.

In summary, we may apply good theoretical assumptions and sound auxiliary assumptions (e.g., to manipulations, measures), but we may stumble at the statistical level. An incorrect statistical assumption could lead to erroneous conclusions about the nature of free will beliefs. Researchers should therefore carefully consider the statistical assumptions being made. Specifically, it is important to check the statistical assumptions bridging the gap from the empirical hypothesis to the statistical hypothesis.

We have now considered three levels of assumptions of the TASI model: theoretical (t), auxiliary (a), and statistical (s). But we still have one final level to consider: inferential (i). This will be discussed subsequently.

Inferential assumptions

The final set of assumptions requiring consideration are inferential assumptions. To make a strong case for a theoretical prediction, researchers are often not content with making statistical assumptions. Instead, researchers wish to generalize to the population level. Inferential assumptions are therefore needed to bridge the gap from the statistical hypothesis to the inferential hypothesis.

To generalize from a sample statistic to a corresponding population parameter, participants should be selected randomly from a defined population. Although generally implied in research studies, whether this occurs is largely questionable. Random sampling has important implications for the ability to generalize from descriptive statistics to corresponding population parameters. In the absence of random sampling, it is easy to argue that such generalization is unwarranted. To illustrate, remember that the goal of free will manipulations is to establish mean differences between experimental and control conditions. And let us assume that a researcher successfully identifies differences in free will beliefs and behavior between these conditions in the hypothesized direction. Without random sampling, it would be wrong to assume that any observed sample difference between means accurately reflects a corresponding population difference between means. Therefore, the relevance of a successful manipulation may be limited to the sample studied. If a researcher wishes to make statements about a population, an inferential assumption is needed to bridge the gap from the statistical hypothesis to the inferential hypothesis.

Another important inferential consideration is that of random assignment, which decreases the probability of systematic differences between conditions. It is possible for a researcher not to select randomly from the population but to randomly assign participants to conditions. However, random assignment is not analogous to random selection. At best, random assignment in the absence of random selection can equate to randomly sampling from a population of possible randomizations. That is because random assignment implies there could have been alternative randomizations. In this case, a researcher could generalize reasonably to the population of potential randomizations but could not generalize reasonably to the population of people. To make inferences to the population of people, the researcher must undertake random selection from that population of people. Despite the necessity of assuming random selection from a population of people, we are unaware of any free will research where this has been accomplished.

Even if participants are selected randomly from a population and assigned randomly to conditions, there are other inferential assumptions to contend with. Careful consideration should be given to the characteristics of participants recruited to studies. Due to accessibility and convenience, it is often the case that psychological research comprises student samples (Arnett, 2008; Hanel & Vione, 2016). The student sample has unique developmental and environmental characteristics (Henry, 2008). For example, as the typical student is transitioning from late adolescence into early adulthood, instability in decision-making, behavior, and routine often follows (Arnett, 2011; Balfe, 2009). Given this uniqueness, reliance on statistical assumptions from student samples limits generalizability to other population parameters (Weigold & Weigold, 2021). Indeed, differences can lead to findings that not only vary in magnitude but differ in direction too (Hanel & Vione, 2016). In relation to free will studies, some research has included participants outside of student samples, but there is similarly an over reliance on students in this research area (Ewusi-Boisvert & Racine, 2018). This could have consequences for the implications that can be drawn from studies manipulating free will beliefs. For example, it may not be the case that reducing free will beliefs influences conformity in populations other than students. Or we may not be able to generalize the supposed effects of diminished free will belief on helpfulness. Reliance on students renders it difficult to make population inferences based on sample statistics. Therefore, we must attach an inferential assumption to the statistical assumption.

Studies in the sciences often involve Western, Educated, Industrialized, Rich, and Democratic (WEIRD) samples (Henrich et al., 2010). Yet, there is variability in the psychology and behaviors of samples across cultures. For example, in research applied to physical activity, Hagger et al. (2007) found the psychological determinants underlying the behavior varied across five national groups with different cultures. There is also preliminary evidence that free will beliefs vary across cultures. Berniūnas et al. (2021) found free will was understood similarly within Western countries (U.S., Lithuania), but differently between Western and non-Western countries (Mongolia, India, China). Due to these differences, it would be difficult to generalize findings about a free will belief manipulation from WEIRD samples. That is, statistics from WEIRD samples might not generalize to population parameters.

For sample statistics to provide good estimates of corresponding population parameters it is necessary to have large enough sample sizes. Under the assumption of random selection, larger sample sizes imply that sample statistics better estimate corresponding population parameters (Trafimow, 2019). Small sample sizes can also undermine the ability to detect true effects due to limited statistical power (Button et al., 2013). Yet, like the aforementioned issues, research in psychology (Open Science Collaboration, 2015), including the free will belief literature, rely on small sample sizes (Trafimow & Myüz, 2019; Trafimow et al., 2020). And even significant findings in studies adopting small sample sizes can be attributed to a type I error (Fraley & Vazire, 2014) and force dramatic effect size inflation. To illustrate, a recent meta-analysis conducted by Genschow et al. (2022) found antisocial behavior was significantly influenced by free will manipulations. However, these effects were mainly driven by studies with smaller sample sizes, with larger samples finding no significant effects on antisocial behavior. Similar patterns were observed in cheating, conformity, and punishment behaviors. Taken together, even if we ensured random selection and participants different from students, there is no reason to believe that inferences from statistical tests can be generalized to the population. An inferential assumption is therefore needed to traverse the distance from the statistical hypothesis to the inferential hypothesis. It is also important to note that large samples do not automatically constitute accurate parameter estimates. Large samples drawn from a restricted population would limit inferences about the wider population.

It is important to note that inferential assumptions are only relevant if one wishes to make population inferences. If a researcher is not interested in making assumptions about a population, the theoretical, auxiliary, and statistical assumptions would suffice. However, there is a high price to be paid if one ignores inferences to population parameters. To dramatize, imagine that no sample statistics generalized to corresponding population parameters. In that case, there would be no reason to expect findings to replicate, no reason to believe that any sample effect pertains to any populations of interest, and no reason to believe that the sample effect is due to the theorized cause. Therefore, our recommendation is not to eschew inferential thinking, but rather to do it better.

Nor have we covered the topic of inferential assumptions. For instance, suppose a researcher performs a traditional null hypothesis significance test. We have already seen that the assumption of random selection from a population is generally false, thereby rendering the null hypothesis significance test likely unsound. In addition, there are issues such as whether to attempt to control for the experiment-wise error rate or the hypothesis-wise error rate, whether to perform a one-tailed test or a two-tailed test, and so on. If a researcher wishes to reject null hypothesis significance testing and opt for a Bayesian alternative, such as Bayes factor, these issues do not disappear. Random selection from the population is every bit as important an inferential assumption for Bayesian statistics as for traditional statistics. More generally, although both traditional and Bayesian statistics tend to be presented in the form of hypotheses to be tested, the truth is that these hypotheses are always embedded in larger models that also contain inferential assumptions (model = hypothesis + inferential assumptions). Even if a hypothesis is correct, such as that the population means of two conditions are equal, the model containing it is almost certainly false due to at least one wrong inferential assumption such as that of randomly selected samples. Even for Bayes factors, where the goal is to show that the data are more consistent with one hypothesis than with the other, the best a researcher can do is show that the data are more consistent with one model (hypothesis + inferential assumptions) than the other. But as both models are likely false due to at least one false inferential assumption, regardless of whether one of the hypotheses is true, it is not clear how to draw a sound conclusion about the relative likelihood of competing hypotheses. A Bayes factor may support one model over another even if the hypothesis contained in the supported model is wrong, or even if the hypothesis contained in the disconfirmed model is correct. A careful consideration of inferential assumptions, as opposed to focusing only on hypotheses, creates potential problems for any sort of inferential procedure.

To summarize inferential assumptions, they are needed to traverse the gap from the statistical hypothesis to the inferential hypothesis. Without considering quality inferential assumptions, it is difficult to make inferences about population parameters based on sample statistics. In a free will manipulation study, not considering inferential assumptions would render unclear whether mean differences in free will beliefs and behaviors between the experimental and control conditions is reflected at the population level. It is therefore important that researchers attend to inferential assumptions, in addition to the theoretical, auxiliary, and statistical assumptions previously described.

Conclusion

There is a growing trend for researchers to establish that free will belief manipulations influence a variety of behavioral variables. Notwithstanding theoretical advances, this work could have important societal implications. However, some studies examining the influence of free will beliefs have demonstrated small effects and others have failed to replicate. This has obvious implications for both appraising the utility of free will belief manipulations and understanding the behavioral consequences of such manipulations. To investigate this, we have suggested researchers pay close attention to assumptions across the TASI levels. By carefully considering theoretical, auxiliary, statistical, and inferential assumptions associated with free will belief manipulations, larger and more replicable effect sizes may ensue. This would enable research in this area to establish both the theoretical and practical implications that free will belief manipulations have on behavior.

As we have outlined, testing theoretical predictions, such as whether free will manipulations lead to behavioral consequences, necessitates the consideration of four levels of assumptions. Because of these four levels of assumptions, two issues require attention when considering the free will belief manipulation debate. First, it is necessary to understand the relevance of each of the four levels. That is, of the theoretical, auxiliary, statistical, and inferential assumptions, one must ascertain which are lacking in the free will belief literature. We may have good theoretical and auxiliary assumptions but poor statistical and inferential assumptions. Or we may have good inferential assumptions but poor auxiliary assumptions. In our view, the theoretical assumptions applied to the manipulation debate are valid. Indeed, despite the limited downstream consequences demonstrated by experimental work, several correlational studies have found a relationship between free will beliefs and socially relevant thoughts and behavior. For example, free will beliefs have shown to correlate with punishment for unethical behavior (Clark et al., 2014; Genschow et al., 2017; Martin et al., 2017), victim blaming (Genschow & Vehlow, 2021), reward for ethical behavior (Genschow et al., 2017), and the perception of intentionality in others’ behaviors (Genschow et al., 2019). This correlational evidence provides some support for the theoretical prediction. However, when one progresses to the auxiliary assumptions, it is apparent that these assumptions have not been fully appraised in the free will belief debate. This is the case in psychological research more generally (Trafimow, 2012), so should come as no surprise. Nevertheless, an accurate appraisal of free will manipulations requires good auxiliary assumptions to be attached to the theoretical prediction. Following this, one is then able to test the statistical and inferential levels.

A second issue worth considering is the pertinence of issues within each of the four levels. Given each level has multiple issues, their contribution to the free will manipulation debate may vary. For example, we suggested auxiliary assumptions are needed to; 1) bridge the gap between unobservable free will beliefs and the observable free will belief measures; 2) bridge the gap between unobservable free will beliefs and the observable manipulation; and 3) bridge the gap between unobservable behavioral constructs and observable behaviors. Thus, considering these three issues requires the consideration of many auxiliary assumptions. It is an empirical question which of the auxiliary assumptions are most important to test whether free will manipulations lead to behavioral consequences, but one can speculate. For example, a personally relevant passage of text arguing against free will would likely demonstrate limited effects if the manipulation is not understood. Similar outcomes may be found if the text is understandable but not persuasive. But that is not to say researchers have neglected all of the issues we outlined. Indeed, auxiliary assumptions applied to free will belief measures have led to the development of newer scales, such as the Free Will Inventory (Nadelhoffer et al., 2014). Nevertheless, it is important that focus is given to the many different assumptions at different levels of the TASI taxonomy, as each may contribute differently to the empirical prediction or be differentially problematic.

Disclosure statement

No potential conflict of interest was reported by the author(s).