Elementary probabilistic operations: a framework for probabilistic reasoning

Abstract

The framework of elementary probabilistic operations (EPO) explains the structure of elementary probabilistic reasoning tasks as well as people’s performance on these tasks. The framework comprises three components: (a) Three types of probabilities: joint, marginal, and conditional probabilities; (b) three elementary probabilistic operations: combination, marginalization, and conditioning, and (c) quantitative inference schemas implementing the EPO. The formal part of the EPO framework is a computational level theory that provides a problem space representation and a classification of elementary probabilistic problems based on computational requirements for solving a problem. According to the EPO framework, current methods for improving probabilistic reasoning are of two kinds: First, reduction of Bayesian problems to a type of probabilistic problems requiring less conceptual and procedural competencies. Second, enhancing people’s utilization competence by fostering the application of quantitative inference schemas. The approach suggests new applications, including the teaching of probabilistic reasoning, using analogical problem solving in probabilistic reasoning, and new methods for analyzing errors in probabilistic problem solving.

Keywords:

• Elementary probabilistic operations
• Computational analysis
• Problem space representation
• Quantitative inference schemas
• Bayesian reasoning

Probabilistic reasoning in general and Bayesian reasoning in particular have been the subject of intensive research in psychology and educational science for many years. In fact, the subject has entered the mathematics curricula of most high schools in western countries (see, e.g., Chernoff & Sriraman, 2014). In the present article, probabilistic reasoning tasks are conceived of as instances of mathematical problem solving that has been studied by Newell and Simon (1972) who adopted the problem space conception (for a similar view of probabilistic reasoning, see Sirota, Vallée-Tourangeau, et al., 2015). We present the framework of elementary probabilistic operations (EPO) to analyze the structure of elementary probabilistic problems, and to explicate empirical findings concerning people’s performance in probabilistic reasoning tasks.

This article is structured as follows: We first present an exposition of the EPO framework. Second, principle applications of the EPO framework are discussed: (a) a problem space representation of elementary probabilistic reasoning is presented; (b) we propose a classification of probabilistic problems on the basis of computational considerations; (c) different methods to improve Bayesian reasoning as well as facilitating effects of numeracy, level of education, and cognitive ability are discussed, employing the EPO approach; and (d) we analyze methods of training Bayesian reasoning. Finally, explanations based on the EPO framework are compared to traditional explanations, and additional applications of the EPO approach are deliberated in the general discussion section.

The framework of elementary probabilistic operations

The conception of probabilistic reasoning as the execution of a set of elementary probabilistic operations dates back, at least, to Von Mises (1972) who reduced probabilistic reasoning to basic operations. More recently, the problem of the efficient computation of specific elementary probabilistic operations has become relevant in the context of expert systems (e.g., Darwiche, 2009; Neapolitan, 1990). Shenoy and Shafer (1990) used the framework of elementary probabilistic operations for the description of computations in Bayesian networks and belief function systems (see also Kohlas, 2003).

In the following, we first present the different types of probabilities that are involved in elementary probabilistic reasoning. This is followed by an exposition of three elementary probabilistic operations for mapping different types of probabilities. These concepts are introduced informally with the help of the example in Figure 1. A formal presentation of the framework is given in the Appendix. We use capital letters to refer to random variables and lowercase letters to denote the values of variables.

Figure 1. Three-variable diagnosis problem: problem description, relevant probabilities, and probabilistic operations.

Types of probabilities

Let $X_1,X_2,\dots ,X_n$ represent n discrete random variables, each with a finite number of values (cf. Appendix, for a formal definition of the probability space underlying the definition of the random variables). For the sake of concreteness, let us consider a probabilistic problem involving three random variables: the hypothesis H and two pieces of evidence E (cf. Figure 1, Part A). There exist three basic types of probabilities (or probability distributions): joint, marginal, and conditional probabilities.

The joint (probability) distribution $P\left(X_1,X_2,\dots ,X_n\right)$ assigns to each possible combination of the values of the random variables a real value in the range $\left[0,1\right]$ in such a way that the probabilities of all combinations sum to one. In our example, the joint probability distribution consists of eight point probabilities (cf. Figure 1, Part C), and is denoted by $P\left(H,E_1,E_2\right)$. For example, the point probability $P\left(h,e_1,e_2\right)$ denotes the probability that a disease is present (h) and the two diagnostic procedures indicate the presence of the disease (e₁ and e₂). We use $P\left(h,e_1,e_2\right)$ as an abbreviation for $P\left(H=h,E_1=e_1,E_2=e_2\right)$.

The joint probability distribution contains the complete probability information about the random variables in question. This enables one to answer any possible probabilistic question about these variables. In addition, the joint probability distribution is defined on the finest partitioning of the sampling space consisting of combinations of the values of the random variables involved. These two characteristics, completeness and most fine-grained partitioning of the sampling space render the joint distribution of focal importance in the case of probabilistic reasoning.

The marginal distribution is the probability distributions over a single variable. In our example $P\left(H\right)$ denotes the marginal distributions over variable H consisting of two point probabilities: $P\left(h\right)$ and $P\left(\overline{h}\right)$. In addition, probability distributions involving more than a single variable but less than the full set of variables are called marginal joint distributions. In our example $P\left(h,e_1\right)$, $P\left(\overline{h},e_1\right)$, and $P\left(e_1,e_2\right)$ denote marginal joint probabilities.

The conditional (probability) distribution $P\left(\left.X_1,X_2,\dots ,X_k\right|X_{k+1},\dots ,X_n\right)$ denotes the set of distributions over the variables $X_1,X_2,\dots ,X_k$, for each combination of values of variables $X_{k+1},X_{k+2},\dots ,X_n$ (assuming $P\left(X_{k+1},\dots ,X_n\right)>0$). In our example, $P\left(\left.H\right|E_1,E_2\right)$ denotes the distributions over the values of H for each combination of the values of $E_1$ and $E_2$. For example, $P\left(\left.H\right|e_1,{\overline{e}}_2\right)$ denotes the distribution of the values of variable H in the case of $E_1=e_1$ and $E_2={\overline{e}}_2$. It consists of two point probabilities: $P\left(\left.h\right|e_1,{\overline{e}}_2\right)$ and $P\left(\left.\overline{h}\right|e_1,{\overline{e}}_2\right)$.

The three different types of probabilities make up the first component of the EPO framework. The second component is given by the elementary operations.

Elementary probabilistic operations (EPO)

There exist three elementary probabilistic operations: (1) combination of information, (2) marginalization, and (3) conditioning. These operators are used to switch between different types of probabilities. The first operation, the combination of probability information, combines probability information of different random variables and consists in the generation of a joint probability distribution by combining conditional and marginal probabilities. In the example in Figure 1D, the operation is applied in two different ways: (a) the information about variable H and E₁ is combined: $P\left(h,e_1\right)=P\left(\left.e_1\right|h\right)\cdot P\left(h\right)$, and (b) this combined information is further combined with information about variable E₂: $P\left(h,e_1,e_2\right)=P\left(\left.e_2\right|h,e_1\right)\cdot P\left(h,e_1\right).$ These two operations can be integrated into a single expression: $P\left(h,e_1,e_2\right)=P\left(\left.e_2\right|h,e_1\right)P\left(\left.e_1\right|h\right)\cdot P\left(h\right).$ This is an instance of the chain rule used for combining probabilistic information (cf. Appendix).

The second elementary probabilistic operation is called marginalization. This operation can be considered as an operation of focussing information on a subset of the random variables. In the case of discrete variables, marginalization is realised by means of a summation performed on the joint distribution. The result of marginalization is a marginal (joint) distribution. The summation is taken over all combinations of values of those variables that do not make up the resulting marginal distribution. These variables are “summed out” of the distribution. In the example in Figure 1D, the operation of marginalization, $P\left(e_1,e_2\right)=P\left(h,e_1,e_2\right)+P\left(\overline{h},e_1,e_2\right)$, focusses information on variables $E_1$ and $E_2$.

The final probabilistic operation is called conditioning. It consists in a re-evaluation of the probabilistic information within a new context (reference class) given by the values of the conditioning variables, and it is realized by means of division. In our example in Figure 1D, the operation $P\left(\left.h\right|e_1,e_2\right)=\frac{P\left(h,e_1,e_2\right)}{P\left(e_1,e_2\right)}$ results in a new evaluation of hypothesis H in light of the fact that both diagnoses are positive: $E_1=e_1$ and $E_2=e_2$.

Level of explanation of the EPO framework

The demonstration of the EPO framework in Figure 1 provides a concrete algorithm for solving a Bayesian problem. Referring to the three levels of explanation of an information processing system of Marr (1982), the formal part of the EPO framework should be considered as a computational level theory. It specifies the basic logic of the computations involved in elementary probabilistic reasoning problems as well as the way they can be carried out (cf. Marr, 1982, Figures 1–4, on p. 25). The EPO approach is not an algorithmic level theory. It is not concerned with details of specific algorithms for implementing the elementary probabilities and representations of the probabilistic information. As exhibited in the next section, the operations of the EPO framework also work with other types of quantities like frequencies.¹ In addition, they can be realized in different ways. For instance, the operation of combination may be performed using probability and frequency trees or by means of frequency arrays (see, e.g., Sedlmeier & Gigerenzer, 2001), and the operation of marginalization is easily accomplished in the context of contingency tables. Finally, the operations are valid within other probabilistic frameworks, like the belief function framework (Kohlas, 2003; Shenoy & Shafer, 1990) or mental model theory (Johnson-Laird et al., 1999).

Figure 2. Possible goal hierarchy of an expert problem solver and operations applied to achieve the goals for the three-variable diagnosis-problem of Figure 1.

Figure 3. Different formats of the general two-variable diagnosis problem.

Figure 4. Bayesian Problems with different probability formats and types of framing.

As a computational level theory, the EPO approach enables an abstract specification of the problem space, underlying elementary probabilistic problems, and a classification of probabilistic problems according to computational requirements for solving a problem. This distinguishes the EPO approach from traditional explanatory approaches that are located on the algorithmic level (see, e.g., Johnson & Tubau, 2015).

Elementary probabilistic operators and quantitative inference schemas

As noted above, the operations of combination, marginalization, and conditioning also work with quantities other than probabilities, like frequencies, length, area, volume or weight. This is due to the fact that these quantities share important characteristics with probabilities. Specifically, similar to probabilities, these quantities conform to the axioms of non-negativity and additivity. Specifically, the axiom of non-negativity states that quantities assigned to real objects are always positive, and the axiom of additivity states that the quantity assigned to the object resulting by joining two or more objects without overlap results in the sum of the quantities assigned to the single objects.² In contrast to probabilities, the norm axiom that states that probabilities are smaller or equal to one does not apply to quantities.

People have acquired quantitative inference schemas. These are problem schemas (cf. Hayes, 1989, p. 13) that are part of the curriculum of every elementary school. They are implementations of elementary probabilistic operations and combinations thereof, respectively, i.e., concrete algorithms that work with specific representations. The following three basic quantitative inference schemas implement the three (probabilistic) operations. They are called the part-whole schema, the concatenation schema, and the (recursive) partitioning/selection schema.

The part-whole schema implements the operation of conditioning: If Object X constitutes a part of Object Y, then the proportion p of the quantity $q_X$, assigned to X, within the whole quantity $q_Y$, assigned to Y, is given by the fraction $p\hspace{0.17em}=\hspace{0.17em}{q}_X/q_Y$. For example, 100 apples (quantity $q_X$) out of a total of 300 apples (quantity $q_Y$) make up a proportion of $p=100/300=1/3$. The concatenation schema accomplishes the operation of marginalisation: If object X is made up exhaustively of disjoint parts $X_1,X_2,\dots ,X_n$ with quantities $q_1,q_2,\dots ,q_n$, then the quantity q of X, conforms to the sum of the quantities of the disjoint (and exhaustive) parts: $q=q_1+q_2\cdots +q_n$. For example, putting two sticks of length 3 and 2 meters together (without overlap) results in a stick of length 5 meters. Finally, the (recursive) partitioning/selection schema implements the operation of combination: Assume that the quantity $q$ is assigned to Object $X$, and $q$ is partitioned according to the proportions $p_1,p_2,\dots ,p_m$ $\left(p_1+p_2+\cdots +p_m=1\right)$. This results in m quantities: $q\cdot p_1,q\cdot p_2,\dots ,q\cdot p_m$ The partitioning schema can be applied recursively, by further partitioning the quantities resulting from a previous partitioning step. In this way, the chain rule (cf. Appendix) is implemented. An illustration of the application of the quantitative partitioning/selection schema can be found in Gigerenzer and Hoffrage (1995, Figure 2 on p. 690) showing the results of cutting a beam recursively. Instead of the recursive partitioning of a quantity the schema may implement the recursive selection of entities (parts) from a given quantity. We, thus, call the schema partitioning/selection schema. For example, assume that there are 9 apples. Alice selects 2/3 of the apples leaving 1/3 for Bob. Thus, the whole quantity (9) was partitioned into two parts, 6 (Alice) and 3 (Bob). In this way, the information which person received how many apples was combined with the original quantitative information, the number of apples.

In addition to these quantitative inference schemas, implementing elementary quantitative operations, people also possess schemas that integrate two elementary operations. For example, if Object X is made up exhaustively of the disjoint parts $X_1,X_2,\dots ,X_n$ with quantities $q_1,q_2,\dots ,q_n$, then the proportion of the quantity of Object (Part) $X_j$ with respect to the quantity assigned to Object X, conforms to the fraction $q_j/(q_1+q_2+\dots +q_n)$. This quantitative inference schema integrates the operation of marginalization and conditioning. This combined marginalization-and-conditioning schema will be called quantitative proportion schema. Referring to the previous example, the number of apples selected by Alice is 6 leaving 3 to Bob. Thus, the proportion of apples possessed by Alice is $6/\left(6+3\right)$.

A quantitative inference schema can be conceptualized as a production rule that operates on a specific quantity. If the condition for applying an operation is satisfied, the respective operation is executed. The quantitative inference schemas are not represented in the abstract manner described above. Rather, depending on their experience with different quantitative domains, people possess specific versions related to particular quantities, like frequencies or volumes. Consequently, quantitative inference schemas are located at Marr’s (1982) algorithmic level. They implement concrete algorithms that work with specific data structures (quantities). This is in line with the observation of McDermott and Larkin (1978) that problem schemas of novices are tied to concrete aspects of a problem. This concludes our exposition of the EPO framework. We, now, turn to applications.

Application of the EPO framework

In the following, we apply the EPO framework in four ways: First, a problem space representation of elementary probabilistic reasoning problems will be presented. Second, we provide a classification of elementary probabilistic reasoning tasks according to the computational requirements for solving the task. Third, the framework is used to explicate why and how different methods for improving Bayesian reasoning work. This comprises a discussion of the effects of individual characteristics, like numeracy, level of education, and cognitive ability, on Bayesian problem solving. Fourth, we employ the approach to reveal the common structure underlying different methods of teaching Bayesian reasoning.

A problem space representation of elementary probabilistic reasoning

The problem space concept is well-suited for explaining mathematical problem solving, like elementary probabilistic reasoning. Newell and Simon (1972) conceptualized problem solving activities as a search through a problem space with a path connecting the initial state, given by the problem description, with the goal state that represents the solution. The problem space comprises five elements (cf. Newell & Simon, 1972, p. 810): (1) a set of knowledge states; (2) a set of cognitive operators that enable one to move between different knowledge states; (3) an initial state; (4) a goal state, and (5) the total knowledge available.

In the case of elementary probabilistic problems, the problem space consists of all sets of probabilities or frequencies that can be computed from the probabilities given in the problem description, making up the initial state, by applying elementary probabilistic operations together with elementary algebraic operations for manipulating equations. The problem can be solved if the goal state, consisting of the required set of probabilities, can be reached from the initial state by applying elementary probabilistic operations.

For other types of problems that can be solved by applying elementary probabilistic operations the states may be made up of relations between probabilities. For example, if the problem consists in proving a statement concerning the relationship between probabilities, like $P\left(\left.x\right|y\right)>P\left(\left.x\right|\overline{y}\right)\iff P\left(\left.y\right|x\right)>P\left(\left.y\right|\overline{x}\right)$ (cf. Tversky & Kahneman, 1982a), the states consist of inequalities between probabilities with the initial state being made up by the left (or right) inequality of the equivalence statement. In this case too, the EPO together with elementary manipulations of equalities and inequalities enables one to reach the goal state, i.e., the right (left) hand side of the equivalence statement.

As with other problems conforming to a problem space representation, like the Tower-of-Hanoi, the different states of the problem space are not all available to the problem solver at the beginning of the problem-solving process. Rather, they are constructed during the process of problem solving by applying elementary probabilistic operations. Moreover, similarly to the Tower-of-Hanoi problem, problem solvers of elementary probabilistic problems may commit illegal moves. These consist in the application of operations other than the EPO. For example, about five percent of the participants in Study 1 of Gigerenzer and Hoffrage (1995) used the multiply-all algorithm: $P\left(e\right)\cdot P\left(h\&e\right)$, and two percent applied the subtraction algorithm: $P\left(\left.e\right|h\right)-P\left(\left.e\right|\overline{h}\right)$, where the symbol h denotes the hypothesis and e the evidence (cf. Gigerenzer & Hoffrage, 1995, Table 3 on p. 695). In general, these operations do not result in proper probabilities. In fact, the subtraction algorithm may even lead to negative values.

In the case of probabilistic reasoning the total possible available knowledge comprises the following elements: (a) elementary knowledge about probabilities, like the fact that probabilities are in the range between zero and one and that the point probabilities of the different values of a random variable sum up to one; (b) knowledge of the different types of probabilities and their relationship, including the different types of events (or sets) the probabilities refer to; (c) general strategies for solving elementary probabilistic problems; (d) quantitative inference schemas and conditions for their application; (e) subjective theorems; and (f) knowledge concerning elementary algebraic operations.³ Which of these knowledge elements actually make up the knowledge of the problem solver depends on her expertise. The knowledge elements available to the problem solver determine, together with the problem description, the representation of the problem, and, by consequence, the path taken through the problem space. However, the given list does not include the knowledge required for setting up the problem space, in particular, the knowledge that is necessary to identify the starting and goal state from the given problem description (see, e.g., Johnson & Tubau, 2015; Kintsch & Greeno, 1985, for a discussion of this aspect).

The sequence of computations in Figure 1D illustrates the strategy of working forward from the given probabilities to the required probability, a mode of problem solving that is typical for experts (Larkin et al., 1980). However, we actually solved the problem in a slightly different way. Figure 2 illustrates the goal hierarchy and the operations applied. The path through the problem space and the setting of sub-goals was determined by four pieces of knowledge: (a) knowledge concerning the definition of the required conditional probability (realizing the operation of conditioning): $P\left(\left.h\right|e_1,e_2\right)=P\left(h,e_1,e_2\right)/P\left(e_1,e_2\right)$, (b) the strategy (goal) to compute the relevant joint probabilities $P\left(h,e_1,e_2\right)$ and $P\left(\overline{h},e_1,e_2\right)$, (c) procedural knowledge of how to compute these quantities from given probabilities by means of combining the relevant pieces of information, and (d) knowledge on how to compute the probability $P\left(e_1,e_2\right)$ from the joint probabilities by means of marginalization. Note that the strategy (goal setting) to compute the joint probabilities was prior to the computation of the marginal joint probabilities $P\left(h,e_1\right)$, $P\left(h,{\overline{e}}_1\right)$, $P\left(\overline{h},e_1\right)$, and $P\left(\overline{h},{\overline{e}}_1\right)$ that are required for computing the relevant joint probabilities. Thus, this is not a working-forward strategy in a strict sense. This example illustrates how knowledge about the probabilities and elementary operations, together with the strategy to compute joint probabilities determine the way through the problem space.

Elementary probabilistic operations and the classification of probabilistic problems

Probabilistic problems can be classified according to the computational requirements for arriving at a solution. The EPO framework provides a computational-level classification of problems by using the set of elementary probabilistic operations that have to be realized for solving a problem as the criterion. This classification imposes restrictions on any solution algorithm in that the latter has to implement the operations required for solving a specific type of probabilistic problem.

In the following, three basic probabilistic problem types will be considered: (1) total evidence problems, (2) conditioning problems, and (3) Bayesian problems. Each of the three problem types has been considered in the literature on probabilistic reasoning, and each of these problem types works with subjective and objective probabilities. We also discuss why the distinction between different types of elementary probabilistic problems is important for psychological research on probabilistic problem solving.

Total evidence problems

Total evidence problems can be solved by applying the elementary operations of combination and marginalization. The conditional probabilities $P\left(\left.X_1,X_2,\dots ,X_n\right|Y_1,Y_2,\dots ,Y_m\right)$ and marginal probabilities $P\left(Y_1,Y_2,\dots ,Y_m\right)$ are provided in the problem description, and the marginal (joint) distribution $P\left(X_1,X_2,\dots ,X_n\right)$ has to be computed. In its simplest form, the target distribution (or target probability) involves a single variable only. Shafer and Tversky (1985) provide a total evidence problem (which they call total evidence design) involving subjective probabilities (for another example of a total evidence problem, involving objective probabilities, see, e.g., Diaz & Batanero, 2009; Item 14). Table 1 exhibits an example with four hypothetical scenarios together with the conditional probabilities of the target event given the different scenario as well as the marginal probabilities of the scenarios. The required probability concerns the target event r = Recovery of the economy from the Asian disease. The four hypothetical scenarios A-D, resulting from the combination of the values of two binary variables Y and Z, are exclusive and exhaustive. The probability of the target event can be found by combining (multiplying) the conditional and marginal probabilities of each scenario resulting in the joint probabilities of scenarios and the target event. The respective joint probabilities are: $P\left(r,A\right)=P\left(r,y,z\right)=0.36$, $P\left(r,B\right)=P\left(r,y,\overline{z}\right)=0.12$, $P\left(r,C\right)=P\left(r,\overline{y},z\right)=0.075$ and $P\left(r,D\right)=P\left(r,\overline{y},\overline{z}\right)=0.0125$. Applying the process of marginalization by summing up these joint probabilities gives the probability of the target event: $P\left(r\right)=0.5675$.

Table 1. Example of a total evidence problem (hypothetical scenarios and associated probabilities): four exclusive and exhaustive scenarios resulting from the combinations of the levels of two binary variables Y and Z.

Scenario (S)	$P\left(\left.r\right|\text{Scenario\hspace{0.33em}}S\right)$	$P\left(\text{Scenario\hspace{0.33em}}S\right)$
A: An effective vaccine is found by the end of this year & strict regulations are installed by the government	$P\left(\left.r\right|y,z\right)=0.90$	$P\left(y,z\right)=0.40$
B: An effective vaccine is found by the end of this year & no strict regulations are installed by government	$P\left(\left.r\right|y,\overline{z}\right)=0.60$	$P\left(y,\overline{z}\right)=0.20$
C: No effective vaccine is found by the end of the year & strict regulations are installed by the government	$P\left(\left.r\right|\overline{y},z\right)=0.50$	$P\left(\overline{y},z\right)=0.15$
D: No effective vaccine is found by the end of the year & no strict regulations are installed by the government	$P\left(\left.r\right|\overline{y},\overline{z}\right)=0.05$	$P\left(\overline{y},\overline{z}\right)=0.25$

r = Recovery of the economy from the effects of the Asian disease.

$\begin{array}{c}P(r)=P(r|y,z)\cdot P(y,z)+P(r|y,\overline{z})\cdot P(y,\overline{z})+P(r|\overline{y},z)\cdot P(\overline{y},z)+P(r|\overline{y},\overline{z})\cdot P(\overline{y},\overline{z})\\ =.9\cdot .4+.6\cdot .2+.5\cdot .15+.05\cdot .25=.5675\end{array}$.

y = An effective vaccine is found by the end of this year.

z = Strict regulations are installed by the government.

Conditioning problems

Conditioning problems require marginalization and conditioning. The given information consists in the joint probability distribution $P\left(X_1,X_2,\dots ,X_n\right)$ and the required distribution is the conditional distribution $P\left(\left.X_1,\dots ,X_k\right|X_{k+1},\dots ,X_n\right)$ of the target variables $X_1,\dots ,X_k$, given the other variables $X_{k+1},\dots ,X_n$. In order to solve this type of problems, the problem solver has to apply the operation of marginalization first by summing over the target variables. Subsequently, the operation of conditioning has to be performed. Typical examples of conditioning problems concern computations within contingency tables. Table 2 exhibits imaginary data concerning the joint distribution of the variables gender (G), extramarital sex (S) and divorce (D). Assume that we want to know the conditional probabilities of a divorce in the case of extramarital sex for men vs. women: $P\left(\left.D=\mathit{\text{yes}}\right|S=\mathit{\text{yes}},G=\mathit{\text{female}}\right)$ and $P\left(\left.D=\mathit{\text{yes}}\right|S=\mathit{\text{yes}},G=\mathit{\text{male}}\right)$. These probabilities can be determined by first computing the marginal probabilities, $P\left(S=\mathit{\text{yes}},G=\mathit{\text{female}}\right)$ and $P\left(S=\mathit{\text{yes}},G=\mathit{\text{male}}\right)$ by marginalizing over variable D (cf. Table 2, last column) The required conditional probabilities are then computed by means of conditionalization, that is, dividing the joint probabilities $P\left(D=\mathit{\text{yes}},S=\mathit{\text{yes}},G=\mathit{\text{female}}\right)$ and $P\left(D=\mathit{\text{yes}},S=\mathit{\text{yes}},G=\mathit{\text{male}}\right)$ by the respective marginal probabilities, for example: $$\begin{array}{c}P\left(D=\mathit{\text{yes}}|S=\mathit{\text{yes}},G=\mathit{\text{female}}\right)=\frac{P\left(D=\mathit{\text{yes}},S=\mathit{\text{yes}},G=\mathit{\text{female}}\right)}{P\left(S=\mathit{\text{yes}},G=\mathit{\text{female}}\right)}\\ =\frac{P\left(D=\mathit{\text{yes}},S=\mathit{\text{yes}},G=\mathit{\text{female}}\right)}{P\left(D=\mathit{\text{yes}},S=\mathit{\text{yes}},G=\mathit{\text{female}}\right)+P\left(D=\mathit{\text{no}},S=\mathit{\text{yes}},G=\mathit{\text{female}}\right)}\end{array}.$$

Table 2. Example of a conditioning problem: Joint distribution of the variables gender, extramarital sex, and divorce (imaginary data).

		Divorce (D)
Gender (G)	Extramarital Sex (S)	yes	no	Σ
female	yes	0.07	0.01	0.08
	no	0.24	0.33	0.57
male	yes	0.02	0.01	0.03
	no	0.14	0.18	0.32

The last term on the right-hand side integrates the operations of marginalization and conditioning into a single mathematical expression. Inserting the respective joint and marginal joint probabilities from Table 2 gives: $P\left(\left.D=\mathit{\text{yes}}\right|S=\mathit{\text{yes}},G=\mathit{\text{female}}\right)=0.07/0.08=7/8$ and $P\left(\left.D=\mathit{\text{yes}}\right|S=\mathit{\text{yes}},G=\mathit{\text{male}}\right)=0.02/0.03=2/3$ .

The class of conditioning problems is one of the most intensely investigated type of probabilistic problems. In experiments on probabilistic reasoning with two binary variables, H (hypothesis) and E (evidence), the joint distribution, that is, the full set of joint point probabilities, $P\left(h,e\right)$, $P\left(\overline{h},e\right)$ $P\left(h,\overline{e}\right)$, and $P\left(\overline{h},\overline{e}\right)$, has been provided in different ways, for example, by means of contingency tables of frequencies or graphical information (see, e.g., Binder et al., 2015; Böcherer-Linder & Eichler, 2019). However, for the most commonly investigated conditioning problem the joint probability information is presented in terms of natural frequencies (Gigerenzer & Hoffrage, 1995; Kleiter, 1994). Figure 3 presents the commonly used general diagnosis problem with two binary variables in probability and natural frequency (NF) format.

Bayesian problems

Bayesian problems are characterized by requiring a complete set of probabilistic operations to obtain a solution. Thus, each Bayesian solution algorithm has to implement each of the three EPO. Prototypical examples are classical Bayesian problems involving two binary variables, like the general diagnosis problem in probability format (cf. Figure 3). In this case, the conditional probabilities (likelihoods), $P\left(\left.e\right|h\right)$ and $P\left(\left.e\right|\overline{h}\right)$, as well as the marginal (prior) probability $P\left(h\right)$ [and, possibly, also $P\left(\overline{h}\right)$] are provided and the problem solver has to compute the inverse conditional (posterior) probability $P\left(\left.h\right|e\right)$. This constitutes the classical Bayesian updating scenario where prior probabilities are updated using the likelihood information.

There exist, however, different variants of Bayesian problems depending on which probabilities are given in the problem description. For example, with two bivariate variables the probabilities $P\left(\left.e\right|h\right)$, $P\left(h\right)$, and the joint probability $P\left(e,\overline{h}\right)$ may be provided (Huerta, 2014). In this case, the solution $P\left(\left.h\right|e\right)$ also requires the execution of each of the elementary probabilistic operators. One important aspect of Bayesian problems consists in the fact that, in the general case, the solution can only be obtained by computing the (relevant) joint probabilities.⁴ Consequently, each path in the problem space from the initial state to the goal state leads through a state comprising the relevant joint probabilities that are required to perform the operations of marginalization and conditioning.

Probabilistic problem structures and natural frequency format

The presented classification regards probabilistic problems in NF format as conditioning problems and not as Bayesian problems since the joint frequencies are presented, making the combination of information needless. There has been a debate of whether solving Bayesian problems with probabilistic information presented in NF format can be regarded as Bayesian reasoning. Howson and Urbach (1993) argued that this is not the case because the usage of natural frequencies makes probabilistic reasoning simpler. On the other hand, it has been argued that the essence of Bayesian reasoning consists in the fact that the probabilistic reasoning process achieves the same function as the application of Bayes theorem (Brase, 2002). Furthermore, Brase and Hill (2015) claim that considering Bayesian reasoning as an application of the Bayes theorem is scientifically unproductive, and that their more general specification of Bayesian reasoning as a process of adaptively updating prior probabilities in the face of new information to arrive at a posterior probability, is more useful from a cognitive perspective.

From the perspective of elementary probabilistic operations, two related aspects have to be considered, a normative and a psychological one. The first one concerns the computational requirements of algorithms implementing the probabilistic operations. As noted above, classical Bayesian updating conceptualizes Bayesian reasoning as the revision of prior belief by incorporating new information. An important step consists in the combination of the prior beliefs with the new evidence. In the case of information presented in NF format this combination is performed in the problem description and not by the solution algorithm. Consequently, it is not sensible to classify solution algorithms for problems with information presented in NF format as Bayesian.

Concerning the psychological point of view, it is important to differentiate between Bayesian and conditioning problems because it is psychologically relevant which mental operations have to be performed in order to arrive at a correct solution (see also Ayal & Beyth-Marom, 2014). As detailed below (cf. the section Simplification of the Problem Structure: Reducing Bayesian Problems to Conditioning Problems), problems in NF format require less conceptual and procedural competencies. Consequently, Bayesian problems in NF format should not be considered as Bayesian problems at all.

Bayesian problems and subjective theorems

For most Bayesian problems, like the general diagnosis problem in standard probability format, the solution requires the execution of EPO. By contrast, for the famous Monty Hall problem (vos Savant, 1990) as well as the problem of the three prisoners (Mosteller, 1965), and structurally equivalent problems, it is possible to arrive at a correct solution by applying subjective theorems, i.e., simple inferential schemas that can be used for solving the problem (Falk, 1992; Shimojo & Ichikawa, 1989). This is possible because the probabilities involved are quite simple: The prior distribution is uniform and the conditional probabilities are either 0, 1, or 1/2. By consequence, employing a subjective theorem may result in a correct solution without applying elementary probabilistic operations.

The crux of this type of problems consists in the fact that it evokes subjective theorems that lead to an incorrect solution. Specifically, the subjective theorem constant ratio is evoked (Shimojo & Ichikawa, 1989, p. 7): If one alternative (out of three) is eliminated the ratio of probabilities of the remaining alternatives is the same as the ratio of the prior probabilities for them. This results in the incorrect answer of 1/2 instead of 1/3, assuming that the host in the Monty Hall problem or the jailer in the problem of three prisoners has no preference for one of the alternatives (see also Baratgin & Politzer, 2010).

By contrast, the subjective theorem irrelevant therefore invariant (cf. Shimojo & Ichikawa, 1989, p. 7) that demands the problem solver to ignore the conditioning event results in a correct solution. Thus, Krauss and Wang (2003) propose to teach the application of this subjective theorem. The fact that the subjective theorem irrelevant therefore invariant (as well as other simple procedures, like repeated playing of the game and taking the view of the different members in the game (Tubau & Alonso, 2003; Tubau et al., 2015)) leads to a correct solution seems to contradict our previous claim that, in general, a correct solution of Bayesian problems requires the computation of joint probabilities. However, that this is not the case is revealed by the fact that simple modifications of these problems lead to a failure of the irrelevant therefore invariant theorem. Assume for the moment, that in the Monty Hall problem the probabilities of the three Doors A, B, and C, locating the car, are 1/4, 1/4, and 1/2 (instead of 1/3, for each door) then the subjective theorem does no longer work since the correct posterior of the car being located behind the chosen Door A, given the host opens Door C, is not 1/4, as proposed by the theorem, but 1/3. Similarly, if the host in the Monty Hall problem has a preference to open a specific door (i.e., he will always open this door if possible) then the theorem does not work. Thus, in general, the computation of joint probabilities, and, by consequence, the application of the operation of combination, is mandatory for solving Bayesian problems.

On the relative difficulty of problem classes

As explicated above, Bayesian problems demand the computation of the joint probabilities (frequencies). By consequence, Bayesian problems may, in general, be considered as more difficult than conditioning problems where the joint probabilities are provided. The facilitating effect of presenting probabilistic information in terms of joint frequencies or in NF format seems to support this conclusion.

However, the difficulty of an elementary probabilistic problem does not only depend on the specific types of elementary probabilistic operations involved but also on how many operations are required and how familiar participants are in performing an elementary probabilistic operation in a specific way. Consequently, presenting problems that require additional elementary probabilistic operations can render conditioning problems more complex than Bayesian problems (see, e.g., Ayal & Beyth-Marom, 2014, and Study 5 of Girotto & Gonzalez, 2001). In sum, the different problem types do not allow for a strict ordering with respect to problem difficulty. Following to this computational analysis of probabilistic problems, we, next, apply the EPO framework to explicate why and how specific methods for improving Bayesian problem-solving work.

Elementary probabilistic operators and methods for improving Bayesian reasoning

Due to the difficulty of Bayesian problems for people without training there have been numerous efforts to improve Bayesian reasoning using different methods. The EPO framework provides new explanations of why and how these methods work. We discuss five attempts to improve Bayesian reasoning: (1) reduction of Bayesian problems to conditioning problems by using NF, (2) problem descriptions fostering the combination of information, (3) methods that facilitate the operations of marginalization and conditioning, (4) the effect of the format of the probabilistic information: frequencies vs. probabilities, and (5) individual characteristics, like numeracy, level of education, and cognitive ability that may result in improved probabilistic reasoning.

Preliminary considerations: types of competencies in mathematical problem solving and quantitative inference schemas

Greeno et al. (1984) identify three types of competencies that are important in mathematical problem solving: conceptual, procedural and utilizational. These competencies are not independent but may develop in a parallel way such that increasing one type of competence leads to an improvement of the other types of competencies (see, e.g., Rittle-Johnson et al., 2001). With respect to elementary probabilistic problem solving, conceptual competence concerns knowledge about different types of probabilities and their relationship, as well as of strategies. Procedural knowledge is concerned with the probabilistic operations and their concrete implementation. Finally, utilizational competence consists in knowledge concerning conditions for the proper application of specific realizations of EPO, in particular of the quantitative inference schemas. According to the EPO framework, successful methods for improving Bayesian reasoning can be classified into two non-exclusive categories: (a) simplification of Bayesian problems such that people with lower conceptual and procedural competencies are able to solve the problem, and (b) methods improving the utilization competence resulting in an increased application of quantitative inference schemas.

Before discussing previous findings, we illustrate how the problem formulation and the probability format may result in an increased use of quantitative inference schemas. Consider the two problem descriptions in Figure 4. Both descriptions are Bayesian problems since, in both cases, the solution requires the application of all three elementary operations. The EPO framework predicts that the first problem description results in a higher solution rate than the second one. This is due to the fact that the first description increases the utilization competence by fostering the application of the proper quantitative inference schemas. This is based on the interplay of four aspects: First, the usage of frequencies: The sample size is given in the first but not in the second description, increasing the availability of acquired quantitative schemas working with frequencies. In addition, the numbers are simple, and, thus, the frequencies are easy to compute. Second, the first problem is framed as a selection problem, that is, subsets are selected from the sets of red and green apples. This should foster the application of the quantitative selection schema since people know how to compute the frequency of the entities within a subset selected from the whole set. By contrast, with diagnostic framing a diagnostic feature is introduced according to which items are selected. This renders the selection of the relevant subsets less transparent. In addition, diagnostic framing increases the likelihood of confusing the sensitivity $P\left(\left.e\right|h\right)$ with the positive predictive value $P\left(\left.h\right|e\right)$. This error is frequently committed in the case of diagnostic framing with participants revealing high confidence of being correct (Lakhlifi et al., 2023). Third, the fact that the selected apples are put into the same basket encourages the focusing on these apples in the basket. Fourth, the question in the first description asks for a proportion and not for a probability. The latter two aspects promote the application of the quantitative proportion schema on the apples in the basket.

Following to this illustration of how the problem description and probability format can improve Bayesian reasoning by fostering the application of quantitative inference schemas, we discuss results concerning different attempts to enhance Bayesian problem solving. In particular, we show how the different methods foster the application of quantitative inference schemas that implement the relevant EPO. We start with an approach that simplifies the problem structure thus reducing the demand on conceptual and procedural competencies.

Simplification of the problem structure: reducing Bayesian problems to conditioning problems

One method to improve Bayesian reasoning consists in presenting joint probabilities. This reduces Bayesian problems to conditioning problems since no combination of information is required with this information structure. The most frequently used method for providing joint probability information consists in presenting the probabilistic information in NF format (see Figure 3). In this case, the joint frequencies and the embedding of the joint events, $e\&h$ and $e\&\overline{h}$, within the marginal ones, $h$ and $\overline{h}$, are presented. However, the information concerning the embedding of events, revealed by NF format, is irrelevant for solving the problem. A different type of nesting of events is important for performing the operations of marginalization and conditioning (cf. the section Methods Facilitating Marginalization and Conditioning).

The presentation of joint frequencies reduces the demand on procedural competence since the solution algorithms are simplified (see, e.g., Gigerenzer & Hoffrage, 1995, 2007). This is due to the fact that the presentation of the relevant joint probabilities, $P\left(e\&h\right)$ and $P\left(e\&\overline{h}\right)$, renders the combination of probabilistic information superfluous. Only the operation of marginalization, $P\left(e\right)=P\left(e\&h\right)+P\left(e\&\overline{h}\right)$, and conditioning, $P\left(\left.h\right|e\right)=P\left(e\&h\right)/P\left(e\right)$, have to be performed. The NF format also reduces the demand on the conceptual competence since knowledge concerning the importance of joint probabilities for solving Bayesian problems is not required: The problem solver does not need to know that relevant joint probabilities have to be computed for solving Bayesian problems.

Improving Bayesian problem solving by fostering the combination of information

The issue of improving participants’ performance on Bayesian problems by fostering the application of the quantitative partitioning/selection schema has, to the best of our knowledge, not been addressed so far. There are, however, studies that may be regarded as instances of promoting the application of the quantitative partitioning/selection schema. In particular, in their first study Johnson and Tubau (2013) provided a Bayesian problem description similar to the example, given above (cf. Figure 4). Specifically, they presented a problem with a selection frame in a frequency and probability format. With frequency format, the frequencies of basic classes (three colors of apples), allowing for simple computations, were provided and the question asked for frequencies. With probability format the probabilities of the three basic classes were presented and the question asked for a probability. The solution rates were 86% with frequency and 27% with probability format (cf. Johnson & Tubau, 2013, Table 1, p. 36). This result indicates that, in most cases, the presentation of problems as selection problems is not sufficient to foster a problem schema that permits a correct solution. Note however, that the solution rate of 27% with probability format is still about as high as the 24% found with problems in natural sampling format (McDowell & Jacobs, 2017). By contrast, presenting, in addition, the problem in frequency format (with simple numbers) resulted in an increase in the solution rate of over 50% (compared to probability format) indicating the fostering of adequate problem schemas, including the quantitative selection schema. However, further research is required to understand precisely the conditions, under which the presentation of Bayesian problems in a selection/partitioning frame promotes the application of the partitioning/selection schema.

Methods facilitating marginalization and conditioning

Despite the fact that the presentation of joint probabilities increases the rate of correct solutions the improvements are far from being perfect. This is not surprising since the problem solution requires the application of the operations of marginalization and conditioning. The various attempts to foster these processes are of two types: (a) Guiding participants through the process of marginalization and conditioning by posing additional questions, and (b) increasing the saliency of the relevant nested set structure.

The method of asking participants to compute the marginal probability $P\left(e\right)$ of the conditioning event e has been applied by Girotto and Gonzalez (2001) as well as by Ottley et al. (2016). After having presented the joint probabilities $P\left(e\&h\right)$ and $P\left(e\&\overline{h}\right)$ in terms of natural frequencies, Girotto and Gonzalez (2001) asked participants to first compute the marginal probability $P\left(e\right)$ and then the conditional probability $P\left(\left.h\right|e\right)$ by presenting the following question

In a group of 100 people, one can expect ____ individuals to have a positive reaction, ____ of whom will have the infection.

This forces participants to first perform the operation of marginalization followed by the operation of conditioning. In particular, asking participants to compute the marginal frequency prompts participants to apply the quantitative concatenation schema. The second part of the question promotes the application of the quantitative conditioning schema implementing the operation of conditioning. The same strategy of first asking for the relevant marginal probability has been applied by Ottley et al. (2016, p. 532). The results of both studies reveal a positive effect of guiding participants through the processes of marginalization and conditioning by means of questioning.

The second approach to promote marginalization and conditioning consists in making the relevant nested set structure more salient. As noted above, the nested set relationships revealed by the NF format is irrelevant for solving the problem since only the joint probabilities are required. The nesting of sets revealed by the NF format does not provide any indication concerning the further path to the solution state. By contrast, the nested set relationship between the marginal event e and the joint events $e\&h$ and $e\&\overline{h}$ is important for performing the operation of marginalization and conditioning. Consequently, various methods have been applied to enhance the saliency of the relevant nesting. These methods may be divided into two types: First, enhancing the nested set relationship by using a problem description that make the embedding salient, and second using visual aids.

The first approach was taken by Brase et al. (1998). They increased the transparency of the nesting of sets by means of specific problem descriptions. For example, in Experiment 2 in the non-salient condition the probability $P\left(e\right)$ represented the number of candy cane halves, whereas $P\left(e\&h\right)$ and $P\left(e\&\overline{h}\right)$ represented, respectively, the number of candy cane halves that taste peppermint and lemon. In the salient condition, $P\left(e\right)$ referred to the number of candy canes and the joint probabilities, $P\left(e\&h\right)$ and $P\left(e\&\overline{h}\right)$, represented the number of peppermint and lemon candy canes. Obviously, the salient condition renders the nesting relationship more transparent. This method and other ones used by Brase et al. (1998) to increase the saliency of the nested set relationships turned out as being successful. According to the EPO framework, increasing the saliency of the nested set structure fosters the application of the quantitative proportion schema (the combined marginalization-and-conditioning schema): To get the proportion of peppermint candy canes, the number of peppermint candy canes has to be divided by the total number of candy canes. With candy cane halves the nested set relationship is less salient, thus resulting in a reduced application of the schema. In their third experiment, the candy canes of the same flavour were put in the same jar thus prompting participants to focus on the respective jar. As noted in the example above, this may foster the application of the marginalization and proportion schema, specifically in case of employing a frequency format.⁵

A second method to increase the saliency of the nested set relationship relevant for performing the operation of conditioning consisted in using Venn diagrams⁶. In particular, in the diagrammatic representation of Sloman et al. (2003, Figure 1 on p. 298) the event $e\&h$ is represented by a circle that is contained within a bigger circle representing event e, thus depicting the nesting of the first set within the second one (see also Barbey & Sloman, 2007; Figure 1 on p. 249). Whereas Sloman et al. (2003) found a facilitating effect of using Venn diagrams, Brase (2009) and Moro et al. (2011) found no such effect. The differing results of Sloman et al. (2003) and Brase (2009) may be due to the different forms of the Venn diagrams: The Venn diagrams of Brase (2009) comprises two overlapping circles, one depicting the sub-population of people having the disease, i.e. representing event h, and the second circle representing the sub-population of persons with a positive diagnosis, i.e. representing event e (cf. Brase, 2009, Figure 1, on p. 374). Consequently, the embedding of event $e\&h$ within event e is less transparent in the diagram of Brase (2009) compared to that of Sloman et al. (2003).

The divergent result of Moro et al. (2011) cannot be explained by differing diagrammatic representations of the sets involved (cf. Moro et al. 2011, Figure 3). The major difference to the study of Sloman et al. (2003) concerns the expertise of the participants: The solution rates in Sloman et al. (2003) were considerably higher than those of Moro et al. (2011) for exactly the same problem formulations. For example, in Exp. 1 of Sloman et al. (2003) the solution rates were 20% for Bayesian problems in probability (P) format, and 48% for Bayesian problems in P format with an additional text that should make the nesting of the sets transparent. The respective solution rates in Moro et al. (2011, Exp. 1) were 0% for both versions. Similarly, the solution rate for the problem in P format with an accompanying Venn-diagram was 48% in Exp. 2 of Sloman et al. (2003) and 0% in Exp.2 of Moro et al. (2011). This suggests that the facilitating effect of using Venn-diagrams is moderated by participants’ expertise. A similar result was observed concerning the effect of participants’ numeracy on solution rates (cf. the section Numeracy, Level of Education, Cognitive Ability, and Bayesian Reasoning).

Taken together, we discussed three attempts to foster the operation of marginalization and conditioning in the case of conditioning problems: (a) guiding participants through the operations of marginalization and conditioning, (b) using problem descriptions that make the nested-set relationship more salient, and (c) employing Venn-diagrams depicting the relationships between different sets. The first two methods turned out as being successful. The facilitating effects of Venn-diagrams seems to be moderated by participants’ expertise with Bayesian problems.

Improved probabilistic problem solving with frequencies

A current debate concerns the issue of whether presenting probabilistic information in terms of frequencies instead of probabilities results in better reasoning performance. The debate comprises two aspects: The first one is concerned with the concrete wording, specifically, whether the usage of the term frequency results in higher correct solutions than the usage of the term chances (see, Brase, 2008, 2014; Girotto & Gonzalez, 2001; Sirota, Kostovičová, et al., 2015).

The second aspect that is (more) relevant with respect to the EPO framework concerns the issue of whether the presentation of frequencies results in better performance than the usage of probabilities. Despite the fact that some studies found no facilitating effect of presenting absolute frequencies (e.g., Fiedler et al., 2000; Macchi, 2000), there is overwhelming evidence of the beneficial effects of absolute frequencies, even if these are not natural frequencies (see, e.g., Ayal & Beyth-Marom, 2014; Cosmides & Tooby, 1996; Johnson & Tubau, 2013). In addition, teaching probabilistic reasoning using frequencies is more efficient than using probabilities (see the discussion below).

The EPO approach explains the beneficial effects of employing frequencies instead of probabilities by the higher capacity of the former to foster the applications of quantitative inference schemas. This should also be the case with absolute frequencies that are not joint frequencies (see the example presented above) since people have learned the application of these problem schemas with frequencies. This explains why the difference between single event probabilities and frequencies is of psychological relevance in the case of probabilistic reasoning (Gigerenzer, 1994): The efficiency of cognitive algorithms, i.e., quantitative inference schemas, depends on the underlying representation of the probabilistic information.

Numeracy, level of education, cognitive ability, and Bayesian reasoning

Several studies examined the association between numeracy, level of education, and cognitive ability, on the one hand, and Bayesian reasoning with different probability formats, on the other hand. The EPO framework, together with the assumption that people with no training do not understand the importance of the joint distribution for solving Bayesian problems, predicts low solution rates for abstract unfamiliar Bayesian problems in probability (P) format, like the two-variable diagnosis problem of Figure 3. Furthermore, since the scales for measuring numeracy (Lipkus et al., 2001; Schwartz et al., 1997) do not measure this type of conceptual knowledge, we expect that numeracy exerts no or only a small effect on solution rates for Bayesian problems in P format. For the same reasons we also assume no or only small effects of level of education and cognitive ability on solution rates.

By contrast, for problems in NF format or simpler and more familiar problems we expect an effect of numeracy, level of education, and cognitive ability, for different reasons: with NF format conceptual knowledge concerning the joint distribution is not required, and for simpler and more familiar problems, there is an increased likelihood that the problem description elicit quantitative inference schemas. This tendency to apply quantitative knowledge is expected to be higher for people high on numeracy, with a higher level of education, and with higher cognitive ability.

These predictions are supported by several studies. First, various studies reveal low performance for Bayesian problem in P format. For example, solution rates of about 2% were obtained by Chapman and Liu (2009). Likewise, Siegrist and Keller (2011) observed one correct solution out of 134 in Exp. 1, and one out of 71 in Exp. 2. Hill and Brase (2012, Exp. 3) and Moro et al. (2011, Exp. 1) found floor effects with zero correct solutions for diagnostic Bayesian problems in P format. Second, a number of studies found no increase in the solution rates with numeracy for problems in P format (Bramwell et al., 2006; Brase & Hill, 2017, Exp. 1; Chapman & Liu, 2009; Hill & Brase, 2012, Exp. 2 and 3, medical problem; Johnson & Tubau, 2013, complex problem formulation). Similarly, Siegrist and Keller (2011, Exp. 1 and 2) observed no increase with level of education for unfamiliar problems, and Stanovich and West (2000) found no effects of cognitive abilities for the diagnosis problem of Casscells et al. (1978) and for the cab problem (Tversky & Kahneman, 1982b). Third, for simpler and more familiar problems or problems in NF format facilitating effects of numeracy (Brase & Hill, 2017; Chapman & Liu, 2009; Garcia-Retamero & Hoffrage, 2013; Hill & Brase, 2012, Exp. 3; Johnson & Tubau, 2013, Exp. 1, simple problem formulation), and education (Bramwell et al., 2006; Siegrist & Keller, 2011, Exp. 1 and 3) have been observed.

However, the results of some studies are in opposition to this general pattern of results. For example, Brase and Hill (2017, Exp. 2) and Garcia-Retamero and Hoffrage (2013) found an effect of numeracy also for problems in P format, whereas Micallef et al. (2012) found no relationship between numeracy and errors for problems in NF format. Moreover, Sirota et al. (2014) observed significant correlations between solution rates for problems in P format and various measures of cognitive ability.

The observed discrepancies may be due to a potential confounding between numeracy, level of education, and cognitive ability on the one hand, and the degree of experience with Bayesian problems, on the other hand. This might also explain the fact that the effects of numeracy, level of education, and cognitive abilities are generally associated with an overall higher performance on Bayesian reasoning problems with P and NF format. To illustrate, in the study of Garcia-Retamero and Hoffrage (2013) professionals (physicians) with higher numeracy revealed higher solution rates also for problems in P format. By contrast, professionals (obstetricians) in the study of Bramwell, et al. (2006) did not perform better than their clients for problems in P format. Notably, the solution rates of the professionals for problems in P format differed substantially between both studies: about 20% in the study of Garcia-Retamero and Hoffrage (2013) and 5% in the study of Bramwell et al. (2006). This indicates a greater experience with Bayesian reasoning problems of the physicians compared to the obstetricians. In fact, the obstetricians of Bramwell et al. (2006) performed about equally well as the patients of Garcia-Retamero and Hoffrage (2013).

This sort of confounding might also explain conflicting results concerning facilitating effects of cognitive abilities for Bayesian problems in P format. Since the time the studies of Stanovich and West (2000) have been conducted, Bayesian reasoning problems have found increased access to curricula in High Schools (see, e.g., Chernoff & Sriraman, 2014; Gage & Spiegelhalter, 2018; Wassner et al. 2004; Weber et al., 2018), and receiving some training in Bayesian reasoning has certainly a greater impact for high ability participants.

In sum, the EPO framework can predict a great number of findings concerning the effect of numeracy, level of education, and ability on Bayesian problem solving with different probabilistic formats. Conflicting results could be due to confounding between numeracy, level of education, and cognitive abilities, on the one hand, and expertise with Bayesian problem solving, on the other hand.

Teaching Bayesian problem solving

According to the present view, except for specifically designed problems like those used in the first study of Johnson and Tubau (2013), people with no training neither have the conceptual nor the procedural and utilizational competence to solve Bayesian problems. This means that they neither have a full understanding of the importance of joint probabilities for solving Bayesian problems, nor are they in possession of detailed knowledge concerning the EPO and their proper application (for a similar view, see Girotto & Pighin, 2015).

There have been various attempts to teach Bayesian reasoning. These endeavors focus on teaching how to combine probabilistic information. This corroborates our claim that combining probabilistic information constitutes a central aspect of Bayesian problem solving. In particular, all of these methods teach algorithms implementing the partitioning/selection schema with different types of quantities. The different algorithms and quantities comprise: (a) teaching natural sampling, that is, instructing people how to translate probabilities into natural frequencies (Kurzenhäuser & Hoffrage, 2002), (b) constructing frequency and probability trees (see, e.g., Gage & Spiegelhalter, 2018; Sedlmeier & Gigerenzer, 2001), (c) a combination of natural sampling and frequency trees (Hoffrage et al., 2015), (d) filling in frequency grids (Sedlmeier & Gigerenzer, 2001), (e) filling contingency tables with frequencies (Talboy & Schneider, 2017), and (f) partitioning of a square resulting in a mosaic plot with frequencies as labels (Talboy & Schneider, 2017).

Concerning the relative efficiency of different methods, the following results were obtained: Frequencies result in superior performance and memory compared to probabilities (Sedlmeier & Gigerenzer, 2001). Similarly, filling in contingency tables with frequencies leads, in general, to a higher transfer performance than creating mosaic plots (Talboy & Schneider, 2017). Finally, presenting natural frequencies and instructing people how to use them to construct a frequency tree, and get the solution from the latter leads to better performance than teaching natural sampling (Hoffrage et al., 2015). These results confirm the superiority of frequencies as quantities on which to perform Bayesian problem solving.

General discussion

The presented EPO framework characterizes elementary probabilistic problems as problems involving a finite sampling space that can be solved by applying elementary probabilistic operations: combination, marginalization, and conditioning. Consequently, the problem space underlying elementary probabilistic problems, with the states made up of sets of probabilities, is closed with respect to the proper application of the three elementary probabilistic operations. Importantly, the three elementary operations also work with quantities other than probabilities, and they can be implemented by means of different algorithms. Therefore, the problem states may be made up of sets of quantities other than probabilities. The given characterization of elementary probabilistic problems differentiates them from problems that test for statistical intuitions (see, e.g., Nisbett, et al., 1983) as well as from statistical problems requiring the computation of statistics or parameters (see, e.g., Catrambone, 1998; Falk & Lann, 2008). In addition, elementary probabilistic problems differ from more complicated probabilistic problems, like combinatorial ones that require difficult counting procedures, problems demanding more or less sophisticated set operations, or problems requiring limit computations. In the following, we first discuss how explanations of the EPO framework concerning the effects of methods for improving probabilistic reasoning differ from traditional explanations. This is followed by a discussion of new applications of the EPO approach.

Comparing explanations based on the EPO framework to traditional explanations

According to the EPO framework, methods for improving probabilistic reasoning are of two different types: (a) simplification of problems (reducing Bayesian problems to conditioning problems) and (b) increasing utilization competence by promoting the application of acquired problem schemas. Traditional explanations are concerned primarily with facilitating effects using natural frequencies (NF). Three types of explanations have been put forward: (a) the reduction of computational complexity, (b) the ecological rationality (ER) account and (c) the nested-set (N-S) explanation. The reduction of computational complexity account states that for problems in NF format computations are simplified thus increasing the solution rates (see, e.g., Gigerenzer & Hoffrage, 1995, 2007). This account is in line with the EPO framework. However, the later provides a precise explanation of why this simplification occurs: with NF, the joint frequencies are presented in the problem formulation, and, consequently, the operation of combination need not be performed. Additionally to the lower demand on procedural knowledge, the EPO account identifies a lower requirement with respect to conceptual competencies since knowledge concerning the importance of the joint distribution for solving Bayesian problems is not required with NF format.

The ER account claims that people are able to perform Bayesian computations if provided with the correct type of probabilistic information. Natural frequencies provide this type of information. This explanation is based on evolutionary arguments that claim that our ancestors encountered probabilistic information in natural frequency format. Consequently, inferential algorithms are tuned to this format (see, e.g., Cosmides & Tooby, 1996; Gigerenzer & Hoffrage, 1995). The ER explanation differs from the EPO account in two respects: First, according to the EPO framework, problems presented in the NF format are not Bayesian problems at all. Consequently, increased solution rates are no indication of people being able to perform Bayesian reasoning. Second, using the principles of evolutionary educational psychology (e.g., Geary, 2002; Tricot & Sweller, 2014), quantitative inference schemas do not constitute biological primary knowledge. This means that they are the product of cultural learning and not acquired automatically.

The N-S explanation claims that the NF format reveals the set structure of the problem and makes the relationship between the nested sets salient. This fosters the understanding of the problem (see, e.g., Barbey & Sloman, 2007; Evans et al., 2000; Mellers & McGraw, 1999; Sirota, Kostovičová, et al., 2015; Sloman et al., 2003). According to the EPO account, the nesting of sets made salient by the NF format ($e\&h$ within event h as well as that of $e\&\overline{h}$ within $\overline{h}$) is not relevant for solving the problem as soon as the joint probabilities, $P\left(e\&h\right)$ and $P\left(e\&\overline{h}\right)$, are provided. Thus, the nested set relationship exhibited by NF format is of no help for solving the problem. The only nesting that is relevant for solving problems with NF concerns the embedding of the joint events $e\&h$ and $e\&\overline{h}$ within the event e (see the discussion in the section Methods Facilitating Marginalization and Conditioning). However, this type of embedding is not revealed by the NF format.

According to a dual-process version of the N-S account, presenting probabilistic information in NF format results in an activation of the rule-based system leading people to process information of the nested set structure (Barbey & Sloman, 2007).⁷ The EPO approach accords with the account of Barbey and Sloman (2007) in one respect: The former claims that specific problem descriptions can foster the application of quantitative inference schemas which are part of people’s acquired rule-based reasoning system. This aspect is, however, of less importance with respect to facilitating effects of the NF format, compared to the accompanying conceptual and procedural simplifications. This is evidenced by the fact that facilitating effects were also observed by presenting joint probabilities instead of natural frequencies (see, e.g., Fiedler et al., 2000; Macchi, 2000; Mellers & McGraw, 1999). In particular, Ottley et al. (2016) found no difference between a problem formulation presenting the complete probabilistic information (i.e., all marginal and joint frequencies) and a structured problem version text that makes the embedding of different sets more salient. The respective solution rates were about 73% and 77% for the unstructured and structured version, respectively. Finally, Moro et al. (2011, Exp. 3) investigated people’s knowledge concerning the relationship between different sets involved in Bayesian problems. Their results indicate that, in general, people had a good understanding of the relationship between the sets involved in the problems. However, this had no effect on the solution rates. Thus, the facilitating effect of the NF format is primarily due to the presentation of the (relevant) joint probabilities and not due to the nesting information.

Further applications of the EPO framework

In this section we discuss three new applications of the EPO framework: Usage of analogies in probabilistic problem solving, teaching probabilistic reasoning, and error analysis.

Analogical probabilistic problem solving

The EPO framework views probabilistic problem solving as analogous to solving other quantitative inference problems that require the application of quantitative operations. According to this conception, probabilities or frequencies are, like length, area, volume, and weight, but specific instances of a general quantity construct. Probabilities differ from other types of quantities only by the fact that the maximum possible value is one.⁸ The fact that quantitative inference schemas are applicable to different types of quantities, suggests that, similar to other types of problems, analogies may be helpful for solving probabilistic problems (e.g., Gick & Holyoak, 1980, 1983; Holyoak & Koh, 1987). Specifically, presenting analogical problems involving quantities other than probabilities may help people to solve probabilistic problems. Figure 5 exhibits an example of an analogical problem that involves volume as the target quantity and requires the application of quantitative inference schemas. Solving analogical quantitative reasoning problems should increase the availability of the respective quantitative inference schemas, and, by a consequence, promote their utilization in subsequent probabilistic reasoning tasks. This requires that people are able to identify the relevant structural similarities between the quantitative source problem and the probabilistic target problem. Quantitative analogies may be most helpful for Bayesian problems that are framed as selection problems rather than diagnostic problems (cf. Figure 4). Future research may investigate the facilitating effect of analogies in probabilistic reasoning.

Figure 5. Quantitative analogy problem.

Teaching probabilistic reasoning

The EPO approach may be helpful in teaching elementary probabilistic reasoning (for various approaches to teaching probabilistic reasoning, see, e.g., Gage & Spiegelhalter, 2018; Chernoff & Sriraman, 2014). This is due to the fact that the EPO approach is basically a computational level theory, focusing on the computational requirements for solving a specific type of probabilistic problem. In this way, it differs from traditional approaches that are located on the algorithmic level, focusing on specific procedures.

In particular, two aspects of the EPO approach seem useful in teaching probabilistic reasoning. First, the conception of probabilities as specific types of quantities subject to the same elementary operations. Current approaches focus primarily on frequencies (e.g., Gage & Spiegelhalter, 2018). However, frequencies have their limitations, like the requirement of huge sample sizes in the case of small probabilities or difficulties associated with incompatible sample sizes (cf., Ayal & Beyth-Marom, 2014, Study 3). For other quantities that are not based on whole numbers the problem of the hugeness of numbers does not appear. In addition, the relationship between probabilities and other quantities explicates the usefulness of various graphical displays and the working of sampling devices, like Roulette wheels and spinners (see Gage & Spiegelhalter, 2018).

The second useful aspect of the EPO framework for teaching probabilities consists in the fact that as a computational level framework, focusing on the computational requirements for solving a specific type of probabilistic problem, the approach focusses on general solution strategies and the respective sub-goals, rather than on specific solution procedures (cf., Novick, 1990). Catrambone and Holyoak (1990) demonstrated that it is more effective, with respect to transfer, to have problem-solving knowledge represented by relatively small sub-goals and methods. The EPO framework addresses this aspect. For example, in Bayesian reasoning the three relevant sub-goals are (in this order): (a) computing the relevant joint probabilities, (b) calculating the probability of the conditioning event, and (c) performing the operation of conditioning (cf. Figure 2). Consequently, the EPO approach allows for heuristic training (Schoenfeld, 1979) by focusing on relevant sub-goals. This should result in more profound and transferable knowledge that enables students to solve more difficult probabilistic problems, like proving the validity of the independence axioms of probability (Dawid, 1979).

Error analysis

Another application of the EPO framework concerns the analysis of errors committed in probabilistic problem solving. The error analysis based on the EPO framework is similar to that of Diaz and Batanero (2009) and Eichler et al. (2020). These studies analyzed the errors in achieving specific sub-tasks that are required to get a final solution.

With respect to error analysis, the differentiation between conceptual, procedural, and utilization competence (Greeno et al., 1984) is useful. In particular, errors may be committed due to a lack of conceptual competence, like missing knowledge about different types of probabilities and their relevance or the inability to identify relevant sub-goals. Moreover, there may be an insufficient understanding of relevant procedures and their application. Participants may be able to identify relevant sub-goals in new problem contexts but fail to apply the appropriate method (Catrambone & Holyoak, 1990). Concerning elementary probabilistic problems, people may, for instance, understand that they have to combine the given probabilistic information. They may, however, employ inappropriate procedures, like the multiply-all algorithm, $P\left(e\right)\cdot P\left(h\&e\right)$, or wrong algebraic operation (e.g., addition instead of multiplication). In this way, the EPO approach constitutes a framework for identifying missing competencies required for solving a probabilistic problem.

Conclusion

The framework of elementary probabilistic operations (EPO) enables a new way of analyzing elementary probabilistic problems and provides new explanations of the functioning of different methods for improving probabilistic reasoning and the working of training procedures. In particular, as a computational level theory that exhibits the computational requirements underlying elementary probabilistic problems, the EPO framework (a) provides a precise specification of the problem space underlying elementary probabilistic reasoning, (b) enables a classification of elementary probabilistic problems according to computational requirements that each solution algorithm has to fulfill, (c) reveals the relationship between elementary probabilistic problems to problems involving other quantities, (d) exhibits that methods to improve Bayesian reasoning are of two types: first, problem descriptions resulting in a reduction of computational and conceptual requirements, and, second, improving the utilization competence by fostering the application of acquired quantitative inference schemas; (e) explains that methods for training Bayesian inference focus on the combination of information using quantities other than probabilities thus capitalizing on people’s knowledge of different types of quantities. Finally, the EPO framework turns out to be helpful in teaching probabilistic reasoning, reveals how to employ analogical reasoning for solving probabilistic problems, and provides a guide for analyzing errors.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Gage and Spiegelhalter (2018) differentiate between frequency and expected frequency. The former denotes frequencies resulting from random experiments, whereas the latter refers to frequencies based on the "true" probabilities. In line with the usage in the psychological literature, we use the term frequency to refer to expected frequencies.

2 Additivity is, in general, not guaranteed for quantities like lengths and volumes (e.g., Cohn, 1980). However, assuming that there are at most countably many objects to which quantities can be assigned (which is the case for real existing things), additivity holds.

3 Algebraic manipulation of equations are required for specific problems. For example, a problem may demand the computation of $P\left(x,\overline{y}\right)$, given the probabilities $P\left(x\right)$ and $P\left(x,y\right)$ (see, e.g., Ayal & Beyth-Marom, 2014; Girotto & Gonzalez, 2001; Huerta, 2014). This can be achieved by using the equation underlying the operation of marginalization: $P\left(x\right)=P\left(x,y\right)+P\left(x,\overline{y}\right)$ and subtracting the term $P\left(x,y\right)$ from both sides resulting in (after rearranging terms): $P\left(x,\overline{y}\right)=P\left(x\right)-P\left(x,y\right)$.

4 In the context of expert systems, sophisticated methods have been developed to avoid the computation of the full joint probabilities (Lauritzen & Spiegelhalter, 1988; Neapolitan, 1990). However, these methods work only in the presence of (conditional) independences between different sets of variables involved in the system. In the absence of such independences the full joint distribution has to be computed.

5 Brase et al. (1998) provide a different explanation by assuming that our probability module works correctly with whole objects only. However, more recent studies have revealed that this explanation is problematic (e.g., Falk & Lann, 2008; Sirota et al., 2014).

6 Sloman et al. (2003) call them Euler circles.

7 The dual-process approach itself is not without criticisms (e.g., Gigerenzer & Regier, 1996; Keren, 2013; Keren & Schul, 2009; Kruglanski, 2013; Kruglanski & Gigerenzer, 2011; Osman, 2013; for a defense of dual-process theories, see Evans & Stanovich, 2013a, b).

8 The relationship between the probabilities and other quantities holds only with respect to the mathematical and objective conception of probability but not in the case of the subjective conception (for a discussion of different conceptions of probability, see, e.g., Gillies, 2000; Mellor, 2005).

Appendix

Formal specification of the probability spaces and probabilistic operations underlying elementary probabilistic problems

In the following, we provide a formal definition of the probability space and the elementary probabilistic operations underlying elementary probabilistic reasoning problems.

Specification of the Probability Space

Let ${\Omega }_1,{\Omega }_2,\dots ,{\Omega }_n$ represent n finite sets of values from various domains, and let $m_i=\left|{\Omega }_i\right|$ denote the size of set ${\Omega }_i$, with $2\le m_i<\infty$ $\left(i=1,2,\dots ,n\right)$. The sample space $\Omega$, underlying elementary probabilistic problems, is defined as the Cartesian product of the n sets: $\Omega ={\Omega }_1\times {\Omega }_2\times \cdots \times {\Omega }_n$. Thus, each element $\omega \in \Omega$ is a tuple $〈x_1,x_2,\dots ,x_n〉$ with $x_i\in {\Omega }_i$ $\left(i=1,2,\dots ,n\right)$.

A basic probability assignment is a function $\pi :\Omega \to \left[0,1\right]$ that assigns the point probability $\pi \left(x_1,x_2,\dots ,x_n\right)$ to the each element $\omega =〈x_1,x_2,\dots ,x_n〉$ of the sample space $\Omega$ such that the assigned point probabilities sum to 1: ${\sum }_{x_1\in {\Omega }_1}{\sum }_{x_2\in {\Omega }_2}\cdots {\sum }_{x_n\in {\Omega }_n}\pi \left(x_1,x_2,\dots ,x_n\right)=1$.

The probability space, underlying elementary probabilistic problems, is a triple $〈\Omega ,2^{\Omega },P〉$, consisting of the sample space $\Omega$, the power set $2{}^{\Omega }$ on $\Omega$, and the probability function $P:2^{\Omega }{}_{}\to \left[0,1\right]$ that assigns to each non-empty set $A\in 2^{\Omega }$ the probability $P\left(A\right):={\sum }_{⟨x_1,x_2,\dots ,x_n⟩\in A}\pi \left(x_1,x_2,\dots ,x_n\right)$, i.e., the probability of A is the sum of probabilities assigned to the elementary events in the sample space that are contained in A. In addition, $P\left(\varnothing \right):=0$. The resulting probability measure $P$ satisfies the probability axioms: (a) the assigned probabilities are in the range $\left[0,1\right]$ of real numbers; (b) $P\left(\Omega \right)=1$, and (c) the measure is additive: Let $A_1,A_2,\dots ,A_K\in 2^{\Omega }$ denote a set of pairwise disjoint sets, i.e., $A_k\cap A_l=\varnothing$, for all k and l $\left(k=1,2,\dots ,K;l=1,2,\dots ,K;k\ne l\right)$, then $P(A_1\cup A_2\cup \cdots \cup A_K)={\sum }_{k=1}^{K}P(A_k)$.

A random variable $X_i:\Omega \to {\Omega }_i$ is defined as a projection function that projects an element $\omega =〈x_1,x_2,\dots ,x_n〉$ of $\Omega$ on its i-th component, i.e., $X_i\left(\omega \right)=x_i$ $\left(i=1,2,\dots ,n\right)$.

Types of Probabilities and Elementary Probabilistic Operations

Elementary probabilistic problems comprise two components: Different types of probabilities and elementary probabilistic operations.

Types of Probabilities

Three types of probabilities (or probability distributions) are involved in elementary probabilistic problems: joint, marginal, and conditional probabilities.

The joint (probability) distribution $P\left(X_1,X_2,\dots ,X_n\right)$ conforms to the probability distribution over the elements $\omega \in \Omega$. It may be regarded as a table comprising $m_1\cdot m_2\cdot \cdots \cdot m_n$ entries. The single entries of the table represent the joint probability of the combination of values of the n variables and are denoted by $P\left(X_1=x_1,X_2=x_2,\dots ,X_n=x_n\right)$, or simpler: $P\left(x_1,x_2,\dots ,x_n\right)$. The joint probability distribution contains the complete probability information about the variables in question, enabling one to answer any possible probabilistic question. In addition, the joint probability distribution is defined on the finest partition of the sampling space consisting of $m_1\cdot m_2\cdot \cdots \cdot m_n$ combinations of the values of the n variables. These two characteristics, completeness and most fine-grained partitioning of the sampling space, render the joint distribution of focal importance in the case of probabilistic reasoning. For elementary probabilistic problems, used in psychological and educational studies, the joint distribution is at the heart of the probabilistic reasoning process.

Given the joint distribution $P\left(X_1,X_2,\dots ,X_k,X_{k+1},\dots ,X_n\right)$ over the set of variables $X_1,X_2,\dots ,X_k,X_{k+1},\dots ,X_n$, the marginal distribution $P\left(X_1,X_2,\dots ,X_k\right)$ is the probability distribution over a proper subset $X_1,X_2,\dots ,X_k$ $\left(k<n\right)$ of the variables.

Given the joint distribution $P\left(X_1,X_2,\dots ,X_k,X_{k+1},\dots ,X_n\right)$ on the set of variables $X_1,X_2,\dots ,X_k,X_{k+1},\dots ,X_n$, and the marginal distribution $P\left(X_1,X_2,\dots ,X_k\right)$, the conditional (probability) distribution $P\left(\left.X_1,X_2,\dots ,X_k\right|X_{k+1},\dots ,X_n\right)$ of the variables $X_1,X_2,\dots ,X_k$, given the variables $X_{k+1},X_{k+2},\dots ,X_n$, is defined by the equation (assuming $P\left(X_{k+1},\dots ,X_n\right)>0$): (A1) $$P\left(\left.X_1,X_2,\dots ,X_k\right|X_{k+1},\dots ,X_n\right)=\frac{P\left(X_1,X_2,\dots ,X_k,X_{k+1},\dots ,X_n\right)}{P\left(X_{k+1},\dots ,X_n\right)}.$$(A1)

The conditional distribution $P\left(\left.X_1,X_2,\dots ,X_k\right|X_{k+1},\dots ,X_n\right)$ may be conceived of as a table with each entry containing the respective conditional point probability, denoted by the symbols $P\left(\left.X_1=x_1,X_2=x_2,\dots ,X_k=x_k\right|X_{k+1}=x_{k+1},\dots ,X_n=x_n\right)$ or $P\left(\left.x_1,x_2,\dots ,x_n\right|x_{k+1},\dots ,x_n\right)$.

Elementary Probabilistic Operations (EPO)

The second component of elementary probabilistic problems constitute elementary probabilistic operations: (1) combination of information, (2) marginalization, and (3) conditioning. These operators are used to switch between different types of probabilities. The first operation, the combination of probability information, consists in the generation of a joint probability distribution by combining conditional and marginal probabilities. The combination is based on Equation (A1)(A1) $$P\left(\left.X_1,X_2,\dots ,X_k\right|X_{k+1},\dots ,X_n\right)=\frac{P\left(X_1,X_2,\dots ,X_k,X_{k+1},\dots ,X_n\right)}{P\left(X_{k+1},\dots ,X_n\right)}.$$(A1) specifying the conditional distribution. Reordering terms results in: (A2) $$P\left(X_1,X_2,\dots ,X_k,X_{k+1},\dots ,X_n\right)=P\left(\left.X_1,X_2,\dots ,X_k\right|X_{k+1},\dots ,X_n\right)\cdot P\left(X_{k+1},\dots ,X_n\right).$$(A2)

Repeated application of Equation (A2)(A2) $$P\left(X_1,X_2,\dots ,X_k,X_{k+1},\dots ,X_n\right)=P\left(\left.X_1,X_2,\dots ,X_k\right|X_{k+1},\dots ,X_n\right)\cdot P\left(X_{k+1},\dots ,X_n\right).$$(A2) on the right-most factor on the right-hand side of Equation (A2)(A2) $$P\left(X_1,X_2,\dots ,X_k,X_{k+1},\dots ,X_n\right)=P\left(\left.X_1,X_2,\dots ,X_k\right|X_{k+1},\dots ,X_n\right)\cdot P\left(X_{k+1},\dots ,X_n\right).$$(A2) results in the chain rule for combining probabilistic information: (A3) $$P\left(X_1,X_2,\dots ,X_n\right)=P\left(\left.X_1\right|X_2,\dots ,X_n\right)\cdot P\left(\left.X_2\right|X_3,\dots ,X_n\right)\cdot \cdots \cdot P\left(\left.X_{n-1}\right|X_n\right)\cdot P\left(X_n\right).$$(A3)

The second elementary probabilistic operation is called marginalization. With discrete variables it is realised by means of a summation performed on the joint distribution. The result of marginalization is a marginal distribution. The summation is taken over all combination of values of those variables that do not make up the resulting marginal distribution. These variables are “summed out” of the distribution. Formally, the operation of marginalization can be represented as follows: (A4) $$P\left(X_1,X_2,\dots ,X_k\right)={\sum }_{x_{k+1}\in {\Omega }_{k+1}}{\sum }_{x_{k+2}\in {\Omega }_{k+2}}\cdots {\sum }_{x_n\in {\Omega }_n}P\left(X_1,X_2,\dots ,X_k,x_{k+1},x_{k+2},\dots ,x_n\right).$$(A4)

The resulting marginal distribution is the (joint) distribution of the variables that have not been summed out. In the case of continuous variables, the operation of summation is replaced by integration.

The final probabilistic operation is called conditioning. It is realized by means of the algebraic operation of division. This operation is based on the definition of conditional probabilities given by Equation (A1)(A1) $$P\left(\left.X_1,X_2,\dots ,X_k\right|X_{k+1},\dots ,X_n\right)=\frac{P\left(X_1,X_2,\dots ,X_k,X_{k+1},\dots ,X_n\right)}{P\left(X_{k+1},\dots ,X_n\right)}.$$(A1) . Conditioning on an event B induces a new probability measure. In particular, given the probability space $〈\Omega ,2^{\Omega },P〉$ and let $B\subset \Omega$ be a non-empty subset of $\Omega$, with $P\left(B\right)>0$. Conditioning on B results in the new probability space $〈\Omega ,2^{\Omega },P_B〉$, with the induced probability measure $P_B$ given by $P_B\left(A\right):=P\left(A,B\right)/P\left(B\right)$, for every $A\in 2^{\Omega }$, where $P\left(A,B\right)$ is the probability measure of the joint event $A\&B$.

In the case of the more general Jeffrey conditioning with uncertain conditioning events, the induced probability measure may be defined as follows: Let $B_1,B_2,\dots ,B_K$ be a partitioning of the sample space $\Omega$, with $P\left(B_k\right)>0$, for all k $\left(k=1,2,\dots ,K\right)$, and let ${\gamma }_1,{\gamma }_2,\dots ,{\gamma }_K$ be probability weights, i.e., ${\gamma }_k\ge 0$, for all k $\left(k=1,2,\dots ,K\right)$, and ${\sum }_{k=1}^K{\gamma }_k=1$, representing the posterior probabilities of the conditioning sets $B_1,B_2,\dots ,B_K$. The corresponding probability measure is given by $P_{B_1,B_2,\dots ,B_K}^{{\gamma }_1,{\gamma }_2,\dots ,{\gamma }_K}\left(A\right):={{\displaystyle \sum }}_{k=1}^K{\gamma }_k\cdot P\left(A,B_k\right)/P\left(B_k\right)$, for every $A\in 2^{\Omega }$. The probability measure induced by simple conditioning is a special case of the probability measure resulting from Jeffrey conditioning: $P_B\left(A\right)=P_{B,\overline{B}}^{1,0}\left(A\right)$, for every $A\in 2^{\Omega }$, where $\overline{B}$ denotes the complement of B with respect to $\Omega$ $\left(P\left(\overline{B}\right)>0\right)$. Thus, conditional probabilities are defined on a different probability space than joint and marginal probabilities. The three elementary probabilistic operations also work with conditional distributions on the new probability space $〈\Omega ,2^{\Omega },P_B〉$.