Searching for causality in educational research

The papers in this double issue address important substantive topics in education. In addition to this, what unites them is that they all speak to causality. To search for causality in educational research is to search for the holy grail. Causality is elusive. For example, in seeking causality, correlational analysis commits a category mistake and is of limited help, and counterfactuals offer only partial assistance (Lewis, 1993). Screening off (Reichenbach, 1956) and controlling out factors, ancestors, and descendants (Pearl, 2009) is useful but inconclusive, and looking for necessary and sufficient conditions in conditional probability does not “prove” causality. Further, causal modelling is simply a heuristic construction – a theoretical suggestion or conjecture derived from a combination of literature review, observation, and a creative imagination – of what is in the researcher’s mind in explaining possible or probabilistic causal linkages between phenomena and their elements. Perhaps, as Mumford and Anjum (2013) suggest, causation is a “primitive concept”, not reducible but only assumed or inferred because it is useful or because that is how we think, and to go further is to risk circularity, that is, causality exists because we think it does and we act on that assumption. The papers in this issue all tussle with these challenges in causal studies.

Problems in considering causality are not only ontological or epistemological but also procedural. How can we investigate it? Let us be a little more mundane and prosaic: How can we identify the possible contenders for possible causal explanations, how do we test them, and how can we judge which explanations survive the most rigorous tests, that is, which fit the data most? Further, in investigating causality we look for the conditions in which causal events occur, but is a condition, be it necessary or sufficient, the same as a cause?

Further still, how far out from, or into, the explanandum – that which is to be explained – must the researcher go in understanding the causation at work in a phenomenon: the familiar macro- to micro-dilemma or, indeed, its reverse: the micro- to macro-dilemma. What macro-, mezzo-, and micro-causes are at work in a phenomenon? Given the infinite regress that causality brings (even though, as Pearl, 2009, avers, we can screen off some possible ancestors), how far back in time must we go in order to explain the causes of a present-day situation? In short, how much and what kind of contextual information do we need in order to come to a satisfactory causal explanation, what are the warrants for that explanation, and what is the origin of these warrants? Again, the papers in this issue all attest to these challenges.

The problems of understanding causality extend further to how we even talk about it, as we are inevitably selective in such discourses. What do we include in a causal discussion? Do we use statistics, qualitative data, mixed methods, longitudinal data, or what? As Powney and Watts (1987) remarked so tellingly in another context, talk might catch some of the issue but not necessarily its heart:

Talk is dynamic – a quality it loses as soon as it is collected in any way. It is somewhat … like catching rain in a bucket for later display. What you end up with is water, which is only a little like rain. (p. 16)

Do statistical analyses and quantitative data, as in the papers in this issue, all of which allude to causality, catch the heart of intended and implied causality and its dynamic nature? Let us be mindful of Goldthorpe’s (2007) contention that statistical methods cannot simply “crank out” causal explanations of phenomena from numbers, however regular they may appear in combination or correlation (p. 17). Regularity and patterning are not causation.

The researcher in education faces severe challenges in understanding and working with causality and offering causal explanations. As you read each of the papers here, ask yourselves questions about the causality in them, both implicit and explicit, for example:

• How is the causal model in the research constructed and warranted?

• How are possible contenders for causal conditions and contingencies ruled in and ruled out – included and excluded – in the study? How is causality “talked about” and explained in these papers, and how much justice does such talk do to the phenomenon in question, catching its causal essence rather than, for example, only its epiphenomena or manifestations?

• What is the role of context and macro- to mezzo- to micro-analysis or vice versa, in understanding causality in the study?

• What role does time play in understanding and explaining causality in the study?

• What factors are included and excluded in explaining the causality at work here, and how can we be sure that these factors really are the causes?

• What are the causal processes at work in the study?

• Are the factors and variables identified actually causal or, rather, the theatres, the sites, or locales, in which other – real – causal factors play out their causal roles and are at work?

These questions are addressed in different ways in each of the five papers in this issue.

Steur, Jansen, and Hofman appeal to the causal role of literature in contributing to a derived theoretical framework for understanding graduateness. They present a tri-partite model of graduateness: reflective thinking, scholarship, and moral citizenship. How does the study arrive at these three important elements? The authors suggest that they come from “developmental theories”, and causal detectives can ask “how causal is the journey from developmental theory to graduateness?” Or, to put it in the authors’ terms: “can growth in students’ perceived abilities be determined for graduateness and in its separate domains (reflective thinking, scholarship, and moral citizenship)?” (italics added). Causality is strongly suggested here (the give-away word is “determined”); how is it warranted?

The paper by Yang Hansen and Gustafsson is explicitly causal (“causes of educational segregation”), and the authors, using a counterfactual approach, a key test of causality, report that “school choice was a determinant of school segregation” in Sweden. Their elegant, careful analysis of large-scale data is both contextualized and longitudinal. Given this, causal sleuths can raise many questions: “How robust are their inferences of causality?” “What is included and excluded in their model?” “What alternative causal explanations are there?” “How are the links in their causal chain established?” “Are there variables causes, contexts, or conditions?” Their paper raises interesting causal questions.

Indeed, by implication, for causal researchers the paper raises the difficult issue of differentiating causality from supervenience (one cannot have an A-difference without a B–difference) (Lewis, 1986), that is, causality is not the same as dependence, as dependence concerns entailment: One cannot have any changes in A-properties (“educational segregation”) without having also changes in their B-properties (“school choice”). Is the study here one of causality or supervenience – a challenge for any researcher on causality?

The paper by Marks, again using large-scale data, offers a rich set of possible causal explanations for inter-domain (e.g., country, school, teacher) and over-time correlations in student achievement. He suggests that “the high inter-domain and within-domain temporal correlations cannot be explained by socioeconomic background”. In other words, socioeconomic status (SES) plays only a small part in the causal drama here. He writes that “[p]rior achievement has a much stronger influence on achievement than students’ socioeconomic background” (italics added), and his conclusion is decidedly causal: “high inter-domain and over-time correlations in student achievement can be attributed to general and domain-specific latent factors with little or no impact from SES and school SES” (italics added). Time and context are relevant here, but the unsettling question for causality is whether they are causal or simply contextual: the theatre or site in which other causal factors are played out. His well-worked paper is thought provoking, both in terms of its substantive findings and in considering causality.

The study by Daza investigates “the effects of students’ social relationships at university on students’ success” (italics added). She reports that: (a) particular forms of social capital contribute to students’ perceptions of success; (b) the bearers of such social capital have differential causal effects on such perceptions; and (c) frequency of social contact has an important bearing on students’ perceptions. She argues that there is an “association between students’ performance at university and social capital”, and that such “association” is causal. Contextualizing this within the Bologna process in Spanish higher education institutions, Daza finds that “social relationships encouraged by universities and access to, and membership of, groups with social capital can be a valuable resource in the educational process of students from environments with low human capital” and that “the use of social capital varied according to students’ social class”. In addition to this useful finding in itself, we can ask “What is the counterfactual evidence for such causal claims?” “What are the necessary and sufficient conditions for the production of the students’ perceptions of success?” “Is social capital a cause, a condition, or the site in which causality operates in influencing students’ perceptions of their progress?” Such challenges beset causal investigators.

The paper by Konstantopoulos and Shen opens with what can be taken as a significant challenge for those working with causality. Whilst the title of the paper suggests causality (“Class Size Effects on Mathematics Achievement in Cyprus”), the paper opens with a causal turn, this time in terms of causal mechanisms: “[c]lass size reduction has been viewed as one school mechanism that can improve student achievement”. In introducing the term “mechanism”, the authors throw down the gauntlet to researchers on causality: Is a mechanism the same as a cause or is it simply the theatre, the locale in which other causal variables are operating (cf. Goldthorpe, 2007, p. 119). Is a mechanism the process of causality at work, or an explanation? Is identifying an explanation the same as identifying a cause? How acceptable is it to consider causality mechanistically, deterministically, or as a linear process? Are there really causal chains or is causality better considered in terms of dynamic, complex causal nets and networks? How much does thinking about causality suggest or require identification of the causal mechanisms at work in a situation: the macro- to the micro-mechanisms? How far into a situation does one have to go to unravel the causality at work?

Let us go further: the often-used sentence “smoking causes cancer” is no more than shorthand for something far more complex and micro-level, getting the researchers into the complex chemistry of the effects of smoke on lung tissue and the conditions that may promote or inhibit the influence of smoking on cancer. It may be so with “causal mechanisms” in relating class size reduction to increased student achievement: It is a useful shorthand for detailed causality. Class size reduction may be an initial or early contributor to causal effects rather than being the actual causal mechanism at work, just as smoking may be the initial or early contributor to the causal effect: lung cancer. Such shorthand harks back to the way in which we talk about causality: We reduce in order to clarify, but such reduction risks losing key parts of the causal processes at work. Or, to echo Powney and Watts (1987) above, rain becomes only water. For causal researchers, the heart, the essence of the causal phenomenon, is easily lost.

What is happening causally between the reduction in class size that leads or does not lead to improved school achievement (in fact, Konstantopoulos & Shen find that “class size was not related with student achievement in Cyprus”)? What are the causal links in the chain here? Class size reduction may raise the likelihood of improving student achievement, but does this mean that it is deterministically causal – an INUS condition (an Insufficient but Necessary part of the condition which is Unnecessary but Sufficient to produce the effect) (Mackie, 1993) – or simply a non-necessary or non-sufficient condition in probabilistic causation? Smoking may increase the likelihood of lung cancer, but many smokers do not contract lung cancer. Their paper makes a valuable contribution to exposing challenges in studying causality.

The five papers in this double issue raise interesting questions for educational researchers working with causality and statistics. In 1994, Pearl and Verma wrote that “statistical analysis can never distinguish genuine causation from spurious covariation” (p. 807) and that believing that this “spurious covariation” can explain causation “has been a major hindrance in the way of developing a satisfactory, non-circular account of causation” (p. 807). How do the five papers here answer the challenge from Pearl and Verma?

The papers here raise challenges for the researcher brave enough to step into the arena of causality:

• Assuming or “proving” causality; causality and human thinking

• How to talk about causality

• Counterfactuals

• Causal contexts and conditions

• Necessary and sufficient conditions

• Screening off, causal ancestors and descendants

• Macro-, mezzo-, and micro-levels of causal analysis

• Inclusion and exclusion of variables

• The limits of statistical analysis of causation

• Causal mechanisms, processes, chains, and links in the chain

• Warrants for asserting causality

• Causal modelling

• Causality and explanation

• Causality and supervenience

• Causality, theatres and locales of causality

• Probabilistic and deterministic causality

Establishing causality is difficult; it takes a courageous researcher to work in this field. The task is hard, but eminently worthwhile. The five papers here attest to this convincingly and provocatively. In addition to raising important substantive educational issues in their own right, they make a valuable contribution to researching causality in education.