Mathematical models: a state of exception

ABSTRACT

Models live in a state of exception. Their versatility, the variety of their methods, the impossibility of their falsification and their epistemic authority allow mathematical models to escape, better than other cases of quantification, the lenses of sociology and other humanistic disciplines. This endows models with a pretence of neutrality that perpetuates the asymmetry between developers and users. Models are thus both underexplored and overinterpreted. While retaining a firm grip on policy, they reinforce the entrenched culture of transforming political issues into technical ones, possibly decreasing citizens’ agency and thus favouring anti-democratic policies. To combat this state of exception, one should question the reproducibility of models, foster complexity of interpretation rather than complexity of construction, and encourage forms of activism aimed at achieving a reciprocal domestication between models and society. To breach the solitude of modellers, more actors should engage in practices such as assumption hunting, modelling of the modelling process, and sensitivity analysis and auditing.

KEYWORDS:

• Mathematical modelling
• sociology of quantification
• science and technology studies
• statactivism
• reproducibility
• sensitivity auditing
• sensitivity analysis

JEL CLASSIFICATION:

• C00
• A14
• D80

1. Introduction: why models live in a state of exception

The thesis of the present work is that mathematical models enjoy a state of exception that results from their versatility and epistemic authority, coupled with the infinite spectrum of methods available to mathematicians and the difficulty of proving a model wrong. The state of exception also results from the pretence of neutrality customarily associated with mathematics, and from the asymmetry between developers and users. Models have a considerable influence on policy, and this often means transforming political issues into technical ones. Models used for the analysis of risk are a typical instance of this process of displacement, a situation well noted by ecologists: a new technology is introduced, and models are deployed to show that it is safe for humans and their environment, thereby displacing the attention from the reasons – and desirability – of introducing the technology in the first place (Marris 2001; Winner 1989).

At times this may lead to surreal predictions, like trying to foresee the risk of leakage from an underground nuclear waste repository million years ahead (Pilkey and Pilkey-Jarvis 2009b)), a profusion of digits such as that generated by mathematical models during the last pandemic (Saltelli et al. 2020a), or the ‘Funny Numbers’ produced by econometricians to assess the risk of financial instruments (Porter 2012). Such errors are not always innocent of consequences. The ‘Funny Numbers’ of speculation on the American mortgages market created the crash of 2008/9 (Ravetz 2008). The rescue operation of ‘quantitative easing’ (zero or negative interest rates) had destabilising consequences that are still with us. And the models of a population explosion in China, retained long after they became inappropriate, led to restrictive policies that now make a catastrophic population implosion all too likely (Qi 2024).

To correct this state of exception, one should question the reproducibility of models, foster complexity of interpretation rather than the complexity of construction, and encourage forms of activism following the French statactivists, aimed at achieving a reciprocal domestication between models and society (Bruno, Didier, and Prévieux 2014). To breach the isolation of modellers, more actors should engage in practices such as assumption hunting, modelling of the modelling process, and sensitivity analysis and auditing.

We will start this analysis by illustrating our claim that models are special compared to other families of quantification. This is based on their exceptional epistemic authority, and on the fact that the sociology of quantification is apparently less active when it comes to these elements of the family of quantification.

1.1. Unparalleled palette of methods/invisible models

Models dispose of a unique repertoire of methods. This statement hardly needs justification. In brief, this view stems from mathematics, the highest ranked among scientific disciplines (Davies and Hersh 1986), considered by the fathers of the scientific revolution the language of God himself. This perspective still endures to this day, and the ongoing effort to integrate mathematics with human experience remains an unfinished project (Lakoff and Núñez 2001). The persistence of this faith has been explained by T.M. Porter (1995) with the role of numbers and mathematics as the source of legitimacy and trust in an increasingly impersonal society.

Models operate without established standards and disciplinary oversight. Until very recently, these features of scientific practice were not recognised in the philosophy of science, and so their absence did not signal a serious problem in the scientific status of a field. The fragmentation of mathematical modelling has resulted in a lack of agreed standards. A book on modelling read by all disciplines would enhance the communication of good practices among different fields. Unfortunately, such a book does not exist. A useful work is that of Foster Morrison (1991), though it is silent about verification and validation procedures. These topics are instead well treated in Santner, Williams, and Notz (2003), and in Oberkampf and Roy (2010). The work of Robert Rosen (1991) contains deep insights from system biology, including discussions on causality and ‘modelling as a craft’. An interesting text on the philosophy of models is offered by Morgan and Morrison (1999). A recent book by Scott Page (2018) draws a broad canvas of different modelling types and strategies, while Saltelli and Di Fiore (2023) tackle the politics of modelling. Still, the problem is that none of these works, regardless of their merit, can command the necessary epistemic authority in the absence of a disciplinary basis, both methodological and social, for modelling.

Padilla et al. (2018), after conducting a survey involving 283 responding modellers, noted that the heterogeneous nature of the modelling and simulation community prevents the emergence of consolidated paradigms, leading to validation and verification procedures being more of a trial and error business. These authors advocate a ‘more structured, generalized and standardized approach to verification’. After reviewing several existing checklists and approaches for model quality, they note that poorly informed users risk remaining ‘unaware of limitations, uncertainties, omissions and subjective choices in models’, with consequent over-reliance on the quality of model-based inferences. Model developers do not fare much better; they may over-elaborate their findings, concealing the uncertainties in their results and recommendations. Jakeman et al. (2006) propose a 10-point checklist for model development. Noticeably, the list starts with the definition of the model’s purpose and context, the conceptualization of the task, the choice of the family of models and uncertainty specification before moving to the nitty-gritty issues of estimation, calibration and verification. Eker et al. (2018) observe the existence of different cultures in model validation – one empirical and data driven, and another qualitative and participatory – which they associate with ‘positivistic’ versus ‘relativistic’ positions. These authors register the prevalence of the former view while acknowledging the usefulness of the latter, especially at the stage of identifying scenarios.

1.2. Mathematical models escape the sociology of quantification

Statistics has a much deeper connection to sociology, and to sociology of quantification in particular (Desrosières 1998; Mennicken and Nelson Espeland 2019; Mennicken and Salais 2022), than mathematical modelling. The sociology of quantification is more concerned with algorithmic governance, metrics, statistical indicators, the quantified self, and governing by numbers, although impact assessment tools such as cost benefit analyses – models of a sort – are a classic theme in this discipline (Porter 1995). An exception is the excellent work on models by Morgan and Morrison (1999), which we will discuss below in relation to ‘models as mediators’.

1.2.1. The epistemic authority of models

The greater epistemic authority of mathematical modelling derives from the widespread assumption that models map onto the underlying state of affairs and therefore decode the mathematical structure of the process of interest. Modellers are regarded as endowed with privileged access to the foundations of reality. Additionally, modellers make use of state-of-the-art computing facilities and thus are at the forefront of technology and science, and this enables them to gain epistemic authority by working on timely and pressing societal issues. Their recommendations are often taken as guidance for policy-making. The already mentioned epistemic authority accruing from the use of mathematics provides models with a better pretence to neutrality than other families of quantification.

1.2.2. Models cannot be falsified

Models do not meet classic (Popperian) criteria of scientificity. Oreskes (2010) has observed that model-based predictions tend to be treated like logical inferences in a classic hypothetic-deductive model. According to this perspective, laws are tested by making predictions which are then verified by experiments. If the prediction is not confirmed, then the law is ‘falsified’ (proven false). For Oreskes, a major problem in our use of mathematical models lies in assimilating them to physical laws, and hence treating their predictions with the same dignity. This is wrong, because in order to be of value in theory testing, the model-based predictions ‘must be capable of refuting the theory that generated them’. Since models are a ‘complex amalgam of theoretical and phenomenological laws (and the governing equations and algorithms that represent them), input parameters and concepts, then when a model prediction fails what part of all this construct was falsified? The underlying theory? The calibration data? The system’s formalization or choice of boundaries? The algorithms used in the model?’

Oreskes’ points are known in philosophy as the Duhem-Quine critique (Duhem 1991; Quine 1951). To the extent that scientific theories themselves are ‘models’, i.e. amalgams of theoretical assumptions, holistic preconceptions, measurement devices, and physical equations, when a prediction is contradicted by data, it needs to be reformulated, but it isn’t always clear how. We cannot even be sure that the problem lies with the model itself, because no element in a theory or model can be tested in isolation. Thus, we may need to revise other parts of the scientific edifice upon which our model rests in order to accommodate our results.

A related problem is that of contrastive underdetermination: even if a model output matches the data,¹ it does not follow that the model is truth-conducive: no finite amount of data can narrow down the range of possible models to just a single function or set of functions. The classic example of oranges and apples (If I have spent $10 in apples and oranges, and each apple costs $2 and each orange cost $1, how many oranges and apples have I bought?) illustrates the problem (Stanford 2023). Under these circumstances, pragmatic and/or non-rational criteria may step in to do the heavy lifting of picking up the ‘best’ model. These factors would include, for example, the charisma and influence of the developer, the availability of resources, adherence to the prevailing scientific paradigm, or the simplicity or parsimony of the model (Feyerabend 1988).

1.2.3. Models as the most effective mediators between theory and reality

Models are regarded as a meaningful simplification of a real system, a device for predicting the unknown behaviour of a system from the operation of its parts, and as a formalization of our knowledge about a system (Hall and Day 1977). Many authors have noted that models mediate between theory and reality, due to their independence from both. Thus, models would be instruments that act and describe things in ways that advance understanding thanks to the tacit craftsmanship of scientists (Morrison and Morgan 1999). For Marcel Boumans (1999), models act as integrators of a broad array of ingredients, including theoretical notions, mathematical concepts and techniques, stylized facts, empirical data, policy views, analogies and metaphors. According to Jerome Ravetz (2023), instead, models are like metaphors that help us understand how we see the world: they express ‘in an indirect form our presuppositions about the problem and its possible solutions’, and can thus assist an extended community of peers in deliberating about social or ecological problems.

2. Consequences arising from the state of exception

What are the consequences of being special? The mentioned versatility of models is crucial because it enables them to colonize all aspects of knowledge. Coupling versatility with the other aspects of state of exception just mentioned leads to important consequences.

2.1. Gross asymmetry between developers and users

Models operate in a context of asymmetry of knowledge between developers and users, a point that is thoroughly elaborated in Jakeman, Letcher, and Norton (2006). One can contest this claim by referring to ‘black boxes’ used in other families of quantification, typically algorithms or statistics. We contend that this asymmetry is especially of concern for large mathematical models (Puy et al. 2022).

2.2. Ritual use

Models lend themselves to ritual use. An important analogy between statistical and mathematical modelling lies in the ‘ritual’ use of methods. The existence of rituals in statistics has been discussed extensively by Gigerenzer (2018) and by Gigerenzer and Marewski (2014). We similarly call ‘cargo-cult’ statistics the use of statistical terms and procedures as incantations, with scant understanding of the assumptions or relevance of the calculations they are based on (Stark and Saltelli 2018). As for rituals in modelling, the best anecdote is perhaps that offered by Kenneth Arrow. During the Second World War, he was a weather officer in the US Army Air Corps working on the production of month-ahead weather forecasts, and this is how he tells the story (Szenberg 1992):

The statisticians among us subjected these forecasts to verification and they differed in no way from chance. The forecasters themselves were convinced and requested that the forecasts be discontinued. The reply read approximately like this: ‘The commanding general is well aware that the forecasts are no good. However, he needs them for planning purposes’.

Along these lines, it has been argued that models universally known to be wrong continue to play a role in economic policy decisions (Martin 2018). This relates to present-day debates about algorithms: authorities may willingly or inadvertently recur to algorithms to implements rules that, if applied by human subjects, would be either legislated or issued through administrative rulemaking. Big data prediction models, instead, can be built and used without policy decisions ever having been the object of a political or at least traceable administrative procedure (Brauneis and Goodman 2018).

Social scientist Niklas Luhmann, as discussed in Moeller (2006), uses the term ‘deparadoxification’ to indicate the use of scientific knowledge to give a claim a pretension of objectivity, so as to show that policy decisions are based on a publicly verifiable process, rather than on an expert’s whim: we all recall the ‘We follow the science’ declaration of politicians during the COVID-19 pandemic (Devlin and Boseley 2020).

2.3. Proclivity for trans-science

Models lend themselves to trans-science. Trans-science refers to scientific practices that appear to be formulated in the language of science, but that science cannot solve because of their sheer complexity or insufficient knowledge (Weinberg 1972).

Examples of trans-science are not difficult to find. Examples include:

• How many people will sit in an autonomous cars by 2050?

• How will the spread of malaria change if global temperature increases by 1.5°C?

• What will be the cost of CO2 averaged over the next three centuries?

Ambitious modelling projects such as the ongoing plans to develop a digital twin of the Earth (Bauer, Stevens, and Hazeleger 2021) may also be seen under this light, where models aspire to become one-to-one maps of the empire as described in the short story Del Rigor en la Ciencia (On Exactitude in Science) by Jorge Luis Borges (1946).

2.4. Grip on policy-making

Models have their own political economy. Like other means of quantification, models encourage strategies of economicism, solutionism, reductionism, and transformation of quality into quantity (Stirling 2023a, 2023b). These strategies have been called out in relation to the use of quantitative methods in impact assessment, as well as in contexts where New Public Management strategies are adopted (Muller 2018; O’Neil 2016; Salais 2022). The political economy of modelling is based on its strong epistemic authority and influence on decision-making, and the fact that models are rarely blamed for failure.

Statistical experiments can be registered, while the preregistration of modelling studies (Ioannidis 2022) is still a long way off. For Chalmers and Glasziou (2009) the percentage of non-reproducible studies in the field of clinical medical research could reach 85%. Nobody can provide a similar figure for mathematical modelling.

Two additional policy features of models are their capacity of ‘navigating the political’ (adapting their inference to an ongoing policy process, thus becoming part of the process itself, as discussed by van Beek et al. (2022), and their ability to act as chameleons (jumping from a context of investigation, i.e. ‘what if’, to one of prediction, and defending against criticism by reverting to ‘this is just a study model’, as seen in Pfleiderer (2020).

The idea that a quantification should conform to both technical and normative dimensions of quality is not new; it goes back to Amartya Sen’s Informational Basis for Judgment of Justice (IBJJ) (Sen 1990). This principle suggests asking the question of whether a system of measurement or analysis provides a fair, informed judgement of an issue. In this framework (capability approach), fairness is related to the possibility for individuals to achieve their goal, once their individual conditions are taken into account. In this way, fairness does not equate to all having the same material means, but rather to all having an equal opportunity to fulfill their aspirations. Salais (2022) resumes this concept and notes that the prevailing unfair quantification schemes originating from new public management approaches contribute to the creation of a state of a-democracy – a system where citizens are confronted with all the signs of a democratic system of governance, but none of the corresponding chances to express agency.

The ideas that regimes of numericization may be ultimately anti-democratic are not new either. This argument has been made in relation to artificial intelligence and the use of big data (McQuillan 2022; Supiot 2017; Zuboff 2019). The models used in operations research may also fit into this category (Muller 2018; O’Neil 2016), with extreme cases of workers exploitation and deprivation (Kantor et al. 2022; Teachout 2022).

2.5. Vulnerability to modelling hubris

The conjecture of O’Neill (1971), see also Turner and Gardner (2015), posits that too simple models may overlook important features of the system, and thus lead to systematic errors. Conversely, models that are too complex (burdened by an excessive number of estimated parameters) may lead to greater imprecision due to error propagation. In this second case, one might speak of modelling hubris, i.e. a situation where the pursuit of improvement leads modellers to over-reach (Saltelli et al. 2020a). In data science, a very similar trade-off is that between under- and over-fitting. For instance, in interpolation, one may use a polynomial whose order is too low, thus underfitting the existing or training points. Instead, if the order is too high, the existing points are fitted too well, often at the cost of performing poorly when new points are added to the set. An array of methods is available in data science to tackle this problem (James et al. 2017).

Other disciplines have different names for this trade-off. In system analysis, Zadeh calls this the principle of incompatibility, whereby as complexity increases ‘precision and significance (or relevance) become almost mutually exclusive characteristics’. Finding the right balance, the point of minimum error shown in Figure 1, is easier in data analysis than in modelling. An old discussion of this conundrum between model parsimony and modelling hubris can be found in Hornberger and Spear (1981), who note that models’ over-parametrization may be fiddled with ‘to produce virtually any desired behaviour, often with both plausible structure and parameter values’.

Figure 1. Increasing the complexity of a model reduces its systematic bias due to the omission of relevant features of the system being modelled. Since descriptive power comes at the cost of additional estimated parameters, increasing the complexity of the model also increases the error due to the propagation of parametric uncertainty to the output.

Note that in cases where models have access to validation data, there are statistical tests available for judging the parsimony of the model (Akaike 2011; Schwarz 1978). When instead the model projects into the future or operates outside the region in the input space where it was calibrated, the assessment of model bias may be challenging. To keep a proper balance between output uncertainty and complexity, modellers working in these scenarios should track the uncertainty build up at each stage of model refinement by calculating the model’s effective dimensions (i.e. the number of influential parameters and the order of the highest-order effect). This may help identify the threshold beyond which the addition of detail and its correspondent boost in uncertainty no longer makes the model fit-for-purpose (Puy et al. 2022).

3. Solutions to the state of exception

3.1. Thinking about the reproducibility of models

Preregistration of mathematical modelling studies is proposed by Ioannidis (2022). Pre-registration, mostly advocated in relation to lab or field experiments and supported by some academic journals in the realms of psychology or human behaviour, is the practice of registering the study plan and objectives in a repository before a scientific project begins. The goal is to prevent malpractices such as HARK [Hypothesis After the Results are Known (Kerr 1998)], whereby different hypotheses are thrown around a set of data until one meets some test of significance. HARK violates the rules of hypothesis testing and, together with P-hacking, may be held responsible for the lack of reproducibility of scientific findings (Munafò et al. 2017; Saltelli and Funtowicz 2017). Whether pre-registration can be applied to mathematical modelling depends on many factors. For predictive models, for example, the time horizon of the prediction is important for the registration to be meaningful: a prediction concerning the performance of a nuclear waste repository over a million-year time span (Stothoff and Walter 2013) clearly falls beyond its scope. Ioannidis believes that preregistration may help build trust in modelling if accompanied by practices. These include specifying what constitutes a successful prediction, outlining the conditions for recalibration, agreeing on validation criteria, promoting team science whereby multiple teams and/or stakeholder participate in the analysis, see e.g. (Lane et al. 2011; Ravetz 2023), and thoroughly documenting present and past model records.

The common assumption is that the models, being mathematical, produce a unique outcome for each set of inputs. Based on the belief that computers only manipulate 0’s and 1’s, this assumption is a strong contributor to the state of exception of mathematical models. In practice, it is safest to treat each run of a model as an experiment. There is evidence that in quantitative analyses, the results can often display a wide range of outcomes depending on who performs the analysis. In Breznau et al. (2022), for instance, researchers from 73 teams produced effects estimate ranging from positive to negative simply by making different modelling choices. It is also worth noting that what made this large experiment possible was the decision of the participating teams to state the objective of their analysis.

3.2. Sensitivity analysis and sensitivity auditing

But the real strength of the models, in my mind at least, were in sensitivity analysis (where one could examine the response of the model to parameters or structures that were not known with precision (i.e. sensitivity analysis), and in the examination of the behavior of the model components relative to that of the real system in question (i.e. validation). By undertaking sensitivity analysis and validation, a great deal can be learned about the real system, including what you do not know (Hall 2020).

For ecologist Charles A. S. Hall, not only is sensitivity analysis an important ingredient of model validation; it is one of the good things one can do with a model.

It is not in the scope of this paper to suggest that sensitivity analysis (SA) alone can lead to a thorough reformation of the field. Reforming modelling will involve both the adoption of methods in the practice of research and their incorporation into teaching syllabuses. It will also require the endorsement of hybrid quantitative/qualitative and transdisciplinary approaches – ‘Cinderella methods’, in the words of Andy Stirling (Stirling 2023b) – which shed light on the normative framworks, categories, value-ladenness and conditionalities of modelling. Sensitivity auditing (Saltelli et al. 2013), with its ‘modelling of the modelling process’ (Lo Piano et al. 2022, 2023), belongs to this class of transdisciplinary methods. If we accept sociologists’ view that every quantification needs to have both a technical and a normative justification of quality (Salais 2022), then we can say that sensitivity analysis and auditing fulfil these criteria (Saltelli and Puy 2022).

SA is especially useful to guide the modeller through the ‘garden of the forking paths’ (Gelman and Loken 2013), shedding light on the model output’s conditionality on the choices made during the modelling process. In an SA framework, all possible alternative modelling choices, from different boundary conditions to parametrizations and model structures, can be simultaneously activated. The modeller can therefore take both left and right at each bifurcation in the garden, like the Chinese author imagined by Borges (1998), and observe how this multiplicity of options impacts the estimation in a modelling of the modelling process (Lo Piano et al. 2022). This may include stages of assumption hunting, where modellers or their interlocutors strive to retrace assumptions made in the construction of the model.

3.3. Following the example of Statactivism

When it comes to examining the quality of their quantifications, few communities have been as active as statisticians. The movement of French statactivists (Bruno, Didier, and Prévieux 2014; Bruno, Didier, and Vitale 2014), rooted in a strong national tradition of sociology of quantification (Bourdieu 1984; Desrosières 1998), has proven able to ‘fight a number with a number’ in domains of policy relevance such as poverty (Concialdi 2014) and consumer prices indices (Samuel 2022). One would very much like to imagine modellers taking the viewpoint of those ‘measured’ into the analysis as advocated by statactivists (Salais 2022), thereby making the invisible visible (Bruno, Didier, and Prévieux 2014), or fully internalizing the double nature – technical and normative – of the quality of a quantification (Mennicken and Salais 2022). Against this perspective, one should note that the statactivist approach comes more naturally in the domain of statistical indicators in socioeconomic systems. Can this be generalized to the many applications of mathematical modelling?

The success of statactivists is also due to the support of important institutional actors such as statistical offices – INSEE in the case of France. What would be the equivalent institution in the field of mathematical modelling? Regulatory authorities concerned with consumer or environmental protection? Or technology assessment offices akin to the now-defunct US institution (Chubin 2001)? We cannot answer these questions in the present work. When it comes to mathematical models, the distance from lay citizens is greater than in the case of socioeconomic statistics, which brings us back to the reciprocal domestication between models and society discussed in the responsible modelling manifesto (Saltelli et al. 2020a). Counter-modelizations similar to those presented by statactivists are not impossible – see e.g. Puy, Lo Piano, and Saltelli (2020) – but at present they are more difficult to disseminate than the counter-statistics of statactivists.

3.4. Reciprocal domestication between models and society

The COVID pandemic of 2020 has dramatically increased the visibility of mathematical modelling, since models were increasingly needed when epidemiological evidence was scarce. Some models were praised for making a convincing case for intervention (Landler and Castle 2020). However, this was accompanied by considerable controversy, either due to the deficiencies of the model, or because of disagreement about the policies they informed (Pielke 2020; Rhodes and Lancaster 2020). See also Saltelli et al. (2020a, 2020b). Interestingly, not only has mathematical modelling jargon entered the common language with expression such as ‘flattening the curve’, but politicians can now be heard discussing mathematical modelling with a notable level of detail (Seely 2022). One can only hope that this trend continues without leading to generalized scepticism toward the use of models for policy decision-making (Saltelli et al. 2023).

3.5. Defogging the mathematics of uncertainty

An important issue in mathematical modelling is the management of uncertainty. Uncertainty quantification should be at the heart of the scientific method, a fortiori when science is used for policymaking (Funtowicz and Ravetz 1990). In statistics, the P-test can be misused to overestimate the probability of having found a true effect. Likewise, in modelling studies certainty may be overestimated, thus producing numbers precise to the third decimal digit even in situations of pervasive uncertainty or ignorance, including important cases where science is needed to inform policy (Pilkey and Pilkey-Jarvis 2009a). Normative and cultural problems naturally compound the problem as for the case of statistics. It is an old refrain in mathematical modelling, first noted among hydro-geologists, that since models are often over-parametrized, they can be made to support any conclusion (Hornberger and Spear 1981).

Building on the notion of ‘Defogging the mathematics of uncertainty’, and more recent insights into the excess precision associated with impact assessment and risk analysis (Stirling 2023a), there is an evident need for greater clarity on how risk numbers are computed. It is essential to unveil the non-neutrality of techniques and of models (Saltelli et al. 2023; Saltelli et al. 2020) so as to allow each party in a conflicted policy issue to make best use of evidence based on numbers.

4. Conclusions

To statisticians (especially those concerned with social statistics) it may seem natural to consider that the quality of a quantification is both technical and normative (Salais 2022). This concept might be effectively transferred to modelling. Most modellers agree that while complex calculations are now being used to support important decisions, these decisions cannot be justified solely based on calculated results without an understanding of the associated uncertainties. At the same time, when confronted with the considerations collected in the present text, some elements of uneasiness do emerge among practitioners.

One is the deep-rooted resistance to ideas of non-neutrality of mathematical modelling. Here there is an important cultural and ritual element at play. The notion that a model can be an avenue of possibly instrumental ‘displacement’, in the sense of moving the attention from the system to its model, as discussed in Rayner (2012), is still too radical for many practitioners to contemplate. Similarly, criticism against excessive or rhetorical complexity is often met with a defence of complex models on the basis of their ability to potentially ‘surprise’ the analyst. Overall, it is noted that modelling is too vast an enterprise to be boxed into a single quality assurance framework. The centrality of mathematical modelling and its insufficient quality control and governance arrangements would appear to offer the occasion to put quality at the core – or at least raise it as a priority theme – of a discussion about ethics of quantification (Di Fiore et al. 2022; Saltelli 2020). Still, we believe that there is sufficient material to give this process a good start (Oberkampf and Roy 2010; Santner, Williams, and Notz 2003).

A second source of discomfort is the sense that the methodologies advocated here may actually have excessive candour. Practitioners may be led to ignore SA and ‘adjust’ their level of uncertainty in the input upon discovering that the output results are too uncertain (Leamer 1985). Worse still, there is a possibility that SA might reveal that the source of a poor (i.e. diffuse) inference lies in an assumption that is hard to identify or to compress to a lower uncertainty. Along the same discomfort axis is the worry that SA may invalidate the use of models by showing the futility of an analysis (such as the cost-benefit modelling mentioned above), or by opening the door to relativism – whereby any framework can be upheld given some sort of evidence.

In the present work, we counter these doubts: we argue that it is better to deconstruct oneself systematically than to be deconstructed in the field. This consideration lies at the core of good scientific practice, as per Merton (1973)’s principle of ‘Organized Scepticism’, whereby all ideas must be tested and subjected to rigorous, structured scrutiny within the community. Richard Feynman elaborated on this principle in his Cargo Cult lecture (1985), emphasizing that a rigorous analyst ‘bends backward’ to offer to potential inquisitors all the elements to possibly deconstruct the analyst’s argument.

In econometrics, Mertonian concerns are known as the ‘honesty is the best policy’ approach (Leamer 2010), and this is the 10^th law of applied econometrics as formulated by Peter Kennedy (2008): ‘Thou shalt confess in the presence of sensitivity. Corollary: thou shalt anticipate criticism’. This reference to SA brings us back to the topic chosen as an illustration of statistics’ potential role in the rescue of mathematical modelling.

How needed is this rescue? Ravetz (1971, 179) prophesized that entire research fields might become diseased, and noted: ‘reforming a diseased field, or arresting the incipient decline of a healthy one, is a task of great delicacy. It requires a sense of integrity, and a commitment to good work, among a significant section of the members of the field; and committed leaders with scientific ability and political skill’. While statistics has proven to possess the disciplinary arrangements and committed leaders to react to a crisis, mathematical modelling seems to lack both.

Declaration of Conflicting Interests

The Authors declare that there is no conflict of interest.

Acknowledgments

AS acknowledges the i4Driving project (Horizon 2020, Grant Agreement ID 101076165). AP was supported by funding from the UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee (project DAWN, PI Arnald Puy, EP/Y02463X/1).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. The relation between models and data is often more symbiotic than adversarial. In climate studies, this relation has been defined as ‘incestuous’, precisely to make the point that in modelling studies using data to prove a model wrong may not be straightforward (Edwards 1999).