Why Monte Carlo Simulations Are Inferences and Not Experiments

Abstract

Monte Carlo simulations arrive at their results by introducing randomness, sometimes derived from a physical randomizing device. Nonetheless, we argue, they open no new epistemic channels beyond that already employed by traditional simulations: the inference by ordinary argumentation of conclusions from assumptions built into the simulations. We show that Monte Carlo simulations cannot produce knowledge other than by inference, and that they resemble other computer simulations in the manner in which they derive their conclusions. Simple examples of Monte Carlo simulations are analysed to identify the underlying inferences.

Acknowledgements

We thank two anonymous referees for this journal and its editor for valuable comments and suggestions.

Notes

Morrison (2009) maintains that many computer simulations, Monte Carlo or not, are experiments because they draw on models in the same way as experiments do. See Giere (2009) for a reply to Morrison.

There are many other fascinating aspects to Monte Carlo simulations. See e.g. Humphreys (1994) and Galison (1997, ch. 8). See also Anderson (1987), Eckhardt (1987), the chapters in the first part of Gubernatis (2003), and Hitchcock (2003) for the history of Monte Carlo simulations.

A more appropriate expression is ‘Monte Carlo method’ or ‘Monte Carlo technique’. See e.g. James (1980), 1147, for a definition. Consult Hammersley and Handscomb (1967), Halton (1970), and James (1980) for reviews of this method.

See e.g. Hammersley and Handscomb (1967, ch. 3), Tocher (1975, ch. 6), James (1980, 1165–1167), and Knuth (2000, ch. 3) for the generation of pseudo-random numbers.

Here we define random numbers probabilistically. That is, a sequence of numbers is random just if they fit the requisite probabilistic model. It is a matter of considerable discussion just what it is for random numbers to fit some probabilistic model. These issues go beyond the scope of the present paper and will not be addressed here. It is sufficient for our purposes that, as a practical matter in Monte Carlo simulations, a large sequence of numbers would not be accepted as conforming to a uniform distribution over [0, 1] if all the numbers are clustered close to some particular value, such as 0.25, even though there is some very small probability that just such a clustering may happen. This sequence and other pathological sequences like it would be rejected in favour of ones whose members are more uniformly distributed over [0, 1].

The N random variables X_i , i = 1, …, N are uniformly and independently distributed over the integers {1, …, M = 1,000,000}. The estimate of the sum is , which has a mean M(M + 1)/2 and a variance . Two standard deviations, expressed as a fraction of the mean, is approximately , which supplies the 1.15% error of the text when N = 10,000.

As explained in section 2 above, this is meant to say that certain frequencies in the random numbers r _i closely match those predicted by the probability model p.

A similar type of argument is carried out by a computer simulation that uses deterministic equations to follow the dynamics of a system (Beisbart 2012).

The only exception is when the random numbers from a randomizer are tested for their statistical properties. If an independence test fails, we do learn something about the randomizer, namely that the trials are not independent. Such tests are necessary to underwrite the premise that the random numbers follow a certain probability model. However, such a test is not a proper part of every Monte Carlo simulation. Once we are confident that a certain randomizer produces random numbers with a certain distribution, we can use the randomizer without testing the random numbers actually produced.

A possible exception is a case in which it is known that a physical randomizer produces the right sort of random numbers, but in which the pertinent probability model is unknown such that one cannot generate the random numbers using software. This case is rather peculiar though and not typical of Monte Carlo simulations. In this peculiar case, we would not object to the suggestion that we have a real experiment.

Humphreys (1994, 112) puts the objection thus: ‘It is the fact that they [Monte Carlo simulations] have a dynamical content due to being implemented on a real computational device that sets these models apart from ordinary mathematical models, because the solution process for generating the limit distribution is not one of abstract inference, but is available only by virtue of allowing the random walks themselves to generate the distribution.’ Humphreys is here concerned with Monte Carlo simulations that trace a probability distribution. The particular interest is the final distribution.

It is true that drawing random numbers (particularly with the aid of a physical gambling device) is not an argument. But a Monte Carlo simulation is much more than drawing random numbers, and what is epistemologically decisive is inference, as we have argued in section 4.

For instance, Dietrich (1996, e.g. 341) reports that certain Monte Carlo simulations were used as a benchmark for theoretical work in the same way experiments are.

As Dietrich (1996, 346) puts it, ‘Monte Carlo experiments share this basic structure of independent variables, dependent variables, and controlled parameters’ known from experiments.

This argument applies not just to Monte Carlo simulations, but to every computer simulation. See Parker (2009, 488–491) for a similar argument.

To outline the objection, we rely on a proposal due to Guala (2002, 66–67) and an emendation by Winsberg (2010, 57–58). A formal analogy contrasts with a material one. The distinction between these analogies goes back to Hesse (1966, 68). Consult also Trenholme (1994) for analog simulations.