Can sample size in qualitative research be determined a priori?

ABSTRACT

There has been considerable recent interest in methods of determining sample size for qualitative research a priori, rather than through an adaptive approach such as saturation. Extending previous literature in this area, we identify four distinct approaches to determining sample size in this way: rules of thumb, conceptual models, numerical guidelines derived from empirical studies, and statistical formulae. Through critical discussion of these approaches, we argue that each embodies one or more questionable philosophical or methodological assumptions, namely: a naïve realist ontology; a focus on themes as enumerable ‘instances’, rather than in more conceptual terms; an incompatibility with an inductive approach to analysis; inappropriate statistical assumptions in the use of formulae; and an unwarranted assumption of generality across qualitative methods. We conclude that, whilst meeting certain practical demands, determining qualitative sample size a priori is an inherently problematic approach, especially in more interpretive models of qualitative research.

KEYWORDS:

• Sampling
• sample size
• qualitative research

Notes

1. We use this term broadly, to embrace ‘codes’, ‘categories’, and similar terms.

2. Some such recommendations relate not to the number of informants but to the number of interviews with an individual informant. For example, Spradley (1979, p. 51) recommends at least six to seven one-hour interviews for an ethnographic study.

3. Although these authors indicate that their model applies to the planning of a study, it is not solely focussed on the prior determination of sample size; they note that the adequacy of the sample size should be continuously reassessed during a study.

4. The binomial distribution is a probability distribution used for binary variables, i.e. those in which an observation can take one of two possible values, such as ‘present’ versus ‘absent’.

5. Monte Carlo simulations estimate the sampling distribution of a particular statistic by drawing numerous random samples from a simulated population of values (Mooney, 2004).

6. More specifically, a theme accumulation curve was constructed, such that a value of 0.05 for this curve (i.e. one new theme for each 20 additional participants) allowed 97.5% of themes to be identified whilst limiting the inclusion of further participants who would not yield further themes; this was proposed as a possible stopping criterion for sampling.

7. Interestingly, an early example of this approach (Romney, Weller, & Batchelder, 1986) focused on calculating the number of participants in terms of investigating their knowledge – the truth or falsity of their responses to specific questions. Moving from this to matters of belief or experience, on which qualitative research characteristically focuses, is questionable.

8. Hagaman and Wutich (2017) are, however, explicit that their themes are descriptive.

9. Higher-order themes, owing to their greater theoretical abstraction and the fact that they are likely to subsume a number of lower-order themes, might be more readily assumed to encompass most or all accounts. However, these numerical and statistical approaches to sample size tend to focus on lower-order themes.

10. The identification of themes by an inductive process is most commonly associated with grounded theory (Glaser & Strauss, 1967). Initially, the process of analysis in grounded theory is indeed inductive, but as theoretical categories evolve, a more abductive logic is employed whereby instances of data are related to the theoretical category with which they best fit (Charmaz, 2009), thereby allowing these categories to be further developed and refined. Accordingly, we are using ‘inductive’ in a broad rather than a narrow sense.

11. Curiously, despite proposing that codes can be foreknown in terms of their number and probability of occurrence, van Rijnsoever (2015) sets his approach within an inductive approach, aligned with the principles of grounded theory.

12. Malterud et al (2016, p. 1754) acknowledge that the determinants may have a ‘mutual impact on each other’, but this is not explicated within the model that they propose.