When You Can't Find the Perfect Match: Using the Accumulated Most Similar Design in Case Studies

Abstract

What happens when you want to use a most similar case study design for a small-N study, but you cannot find a particular pair of cases where all of the relevant, competing explanations are held constant? It is proposed here that scholars and teachers could employ or teach the “accumulated most-similar/crucial case design.” This design uses two or more partially flawed most similar designs to work like a “prefect” most similar design. This method allows you to hold all the rivals or alternative explanations constant in at least one match. It also can be employed with a crucial case aspect. This methodology is outlined theoretically, and then it is illustrated with the study of trade orientation in four matched pairs of sub-Saharan African countries from 1966–1986. The article shows how you can “fix” the most similar design by expanding it when the “perfect” design eludes us. This approach should be of interest to scholars and teachers of comparative politics, international relations, methodology, regional experts, and anyone interested in using or teaching small-N case designs. It can be taught to upper division undergraduates or graduate students as well as be used in research.

Keywords:

• comparative case studies
• methodologies
• most similar design
• overdetermination
• pedagogy
• small-N studies

Notes

Bold letters show where variables are held constant.

∗Crucial case (a variable that makes the opposite prediction for the dependent variable than does the critical independent variable [MSO]).

⁺After 1978 in Nigeria and after 1980 in Zimbabwe.

⁻The first letter refers to the first country in the match; the second letter refers to the second one.

For descriptions of the most similar case designs, see Przeworski and Teune (1970). For similar ideas, see Lijphart (1971), Eckstein (1975), George (1979), Geddes (2003), and George and Bennett (2004). See also Manheim, Rich, and Willnat (2002). For science behind case studies, see King, Keohane, and Verba (1994), esp. pp. 118–121. For critiques of King, Keohane, and Verba, see Laitin (1995), Caporaso (1995), Collier (1995), Rogowski (1995), and Tarrow (1995). For response, see King, Keohane, and Verba (1995). For a deep review of the strengths and weaknesses of the most similar design, see George and Bennett (2004), also King, Keohane, and Verba (1994). For qualitative approaches to case studies, see Yanow and Schwartz-Shea (2006).

We use the term most similar design as does Przeworski and Teune (1970), though Mill's (1843/2002) original term for a most similar design was the “method of difference.” For Mill, the difference was in the dependent variable; for Przeworski and Tuene, the similarities were in the rival/alternative explanations. For further explanation, see George and Bennett (2004, 153–160).

Positive degrees of freedom (df) is calculated with the equation: Number of cases – number of variables = degrees of freedom. Most feel that case studies must have at least one degree of freedom to draw inferences. Large-N studies usually have 30 or more degrees of freedom. For an overview of this idea, see George and Bennett (2004, 28–34).

See Huntington (1991) for snowball effect.

For diffusion and escalation of ethnic conflict see Lobell and Philip Mauceri (2004) as well as Lake and Rothchild (1998). For diffusion in Africa, see Quinn (2004).

In my undergraduate comparative class, and over a dozen years, I have had only two or three students who used the most different design, in contrast to nearly all the others choosing a most similar design or a large-N study.

For a discussion and review of the problems of overdetermination see Lopez (1992).

That is eliminating alternative explanations though controlling them as opposed to logic or counterfactuals or process tracing and the like.

Term (and design) taken from Quinn (2002). It is the only example of the accumulated most similar/crucial case that we are away of. The term aggregated most similar (crucial case) design would also be descriptive.

For a deeper description of a crucial case, see Eckstein (1975). For critique, see King, Keohane, and Verba (1994, 209–213).

For crucial cases, see especially Eckstein (1975) and George (1979). Strictly speaking, a rival hypothesis is one in which the original hypothesis and the contending one are mutually exclusive. However, alternative explanations are ones that may be part of the explanation along with the original hypothesis. This analysis will use both terms interchangeably as alternative; though both are addressed as important in the research design.

For theory infirming and confirming, see Collier (1993). Crucial here does not indicate the importance of the case, rather it is a case where two rival explanations are in the same case and they make opposite or competing predictions about the dependent variable. It is crucial insofar as one or another rival will be infirmed or confirmed, or mixed results will emerge.

Cases should not be picked by their outcomes, strictly speaking. See Geddes (2003), especially Chapter 3.

The design is quasi-experimental and not experimental, as no pretesting is possible and the experimenters have no control over the administration of the independent variables.

See endnote # 4.

Ideally, there would be no alternate explanatory variables, but this is often not the case, but one can still discount the variable that did not predict the correct direction of Y.

Przeworski and Teune preferred the most different systems design because any most similar match can be overdetermined. We are trying to address this very issue with this approach.

Logic and counterfactuals cases can also be introduced; though none are part of a quasi-experimental design where we act as if we had control over all of the important independent variables. This does not suggest that they are not rigorous, just not part of normal quasi-experimental design. For example, see Fearon (1991). Also we are not dealing with problems of endogeneity, as neither does a “perfect” most similar case study. For issues of endogeneity, see King, Keohane, and Verba (1994).

Manheim, Rich, and Willnat (2001, 23).

However, it will not necessarily eliminate unknown causal agents as these are matched pairs and not randomly selected, necessarily (unless the unknown covary with the known in the proper direction).

We also assume that rival or alternative hypotheses are held to be generalizable throughout a region or for a period of time (and are therefore at least midlevel theories). This allows them to be rigorously tested against a new, competing hypothesis using this design. For problems with Mill's method of difference/most similar design, see George and Bennett (2004, 155). They state that three demanding assumptions must be met: “First the causal relation… must be a deterministic regularity involving only one condition that is either necessary or sufficient…. Second, all causally relevant variables must be identified prior to analysis… Third, cases that represent the full range of all logically and socially possible causal paths must be available for study.”

See endnote 21. We feel that this language is more accessible.

For majority state ownership as a concept, see Quinn (2002). Others that have written on state ownership and economic policy, including the World Bank (1981), Sandbrook (1993), Grosh and Mukandala (1994); though none of these have used the state as level of analysis. Quinn (2002) argues that political elites came to own scarce factors of production (capital intensive industries) and in Africa these sectors are scarce. Following Rogowski's (1989) logic, these political elites should prefer inward-oriented policies as would private owners. But see Quinn (2002) for full argument.

So Quinn established the covariance in an alternative setting, though it need not have been done ahead of this use of cases. The covariance could come from the cases themselves, but this book uses a series of mixed methodologies to increase rigor.

For theories of urban bias see Lipton (1976), Bates (1981), Berg in World Bank (1981). For retrospective views, see Varshney (1993). In a nutshell, urban groups overcome their collective action costs and obtain their preference policies of cheap imports and cheap food. The agricultural groups cannot overcome their collective action costs and do not obtain their preferred export-promoting policies. See previous citation as well as Olson (1965).

For a description of the importance of large-scale farmers in policies see Bates (1981) and Skålnes, (1995). It is assumed that these groups overcame their collective action costs and could get their preferred policies.

For the role of elites as large-scale farmers see Lofchie (1989) and Widner (1993). Here too, it is assumed that political elites that own land can implement policies that favor their economic interests as they easily overcome collective action costs. And most agriculturalists in each of these countries grew export crops.

See Young (1982), Killick (1978), and Arndt (1987, esp. Chapter 5); and Sigmund (1972). The assumption is that more classically liberal states follow outward-oriented policies and that more communists ones are more autarkic. Others fall in the middle.

See especially, Easterly and Levine (1997), World Bank Mimeo (1996, 2), available at http://www.worldbank.org/html/prdmg/growth_t.htm. For political ties see Mozaffar, Scarritt, and Galaich (2003), as well as Posner (2004). The argument for Easterly and Levine is about policies that create patronage to be dispenses. The more fragmentation, the more the number of players needing patronage, the more inward-oriented the policies as these policy distortions created points of patronage.

Given this choice of times and countries, many other variables are held constant as is the case with regional studies within a particular time: e.g., colonial past, British colonial past (Zambia/Kenya), French colonial past (Congo/Côte d'Ivoire), low level of initial development for all of Africa, relative autonomy for political elites to set policy as structural adjustment was introduced systematically in the late 1980s (Lancaster 1989), and real enforcement did not begin until the 1990s (van de Walle 2001). Also, Englebert's (2000) work on legitimacy was not directly addressed; though in all of the matches besides the one with Botswana, it was held constant (i.e., Zambia/Kenya, Nigeria/Zimbabwe, and Congo/Côte d'Ivoire—none of these countries were “legitimate” as he defines it). However, his variable would not have been held constant in the Zaire/Botswana case, but it was in each of the others. This period also controls for some ideological changes and the effect of changes following the end of the Cold War by taking place prior to this transition.

No single variable is accepted as the best measure of inward- or outward-oriented policies. Therefore several indices were offered and the reader could judge the relative inwardness or outwardness of the policies for each country.

The sources for each data source are available in Quinn (2002).

The book listed post 1980 Zimbabwe as not having a political elite with significant land ownership, and this was called into question. However, regardless of how this is coded, the conclusions from these matches would remain the same. Elite ownership of land was controlled for in several cases. In fact, the inclusion of a “yes for elite ownership” would make this match a most similar for this variable, instead of a crucial case. Regardless of the coding, the overall inferences remain intact. The results from the matches would not change whether or not Zimbabwean (ZANU/ZAPU or ZANU-PF) leaders had significant farming interests, since it would then become a most similar design for the latter period for both countries. At the time, little empirical evidence was available to make this claim. But clearly there was elite ownership of significant agricultural resources in Rhodesia.

Interestingly, as a share of total capital the government of Côte d'Ivoire had a large amount of ownership, though almost always a minority share in large-scale industry. See Rapley (1993) and Quinn (2002).

Since currency overvaluation was a key indicator of inward-oriented policies, we matched Côte d'Ivoire with Congo instead of Ghana, as so many do. The CFA currency union had a majority French vote and African political elites could not pursue this policy. See Bach (1983) for an explanation and background. CFA stands for Communauté française d'Afrique (French Community of Africa).