Comparative multilevel analysis: proposal for a methodology

Abstract

This article presents a new methodology for multilevel analysis using a small number of cases, named Comparative Multilevel Analysis (CMA). A classic problem in comparative studies has been the presence of too many variables and too few cases. One traditional solution to this problem has been to study subsystems within a system. However, the approach has fundamental limitations: it cannot analyse subsystems from different contexts, nor can it determine how conditions on the system level influence subsystems. By proposing four additions to traditional methodology, this article offers a new method of comparing subsystems from different contexts in order to analyse the effect of context on subsystems. The author also illustrates how CMA can be combined with Qualitative Comparative Analysis and Fuzzy‐set, thereby enabling these methods to be used in the study of subsystem and context effects.

Keywords:

• Comparative Methods
• Multilevel Analysis
• Small–N Methods
• QCA
• Fuzzy Set
• Regional Analysis

Notes

1. One can also do an intracontextual comparison if cases with the same contextual properties from different systems are included in the analysis.

2. The aim of this example is not to actually explain regional differences in voter turnout. Instead, the aim is to illustrate the method of intrasystem comparison.

3. QCA has also been criticised in several aspects (Peters, 1998). For example, the outcome of the analysis is sensitive to (1) how the dichotomisation of the variables is done and (2) cases that are deviant or outliers; and the outcome also has a tendency to (3) overestimate the difference between necessary and sufficient variables, and (4) underestimate the variation between cases or factors (as a consequence of the dichotomisation).

4. In Boolean logic, multiplication means ‘and’ and addition means ‘or’. For example AC means ‘A and C’, while A + C means ‘A or C’.

5. CMA may also complement statistical methods for multilevel analysis, which make it possible to analyse sets of surveys from different contexts and study how different attitudes or behaviours are influenced by context (Goldstein, 2003; Iversen, 1991; Kreft & de Leeuw, 1998; Snijders & Bosker, 1999). CMA also provides opportunities for using multilevel design in system comparisons with few cases.