Doing comparative case study research in urban and regional studies: what can be learnt from practice?

ABSTRACT

Recent years have seen a vivid debate on the epistemological foundations of comparative urban research. Remarkably, comparative case study research practice has remained unaffected by these wider debates and empirical research processes often stay a ‘black box’. Thus, we identify an unmet need for a critical and transparent reflection of conceptual foundations and empirical processes. Based on a review of EU-funded projects in the field of territorial cohesion, we discuss minimum standards of comparative case study research. These standards encompass the theoretical framework of the study, the objective of comparison, questions regarding the ambition to generalize, the case study selection strategy, and potential trade-offs. We conclude that researchers should be more explicit in their way of carrying out comparative research. Eventually, this transparency supports both a fruitful debate on comparative case study designs and the soundness of academic and policy conclusions.

KEYWORDS:

• Comparative case study research
• methodological minimum standards
• transparency
• territorial cohesion
• urban and regional studies

1. Introduction

‘At the very least, comparative urbanism must be practiced in a conscious manner: comparative conceptual frameworks and comparative methodologies must be explicated and argued’ (Nijman, 2007, p. 3).

This citation skilfully discloses the challenges associated with comparative research and it also applies to comparative case study research. As recent years have seen a renaissance of comparative approaches and an ever-growing body of comparative studies, Nijman’s plea for practicing comparative studies ‘in a conscious manner’ (Nijman, 2007, p. 3) is more than ever relevant. Yet, there is an evident gap between the vivid debate on theoretical demands of comparative studies (Robinson, 2016; Scott & Storper, 2015; Ward, 2010) and the way in which comparative case study research is practiced. In research practice, much remains implicit.

This paper was initiated by our search for guidance on the design of a European comparative case study research on territorial cohesion.¹ Having screened the established theoretical contributions, such as Fontana, Afonso, and Papadopoulos (2008), Blatter and Blume (2008) or Kantor and Savitch (2005), not to mention numerous text books (e.g. Gerring, 2017; Goertz, 2017; Yin, 2014), we reviewed empirical literature on EU-funded projects to find further guidance. We realized, however, that concepts and approaches behind the comparative case study research were rarely discussed. Most comparative case studies are published without a clear explanation of the comparative methodology. This finding is in obvious contrast with the current renaissance of comparative case study research approaches in urban and regional studies. We would have expected these EU-funded case study analyses to have a clear conceptual and empirical basis as particularly projects under the 6th and 7th framework programme (and later Horizon 2020) are applied basic research. Likewise, a critical reflection of conceptual and empirical foundations of comparative case study research seems highly promising for both sound policy implications and academic learning.

For the field of territorial cohesion, we investigate how researchers in completed EU-funded research projects handled the challenge of comparative case study research to see what is done in empirical work. Territorial cohesion is a suitable field for investigating these practices because it is a concept which is clearly linked to EU integration process and EU policy (Sykes, 2008; Zaucha & Böhme, 2019). Likewise, a comparative case study research approach is specifically well suited to offer in-depth insights into promoting and limiting factors in achieving better cohesion. Therefore, research results are of great interest to policy-makers and public funding bodies.

This paper contributes to the current knowledge of empirical comparative case study research in different ways. It first discusses the state-of-the-art of conceptual and epistemological approaches to comparative case study research in urban and regional studies using a wide body of scholarly literature in political science, social science and urban and regional geography. The paper second systematically investigates how these aspirations are addressed in empirical research. We identify an unmet need for transparency and critical reflection in the practice of doing comparative case study research. Based on these findings, the paper third suggests minimum standards which promote transparency regarding conceptual foundations of comparative case study research and their translation into empirical practice.

It should be noted that we discuss the practice of comparative case study research in the light of theoretical demands rather than provide a theoretical contribution to the methodology of comparative case study research. We are convinced that a critical discussion of the means, methods and limits of comparative case study research support both a fruitful debate on comparative case study designs and the soundness of consequent policy recommendations.

The remainder of the paper is organized as follows: Section 2 introduces the debate on the epistemological foundations of comparative case study research and briefly discusses territorial cohesion as an interesting field for investigating the practice of comparative case study research. Section 3 explains how we identified EU-funded research projects on territorial cohesion, how we collected these projects’ scholarly output and how we analysed it. Section 4 provides the results of our analysis and develops minimum standards for empirical comparative case study research. Section 5 concludes.

2. Less than obvious: carrying out good comparative case study research

2.1. Current discussions in comparative case study research

Comparative research has become an established research strategy in urban and regional studies. Pierre (2005, p. 449) argues that comparative research is the ‘most rewarding research strategy […] for uncovering causal patterns of explanation’. It can help to understand the paths of different regions or nations and to detect patterns of convergence and divergence across these (Kantor & Savitch, 2005, p. 135; Nadin & Stead, 2013, p. 1543). Likewise, comparative research is necessary to inform policy makers on alternative policy approaches when facing similar societal problems. For both this reason and its merits of deducting conclusions based upon the understanding of commonalities and differences across places, there is an immense policy interest in comparative research. This interest is evidenced by cross-national Joint Research Programmes or Open Research Areas, such as ESPON projects, or research within the Framework Programmes announced by the European Commission.

Researchers face quite a challenge in implementing a comparative research design as it is not immediately obvious what constitutes ‘good’ comparative case study research. Notwithstanding exceptions mainly from political scientists, few scholarly contributions (e.g. Fontana et al., 2008; Kantor & Savitch, 2005) have taken an analytical and reflective stance on how theory-building has been linked to empirical work in comparative case study research. Strikingly, many comparative project reports of research projects funded by the EU are published without specific reference to the comparative research methodology literature. This finding applies to both the understanding of comparison, the different approaches to comparing and the methods of analysis. Whereas scarce methodological explication might be expectable in applied research programmes, such as Interreg, it is quite surprising in EU basic research programmes designed to draw far reaching policy conclusions.

We see different explanations why empirical comparative case study research is mostly done without publishing how unexpressed – but fundamental – assumptions and frames of reference influence the practice of doing this kind of research. Healey (2012, p. 192) highlights that the own foundations are seldom interrogated as implicit frames of reference. However, these frames of reference need more explicit argumentation today as different epistemological strands exist in comparative case study research (see Table 1).

Table 1. Different epistemological starting points to theorizing in comparative case study research.

‘Traditionally’	‘Recently’
• Conceptualizing the constitution and development of a place as subject to universalized laws	• Space-specific emergence and reproduction of a place; non-linear development trajectories
• Rather fixed theorization of space and scale	• Place is theorized as interconnected; attention to constitutive processes and practices
• ‘Measuring’ of findings against a-priori defined variables; the case as a variation	• A commitment to seeing difference; working with repetition • Including diverse urban experiences across the global North-South divide
• Expectation to draw generalizable findings	• Not necessarily generalizing from the cases/different degrees of generalization
• Affinity to variable-centred comparative case study research	• Affinity to (constitutive) process-centred comparative case study research

Source: own elaboration.

Various authors criticize the rather disappointing results of comparative studies regarding the explanatory value of their findings (Kantor & Savitch, 2005, p. 135; Pierre, 2005, p. 446). Kantor and Savitch (2005, p. 135) summarize that there is ‘not much, not very comparative’ research. They argue that a comparative analysis cannot achieve its potential in exploring, clarifying and explaining specific phenomena across countries and regions without a robust and rigid analytical framework (see left column of Table 1). Any comparative analysis, they claim, needs to be systematic from the beginning, with an explicit a-priori theoretical construct, testable propositions and the use of ‘common categories, concepts or variables that can be measured’ (Kantor & Savitch, 2005, p. 136).

However, such propositions for a-priori hypothesis building and empirical testing have recently been criticized (see right column of Table 1). At the base of the critique is the question how to – if at all – generalize from cases. Hart (2002, pp. 13–14) argues against a traditional concept of comparison that ‘measure[s] “cases” against a universal yardstick’. Robinson (2016, p. 188) states that ‘[t]he starting points as well as the cultures of theorizing’ are at stake in the discourse on the appropriate strategy in comparative research. Rather than ‘trying to “control for difference”’ she argues for a new methodological approach ‘[t]hinking with variation and repetition’ (Robinson, 2016, p. 188) for theory building. Notwithstanding conceptual differences between different authors, current approaches to theorizing in urban geography and planning share a suspicion towards overarching, universally valid assumptions or a general theory of the urban (Hart, 2018, p. 12).

However, and in contrast to traditional approaches, no substantial advice has been given how to link theory building to empirical analyses based on a relational understanding of space. Thus, Lees (2012, p. 157) notes that ‘[r]ecent writings are thick with idealism but thin with the practicalities of everyday urban research’. This might be another explanation why only few scholarly contributions offer substantial advice regarding theoretical and methodological questions in comparative case study research.

2.2. Territorial cohesion as a suitable field of study

Comparative case study research is a rewarding strategy for territorial cohesion research as it can deliver in-depth insights how processes, outcomes or relations are interlinked. Findings on potential (process) mechanisms are invaluable to policy-makers and public funding bodies, who are searching answers to combat persistent territorial disparities in Europe (Rau, Goggins, & Fahy, 2018, p. 266). Consequently, several territorial cohesion studies have been commissioned by the European Commission over the past years. Territorial cohesion is linked to the aim of a harmonious development across the European territory and thus implies a comparative perspective. Hence, it is an interesting field for investigating the practice of comparative case study research.

The origin of the concept of territorial cohesion is linked to EU cohesion policy (Abrahams, 2014, p. 2136). The Lisbon Treaty agreed upon in 2007 consolidated the relevance of territorial cohesion as a key policy aim alongside economic and social cohesion (Nosek, 2017, p. 2158). Following the European Commission, ‘[t]erritorial cohesion aims for more balanced and sustainable development’ (European Commission [EC], n.d.). Despite its long tradition in the policy discourse, territorial cohesion has remained fuzzy: there is no clear-cut and widely accepted understanding, neither conceptually nor regarding its operationalization (Davoudi, 2005; Faludi, 2013; Zaucha & Böhme, 2019). Different actors associate different meanings with territorial cohesion, ranging from an economic competitiveness storyline to a socio-economic convergence storyline, each with different interpretations within and between European member states (Nosek, 2017, p. 2171).

3. Research design and research methodology

As noted earlier, we seek to highlight what can be learnt from the current state-of-the-art of practiced comparative research rather than to provide a theoretical contribution to the methodology of comparative case study research. In this light, we discuss how transparency and reflection can be further encouraged in such a way that empirical case study research contributes more to theoretical, conceptual or methodological foundations of the same. Furthermore, this transparency and reflection should stipulate a similarly vivid debate about ‘doing case study research’ as we find it on theoretical demands of comparative study research.

To reach this aim, we reviewed a sample of EU-projects (Section 3.1) and carefully analysed relevant documents (Section 3.2). Scholarly literature on developing a comparative case study research design (such as Blatter & Blume, 2008; Gerring, 2017; Goertz, 2017; Kantor & Savitch, 2005; Yin, 2014) served as background knowledge and helped to formulate our research questions for the empirical analysis (Section 4). Rather than following a streamlined and deductive process, our research findings were produced in an iterative process, revisiting both theoretical demands on comparative case study research, the documents of the selected EU-projects and data evaluation in a continuous research process.

Reflecting on our methodology, we would like to mention three aspects: First, we refer to the selected projects’ reports as a source for an informed understanding of the current state-of-the-art of the comparative approach rather than pursue an in-depth analysis of the projects’ methodology and results. Second, our intention is not to criticize colleagues’ studies for their missing reference to comparative methodology, but to explore what can be learnt from a reflection of the current state-of-the-art of doing comparative case study research. Third, we do not suggest that there is a universal strategy for comparative case study research. We recognize the variance in epistemological standpoints and framing theories and we argue for the diversity of approaches. Nevertheless, we claim that choices and decisions underlying the final research design need to be transparent to facilitate a critical debate.

3.1. Selection of EU-projects

For investigating the question how researchers handle the challenges of doing comparative case study research, we reviewed a selected set of research projects addressing territorial cohesion. The selection of research projects was guided by three criteria:

1. All projects are funded by the European Commission’s Research Framework. The reason for this limitation is that funding structures are comparable, which would not have been the case if we had included nationally-funded studies conducted anywhere in the world. The focus on projects funded under the 6th or 7th Framework Programme highlights an applied but still basic research aspiration as opposed to schemes such as Interreg, which have a strong application orientation. FP 6 and 7 are designed to facilitate policy recommendations.

2. The selected projects follow a multi-methods approach. The reason is that the empirical challenges would have been too different if we had considered all possible empirical research designs.

3. For similar reasons, we focus on comparative case study research designs.

The projects were identified using the Community Research and Development Information Service (CORDIS) Website² and searching for ‘territorial + cohesion’. The resulting project list was adjusted, excluding those projects which were still ongoing, had a purely quantitative or purely qualitative research design, had a main focus on social or economic rather than territorial cohesion, or which had no regional or urban case studies. Table 2 provides an overview of the chosen research projects.

Table 2. Final set of documents used to find answers how to do good comparative research.

Project acronym (time span) funding project ID (‘grant number’)	Topic/research interest of the project	Analysed number of documents pr. source (date of document extraction: MM/DD/YY)
		Web of Science (11/26/18)	Scopus (11/21/18)	Google Scholar (11/21/18)	OpenAIRE (11/26/18)	Further sources
PROFIT (2004–2007) EU FP6^a 506245	Overcoming the inter­generational transmission of inequalities	0	0	11	0	1
ACRE (2006–2010) EU FP6 028270	Creating or stimulating ‘creative knowledge regions’ in the EU	0	0	9	0	0
SHRINK SMART (2009–2012) EU FP7^b 225193	Social cohesion and the governance of urban shrinkage	0	5	24	0	2
COPE (2012–2015) EU FP7 290488	Minimum income pro­tection policies that help to alleviate the risk of poverty	1	6	14	4	0
IMPROVE (2012–2016) EU FP7 290613	Social cohesion, social policy and social innovation	9	3	34	35	3
CITISPYCE (2013–2015) EU FP7 320359	Combating inequalities through innovative social practices of and for young people in cities	0	0	7	0	1
DIVERCITIES (2013–2017) EU FP7 319970	Increasing the (perception of) positive effects of urban diversity	5	2	11	16	3

Source: own, based on the data sources as provided in the table.

^aEU FP6 – European Commission – Research: The Sixth Framework Programme (2002–2006).

^bEU FP7 – European Commission – Research: The Seventh Framework Programme (2007–2013).

We are aware that EU project consortia and their proposals have to meet certain criteria which are not oriented toward scientific excellence only. Project consortia might be – we purposely exaggerate here to clarify the point – rather partnerships of convenience or necessity than a homogeneous group of researchers sharing a consistent epistemological foundation. Similarly, the choice of cases to be analysed in these projects might follow other than theoretically or conceptually ideal objectives. Yet, against the background of undoubted policy relevance and recommendation aspiration (and expectation) of FP 6 and FP 7, conceptual and methodological clarity should be self-evident.

3.2. Strategy for identifying relevant documents originating from the selected research projects

Our research questions posed to the analysis of the selected projects were: How have researchers discussed their comparative research methodology? How have they reflected upon their frame of reference? How have they dealt with challenges?

We generated a data basis of publications that were produced in the selected EU projects (Table 2) and we reviewed these publications to collect answers to our research questions. This data basis has been created by a search of four sources: Web of Science Core Collection (henceforth Web of Science), Scopus, OpenAIRE and Google Scholar. Additionally, we considered further sources such as CORDIS or the projects’ individual websites, if available.

Our search parameters within these sources were Project Title, Project Acronym and Project ID of the chosen research projects, both individually and in combination. Using these we found an impressive number of 527 documents. We checked these documents to identify implausible or misleading results, such as no project relation, and removed them manually (see Belcher, Rasmussen, Kemshaw, & Zornes, 2015, p. 4 for a similar procedure). This procedure resulted in a reduced data basis of 503 documents. The data collection and cleaning activities were executed and finished in the end of 2018.

We obtained the final set of documents by manually screening both title and abstract of the 503 documents. Precisely, we checked if a document either empirically investigated more than one case in a comparative perspective or referred to the methodology of comparative case study research. If one of these issues was confirmed, the paper was kept in our data basis. This procedure revealed 206 relevant papers and these establish our final set of documents. We scanned the full texts of those ‘final’ 206 documents by searching ‘compar’ (-ison, -e, -ing, -ative), ‘case’, ‘method’ (-ology, -ological, -s) and screening the context of its use.

We are aware of some limitations of this procedure, such as a substantial time lag between empirical project work and the results’ publication. Thus, there might be manuscripts still ‘under review’ implying that research has been conducted but that it is not yet publicly available. However, this time-related limitation applies most probably to just two research projects (ImPRovE, DIVERCITIES) as the other projects ended more than three years ago. We tried to mitigate this aspect by looking at several indices including Google Scholar, which is capable of identifying working paper versions, and OpenAIRE, which contains all official project reports.

4. Empirical analysis

Based on a thorough screening of all 206 documents, we explore in this section how scholars have dealt with theoretical, conceptual and empirical challenges while applying a comparative case study design when analysing territorial cohesion. We first ask how researchers develop their framework for comparing regarding the theoretical and conceptual foundations of comparative case study research (Section 4.1). We then turn to the practice of choosing cases (Section 4.2). Finally, we explore empirical challenges in comparative case study research and how researchers have addressed these in their projects (Section 4.3).

Our analysis is based on (Excel) evaluation tables, which collect the document-based findings according to three categories following Section 4’s structure. In each of Section 4's subsections, we first review which aspects we would expect to be addressed in the project documentation following the recent literature discussion. We second analyse how comparative case study research is done in our selected projects. Based on this analysis, we third suggest standards for comparative case study research that future projects should address to both improve the practice of doing comparative case study research and foster a more engaged reflection and discussion of the same.

4.1. Reference frame and conceptual foundations

As explicated in Section 2, the most influential contributions on comparative case study research originated from political science (DiGaetano & Strom, 2003; Kantor & Savitch, 2005, see Sellers (2005) for urban and regional governance). These contributions were complemented with important contributions from further social science disciplines (Flyvbjerg, 2006; Oxley, 2001; Ragin, 2014). However, later contributions from urban geography and planning theory argue for a fundamental shift in comparative research following a relational, contextualized understanding of the production of space and place (Hart, 2018; Healey, 2012; Robinson, 2016; Ward, 2010). In the light of this debate, we would expect researchers to explicate their own positioning with regards to a particular epistemology.

Referring to our project sample we find scarce reference to a methodological reflection of the researchers’ own positioning towards either traditional or more recent approaches to theorizing (see Table 3 for an overview). This finding may suggest two things: either the own epistemological frame of reference is taken for granted, i.e. researchers perceive no need to outline it. Or essential decisions made during the research process are neither interrogated nor – probably consequently – clarified. However, there is a scholarly debate on the question how theoretical and conceptual foundations of comparative research influence decisions in the research process, the way how to draw conclusions (‘how to theorize’) and whether, and if so how, to generalize (Blatter & Blume, 2008; Robinson, 2016; Ward, 2010).

Table 3. The theoretical framework for comparing.

Project	Positioning towards comparative methodology and generalization aspirations
PROFIT	• Statement on project’s methodology and its aspirations to generalize: explicitly excluding a claim to generalize on the basis of cases (Warzywoda-Kruszyńska et al., 2007, p. 12) • Affinity towards a theory-testing/hypothesis-testing approach (Warzywoda-Kruszyńska et al., 2007, p. 22)
ACRE	• Hypotheses-building and empirical testing (Kovács, Murie, Musterd, Gritsai, & Pethe, 2007, p. 14; Musterd & Gritsai, 2010, Section 2) • No explicit statement on comparative case study approach, but encompassing description, e.g. in Musterd and Gritsai (2010, Section 5)
SHRINK SMART	• Developing working models and empirical testing (model provided in Haase et al., 2016) • Shift over the course of the project towards process-orientated, context-sensitive research (Bernt et al., 2014; Haase et al., 2016, p. 99) • Explicit statement on comparative case study approach utilizing the model developed in Haase et al. (2016) (Cortese, Haase, Grossmann, & Ticha, 2014, p. 2054; Haase, Rink, Grossmann, Bernt, & Mykhnenko 2014, p. 1521)
COPE	• Research questions guided by theory (Jessoula, Sabato, Agostini, & Madama, 2014, p. 34) • Causal Process Tracing (Jessoula et al., 2014, p. 61) • Information on means of comparison, associated challenges and guidelines for field work (e.g. Angelin, Johansson, & Koch, 2014; Jessoula, 2015, p. 497; Kozek & Kubisa, 2014, pp. 78–79)
IMPROVE	• Explicit reference to variation-finding, hypothesis-testing in the beginning, changing to ‘individualising comparing strategy’ (Oosterlynck et al., 2016, pp. 4–5) • Contextualized, ‘contingent’ insights within the case study research (Sabato & Verschraegen, 2016, p. 9) • Extensive discussion of the project’s positioning and comparative approach (Oosterlynck et al., 2016)
CITISPYCE	• Contextualized insights; ‘triangulation approach’ (Jubany & Güell, n.d. (2016), p. 49) • Finding commonalities and differences based on qualitative analysis of case study reports (Hussain & Higson, 2014) • Comparing along analytical categories and research dimensions • Explicit reference to action-research approach and qualitative framework of analysis (Jubany & Güell, n.d. (2016))
DIVERCITIES	• Contextualized, ‘contingent’ and not necessarily generalizable insights (Tasan-Kok et al., 2013, pp. 21, 63) • Research questions guided by theory (Tasan-Kok et al., 2013, p. 61ff.) • Explicit reference to relational perspective on cities and neighbourhoods (Bolt, 2017, p. 6; Tasan-Kok et al., 2013, p. 61ff.) • No explicit statement on comparative case study approach found

On one end of the spectrum of how to do comparative research is a variable-centred (co-variational) approach, often called ‘traditional approach’ (see Table 1). Here, ex-ante propositions about mechanisms and causalities are empirically tested in case studies. Hence, the research process is deductive. Differences or commonalities between cases are explained based on observed co-variation between factors (independent variables) and effects (dependent variables). Defining variables is commonly associated with quantitative research, but it has been done in mixed-method studies (for instance, see Kantor & Savitch, 2005), too. Co-variation-based research aims to draw general conclusions.

On the other end of the spectrum of how to do comparative research are more recent approaches as shown in Table 1 that either focus on explaining and understanding causal chains within a specific context or concentrate on understanding the constitutive structures of the researched cases. Thus, comparing is based on understanding the case and its constitutive structures within its context. Hence, conclusions are contingent upon time and space. Utilizing the identified case-specific insights researchers then look for commonalities and differences across their cases. In doing so, their research questions work as an analytical framework or ‘guideline’. Theorizing thus goes from the specific to the general (inductive process) and there is not necessarily a claim to generalize.

We find characteristics of both approaches in our sample (see Table 3). One could place the ACRE and the SHRINK SMART project within the area of a rather variable-centred and co-variational approach. Hints for this positioning may be found in Musterd and Gritsai (2010, p. 2) or in Rink, Haase, and Bernt (2009, p. 3). In contrast, CITISPYCE, ImPRovE and DIVERCITIES are rather situated at the other end of the spectrum due to their claim for contextualized insights (see e.g. Tasan-Kok, van Kempen, Raco, & Bolt, 2013, p. 65 for the DIVERCITIES project).

However, assigning the projects to certain approaches is tricky using the publicly available project documents only because the corresponding researchers’ positioning is rather implicit than explicit there. While the research aim to compare is obvious, it is not sufficiently clarified how comparisons are done. In fact, researchers rarely specify their conceptual framework and their comparative research design. We identified two projects, ImPRovE and Shrink Smart, where the comparative case study approach was explicitly discussed (Kazepov, Saruis, Wukovitsch, Cools, & Novy, 2013; Rink et al., 2009, pp. 27–28). Brief information is also available for the COPE (e.g. Johansson, Panican, Angelin, & Koch, 2013, pp. 18–19; Kozek & Kubisa, 2014, pp. 78–82) and the PROFIT (Warzywoda-Kruszyńska et al., 2007, Section 2.2; p. 12) project. Among the later projects, DIVERCITIES explicitly refers to a relational perspective on cities and neighbourhoods (Bolt, 2017, p. 6).

Interestingly, there are also shifts in the comparative strategy of two other research projects, which might relate to recent discussions on the way how to theorize in comparative research: We identified a shift from ‘variation finding to an individualizing comparative strategy’ (Oosterlynck et al., 2016, p. 5) in the ImPRovE project and a shift from a variable-centred approach towards an approach that gives greater attention to the constitutive processes of cases in the Shrink Smart project (e.g. Bernt et al., 2014, p. 1753; Haase, Bernt, Großmann, Mykhnenko, & Rink, 2016, p. 99).

Summarizing the empirical evidence obtained from the selected projects’ documents, the approach to comparing is fairly undisclosed as the information is scattered in different documents. Most often, we needed to scan all project-related documents before being able to assume which epistemological approach to comparative case study research the teams had taken. We purposefully wrote ‘assume’ rather than ‘find’ or ‘conclude’ because much of the necessary information is implicit. We thus suggest that projects address two questions, which establish our first two out of five minimum standards:

• (1) What is the study’s frame of reference and starting point, ideally regarding concept, epistemology and ontology? Both researchers and addressees of the research findings should be clear about main cornerstones of the work, their rationale and limitations. These cornerstones comprise both a positioning of the study within the landscape of current debates on comparative research and a justification why researchers have chosen their approach.

• (2) How is theorized and is there an ambition to generalize? We ask researchers to reflect how to move from the specific to the general and, if so, how to generalize from cases. Thus, if generalization is an objective, we ask researchers to critically reflect on the coherence between the aspirations towards generalization and theory building on the one hand and the empirical and methodological case study research design on the other hand.

4.2. The practice of choosing cases

Two prominent strategies for selecting cases are the ‘most different’ and ‘most similar’ systems design. Both of them are rooted in pre-defined similarities or differences between cases, but they are not uniquely defined. Rather, several varieties exist (Fontana et al., 2008, p. 524; Seawright & Gerring, 2008; Tilly, 1984, pp. 81–83; Ward, 2010, p. 474; Yin, 2014). Thus, the most similar/most different system design plays a role in a ‘traditional’ variable-centred approach. Case selection is often grounded in statistical analyses that quantify the cases and thus reveal the rationale for which they were chosen. Based on a hypothesis, which defines the relationship between independent and dependent variable(s), either very similar or very different cases might be chosen, while controlling for remaining factors which might influence the outcome.

In recent, relational approaches to comparative research, the most similar/most different systems design may play a role, but more essential is the cases’ promise of crucial insights into the phenomenon or causal configuration of interest. Researchers who theorize space in a relational way will be sceptical regarding a selection of cases based on similarities or differences because ‘geographical relational methods read places through one another (rather than cataloguing similarities and differences)’ (Elwood, Lawson, & Sheppard, 2016, p. 749). Rather, researchers draw on the diversity of urban or regional experiences and include especially those cities or regions that have been largely disregarded in comparative case study research as these cities/regions have been perceived to be too different in their cultural, political or economic contexts to be reasonably compared.

Looking at our sample of research projects (see Table 4), we see that some content-wise explication is provided regarding the choice of cases. The ‘most different’ and ‘most similar’ systems designs play a role but in a pragmatic rather than straightforward way. Thus, predefined characteristics of similarity in relation to the respective research interest (e.g. shrinking/declining places) and/or existing classifications of differences in context (e.g. typologies of different European welfare systems based on Esping-Andersen’s 1990 work) are used as a starting point.

Table 4. The practice of choosing cases.

Project	Cases	Case study selection
PROFIT	8 medium-sized towns	• ‘deliberately selected towns’, as data basis too limited for controlled choice • Mix of similarity (basic city characteristics, disadvantaged cities) and difference (social welfare systems; social policies; East-West) (all information based on Warzywoda-Kruszyńska et al., 2007, pp. 5, 12)
ACRE	13 city regions	• Thematically chosen (creative knowledge regions) for hypotheses testing • Familiarity seems to play a role (cases are often equivalent to the location of academic project partners) (Musterd & Gritsai, 2010) • Mix of similarity (second range global cities or below) and difference (geographical and structural variety across Europe) (Musterd & Gritsai, 2010, p. 23)
SHRINK SMART	7 urban regions	• Thematically chosen (shrinking/shrunk cities), although acknowledgement of some pragmatism in the choice of cases • Mix of similarity (shrinking/shrunk cities) and difference (geographical variety across Europe; both cities having experienced long-term shrinkage and rapidly shrinking cities) (all information based on Bernt et al., 2014, p. 1753; Haase et al., 2016, p. 91)
COPE	5 post-industrialist cities	• Thematic choice (precisely, cities which implement or adopt national active inclusion policy) • Establishing common criteria and let national partners make the actual choice of the case study • Selection criteria to also ensure variation in terms of (welfare) policy regimes and relationship to Europe (Jessoula, 2015, p. 492; Johansson et al., 2013, p. 17)
IMPROVE	31 social initiatives in 8 countries	• Choosing thematically ‘interesting’ cases in accordance with a clear-cut theoretical background (Kazepov et al., 2013; Sabato & Verschraegen, 2016) • Cases partly selected from a call that the researchers sent to many different civil society actors • Ensuring variation in terms of geographical and structural context
CITISPYCE	2 neighbourhoods in each of 10 selected cities	• Two-stage process for city selection, in which familiarity and active engagement of cities played a role (some of these cities are consortium partners themselves) (‘Citispyce Repository of Case Studies’, n.d.; Robinson, n.d. (2016), p. 2) • Mix of similarity (focus on young people in disadvantaged neighbourhoods) and difference (geographical and structural context)
DIVERCITIES	13 European cities, including 10 policy initiatives in each city	• Familiarity seems to play a role (cases are often equivalent to the location of project partners) (Maloutas & Souliotis, 2015) • Selection of policy initiatives within selected case study area following comparable criteria (see e.g. Barberis, Grossmann, Kullmann, Skovgaard Nielsen, & Hedegaard Winther, 2019, pp. 965, 969 for details) • Mix of similarity (larger cities, deprived, diverse and dynamic research areas) and difference (geographical/structural context of research areas; themes and nature of policy initiatives) (Barberis et al., 2019)

Rather than selecting cases based on encompassing statistical analyses or a systematic literature review, pragmatic reasons seem to be a frequently chosen strategy. We acknowledge that there might be good reasons for choosing a case the researcher is familiar with (see also Blatter & Blume, 2008, p. 340) who write that ‘access to sources and actors is indispensable’). However, to the best of our knowledge these familiarity aspects are not mentioned as a strategy and seldom reflected in the corresponding project documents. As an exception to the rule, Haase et al. (2016, p. 91) reflect that ‘[t]aken together, the 10 cases represent a collection of individual exemplars, rather than a strictly systematically selected and methodologically controlled sample. There are both theoretical and practical reasons for this’, referring to both the theoretical discussion and practical constraints in finding adequate cases.

Nevertheless, the case study selection procedures remain shallow in the analysed projects because readers cannot see from these projects’ publications which selection strategies were available, which selection strategy/-ies have been chosen, which cases have been shortlisted following the selected strategy/-ies and which of these cases have finally been chosen. Together with the case study selection procedure, decisions have to be made what to compare within and/or across those cases, i.e. what are object and objective of comparison? While the object of comparison is naturally defined by the research questions, the objective may be less clear cut.

Researchers should thus – and this is our third minimum standard – answer the question: (3) How have cases been chosen? We are aware that ‘[i]n actual research practice, of course, cases are chosen for all sorts of reasons, from convenience and familiarity to fascination and strategy. Once chosen, however, the case must be justified – shown to be a case of something important’ (Walton, 1992, p. 125). According to both this and our own analysis, the case study selection strategy should be discussed and the selection process reflected as it may crucially influence the policy recommendations based on the (comparative) case study analysis.

A related key question, and thus a fourth minimum standard, is to explicate what is being compared: (4) Within each case, what constitutes the case, i.e. the objective of comparison? Is it, for instance, a process, a connection, a situation, an outcome or any combinations of these? All these objects of comparison can eventually constitute a case. These questions are certainly dependent on the research interest, but they nevertheless require consideration and influence case study selection. At the very least, there should be a discussion what should reasonably be compared within the research context.

4.3. Challenges in the research process

Based upon our literature review, we see several challenges which are specific to comparative case study research. We discuss two aspects here, one of a more methodological nature (the so-called ‘depth versus scope’ debate, see Kantor & Savitch, 2005) and the other one linked to the practicalities of working in inter- and transdisciplinary projects (see e.g. Rau et al., 2018).

All research consortia are confronted with the question how many cases to choose and how to balance contextualized case study research with the demand for generalizing conclusions (i.e. depth vs. scope). The latter aspect particularly pertains to explanatory case studies rather than to case studies whose aim is an informed juxtaposition of affairs. It may be assumed that this ‘depth versus scope’ trade-off is more relevant for traditional variable-centred approaches than for more recent process-centred approaches. The thickness of case studies in a variable-centred approach is only achieved at the expense of abstracting and generalizing. Contrastingly, the thickness of case study work is the condition for abstracting and generalizing in process-centred approaches to comparative research (Blatter & Blume, 2008; Robinson, 2016). Consequently, the choice of the number of cases inhibits a trade-off between studying a case in its individual context and a cross-case analysis in both approaches. In our sample of projects, there was no reference to the ‘depth versus scope’ debate although we identified hints towards one or the other ‘end’ (see Table 5).

Table 5. Dealing with empirical challenges.

Project	Reflective statements on/discussion of empirical challenges and other issues
PROFIT	• Problems in comparability of data • Preferring depth to scope
ACRE	• Limits in the generalisability of findings (Musterd & Gritsai, 2010, p. 32) • Admission that interview-based quantified indicators may be subjective (Musterd & Gritsai, 2010, p. 42)
SHRINK SMART	• Learning processes in the project’s conceptual approach to comparing (Bernt et al., 2014, p. 1753)
COPE	• Data collection challenges regarding their comparability and functional equivalence, respectively (Johansson et al., 2013, p. 19) • Statements on terminology, terminological and conceptual differences across disciplines to establish a common ground for the project (Hvinden & Halvorsen, 2012)
IMPROVE	• Challenges linked to a mixed-method approach: issue of timing and synchronization of different work (Kazepov et al., 2013, p. 43) • Learning processes in the project's conceptual approach to comparing and justification on the adaptation of the project’s strategy (Oosterlynck et al., 2016, p. 5)
CITISPYCE	• Adaptations in guidelines for empirical work (Hussain & Higson, 2014, p. 4)
DIVERCITIES	• No explicit statements found

Researchers in the ImPRovE project reflect ex-post upon the challenges linked to a mixed-method approach and the gap between intentions and final outcomes (Kazepov et al., 2013, p. 43). We did not find similarly open and reflective statements for other projects. Another example is COPE where they establish a common terminology or at least understanding of terms and concepts at a rather early stage of the project. This is useful, but a reflective stance towards the end of the project, with a focus on what worked out and/or necessary modifications would have been even better.

Given the project time frames and the available resources (full-time equivalents, financial endowment etc.), we assume that a compromise on how many cases to choose and thus an ‘aspiration vs. pragmatism’-consideration was also likely. But there is little self-reflectivity on these aspects in the official project reports or further documents.

Similarly, we assume that different disciplines and disciplinary standards within the research team may have led to negotiating aspiration vs. feasibility vs. pragmatism, too. Yet, these negotiations are rarely disclosed in comparative studies. This finding is remarkable as the European Commission has advocated for inter- and transdisciplinary production of knowledge over the last years. The research consortia behind the projects in our sample were thus faced with the practicalities of working with different disciplines and disciplinary standards.

To sum up, there is just sporadic reasoning and reflection across the project sample on project-specific challenges. For the sake of clarity, and indirectly credibility, we therefore encourage researchers to answer – as a fifth minimum standard –the question: (5) Which trade-offs occurred and how have they been addressed? Trade-offs may arise between case study research requirements and comparative analysis, or between seizing the potentials of interdisciplinarity and following a pure, discipline-specific tradition. In a similar vein, if a self-reflective perspective on project-specific practical experiences and related learning processes became a routine, this would facilitate an in-depth understanding of the study under consideration and permit other scholars to further develop the selected methodology.

5. Conclusions

Our study has raised remarkable observations regarding the conclusions for researchers and for EU-research policy: Searching for guidelines how to do good comparative case study research, our starting point was a 10-year-old call for a ‘conscious’ conceptual and methodological development of comparative urban research (Nijman, 2007; Ward, 2008). Based on a literature review and the analysis of recent research projects on territorial cohesion funded by the EU, we reveal a persistent missing reflective stance in empirical comparative case study research.

But such a reflective stance is more relevant than ever considering recent epistemological debates and the relational turn in urban geography, regional and planning studies. Therefore, an increased need exists for an open and critical debate about different approaches used in urban and regional research, about their opportunities and limitations. While not declaring ourselves in favour of one or the other approach to comparative case study research, we argue that the choices and decisions underlying the research design need to be clarified. Whatever approach is taken, the produced knowledge will be somewhat limited. Therefore, other researchers can only utilize and build upon this knowledge if approaches and choices are deliberated, if consequential limitations in the results and their interpretations are acknowledged and if these reflections are disclosed.

Therefore, we have suggested five minimum standards in the form of five questions to be answered regarding cornerstones in the research process. These minimum standards serve three purposes: First, other researchers can easily follow the line of argumentation of projects and thus better contextualize or evaluate the obtained results. Second, transparency in the methodological decisions facilitates debates and further development of empirical research approaches as it permits insights into opportunities and limitations of certain research designs. Third, this transparency provides a link between empirical research (pragmatism) on the one hand and theoretical considerations of ‘ideal’ comparative case study research on the other hand, thus enhancing mutual progress.

We are aware of some challenges and potential limitations that our minimum standards entail when they are to be put into practice in future empirical case study work. There is a gap between theoretical debates, empirical case study research and funding practice and (implicit) expectations, which might make it too demanding for entire research consortia to explicitly position their research in the epistemological and ontological context of current debates. It would nevertheless help others (for instance, for project review or meta-analysis) to document the chosen approach and be as open and transparent as possible about the cornerstones and decisions in developing the comparative research methodology for the project.

Based on our findings, and so to say a ‘spin-off’ result of our research, we argue that more emphasis is needed regarding the documentation of EU-funded research projects to make results detectable. While creating our list of documents from the selected EU projects, we realized that neither CORDIS nor OpenAIRE provided us with a complete list of all project-related documents. Obtaining this list turned out to be tedious: some reports are available via the project’s (temporary) website, other, rather technical reports via CORDIS, and yet other publications are not listed there at all. For these reasons, valuable research output in terms of comparative findings, drawbacks, methodology, best practices etc. remains unwarranted.

This finding calls for further consideration from both the research community and EU funding institutions as EU-funded research has allowed to create an enormous wealth of knowledge on comparative case study research in the field of European territorial cohesion that ought to be increasingly brought into the discussion. Meta-analyses of the research results in general and the applied research methodology in particular would permit both scholars and the study’s addressees to benefit from this pool of knowledge.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the European Union's H2020 Society research and innovation programme under Grant Agreement N° 727097. The project's title is “Resituating the Local in Cohesion and Territorial Development”, its acronym is “RELOCAL” and the official Project-ID is “727097”. Earlier versions of this paper have been presented at the Association of European Schools of Planning (AESOP) Annual Congresses 2019 and 2017 as well as the Annual Meeting of the Association of American Geographers (AAG) 2018.

Notes

1 The study is called RELOCAL (Resituating the Local in Cohesion and Territorial Development). For more information visit www.relocal.eu.

2 https://explore.openaire.eu/search/find/projects.