Methodological Challenges for Comparative Welfare State Research: Capturing Intra-Country Variation in Cross-National Analyses

Comparative Welfare State Research: beyond Methodological Nationalism

Scholars of welfare state and public policy have repeatedly emphasized the value and need for comparative approaches (Engeli and Rothmayr Allison 2014; Geva-May et al. 2018). In contrast to other fields of policy studies, welfare state research has taken a decisive comparative turn already in the 1990s, which has been productive in terms of theory development and conceptual and methodological innovation. Two sets of questions have been particularly central to the field: whether cross-national variation in social policy outputs and outcomes could be meaningfully synthesized through welfare regimes typologies; and how to explain different national trajectories of development and reform of welfare state institutions. In tackling these questions, welfare state scholars have adopted a wide array of methods including historical analysis, case studies, configurational and statistical techniques, but comparisons have generally been limited to national welfare states. The choice of countries as the main unit of analysis is grounded in the historically close ties between the welfare state and the nation state. While the boundaries of welfare states were circumscribed by the nation’s borders and limited to their citizens or residents, they also served to strengthen the cohesion of nation states and enhanced their legitimacy (De Swaan 1994). However, the common use of welfare state regime typologies and longitudinal analyses based on country averages reflects a methodological nationalism which implicitly assumes that national welfare states are congruent entities, homogeneous within countries and across social policy domains. This assumption has been criticized by new conceptual and methodological perspectives, which emphasize the need for more nuanced analysis of national contexts and the inclusion of other analytical levels both at the supra- and subnational levels as well as relations between nation states. Criticism has drawn attention to important sources of internal heterogeneity of national welfare states such as territorial disparities (Greer et al. 2015), programme-specific dynamics (Trein 2017), and policy hybridity (Ciccia 2017). The exclusive focus on nation states is particularly problematic for the study of social investment-type policies such as childcare, education and labour market policies for which decentralized implementation, financing and delivery are the norm (Javornik 2014; Greer et al. 2015), but it also has implications in other policy domains because of the co-existence of distinct structural contexts, historical legacies and socio-political cleavages within nation states. By focusing on nation states, comparative research neglects important differences in welfare arrangements and the full range of actors and processes involved in the development, organization, and delivery of social policies. Moreover, the increasing relevance of multilevel governance and the retreat of the state from many sectors makes the challenge of intra-country heterogeneity even more pressing.

Far from denying the importance of national contexts, this special issue offers an explicit reflection on how the focus on the national level can be extended and specified in comparative analysis. It shows that the challenges of capturing intra-country variation in cross-national analysis are both conceptual and methodological and span multiple levels and units of analysis including local governments, supranational institutions, street-level bureaucracies and employers. The collection of papers in this special issue focuses in particular on three sources of heterogeneity within national welfare states, relating to processes of policy implementation and the territorial politics of welfare states, the interaction between public and private provisions, and the understanding of processes of welfare state change.

Policy Implementation and the Territorial Politics of Welfare States

The first source of intra-country heterogeneity relates to the relationship between various phases of the policy process, in particular the importance of policy implementation and the territorial politics of welfare states. The idea that policy making consists of interrelated stages ranging from agenda setting to policy formulation, decision making, implementation and evaluation provides a general framework for understanding policy development. However, comparative welfare state research has generally focused only on policy design and the adoption/formulation of policies. Policy design figures prominently in typologies of welfare state regimes, which are typically developed based on codified data about the institutional design of legislated policies (Esping-Andersen 1990; Korpi 2000). In contrast, longitudinal studies of welfare states have generally concentrated on understanding differences in policy adoption across countries (Huber and Stephens 2001; Pierson 2001). By focusing on the early stages of policy process, further cross-national variation deriving from the way policies are implemented, delivered and used has remained almost completely undetected. The identification of the implementation stage as a missing link is considered as one of the most important innovations in policy research since the 1970s. Implementation studies show that once polices are adopted, they continue to change and this leads to considerable divergence from the initial goals and intensions of policy makers. The first generation of implementation research considered this as a problem of policy design and the lack of adequate bureaucratic procedures leading to policy failure (Pressman and Wildavsky 1973). Subsequent research has come to the view that “perfect” implementation is hardly achievable. In particular, the development of bottom-up approaches, emphasizing the need of studying policies at the level of recipients and the network of actors involved in their delivery, rejected the idea that policies were essentially defined by central governments. In this view, discretion in the implementation stage was not only inevitable, but potentially beneficial because it enabled practitioners to better deal with the problems encountered on the ground (Lipsky 1980).

The range of actors and institutions involved in the implementation of social policies includes supranational and subnational legislators and administrators, street-level bureaucrats, judicial courts, employers and workplace cultures and situated social practices shaping people’s use of policies. As policies move through this multi-layered process, significant intra-country variation is produced in policy outputs and outcomes.

The territorial dimension of social policy is particularly relevant for comparative analysis because social policies are essentially delivered and experienced at the local level. Furthermore, welfare states have undergone deep transformation over the past decades, which have often entailed the adoption of decentralized forms of governance and the increased involvement of networks of private and public actors. These reforms have unfolded along two main lines: (1) the devolution of regulative powers from central government to local organizations (vertical subsidiarity); (2) the multiplication of actors involved in designing, managing, and implementing social policies (horizontal subsidiarity) (Kazepov 2010). Despite such (d)evolutions, cross-national analysis of local welfare arrangements remains underdeveloped.

The challenge of accounting for subnational variation in cross-national comparative research is both conceptual and methodological (see Mätzke; Satyro and Cunha in this issue). Conceptually, it involves identifying the key levels at which policies occur, combining and contextualizing information on different analytical levels. Methodologically, territorial disparities can often be effectively accounted for by applying conventional cross-national approaches (e.g. case-based analysis, historical analysis, statistical methods) to subnational units. However, these analyses are fraught with difficulties because of limited commensurable data at subnational levels.

A stronger connection between comparative welfare state studies and implementation research can be mutually beneficial. On the one hand, implementation research would benefit from connecting to research on earlier stages of the policy process, and the vibrant methodological debate in welfare state studies would provide new tools for comparative implementation research (Hupe and Saetren 2015). On the other hand, comparative analyses of welfare states could develop a more nuanced picture of cross-national differences through an improved understanding of the territorial politics of social policies and the complex processes that link policy design to practices and outcomes.

Employers as Providers of Welfare: Interactions between Levels of Provisions

The second often overlooked source of intra-country variation in cross-national analyses derives from private welfare providers. While the role of the family is an established research theme and a recognized contribution of feminist scholarship to the field of comparative welfare state studies (Ciccia and Sainsbury 2018; Satyro and Cunha in this issue), attention to the employers as welfare providers is a more recent addition. Early studies on employers and their organizations focused on their influence on social policy development (Swank and Martin 2001; Mares 2003), but the interactions between levels of provision at the national, industry and workplace level have received less attention in comparative research (Ollier-Malaterre et al. 2013). Welfare state research (implicitly) assumes that the national context shapes workplace policies and provisions; however, employers can depart from the legislation either by extending public policies (e.g. enhanced pension and leave provisions) (Javornik and Oliver forthcoming); offering additional benefits such as healthcare packages or flexible working; or restricting workers’ access to statutory entitlements. Conceptually, the capability approach (Sen 1992) provides a contextualized account of the complex interplay between social policy and macro- and meso-level institutional settings, while also recognizing the importance of company-level provisions and practices as conversion (i.e. explanatory) factors. A growing body of comparative literature on organizational practices shows significant variation between organizations and explains how workplace provisions are embedded within national-level policies and societal contexts. It finds that companies in the same country display systematic differences across sectors, size, and workforce, with the public sector and large organizations leading in the adoption of workplace policies (Den Dulk et al. 2010). Different traditions of employment law and employment relations determine what policies people can effectively access in a specific country/company (Javornik and Oliver forthcoming). This is relevant when policy success depends on workplace characteristics or supportive practice such as in the case of work–family policy, which intertwines many intersecting domains (family, employment, gender equality, and child well-being). Methodologically, the growing availability of comparative survey data on company-level provisions and practices offers opportunities to explore these issues with statistical techniques such as multilevel modelling. For example, Chung (in this issue) shows that only a small percentage of variance in workers’ access to schedule control is due to the country-level policy, and that company-level provisions further enhance generous national policies. Other promising developments concern the use of case study analysis to investigate how company-level factors shape individuals’ work–life capabilities (Hobson 2014). This discussion has, however, remained confined to issues of work–life balance.

Measuring Welfare State Change: Capturing Variation across Time

In this section, we reflect on the implications of the prevailing approaches in longitudinal analyses of welfare state change. In the last decades, the use of macro-comparative data and panel data methods, in particular time-series cross-section (TSCS) models, has become common practice. This approach has produced many new insights on the socio-economic and political processes underlying reforms, but has also received some criticism (Kittel and Winner 2005). Here, we focus in particular on two issues with particular implications for intra- and cross-country heterogeneity in causal processes: the level of the analysis, and the opportunities to combine variable and case-oriented approaches.

To measure change in TSCS, researchers have used either total social expenditure data at the country level as a measure of welfare effort or data on the structure of social rights as an indicator of the different institutional logics of welfare regimes (Clasen and Siegel 2007). While these measurement approaches have traditionally been applied at the level of the whole of the welfare state, researchers are now increasingly turning to disaggregated data to investigate the dynamics of specific policy domains. The use of aggregate data at the country level runs the risks of masking the extent of changes due to interactions and compensation effects between policy domains (e.g. cutbacks in unemployment insurance which remain concealed because of increased provisions in other sectors) (Ciccia 2017). Theoretically, there is a growing recognition of the challenge of inferring political decisions and processes from macro-level changes because reforms and political debates generally concern specific measures and not the whole of the welfare state (Hinrichs 2000). However, studies of particular policy sectors may be biased towards the constellations of actors and factors which are peculiar to that specific sector, and thus hold limited validity in other areas. A further difficulty in understanding dynamics of change derives from the neglect of subnational political units and the effect of regional funding mechanisms on social policies. The traditional approach of TSCS analysis is to treat decentralized allocation of authority as an attribute to the state and an obstacle to social policy development (e.g. a veto point) despite case study analyses demonstrating that the design of intergovernmental relations, law and finance are important for the welfare state (Greer et al. 2015). To overcome these limitations of macro-comparative analysis, a combination of levels of analyses linking information about the policy, sector, subnational and national levels is needed (Ciccia 2017). This would provide a picture of the many facets of reform processes, which derive from the relations between constraints and opportunities at the macro and meso levels and interactions between sectorial and territorial dynamics.

In social sciences, it is common to distinguish between variable and case-oriented approaches. TSCS, with its focus on establishing generalized relationships between variables, belongs to the first family. Its general aim is to identify the average effect of independent variables net of country- and period-specific effects and as such it provides a limited understanding of context- and time-specific dynamics. Moreover, the practice of increasing the number of observations by including a large number of countries and time points in pooled analyses runs the risks of confounding causal heterogeneity. These aspects can be better evidenced through configurational methods such as Qualitative Comparative Analysis (QCA), which has found increased application in the analysis of policy processes. A number of features make QCA particularly attractive for the analysis of welfare state change. First, QCA is suitable to analyse a small to medium number of observations such as those typically encountered in welfare state analysis. Second, it does not assume that cases need to be independent – a core assumption of TSCS – and thus is apt to investigate cases such as countries which are interconnected through forms of regional governance, economic and political integration as well as subject to policy transfers and diffusion. Finally, QCA handles more easily than multivariate statistical methods complex causal relations, characterized by complex sets of factors interacting at multiple levels such as those implied in policy change. In particular, it detects both equifinality (coexistence of multiple causal paths) and conjunctural causation (conditions having different effects depending on the context), which fits well with core theories of welfare state change. Nonetheless, the use of QCA in welfare state research has been generally limited to synchronic analysis because of the difficulties this method has traditionally encountered in incorporating the time dimension. However, there have been many technical developments which now enable us to deal with the issue of temporality in QCA. For example, Fischer and Maggetti (2017) identified five strategies including the introduction of sequences in the configurational analysis, the operationalization of non-commutative combinations of conditions, and comparative assessments at several points in time. In this view, configurational methods could be used to contextualize and validate the findings of traditional statistical approaches (see Flaherty in this issue).

Contributions to the Special Issue

Articles in this special issue highlight important issues involved in capturing intra-country variation in comparative analysis of welfare states and suggest useful conceptual and methodological strategies to deal with policy heterogeneity at subnational levels. Using cross-national analyses and single case studies, and applying a range of qualitative, statistical and interdisciplinary methodologies, this collection provides a picture of intra-country variation across a number of social policy domains such as work–life balance, family, social assistance, social protection, and income inequality.

The article by La Barbera and Lombardo titled “The Long Winding Road: A Comparative Policy Analysis of Multilevel Judicial Implementation of Work–Life Balance in Spain” investigates the impact of policy implementation through judicial litigation in shaping work–life policies in Spain. By looking at different stages of the litigation process at the national and European levels, the authors show that courts have considerable discretionary powers when applying legislation. Methodologically, they adopt an interdisciplinary approach, combining legal and critical frame analysis. Their findings demonstrate that a number of material and discursive opportunities and obstacles in the wider legal environment contribute to maintain traditional gender norms and limit the transformative potential of legislation.

Heejung Chung’s article, “National-level Family Policies in a European Comparative Perspective: Crowding Out or In, and for Whom?”, examines the interaction between national- and company-level family policies across 27 European countries. Focusing on schedule control and using multilevel techniques, the author finds large variation across different groups of workers, which runs counter to general assumptions about the uniform effect of social policies. The study demonstrates that childcare services at the national level “crowd in” company-level provisions, particularly for high-skilled workers. However, generous statutory leave entitlements “crowd out” workers’ schedule control at the company level. The author shows that policy outcomes are shaped by complex interactions between the characteristics of individual workers, workplace provisions and national-level policies.

Margitta Mätzke’s article titled “Comparative Perspectives on Childcare Expansion in Germany: Explaining the Persistent East–West Divide” leads us to consider the implications of changing boundaries of nation states and conceptual issues around levels and units of analysis. Using the case of “two Germanies” along the east/west divide, her study carries considerable weight, suggesting that more attention needs to be given to the subnational levels, local historical legacies and socio-cultural contexts as well as individual-level behavioural and attitudinal factors. Methodologically, it underlines the importance of considering the properties of both national polities (necessary conditions) and subnational units (sufficient conditions) to understand intra-country variation. By using the method of contextualized comparisons, Mätzke demonstrates how local supply and demand factors interact to produce marked disparities in childcare regimes at the regional levels.

The article by Natalia Guimaraes Duarte Satyro and Pedro Schettini Cunha, “The Coexistence of Different Welfare Regimes in the Same Country: A Comparative Analysis of the Brazilian Municipalities’ Heterogeneity”, demonstrates the presence of vast differences in welfare arrangements within Brazil. The study argues that the focus on cross-national variation in the welfare regime literature limits our understanding of within-country variation, particularly in countries from the Global South. The authors use the grade of membership methods (a fuzzy clustering technique) and factor analysis to analyse 5565 municipalities along five dimensions: expenditures, coverage, private spending, family structure, and poverty. They identify six subnational regimes: two pure models (corporatist, social assistance), three hybrids and a distinctive family insurance regime. These findings clearly demonstrate that subnational dynamics do not necessarily fit the supposed characteristics of the national regimes and can, instead, exhibit characteristics of other welfare regimes.

Eoin Flaherty’s article titled “Varieties of Regulation and Financialization: Comparative Pathways to Top Income Inequality in the OECD, 1975–2005” elaborates on the application of case-based analysis to explore the dynamics between financialization and income inequality. Methodologically, the paper adopts a time-series approach to Qualitative Comparative Analysis, using period-specific calibrations and analysing data in three historical waves. The results show that, contrary to prevailing arguments, countries exhibit diverse causal pathways towards income polarization. Flaherty’s article offers an explicit reflection on the issue of causal complexity in longitudinal analysis.

Acknowledgements

Guest editors wish to thank the Advisory Committee Members, Mara A. Yerkes, Dirk Schubotz and Niels Spierings, the reviewers, JCPA editors and all authors for their valuable support.

Additional information

Funding

This research has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 746168 – AGenDA. Any dissemination of results reflects only the author’s view and the Commission is not responsible for any use that may be made of the information it contains.

Notes on contributors

Rossella Ciccia

Rossella Ciccia is Lecturer in Social Policy at Queen’s University Belfast, and Marie-Sklodowska-Curie Fellow at the Scuola Normale Superiore. Her research focuses on the comparative political economy of inequalities and social policy.

Jana Javornik

Jana Javornik is Associate Professor in Work and Employment Relations at University of Leeds. Her research contributes to comparative social policy analysis and indicator development, and focuses on workplace sustainability, equality, diversity and inclusion in the labour market.