Policy effectiveness and capacity: two sides of the design coin

Abstract

Policy capacity and effectiveness are two themes that have opened new pathways for academic and empirical enquiry throughout the policy sciences. In the contemporary discourse of policy design, effectiveness has taken on a more foundational meaning that goes beyond what is understood as only the attainment of specific policy goals. Rather, it has come to occupy a central position in the study of policy design, signifying the broader logic of deliberate policy action used to articulate policy problems and present alternative ways of addressing them. Effectiveness thus signals both effectual processes as well as successful policy outcomes. However, what constitutes effective design is a question that still reflects a largely dispersed body of research within the policy sciences. This article and others in this special issue, aim to address the topic of effective design from the perspective of capacity, defined as the inherent analytical, managerial and political capabilities of policymakers to bring about effective policy solutions.

Keywords:

• Policy design
• policy effectiveness
• policy capacity

1. Introduction

The modern era is characterized by unprecedented sociopolitical, environmental, and economic changes. The urgency to develop sound policy responses that are able to harness fresh opportunities, whilst addressing new challenges borne of these sweeping changes, defines the work of modern policy design and ultimately of public policy. While the topic of design has attracted a notably invigorated level of attention within the policy sciences over the last decade, certain elemental aspects of the design process still elude academic study. Understood as the purposive, knowledge-driven endeavor of governments to formulate policies, the feat of policy design rests crucially on how well policy alternatives are identified, assessed and deployed to meet complex social goals. Questions about why design takes place during formulation and how suitable, sustainable, and feasible designs can be achieved given political and contextual realities remain largely unanswered in contemporary studies of policy formulation. This is the case not only for individual concerns about what constitutes effective policy design or what define the capabilities necessary to make design happen, but also for how the interaction of these two factors can shape policy design in practice. This remaining gap suggests, perhaps, that the time is ripe for exploring the explicit relationship between policy effectiveness and capacity and what their interaction means for sound policy design.

A wave of recent studies, echoing the “new” design orientation in the policy sciences, has reiterated the importance of “effectiveness” as an overarching goal that should guide policy deliberations (Peters et al. 2018; Howlett and Mukherjee 2018; Bali, Capano and Ramesh 2019). What does policy effectiveness mean for practice? How can effectiveness be operationalized across different policy domains? What kinds of capabilities do designers need to produce sophisticated formulations, and ultimately more successful public policies? As showcased by the contributions to this special issue, these questions offer several promising pathways of empirical investigation to inform both the study and practice of policy formulation, by examining how capacities enable effective policy design.

2. Understanding policy effectiveness and capacity

2.1. Policy effectiveness and capacity: a necessary coupling

The concept of policy effectiveness echoes a central tenet of the contemporary policy sciences: problem-solving. That is, the fundamental goal of public policy is to address or solve societal problems or improve policy outcomes through a deliberative process (Peters et al. 2018). Accordingly, if public policy is primarily about problem-solving, then policy design is essentially about developing policy solutions in a deliberate manner that accomplish that purpose (Howlett et al. 2018). However, developing effective policies requires designers and those involved in implementation to have the requisite skill-sets and competencies. Studies about the formulation and implementation of policy in general have concluded that an essential condition for success in policy design activities rests on the interplay of analytical, managerial, and political capacities on the part of individual policy actors, regulatory organizations, and the general policy system (Table 1) (Wu et al. 2017). These policy capacities (understood as skills and competencies) rest on the availability of a variety of governance capabilities. First, analytical proficiencies are needed to match policy goals to existing policy means. Second, managerial abilities are required to marshal state resources toward policy priorities. Third, the effective design of policy is contingent upon political endowments which allows policymakers and administrators to coordinate, create, and implement their policy plans (Wu et al. 2017; Howlett and Ramesh 2016). These various resources at different levels of policymaking yield nine distinguishable types of overall policy capacity, required for effective policymaking. (Table 1)

Table 1. Dimensions and levels of policy capacity (Howlett and Ramesh 2016).

	Level
Dimension	Individual level	Organizational level	System Level
Analytical skills	Policy Analytical Capacity: Knowledge of policy substance and analytical techniques and communication skills	Organizational Information Capacities: Information and e-services architecture; budgeting and human resource management systems	Knowledge System Capacity: Institutions and opportunities for knowledge generation, mobilization, and use
Operational skills	Managerial Expertise Capacity: Leadership; strategic management; negotiation and conflict resolution	Administrative Resource Capacity: Funding; staffing; levels of intra-agency and inter-agency coordination	Accountability and Responsibility System Capacity: Rule of law; transparent adjudicative system
Political skills	Political Acumen Capacity: Understanding of the needs and positions of different stakeholders; Judgment of political feasibility	Organizational Political Capacity: Politicians’ support for the agency; levels of inter-organizational trust and communication	Political Economic System Capacity: Public legitimacy and trust; adequate fiscal resources

At the individual level, analytical capacity entails various technical skills; managerial capacities involve leadership strategies; and political competences are embodied by the individual acumen of policymaking actors to assess the needs and interests of stakeholders. For organizations, analytical skills include information dissemination and the creation of an information-sharing architecture within and across administrative agencies; managerial competences encompass coordination of resources and personnel among agencies; and political aptitude concerns support and trust within and for public organizations. At the level of policymaking systems, analytical endowments have much to do with the institutions that exist for knowledge generation and use; operational competences affect overall accountability and transparency; and political capacities directly impact public legitimacy and trust (Wu et al. 2017; Howlett and Ramesh, 2016).

The implications for practice in Table 1 are at least twofold. First, the framework provides a useful heuristic for policy practitioners to identify specific skills and competencies required across different policy activities or tasks. For example, if the goal is to improve the resilience of specific programs (discussed below), what specific policy capabilities does this entail? Second, it enables agencies to undertake a “capacity benchmarking” to identify extant capabilities and deficiencies. While deficits in some individual skillsets, in specific agencies, and at specific levels of government can be addressed and offset by relative strengths in other areas, scholars have argued that certain critical capacity deficits are challenging to overcome and can undermine the policy or program entirely (Howlett and Ramesh 2016).

2.2. Three levels of analysis

Recently, scholars of policy formulation have suggested that observing effectiveness and capacities in policy design should entail looking at three levels of analysis (Peters et al. 2018). The first and broadest analytical lens is that of formulation “spaces” or political environments that enable sound design. Questions about effectiveness at this level can involve a discussion of, for example, the nestedness of design choices contained within established policy regimes that determine how policy objectives are decided and what types of instruments are preferred (Howlett 2009; Parsons 2004). These considerations echo a jurisdictional stance as well, so as to gauge the enabling conditions in existing national policy regimes, which can influence why and how certain policy designs are chosen over others. Effectiveness at this level of design also appeals to the increasingly transboundary nature of public problems that occupy complex, multi-level regimes (Overdevest and Zeitlin 2014). Overcoming transboundary policy design dilemmas has a lot to do with facing the steadily increasing problem of regime complexity “in which a proliferation of regulatory schemes operate in the same policy domain, supported by varying combinations of public and private actors.” (Overdevest and Zeitlin 2014: 22) Therefore, an exploration of effective design spaces also involves understanding how they are delimited by on-the-ground political realities and prevailing modes of governance that can shape policy instrument preferences and the logic behind their implementation. Indeed, the legacy of past policy choices and political conditions can have a strong impact on whether policy choices and changes follow a “design” orientation, and if so, whether or not design occurs wholly or in part (Wu, Howlett, and Ramesh 2017; Mukherjee and Bali, 2018; Howlett, Mukherjee and Rayner 2017).

A fundamental determinant of effective policy design “spaces” is the degree to which they are commensurate with existing governance capacities. At the individual level of policy officials, administrators, and policy analysts these capacities are contained within their existing knowledge of an issue area, communication, leadership, and negotiation skills, as well as their political acumen. For organizations such as taskforces, advisory committees, administrative departments, and partnerships, policy capacity is reflected in the degree and ease of information dissemination and mobilization, administrative zeal for ensuring coordination and capabilities for garnering political support. At the broadest and systemic level of governance, overarching capacities such as institutions for knowledge creation, accountability, and political legitimacy can determine effectiveness across modes of governance – whether defined by legalism, a market-orientation, or horizontally coordinated networks of policy communities (Howlett and Ramesh 2016; Wu, Howlett and Ramesh 2017).

A second analytical perspective through which design effectiveness can be understood is at the level of policy instrument mixes or policy programs that unite a portfolio of policy tools toward meeting policy objectives. In striving for effectiveness, such policy mixes must ensure that their various components and multiple objectives display a range of design attributes such coherence, consistency, and congruence (Howlett, Mukherjee and Rayner 2015)¹. The success of policy programs that deliberately combine policy instruments toward meeting a common purpose (e.g. expanding healthcare coverage, improving access to electricity, or designing the boundaries of marine parks) can be profoundly impacted by how much and how well the various capabilities of relevant policy practitioners match.

At this level of policy design where purposive arrangements of policy instruments must be put to work, political, analytical, and operational capacities on the part of officials and practitioners can determine how well instruments are chosen for meeting stated, and often complex, objectives. Their capacities can shape success in how complementarities are maximized and conflicts reduced, how synergies are tapped, and how contradictions are avoided. This is especially so in the case of policy programs in which local design decisions need to be aligned with extraneous uncertainties that undermine programs success. An increasingly visible example of this kind of design can be found in agricultural and environmental management sectors that are continuously impacted by variations in climatic conditions. The successful deployment of policy program toolkits gauging ecological impacts, for example, requires a constant estimation of multiple possible socio-economic and political futures stemming from present resource allocation decisions (McBain et al. 2018), requiring a unique blend of analytical, organizational, and political aptitudes on the part of program designers and decision makers. Unique policy mixes can therefore demand a customized mix of capacities.

At a third, more operational level of scrutiny are individual policy instruments. Much of this discussion has benefitted from the growing conversation about effective mixes and what has worked well with instruments that are united within (Droste et al. 2018). For example, Hou and Brewer (2010), in their exposition of instruments deployed to stabilize state budgets against economic downturns, have commented on the substitutability of individual instruments and their effectiveness during the initial stages of application – along with related tradeoffs across the policy development process. While these and other studies have alluded to the internal efficacy of an instrument’s procedural settings and calibrations, other studies have contemplated the external effectiveness of an instrument, or how best it contributes to intended policy outcomes and brings about change in the actions of policy targets. In this vein, Thomann (2018) highlights in a comparison of organ donor policies in Europe the explicitness of behavioral policy instruments and the degree to which they make their goals obvious. Specifically, “the explicitness of an instrument results from two questions: first, does the instrument specify a behavioral change? Second, does the instrument attach valence to this behavior” (Thomann 2018: 433). Similarly, D’Adda et al. (2017) use experimental methods to study the ability of behavioral instruments to persist over time and gradually spill across contexts. These discussions implicitly indicate an emphasis on the temporality and tenure of instrument-level effectiveness. That is, they raise the question: how long after design and deployment of certain policy tools do their effects endure?

Capacities for effectiveness among ground-level policy instruments can thus rest heavily on analytical aptitude that is supported by relevant institutions and enabling managerial resources. As design is undertaken at this level within a more tightly specialized community of policy makers, officials analysts, and individual policy actors need to possess analytical skills, issue expertise, and knowledge gathered by practice or formal training (Meltsner 1975; Howlett and Lindquist 2004; Wellstead, Stedman, and Howlett 2011). These analytical traits are reinforced and facilitated by higher level organizational capabilities that allow designs to be robust yet flexible, in order to most effectively anticipate and respond to contextual uncertainties (Considine, Alexander, and Lewis 2014; Dunlop and Radaelli 2017; Howlett and Mukherjee 2014; May, Koski, and Stramp 2016; Capano and Woo 2017, 2018).

Bringing these various elements of effectiveness together at the instrument-level, Bali, Capano and Ramesh (2019) offer a framework to guide practice that focuses on the efficacy of instruments and the capabilities of implementing agencies to utilize these tools along three dimensions (Table 2).

Table 2. Instrumentality and capacity considerations for design effectivness (Bali, Capano and Ramesh 2019).

Dimension	Instrumentality considerations	Design capacity considerations
Analytical	Is/are the instrument(s) capable of solving the problem?	Does the agency know which tool to use? Can the agency calibrate and use the policy tool?
Political	Is the instrument socially acceptable/ politically viable to use?	Does the agency have the legitimacy/ ability to reconcile political differences or deal with political opposition?
Operational	Is the instrument operationally feasible?	Does the agency have accountability mechanisms, coordination mechanisms, and a trained bureaucracy?

Some studies offer a meta-view of effectiveness. For instance, Chindarkar, Howlett, and Ramesh (2017) argue that effectiveness relates to the extent to which a policy achieves both technical goals (i.e. addresses the substantive problem at hand) and advances the political goals of the government (e.g. retain political office or strengthening legitimacy). The overarching lesson for practice from their analysis is that both political and technical feasibility are necessary conditions for effectiveness. In similar vein, Compton and ‘t Hart (2019), building on Bovens and ‘t Hart (1996) and McConnell (2010), examine policy effectiveness (success) in four dimensions. These are programmatic (the degree to which a policy achieves its stated goal), process (the extent to which the design process is socially appropriate and perceived as being just), political (the extent to which there is widespread political support), and temporal (the extent to which a policy sustains its performance in the face of changing circumstances).

There also exists the notion of “dynamic” policy effectiveness, i.e. ensuring that the designs are able to adapt to changing policy conditions and circumstances. This requires designers to accommodate for uncertainty and policy “surprises” in the operating environment. Examples of these include the use of procedural policy tools that allow for automated or semi-automated calibrations to be made. This in turn requires designs that not only address routine policy errors in implementation but also accommodate unpredictable policy situations. (Capano and Woo 2018; Nair and Howlett 2017). Developing such “resilience” and “robustness” in designs, i.e. “the property of policies that allows them to continue to deliver, overtime, their intended functions, purposes, and objectives, even under negative circumstances” (Howlett, Capano, and Ramesh 2018), requires skills and competencies that allow for anticipatory designs.

Such studies view effectiveness as developing anticipatory designs which at t₀ is capable of addressing and preparing for what will happen at times t_1…2,3 in a manner that is consistent with an agent’s expectations over that time period (Bali, Capano and Ramesh 2019). This requires “a system of institutions, rules and norms that provides a way to use foresight for the purpose of reducing risk and to increase capacity to respond to events at early rather than later stages of their development” (Fuerth 2009, 19; Quay 2010; Karinen and Guston 2010; Miller 2012).

Formulating and implementing policy processes and programs that promote agility, anticipation, resilience, and address contemporary uncertainties is not automatic, however, but requires care and forethought. Designers must be able to understand the types of uncertainty characterizing a particular problem in order to recommend and design strategies and policy options to manage it effectively (Capano and Woo, 2017; Howlett et al. 2018).

Correspondingly, failing to correctly identify the bounds and range of these uncertainties is a major cause of policy failures due to over- and under-reaction (Maor 2012a, 2012b) and to over- and under-policy design. Enhancing the discretion of managers or street-level administrators working in traditional bureaucratic organizations to allow them to adapt policies on the ground, for example, could work in some situations.

Attaining responsive policies involves accurately assessing underlying levels of uncertainty and the design of appropriate tools to address them; this requires a great deal of governmental policy capacity (Howlett, Mukherjee and Woo 2015; Bali, Capano and Ramesh, 2019). Designers seeking such solutions need to overcome institutional constraints such as the availability and access to data to conduct relevant analyses, as well as deploy additional resources to manage future known and “unknown unknowns” (Nair and Howlett 2017). In dealing with more abstract levels of uncertainty, policymakers now have to operate as “continuous policy-fixers” oscilating between a policy “architect,” and “learner” “adjusting” policies in response to changing conditions. over time. Overcoming both short-term electoral orientations and the emphasis in public administration toward routinization and narrowly defined considerations of efficiency, are also required if such foresight and adjustment is to occur (Bason 2014; Howlett and Mukherjee 2014; Mulgan 2008).

3. Lessons for practice: articles in this special issue

Contributions to this special issue reflect the multiple perspectives of effectiveness discussed above. Situating their discussion of capacity and effectiveness in policy design at the broadest and most systemic level of political environments are the contributions by Bajpai and Chong (2019), Hartley, Kuecker, and Woo (2019), and O’Flynn (2019).

Bajpai and Chong offer a qualitative assessment of India’s foreign policy system by exploring the strengths and weaknesses of the Indian Foreign Service (IFS) and the Ministry of External Affairs (MEA). The central question they raise is whether India can be effective in attaining its geo-strategic ambitions. The authors argue that India’s geo-strategic ambitions are constrained by the operational capabilities of the IFS. These weaknesses include a critical mass of capable trained officers, the infrastructure for collecting and processing information, the international organization of the MEA, and the extent of communication with governance partners and the public. The authors suggest that it is vital for the government of India to pursue reforms at the organizational level if it aspires to play a more influential role internationally. Bajpai and Chong approach design effectiveness from the viewpoint of overcoming critical deficits in policy capabilities, which would entail addressing administrative inertia that hinders the development of a new governance relationships, themes that are also explored by Hartley et al. and O’Flynn.

Echoing the importance of anticipatory designs in complex and unpredictable policymaking contexts, Hartley et al. argue that policy designers need to overcome an epistemic legacy focused on solving discrete problems. Rather, policy designers need to develop a new set of capabilities to respond to the convergence of wicked, synchronous, and interconnected problems such as climate change – a challenge faced by governments across the world. The authors approach effectiveness from the perspective of cultivating policy capabilities (e.g. predicament thinking) that can not only address present policy challenges but also rise to meet unprecedented future policy complexities. The authors develop a series of recommendations to guide practice focused on strengthening analytical capacities (training in methodologies, use the foresight units, protocols for information sharing) as well as political capacities (interdepartmental rotation, collaborations, and developing conduits of influence and exchange).

O’Flynn (2019) confronts a central debate in contemporary public management: procurement. The average OECD economy spent 13% of GDP in public procurement in 2018. O’Flynn argues that, despite the widespread use of contracting in policymaking, governments have a tendency to conceptualize contracting in an extremely narrow and relational manner. This restricts the ability of governments to address citizen aspirations and ultimately the effectiveness of these contracts. Her paper echoes the arguments advanced by Hartley et al. on epistemic lock-in and preference for contracts as a dominant instrument. Improving the efficacy of contracting would require the cultivation of strategic relationships and a shift in mindset, which in turn require specific capabilities. These include competencies of the individual public servant: the skills, attributes, and characteristics of those that manage relationships and make externalization decisions; capabilities of public sector organizations: such as structures, processes, protocols that facilitate relationships and mitigate obstacles and lastly capacities to develop an enabling environment: how to configure government-wide structures and processes so that the other two levels can be more effective.

O’Flynn’s concerns about narrow operationalization of policy effectiveness at the systemic level is mirrored by Virani (2019) who focuses on the level of policy programs. Using the example of a sub-national public private partnerships (PPPs) in the health sector in India, he explains the inherent differences (and contradictions) in the policy expectations of different actors and shows how PPPs, if poorly designed, tend to produce outcomes that are inequitable and inconsistent with their original motivations. This, he argues, can be overcome by proactive design efforts to align competing objectives and by building capacities that better orchestrate the use of procedural instruments at key stages in the policy life cycle: the use of partnership agreements, task bundling, governance boards, project coordination committees, public feedback, consultation committees, and relational contracting.

The papers by Saguin (2019) and Katsonis (in press) approach effectiveness at a more instrumental level. Their papers add to the recent scholarship in the policy sciences on using policy tools more effectively (Bali and Ramesh 2019; Bali, Capano, and Ramesh 2019; Howlett 2018). Saguin uses the example of the Philippines’ Education Service Contracting Program to advance the concept of completeness in policy mixes and the requisite capacities to better combine substantive and procedural instruments of design. He argues that policies often fail to realize their objectives absent vital procedural tools and capacities that are critical in ensuring that policy mixes remain effective. The overarching lesson for practice is that effectiveness of policy mixes rests heavily on carefully combining substantive instruments (e.g. fees, taxes, licenses) and associated procedural tools (e.g. accountability mechanisms and mechanisms to reduce errors of inclusion and exclusion). In addition to the specific case in the Philippines, the paper adds to the literature by conceptualizing a design attribute (completeness) that designers must account for during the formulation stage when policy and programs are layered, assembled, and calibrated (see Howlett 2018; Bali and Ramesh 2018).

The proliferation of collaborative and participatory designs – a tenet of contemporary formulation practices in many societies – requires fundamental capacities of public engagement on the part of policy designers. Katsonis uses the empirical case study of community engagement in Melbourne to outline the capacities underlying effective public engagement. She argues that this requires the commitment of resources (fiscal and people) at an organizational level to embed engagement at all stages of the design process, define and communicate clear engagement objectives, create meaningful opportunities for public participation, provide a supportive environment for engagement and feed the inputs back into policy design. These tasks require an array of professional, strategic and innovative capabilities including strong leadership and a commitment towards inclusion, participation and transparency in decision making. The paper concludes with insights for effective public engagement including the need for strong leadership from decision makers, whether elected officials or senior public servants, and a commitment towards inclusion, participation and transparency in decision making.

What our paper and others in this special issue suggest, is that effectiveness and capacity are invariably interlinked and there are multiple levels at which both can be understood, analyzed and effectuated. The contributions in this special issue provide a practical lens to facilitate more sophisticated policy deliberation and designs across heterodox policy sectors and activities. A fine-grained dissection of policies and programs at the various levels discussed above can result in a clearer understanding of the relationship between policy goals, means and outcomes.

Disclosure Statement

No potential conflict of interest was reported by the authors.

Notes

1 See Bali and Ramesh (2018) for a review.