913
Views
5
CrossRef citations to date
0
Altmetric
Articles

Regulatory impact assessment in Slovakia: performance and procedural reform

Pages 214-227 | Received 06 Jan 2016, Accepted 22 Mar 2016, Published online: 06 Jun 2016

Abstract

Regulatory reforms in Europe and OECD countries in the last decade have focused on various tools that would improve ‘regulatory quality’ by bringing more evidence, science and analysis into policy-making. This trend was also followed by Central and Eastern European Countries where regulatory impact assessment (RIA) was institutionalized on a formal level. In order to increase efficiency of RIA, several procedural changes, such as standardization of reports, two-phase system and oversight mechanism are considered. This article looks at these changes and discusses whether any qualitative improvements can be observed in evidence utilization. The paper focuses on RIAs conducted in Slovakia from 2007 to 2013 in order to observe any patterns. Most significant qualitative improvement is due to the standardization of RIA and clear methodologies. The effect of an oversight mechanism with limited powers and improper timing of its control function not only does not bring the expected qualitative improvement of RIA output, but can create loopholes for avoiding RIA altogether.

1. Introduction

The concept of ex-ante regulatory impact assessment (RIA) was introduced in the late 1990s in OECD countries, followed by Central and Eastern European (CEE) countries in early 2000. The OECD declaration on regulatory quality in 1995 (OECD Citation1995) provided the first set of international standards in this policy area and was endorsed at the highest possible level by its member states. Typically, a RIA describes a policy problem, identifies alternative solutions to achieve the policy objectives, assesses possible effects and describes measures to be taken (OECD Citation1997). The RIA should help decide whether action is appropriate and if regulation is the best method of addressing the problem (hence RIA). RIA became a fundamental component of the smart regulatory state advocated by international organizations (OECD Citation2012) because of its systematic consultation, criteria for policy choice and economic analysis of how the costs and benefits of proposed regulations affect a wide range of actors. Many countries have introduced the obligation of regulatory authorities to perform an ex-ante impact assessment during the process of drafting and prior to law approval.

RIA is currently experiencing a remarkable level of attention in many countries (see Radaelli Citation2004, Citation2005; De Francesco Citation2012) under different names in governmental policy-making processes, utilizing terms such as impact assessment, impact analysis, impact evaluation, policy appraisal or policy assessment. In the European Union (EU), impact assessment – the EU does not use the term ‘RIA’ – is seen as a major part of the drive for Better Regulation and Smart Regulation (European Commission Citation2010, Citation2012; OECD Citation2012). Thus, many European member states have adopted Better Regulation programmes and legislative frameworks for the introduction of RIA as a tool for the improvement of regulatory processes. The CEE countries have followed a different path in this respect, since in most of them RIA was adopted prior to the development of better regulation programmes due to international organizations being the single-most important trigger (Staroňová Citation2010). Discussions of RIA on the international level are relatively coherent in terms of the nature and purpose of the tool (e.g. Radaelli & De Francesco Citation2007; OECD Citation2008; European Commission Citation2015): rational, evidence-based, participatory, transparent, looking at options and bringing stakeholder views.

Regardless of the impetus for the introduction of RIA methods and rules, the advocates of RIA argue that it introduces ‘new discipline and rigor to the rule-making process’ (Harrington & Morgenstern Citation2004, 3) as it provides decision-makers with empirical data and evidence as well as with a comprehensive framework in which they can assess their options and the possible consequences of their decisions. A poor understanding of the problems at hand or of the indirect effects of government action can undermine regulatory efforts and result in regulatory failures. Many previous studies assume that the availability and use of information from RIA leads to changes in the outcome of policy-making, notably to better law-making and regulatory quality (Hahn & Litan Citation1997; OECD Citation1997, Citation2008; Mandelkern Group Report Citation2001). It should also improve the accountability and legitimacy of any policy- and law-making system due to the factual efficiency provided by adequate information (Hahn & Litan Citation2003). In addition, proponents also believe that RIA increases policy coordination and stronger stakeholder involvement (European Commission Citation2009).

In this paper, we want to take institutional context approach – explore the wider political and institutional context within which RIA system operates. The relationship between evidence, knowledge, information and governance and institutions is extremely complicated. Both international organizations (e.g. the World Bank, OECD and EC) and academics have called for more evidence in national policy-making. However, existing studies (Bosswell Citation2008, Citation2009; Jacob et al. Citation2008; Turnpenny et al. Citation2008, Citation2013; Schrefler Citation2010) show how difficult it is to connect evidence with policy-making processes. RIA processes are often formal box ticking exercises (Staroňová Citation2010), evidence is often ignored, used selectively for preferred decisions (Radaelli Citation1995, Citation2010) or simply regarded as an additional administrative burden on civil servants in policy-making (Jacob et al. Citation2008). Thus, political context within which individual countries institutionalize RIAs shapes actual implementation and practice. It has been argued that institutional arrangements play a considerable role as they shape the incentive structures for the use of RIA tools, including how the assessment is being timed in relation to the policy-making process and how closely linked the assessment is to the political arena (Nilsson et al. Citation2008; Turnpenny et al. Citation2008). Various authors have tried to identify important institutional factors influencing the implementation of RIA, such as differing bureaucratic and organizational traditions (Turnpenny et al. Citation2008), governmental capacities and policy processes (Radaelli & De Francesco Citation2007), sectorization of policy-making (Turnpenny et al. Citation2008), phasing of the assessment process, the strength of institutional oversight and coordination (Staroňová Citation2010) and existence of guidelines for standardizing the RIA (Staroňová Citation2010).

In response to these challenges, many countries have reformed their processes to increase the quality of RIA output, including Central European countries. One of the major implications for the reform of RIA processes is whether an oversight body on RIA exists. Indeed, a widespread consensus seems to be emerging on the merit of having a strong oversight body for RIA (Jacobs Citation2006; Renda Citation2006; Staroňová Citation2010; Fritsch et al. Citation2012). Also, through the soft-law mechanisms of RIA policy recommendations, the OECD strongly encourages the establishment of a central oversight body as the single most significant quality assurance mechanism (OECD Citation2008, Citation2009, Citation2012). Such a body should have a mandate for issuing guidance and checking that the quality of the analysis is satisfactory, and for ensuring transparency and coordination. There exist many types and models of oversight mechanisms; perhaps the most studied ones are the US Office of Information and Regulatory Affairs, the UK Better Regulation Executive and the European Commission’s Impact Assessment Board. The last of these was created in 2007 within the Secretary General to check the quality of draft IAs. The Secretary General also coordinates the machinery of inter-service consultation and manages the Commission’s policy on transparency and access to documents.

RIA should ideally work as an iterative learning process which starts early in the policy-making process and runs in parallel with the decision-making process. As the assessment evolves, new questions may emerge or new data may require the reinterpretation of original assumptions. One approach to this issue may involve the official division of the impact assessment process into different phases. Some phases must be compulsory, while others can be optional. The adoption of a two-phase (‘dual stage’) system was also recommended by the Mandelkern Group (Mandelkern Group Report Citation2001) and can be found, among others, in the European Commission’s impact assessment procedures. The Commission applies this approach to RIAs on the basis of a so-called ‘principle of proportionate analysis’ (European Commission Citation2009). This translates, since 2005, into practice the obligation to produce preliminary ‘roadmaps’ on all proposals, while more comprehensive impact assessments are carried out only on selected items. It is the Secretary General, the Impact Assessment Board and individual Directorates General that decide on an annual basis which proposals undergo an impact assessment. The services are then invited to match the depth of the necessary analysis with the time and financial resources at disposal. The EC guidelines (European Commission Citation2009) explain in detail the nature and various steps, and provide advice on how and when to prepare it.

Many authors call for RIA and policy analysis not to be perceived as a ‘purely scientific process’ (Hertin et al. Citation2009, 9), but rather as a ‘discursively rational’ and ‘participatory’ process (Fischer & Forester Citation1993). This transformation of policy analysis into an ‘argumentative’ form (Fischer & Forester Citation1993; Hoppe Citation1999) leads to the emergence of wider concepts of what evidence actually means: not only the generation of data and information by expert analysts in purely scientific processes, but also a discursive process encouraging dialogue and the gathering of ‘lay knowledge’ via consultation process.

The main aim of the research discussed in this paper is to assess various institutional procedural changes introduced in order to increase the quality of RIA output, such as the regulatory oversight, two-phase system, standardization of RIA reports, guidelines for the use and interpretation of RIA. It aims to answer the following questions:

Are the institutional procedural changes an effective tool for strengthening evidence-based policy-making?

How are RIA process changes reflected in the quality of contents of RIA reports?

Have the policy ammendments failed, and if so, what are the reasons for this failure?

In order to explore these sets of questions, we focus on RIA practice using Slovakia as a case study. We have chosen this jurisdiction because it has introduced RIA process in early 2000s, nevertheless with dubious results reflected in extreme formalism (Staroňová Citation2007, Citation2010). Many CEE countries, including Slovakia, have reformed its processes in order to overcome the formalism. Thus, the early phase of RIA adoption was followed by a phase of RIA process changes. Nevertheless, it is useful to look at Slovak case because only here the procedures have been changed on a number of occasions and with shifting rationales. The article checks the theoretical expectations of improvement in RIA output (data and analysis in the RIA reports) due to the RIA process changes (the introduction of regulatory oversight, two-phase system, standardization of RIA reports, methodology) against empirical evidence from impact assessments reports from the period of 2007 to 2013. This period was selected to reflect the broad trends conducted against the backdrop of repeated procedural changes (see Table ). Therefore, we want to observe if qualitative changes occurred throughout this period in the content of the RIAs. The paper looks at two levels: firstly the institutional setting of the RIA process, i.e. the formal requirements for RIA, and secondly the actual practice of RIA as manifested in the impact assessments produced between 2007 and 2013. To this end we have coded information on RIA, as attached in Explanatory Memoranda to the draft legislation. Specifically, we ask how the quality of RIA has been effected by the regulatory reforms over the years and what main trends can be observed since the introduction of the institutional changes in RIA process.

Table 1. Key milestones in RIA system institutionalization in Slovakia.

2. Institutionalization of RIA in Slovakia

This section looks at how the RIA process has been institutionalized in Slovakia and examines how the RIA procedures and guidance has changed over time, the case study studied here. Table summarizes the main steps of RIA institutionalization.

The preparation of material for government sessions is guided by several documents, setting the general requirements for presenting the assessment of the possible impacts of draft laws. The most important ones are the Legislative Rules of Slovakia, Guidelines for the Preparation and Submission of Material for Government Sessions of the Slovak Republic and since 2008 the Joint Methodology for the RIA. All three documents have been amended several times to reflect developments in RIA (see Table ).

The first requirement to apply ex ante RIA to draft legislation was introduced by amending the Legislative rules of the Slovak government in 2001 following the recommendations of SIGMA OECD, a think tank and a joint initiative of EU and OECD. In contrast to the experience of other OECD countries the adoption of RIA took place prior to the development of the Better Regulation Program. The main document on impact assessment that brought about significant RIA procedural changes is the 2008 Joint Methodology of the Assessment of Impacts standardizing the RIA reports and providing clear requirements and expectations. On this basis the government introduced two-phase system for RIA development, oversight mechanism by entrusting four line ministries with the development of unified joint methodologies for particular areas of RIA: the Ministry of Finance for both fiscal RIA and RIA on e-democracy; the Ministry of Economy for RIA in the business environment; the Ministry of Environment for environmental RIA; and the Ministry of Labour, Social Affairs and Family for social RIA. The document has been modified after its piloting in the first two years (2008–2009). The revised Joint Methodology came into full effect in February 2010. As of 2012 two-phase system was made voluntary through an amendment of the Legislative Rules of the government.

When looking at the overall framework in place, the requirements for the RIA outcome presentation evolved with procedural changes. During the first phase of RIA adoption (2001–2007) the information on RIA-related issues were not standardized and were found in various parts of the Explanatory memoranda, creating confusion for the reader (Staroňová Citation2007, Citation2010). The introduction of Joint Methodology in 2008 brought standardization for presenting data and information in a formal RIA report which has two distinctive formats depending on the RIA phase and corresponding depth of analysis conducted: Note on Selected Impacts (partial RIA) and Note on the Analysis of Impacts (full RIA). The former one is utilized for the first phase and initial scanning for potential impacts, whereas the latter one is utilized in the second phase of a more in-depth RIA analysis if any impact has been identified. Note on Selected Impacts has a very simple format, identifying impacts for five requested areas (fiscal, economic, social, environmental, e-government) by ticking the relevant box: positive, no, negative impact. Only if positive/negative impact is ticked, further Analysis is conducted, but only in the selected RIA area.

One of the biggest methodological concerns is related to narrow understanding of RIA as assessment of impacts only. As seen from Table , the assessment of impacts is not seen in the context of problem analysis and subsequent justification of intervention and goal setting. Thus, these vital pre-conditions for effective options development, monitoring and effective policy assessment are not in place. Consequently, the existing formal framework does not provide adequate space for looking at options, although in contrast to the past, the changes now at least provide space for options development (but not comparison). Rather, current form focuses on specific issues of effects than providing incentives to analyse and see trade-offs between options, or wide range of potential impacts, including side effects. The selection of questions focuses on producing ‘hard’ data, particularly in the fiscal area (such as costs), and no space is given for exploratory methods. In none of the cases do the methodologies ask for the provision of analytical methods and sources for data or any type of evidence to substantiate the answer (e.g. why ‘no’ has been ticked). This can have severe consequences on the way how RIA is conducted, namely providing analysis of effects only for the option already chosen.

Table 2. RIA components required by formal guidance.

Contrary to EC and OECD terminology and practice, there is no specific requirement in Slovakia for consultations outside of the bodies of public administration. The participation of stakeholders is not foreseen in the design of RIAs, even after the reform. To the contrary, the consultation process is related to a formal inter-ministerial review process which, however, is a different process compared to the consultation. It is reserved for the gathering of legislative opinions on the draft legislation primarily from central agencies rather than on the design of regulation, and its mechanism appears very late in the process of policy-making. Although this process was opened up to public scrutiny thanks to the Free Access to Information Law, it still does not fulfil the consultation function. Thus, external actors have to rely on a formal inter-ministerial review process which comes at a late stage in the whole process; most such materials are already prepared in legal paragraphs for approval by the government, and civil servants are reluctant to deal with opinions from the public at such a late stage. Further, this formal procedure must be observed by the public participating in the process of inter-ministerial review. In practice, this means that the legal formulation of opinions, including those from the public, is extremely important which contradicts the idea of public consultation.

3. Methods and data

Data collection took place on three principal levels: (a) changes anchored in official regulations guiding RIA process (b) RIA documents coding (Notes on Selected Impacts, Notes on the Analysis of Impacts and the Opinions of the Supervisory Bodies) taken from the government’s official site for documents presented and approved at cabinet meetings, and (c) five semi-structured interviews with heads of the supervisory units for five RIA areas (fiscal RIA, e-democracy RIA, economic RIA, social RIA and environmental RIA) from four Ministries that were designed as regulatory oversight: Ministry of Finance, Ministry of Economy, Ministry of Labour, Ministry of Environment. All draft laws and amendments with their explanatory memoranda and accompanying statements of impacts proposed at cabinet meetings from 2007 to 2013 were analysed in terms of the RIA process and RIA information (in total 668 draft law and law amendments with RIAs). Although the joint methodology requires RIA for all material going to government sessions, the sole focus of this paper is on the RIA reports accompanying draft laws and amendments proposed to the cabinet in meetings in the selected period. The analysis omitted all other material that goes for discussion to government sessions, such as law intentions, concept papers, information, action plans, bilateral agreements and loans. The reason for this focus is twofold: firstly, most of the policies in CEE countries take the form of a legal document which is binding for everyone in the country. Thus, draft laws and amendments usually have a significant impact on the lives of citizens. Secondly, it is the legislative process that is formally regulated which allows the author to evaluate the degree of compliance with international standards. Draft laws debated more than once within government sessions are calculated as one unit, simultaneously taking the characteristics of all the materials into account.

Data on each RIA document was collected for both quantitative and qualitative analysis by employing a single coding template to cover (a) institutional and context variables (background variables) (b) 120 variables organized around variable groups: basic RIA content components variables (see Table ), stakeholders and consultation, presentation of the RIA report (number of words, organization, transparency), phasing of RIA, oversight body (support, opinions, veto), RIA tools, specific RIA areas (quantification and monetization). Each RIA document is coded manually following the template. By collating and systematizing the findings under the above mentioned categories, the analytical framework allows to go beyond a descriptive presentation and assessment of the quality of contents of RIA report towards a more analytic understanding of key variables. Once all the manual codes were compiled, statistical reliable inferences could be drawn. This process enables to create a database that can be utilized for more countries and comparative quality assessment of RIA documents can be conducted in future.

Table 3. The depth of RIA – the average number of words.

The variables were derived from the guidelines of the European Commission (Citation2009, Citation2015) and best practice of OECD countries. In terms of the variable group related to criterion of basic content components of RIA, EC suggests the following issues to be assessed by the RIA: justification of intervention (purpose), measurable objectives, options for intervention (including non-regulatory ones), assessment of impacts, comparison of options, monitoring and ex-post evaluation measures and identification of effected groups and its consultation. The second criterion of impact analysis looks into details of the analysis conducted in social, economic and environmental areas. To this we have included also traditional fiscal impact assessment (namely impacts on state budget) to better differentiate among ‘internal’ (impacts on state) and ‘external’ (impacts on society) impacts. In this sense, we look at identification of impacts in each area, provision of data (qualitative or quantitative) in each area, identification of costs, identification of benefits and quantification of costs and quantification of benefits, as well as mutual trade-offs. The third criterion of consultancy process looks into the way how affected parties have been identified and involved into the consultation process as well as how the results of the consultation process are presented in the RIA report.

The coding involves only RIA reports accompanying ministerial proposals which have been approved by the cabinet in its meetings. There is no obligation to prepare RIA for parliamentary initiatives. Therefore, the research presented in this article is based on content analysis of government-initiated draft laws and amendments.

4. Evidence from RIA performance

Formally speaking, almost all draft legislation in Slovakia complies with the requirement of attaching a Note on Selected Impacts for assessing potential impacts to the material that goes to government sessions. The analysis of the RIA documents (see Tables ) show that the biggest problem is not compliance so much as the scope and depth of the further analysis, although even this did improve significantly with the standardization of documents and provision of specific questions in each area since 2010.

Table 4. The scope of RIAs.

Table 5. Number of Full RIAs (Notes on the Analysis of Impacts) that identify and quantify costs and benefits.

Table 6. The two-phase system in practice.

The introduction of standardized forms with questions has forced the civil servants to answer these questions, and although the answers still do not provide the used analytical methods or data sources, they do show the costs and benefits, and they discuss issues in more depth. When discussing the scope and depth of RIAs conducted prior to changes (2007), during the piloting period without having standardized forms (2009) as well as after the revisions (2010 onwards), it is worthwhile looking at a proxy indicator – the average number of words used in the RIA document. Although it is not a direct indicator of the depth of analysis (in-depth analyses can be just a few pages long), it is still an indicator of the length of the document and thus an indirect measurement of what such a document can potentially contain (in the case of forms we have deducted the words in the questions to get a ‘pure’ word count). A typical A4 page with double spacing and Times New Roman Font contains approximately 300 to 330 words. Table shows that the average number of words in the individual Notes on Impacts dramatically increased and by far exceeds the average number of words in the Statement of Impacts covering all four areas (financial, economic, environmental and social), which ranges from 340 to 730 words. In other words, what used to be one to two A4 pages, or half a page per RIA area, is now several pages long for each area, particularly in the area of finance and the economy. This can be considered a clear benefit of rigidly following a given format.

Yet, another benefit of standardized forms for RIA documents was observed in the increase of non-fiscal areas covered and in the increase of the number of areas covered simultaneously (Table ). While prior to the introduction of joint methodology, fiscal analysis was the only RIA conducted in depth, other RIA areas became part of in depth analysis. However, the fiscal area is still a dominant area covered by RIAs and it is still equated with expenditures and costs that fall on the state budget, due to the strong supervisory position of the Ministry of Finance. Full RIAs (Notes on the Analysis of Impacts) are followed by economic analyses, which prior to the joint methodology were basically absent. Environmental analyses remain the weakest both in terms of number of RIAs conducted and depth of analysis provided (see Table ). This finding contradicts the findings of Zvijáková et al. (Citation2014). Still, close to the half of all RIAs show that there is ‘no impact’ foreseen (Table ), although this number has decreased by 15% since the introduction of Joint Methodology. The lowest number of RIAs showing ‘no impact’ is only in 2011 (only 30% which corresponds to OECD estimates of the percentage of regulations that may not have a significant impact)), a year when three new procedural measures collided: introduction of standardized questions for each RIA area, oversight mechanism in place in practice, obligatory two-phase process.

Turning to quantification, data show that prior to the procedural changes, the biggest problem was the superficial way of filling in the documents with simple statements such as ‘no impact’, ‘will bring positive impact’ and ‘no serious impacts’, and numerical information with no further explanation as to how this number had been calculated or achieved. There was no effort to analyse or quantify information, except for quantification of costs on state budget (Staroňová Citation2007, Citation2010). As noted before, with the introduction of Joint Methodology there is an immediate increase in full analyses in other than fiscal areas. However, this surge of full RIAs was not accompanied by increase of quantification of costs or benefits in any of the areas. Thus, although it might seem from Table that in 2007 and 2009 significantly more RIAs were conducted in the fiscal area, this number also covers simple monetization with no provision of evidence or logic as to how that monetization had been arrived at (except for costs on state budget). Thus, also increase in social and economic full RIAs meant identification of primarily benefits with phrases such as ‘positive impact on citizentry/society’.

Regarding the further step of quantification of costs and benefits, the data reveal increase in 2011 with the introduction of standardized questions in each area (as of 2010). The rigour of quantification of both costs and benefits became a standard in fiscal area (as opposed to prior focus on costs solely) and there is effort noticeable in other areas (as opposed to zero previously). The finding confirms that institutionalization of assessment system (via questions and methodology) brings results in short term. This finding corresponds to previous studies of other RIA systems, such as European Commission (Radaelli & Meuwese Citation2010). Nevertheless, quantification of social and environmental costs and benefits is lagging behind the identified effects despite the questions and methodology provided. Other research reveals that indeed quantification often focuses on easily quantifiable and typically neglects social and environmental aspects which are not easily quantifiable (Renda Citation2006). In addition, civil servants are also reluctant to engage in quantifications of administrative burden (economic IA) which typically belongs to easily quantifiable aspects. This could be attributed to the fact that despite existing guidance in the form of questions and examples, if no trainings are provided, civil servants ignore the requirement.

5. Discussion: effectiveness of institutional changes

Our data suggest that there has been some improvement in the performance of RIAs, both in quantitative and qualitative aspects. We discuss in this section how changes in RIA procedures positively effect the output, or to the contrary, create obstacle to how RIA knowledge is supposed to be produced and used in the policy process.

We can observe an improvement in providing full RIA reports across all RIA areas from 2009 onwards when joint methodology was introduced. However, only clear methodology and set of questions in each area meant that not only ‘identification’ of impacts was mentioned, but also more in depth-analysis in quantification of costs and benefits started to occur since 2011. Thus, major element of institutionalization and learning occurred through the publication of RIA guidance documents. Second, the standardization of the RIA documents helped to systematically locate information in one place and force civil servants to provide information in structured ways thanks to the questions in the form. In general, transparency has improved, although there still exist ad hoc cases where there is a reference to a detailed analysis in an external source (analysis, modelling, etc.) which, however, is not attached, thus making it impossible to check the results of the analysis.Footnote1 Surprisingly, the oversight bodies did not object to such a practice. This can be attributed to low authority and inadequate powers vested in oversight bodies.

Nevertheless, standardization has also its limits; RIA report does not provide specific information unless such information is asked for. Unfortunately, the design of the RIA documents is strongly focused on and reduced to assessing impacts (Table ) rather than detailing the objectives, problem at hand and providing a rationale and options for solving the problem. The concept of RIA follows a rationalist idea: the assessment is perceived as a value-free technical exercise where context (assumptions, problem framing, objectives, dominant discourses, other options and other stakeholders’ views) lies outside the scope of RIA.

Thus, even the renewed RIA documents reflect the old unreformed formal legislative process where RIA was typically a one-off activity which took place at the wrong place and time within the decision-making process. Prior to procedural changes RIA took place towards the end of the policy formulation process, notably once a draft law was ready and written in articles with no time left for options or consultations. To have policy formulation constrained before the decision-making has even started is not limited to CEE countries only. Hertin et al. (Citation2009), Russel and Jordan (Citation2007) mention administrative, legal and international (EU) procedures and commitments that narrow down the alternatives consideration.

In general, we could not observe a steady improvement over time in scope (number of areas covered) or depth of analysis (number of full RIAs and percentage of quantification of costs and benefits). Although patterns are rarely linear, we can see some clusters of improvement and degradation. It seems that in 2009 most integrated full RIAs were conducted (Table ), while many of these remained formal. 2011, on the other hand, witnessed the peak of quantification of costs and benefits. In addition, the data show that the preliminary reports still formally state ‘no impact’ (Table ) in more than 40% of RIAs, except for one year in 2011. However, these results were not reached in subsequent years of 2012 and 2013. Why further improvement and learning did not take place? There are several possible explanations for this failure, related to procedural change of regulatory oversight and phasing.

First, there seem to be resistance among both politicians and civil servants against any attempts to make the RIA process into more iterative one by gradual deconstruction of the phasing system. The procedural changes implemented a two-phase process to all material, except for those implementing EU directives. The output of the first phase (‘Note of Preliminary Impacts’) is sent to the four supervising bodies. If this quick test identifies any type of impact in five areas (fiscal, economic, social, environmental and e-democracy), a more detailed RIA (‘Analysis of Impacts’) is prepared in a second phase. While the draft of the reform was being prepared, there was a discussion on the table to connect the RIA process more tightly to the existing legislative procedure by requiring RIA very early in the process at the phase of the preparations of the annual list of bills (interview 1). However, this idea was immediately dismissed by law drafters in the legislative departments of individual ministries on the basis of ‘creating additional administrative burden’ (Interview 1) and never became materialized. During the pilot period from 2008 to 2009 none of the material underwent the first phase (see Table ). Simply, the old routine of presenting RIAs only once a draft act or amendment was ready and had gone through a regular inter-ministerial review process remained. After the pilot, the procedural change of phasing came into full effect on 1 February 2010, with additional changes that again narrowed down the iterative aspect: the first phase (preliminary review procedure) merged the preparation of the two types of RIA reports into one step and shortened the amount of time available for supervisory bodies to provide feedback. The reluctance of line ministries to start early with the RIA process culminated in final deconstruction step as of January 2012 when the first phase (preliminary review process) was changed from being obligatory to voluntary. Making this phase voluntary factually terminates the two-phase process. Table shows that the number of RIAs submitted to the first phase (though never in full compliance) dropped from over 72% (when EU-related items are deducted, as they were exempted) in 2011, when the reform was in full effect, to 10% in subsequent years, when the process was made voluntary. This confirms that RIAs are not conducted in an iterative way anymore, at least prior to having the draft law proposed written in legal language in articles. Rather the process stays as it used to be – at the wrong time and place at the end of the decision-making process.

Thus, although a two-phase system today still exists on paper, the reality is different for several reasons. Firstly, merging the preliminary and full impact assessments into one step is contrary to the idea of identifying the material that requires an in-depth analysis early enough for there to be sufficient time to conduct the analyses based on a plan with a time schedule for collecting additional data and information (e.g. via consultation with stakeholders). Secondly, shortening the period to provide feedback by the supervising bodies does not give sufficient time to both supervisory bodies and RIA writers to reflect upon the comments received. Thirdly, although the timing is set for a minimal period (20 days), most of the line ministries use even less than the minimal period by utilizing a ‘short-track procedure’, which in practice means that ministries submit material to the first phase five days prior to the second phase starts – regular inter-ministerial review process which is far too late for all supervising bodies (Interviews 1, 2, 3, 4 and 5). In fact, supervising body for environmental RIAs state that this is one of the core reasons why environmental RIAs are so lagging behind in comparison to other RIAs:

Environmental RIAs take time, they cannot be done hastily… we are used to [from strategic environmental RIA] to at least 100 days for conducing environmental RIAs ... and also our methodology assumes more time available ... but originating ministries when told during the first phase they have to postpone submission of the law due to more in-depth environmental RIA, they do not do that. There is no time ... the timeframework within which ministries work is too short. (Interview 5)

Fourthly, EU-related items are exempted from the preliminary review process, which gives yet another possibility for the originating ministries to skip this phase by simply labelling the item as ‘EU-related’. In an interview (1, 4), it was confirmed that nobody checked whether the item was really EU-related or simply a strategy to avoid the development of RIAs. In sum, the system provides a number of loopholes for ministries to avoid the early signalling of analysis and review opportunities for the supervisory bodies. Even in the best scenario, there is no time to conduct a real in-depth analysis and review of the conducted analysis based on supervisors’ opinions.

Such a system creates problems for supervising ministries as well. The first time a supervisor sees a draft and has the opportunity to comment and ask for a correction to the identification of potential impacts before reviewing them (or asking for a plan for data collection) is also the very same time that the actual in-depth analysis is undertaken. In addition, this timing is too close to the regular inter-ministerial review process when ready-made draft laws or amendments are proposed. Moreover, since 2012 the supervising ministries usually see RIAs for the first time only during the second phase – regular inter-ministerial review process. This is only few days prior to government meeting and with no real possibility to check the quality (and validity) of the answers. One supervisory body observed:

I have less authority when providing feedback to the originator of RIA during the inter-ministerial review process than during the first phase [preliminary review procedure], since I am one of the many subjects providing formal opinions on draft legislation [and RIA] …. And I do not have any opportunity or chance to check whether the opinion was taken seriously or into an account …. I do not feel like a supervising body anymore, but only another part of the ministerial machinery. (Interview 2)

Thus, opinions provided during the regular inter-ministerial review process are tackled as opinions of a ministry (not a supervising body) with no obligation on the part of the originating ministry to deal with it until it is marked as ‘important’.

Second, contrary to the title joint methodology, both the methodologies (i.e. the five areas of formatted questions for a more in-depth note on the analysis of impacts) and the oversight is broken down to four ministries depending on the type of RIA (the Ministry of Finance oversees impacts on the state budget and the informatization of society; the Ministry of Economy oversees impacts on the business environment; the Ministry of Labour, Social Affairs and Family oversees impacts in the social area; and the Ministry of Environment oversees impacts on the environment) with limited, if any, mutual coordination. In fact, each supervisory body provides its ‘own’ opinion with no coordination:

It would help me if I could have access to opinions provided by other supervisors, particularly Ministry of Finance since fiscal RIA is usually well done. If they see something, it helps me to see if I overlooked something, but until I see their opinion my has to be ready and out, so it is too late for me. (Interview 2)

This approach directly contradicts the efforts of the EU for an ‘integrated approach’ in the European Commission’s proposal, Impact Assessment: Next Steps (2002), where it urges the consideration of social, economic, financial, environmental and administrative aspects in an integrated and balanced manner in order to avoid sectorialism. The barriers to integration appear at micro, meso and macro levels (Hertin & Berkhout Citation2003; Turnpenny et al. Citation2008) and range from lack of resources or training available to civil servants, to sectorialism but also to perceived role of analysis in a particular RIA area and capacity development.

In order to strengthen the capacity of the supervising ministries, the procedural changes in 2010 obliged them to create specific analytical units whose task is to assist and supervise the quality of the conducted RIAs. The Ministry of Economy has created the Economic Analysis Unit; the Ministry of Finance has created the Public Administration Budget Unit and E-government Unit; the Ministry of Labour, Social Affairs and Family has created the Analytical Unit; and the Ministry of Environment has created the Environmental Impacts Unit. Nevertheless, these units are understaffed (having only two to three people who have other tasks in addition to RIA oversight). The only exception is the Ministry of Finance, where the PA Budget Unit has three subunits and is part of the Department on Budgetary Policy. The workload at the Ministry of Finance is distributed according to the specialization of the proposal (also outside of the department, e.g. to the tax department), and RIAs and all documentation are circulated and archived in an internal database. There is a clear role division with an assigned responsibility. At the same time, the document development is traceable in the database. The expertise of the Ministry of Finance has a long tradition and reputation which is strengthened by a clear division of responsibility within a shared case management. The overall analytical capacity of the Ministry, its institutional internal system of work division and document circulation and sharing is also strengthened by the fact that the Ministry of Finance is the least politicized ministry with the most stable staff that survives the turmoils of elections and ministerial changes (Staroňová & Adamicová Citation2016).

Despite the efforts to supervise the quality of RIA, in addition to the above-mentioned capacity, the supervising ministries are facing hierarchical problems vis-à-vis other line ministries due to several factors. Firstly, opinions have a non-binding character. Secondly, the first phase where the oversight might bring results was made voluntary, which reduced the number of proposals going to the oversight bodies to 10% of the overall amount going to cabinet sessions. Thirdly, the opinions provided during the second phase – the inter-ministerial review – are not considered to be the ‘supervisory body’s opinions’, but are collected along with other substantive opinions of the ministry and presented in an aggregated manner together with opinions from other line ministries towards each proposal (legislative item) prior to its submission to the cabinet. Thus, even if opinions on RIA quality are provided, there is no time or place for the supervisory body to check whether the proponent reacted and redrafted the RIA.

Our data show that fiscal RIAs are both quantitatively and qualitatively superior to any other RIAs. This can be attributed to the fact that the Ministry of Finance has a specific position and strength, both formal and informal, as a supervisory body over fiscal RIA. The formal strength is derived from its role in the preparation of the state budget which is also anchored in the Law on the State Budget and which requires the originating ministry to consult any aspect of a proposed law that might effect the state budget with the Ministry of Finance and have its approval. At the same time the Law on the State Budget enables the Ministry of Finance to ask the originating ministry for any additional explanations, calculation or other data to explain the calculations backing the proposal and the Ministry of Finance may do so at any stage of the legislative process. This mechanism is derived from long tradition of linking policy decision-making to implications for the state budget. At the same time, the Ministry of Finance has a very strong informal position (confirmed by all interviewees). Its strong informal position comes from expertise developed inside the ministry and accepted by both politicians and civil servants. As a result, it is the only supervisory body that is able to see how the opinion towards RIA was incorporated and return draft law (RIA) if they are of low quality in terms of (a) inadequate analysis on fiscal impacts or (b) inadequate information on how finances are to be secured for any fiscal implications (Interview 3). This is done in an informal way; prior to the cabinet meeting. The minister is briefed by his or her experts from the supervisory unit to what extent the opinion was taken into account and on the basis of this he or she is able to veto the proposal at the cabinet meeting level. No other minister has such a strength and/or will to practice this (and/or internal informal mechanisms established):

We have already authority in this …. Informally, we ask the originating ministries to send us information on how our opinions have been tackled. They always do that. And we report – in writing – to our minister on how satisfactory that was done … so that he is prepared for the negotiations at Cabinet meeting, or Legislative Council meeting or even Parliament ….we have special position – it is about finances after all. (Interview 3)

Overall, the introduction of supervising ministries is of the utmost importance, but the lack of real authority (no binding power and no sanctioning power) combined with inadequate communication between the four supervisors and the bad timing of opinion provision means that the opinions are not taken seriously, with the exception of the Ministry of Finance, which has a different status and informal ways utilizing its power. In addition, opinions provided during the regular inter-ministerial review process do not even have the status of a ‘supervising body opinion’, and there is no real space for the supervisor to find out if the originating ministry has taken the opinion seriously (Interviews) unless they do so in an informal way as Ministry of Finance.

6. Conclusion

The research clearly confirms some advancement in reforming RIA process by having an oversight mechanism incorporated into the system and RIA sequencing in legislation procedure. The first finding of the study draws attention to the importance of standardization of RIA documents that force the civil servants to provide more elaborative answers and broadening the overall scope of RIAs. The standardization brings structure and transparency into the way how evidence and information is provided. As a result, standardization increased RIA quality both quantitatively and qualitatively. Nevertheless, standardization has also its limits: conductor of RIA does not provide specific information unless such information is asked for. Thus, if the oversight body has limited understanding of the RIA as a tool, it can focus on unnecessary exactness of calculations of impacts rather than overall justification of regulation as such. The Slovak case has clearly shown that. Therefore, the emergence of oversight body should go hand in hand with clear understanding of RIA functions and training provided to oversight body staff.

Second, the Slovak case study illustrates how RIA is still not commonly perceived as an essential element in the whole system of policy-making or as a tool for improvement in policy design and coherence by presenting evidence to the decision makers despite of the reform efforts. This is manifested by predominant focus of RIA methodology on prediction (and the unnecessary exactness) of impacts without considering the context which narrows down the scope of RIA to a pure assessment of impacts. As a consequence, RIAs as a process is methodologically weak in ‘alternative’ consideration of policy options when preparing draft policy proposals, including limited deliberation with stakeholders to gain ordinary knowledge. Low embeddedness of RIA in the overall policy-making process results in the dominance of assessments in the fiscal area, with emphasis given to quantification rather than justification.

Third, the case study has shown that the system is easy to be bended and a number of loopholes arise for ministries a) to avoid the RIAs altogether b) conduct RIAs after the preferred regulatory option has been put into the legal articles c) downplay early signalling of analysis and review opportunities for the supervisory bodies. This is one of the research areas which future investigations should focus on. The Slovak case shows that the main challenge still seems to be proper linking of the RIA process to the existing legislative process to ensure a timely ‘flow’ of relevant and objective findings from the various analytical stages. The organizational culture, including the strong legalistic tradition typical for CEE countries, constitutes a major barrier to the effective use of RIA. Although Slovakia has tried to achieve a smoother process by introducing a two-phase RIA process, gradual pressure from the ministries reversed the reform efforts to a ‘box-ticking’ exercise that is done at the end of the process, once a draft law has been prepared, with no real possibility for a supervising body to step in and point out the analytical drawbacks. Also, the Slovak case has pointed out the potential for loopholes, such as the misuse of fast-track procedure and EU related agenda to avoid RIA process altogether, which is worth researching in the future. As a result, RIAs are to be made ex post in a bureaucratic manner to fulfil obligations rather than during the process of policy-making, which would assist the decision maker to make an evidence-based decision.

The other important issue is the strength of the oversight body that can enhance RIA capacity by means of support and sanctions if low-quality RIAs are produced. The Slovak case study has shown that the reason why oversight has not been effective is because the oversight body does not have the power to challenge low-quality RIAs, with the exception of the Ministry of Finance which utilizes most of its power in an informal way. In addition, the quality control function is fragmented among the line ministries which in combination with bad timing and communication of opinions is the reason why RIA reforms have not been effective and the quality of RIA reports have not improved. This finding confirms that sectorism is a barrier not only to integrated RIAs and policies (Hertin & Berkhout Citation2003, Turnpenny et al. Citation2008) but also to integrated oversight which decreases the overall effectiveness of the RIA process.

In addition, the study showed a very important factor that determines the effectiveness (and strength) of reform – internal expertise and political support for using RIA. The Slovak case has showed that lack of politicization strengthens the expertise of civil servants and in return the political leader (minister) trusts and backs the decisions of them. Thus, if political leader supports and considers RIA to be a priority, he/she will find even an informal way for influencing the decision-making based on data from RIA oversight. This approach is then reflected in the institutional set up of the oversight mechanism which needs to rely on effective internal processes of data sharing and processing (as well as workload division and capacity issues). This is not only important for understanding the internal process of effective oversight function but also for understanding the formal – informal relations related to influencing the decision-making.

These analytical results of the formal existence of RIA in Slovakia only confirm the notion by Radaelli (Citation2005), who argues that the RIA policy process is shaped by context in terms of dimensions and mechanisms. He claims that the particular context of public administration institutions and bureaucracy in continental Europe is different from the Anglo-Saxon one where RIA originated. In this sense, ‘efficiency still comes second to formal respect of legitimate procedures in the list of criteria used by bureaucracies’ (Radaelli Citation2005, 11). In addition, a transition country (or a country with a legacy of transition) constitutes yet another specific context. Firstly, the bureaucracy still bears the legacy of ‘non-activism’ and thus increases the chances for the presence of formalism where the RIA process is reduced to a bureaucratic ticking-off exercise and a political tool for substantiating a preferred option. In other words, it is the politicians who set the course of actions to be followed and bureaucrats are not expected to do any analytical work. Secondly, the systematic data collection and analysis is still in the process of being established. Whatever the reason for the specific context, ignoring the importance of RIA may increase the risk of there being an inadequate basis for decision-making, leading to poor and/or symbolic policies.

Funding

This work was supported by the Slovak Research and Development Agency (APVV) [grant number APVV-0880-12].

Notes

1. A notorious case of such a practice is the big reform of the state administration (known as the ‘ESO reform’) to increase efficiency in local state administration by creating ‘one-stop’ offices. These announced (and approved) administrative changes will have a significant impact. Nevertheless, the RIA documents were not attached or provided to the opponents of the reform, and the draft law was approved without any analysis being made available to the decision-maker.

References

  • Boswell C. 2008. The political functions of expert knowledge: knowledge and legitimation in European Union immigration policy. Journal of European Public Policy. 15:471–488.
  • Boswell C. 2009. The political uses of expert knowledge: immigration policy and social research. Cambridge (UK): Cambridge University Press.
  • De Francesco F. 2012. Diffusion of regulatory impact analysis among OECD and EU member states. Comp Political Stud. 45:1277–1305.10.1177/0010414011434297
  • European Commission. 2015. Better regulation guidelines. Brussels: European Commission. Available from http://ec.europa.eu/smart-regulation/guidelines/toc_tool_en.htm
  • European Commission. 2012. EU Regulatory Fitness. Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions, COM. 2012. 746 final. Strasbourg: European Commission.
  • European Commission. 2010. Smart Regulation in the European Union. Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee of the Regions. COM (2010) 543 final. Brussels: European Commission.
  • European Commission. 2009. Impact assessment guidelines. SEC (2009) 92. Brussels: European Commission.
  • Fischer F, Forester J. 1993. The argumentative turn in policy analysis and planning. Durham (NC): Duke University Press.
  • Fritsch O, Radaelli CM, Schrefler L, Renda A. 2012. Regulatory quality in the European Commission and the UK: old questions and new findings. Centre for European Policy Working Paper, (362).
  • Hahn RW, Litan RE. 1997. Improving regulatory accountability. Washington (DC): Brookings Institution Press.
  • Hahn RW, Litan RE. 2003. Recommendations for improving regulatory accountability and transparency. Testimony before the House Government Reform Committee.
  • Harrington W, Morgenstern RD. 2004. Evaluating regulatory impact analyses. Washington (DC): Resources for the Future.
  • Hertin J, Berkhout F. 2003. Analyzing institutional strategies for environmental policy integration: the case of EU enterprise policy. J Environ Policy Planning. 5:39–56.10.1080/15239080305603
  • Hertin J, Jordan A, Turnpenny J, Nilsson M, Russel D, Björn N. 2009. Rationalising the policy mess? Ex ante policy assessment and the utilisation of knowledge in the policy process. Ex Ante Policy Assessment and the Utilisation of Knowledge in the Policy Process (March 17, 2009).
  • Hoppe R. 1999. Policy analysis, science and politics: from 'speaking truth to power' to 'making sense together'. Sci Public Policy. 26:201–210.10.3152/147154399781782482
  • Jacob K, Hertin J, Hjerp P, Radaelli C, Meuwese A, Wolf O, Pacchi C, Rennings K. 2008. Improving the practice of impact assessment. Evaluating Integrated Impact Assessments. EVIA Project Report 028889. European Commission Sixth Framework Program.
  • Jacobs S. 2006. Current trends in regulatory impact analysis: the challenges of mainstreaming RIA into policy-making. Washington (DC): Jacobs and Associates.
  • Mandelkern Group Report. 2001. Final Report, Brussels, Nov 13. Available from http://www.cabinetoffice.gov.uk/regulation/docs/europe/pdf/mandfinrep.pdf.
  • Nilsson M, Jordan A, Turnpenny J, Hertin J, Nykvist B, Russel D. 2008. The use and non-use of policy appraisal tools in public policy making: an analysis of three European countries and the European Union. Policy Sci. 41:335–355.10.1007/s11077-008-9071-1
  • OECD. 1995. Recommendation of the council on improving the quality of government regulation. Paris: OECD.
  • OECD. 1997. Law drafting and regulatory management in Central and Eastern Europe. Paris: SIGMA Papers No. 18.
  • OECD. 2008. Building an institutional framework for regulatory impact analysis (RIA). Guidance for policy makers. Paris: OECD.
  • OECD. 2009. Indicators of regulatory management systems. Paris: OECD.
  • OECD. 2012. Recommendation of the council on regulatory policy and governance. Paris: OECD.
  • Radaelli CM. 1995. The role of knowledge in the policy process. J Eur Public Policy. 2:159–183.10.1080/13501769508406981
  • Radaelli CM. 2004. The diffusion of regulatory impact analysis – best practice or lesson-drawing? Eur J Political Res. 43:723–747.10.1111/ejpr.2004.43.issue-5
  • Radaelli CM. 2005. Diffusion without convergence: how political context shapes the adoption of regulatory impact assessment. J Eur Public Policy. 12:924–943.10.1080/13501760500161621
  • Radaelli CM. 2010. Rationality, power, management and symbols: four images of regulatory impact assessment. Scand Political Studies. 33:164–188.10.1111/scps.2010.33.issue-2
  • Radaelli CM, Meuwese AC. 2010. Hard questions, hard solutions: proceduralisation through impact assessment in the EU. West Eur Politics. 33:136–153.10.1080/01402380903354189
  • Radaelli C, De Francesco F. 2007. Regulatory quality in Europe. Concepts, measures and policy processes. Manchester: Manchester University Press.
  • Renda A. 2006. Impact assessment in the EU: The state of the art and the art of the state. Brussels: Centre for European Policy Studies.
  • Russel D, Jordan A. 2007. Gearing-up governance for sustainable development: patterns of policy appraisal in UK central government. J Environ Planning Manage. 50:1–21.10.1080/09640560601048267
  • Schrefler L. 2010. The usage of scientific knowledge by independent regulatory agencies. Governance. 23:309–330.
  • Staroňová K, Adamicová N. 2016. Politicization of civil service in Slovakia: case study of selected ministries in 2004–2014. In: Current trends in public sector research. Brno: Masaryk University; p. 86–93.
  • Staroňová K. 2010. Regulatory impact assessment: formal institutionalization and practice. J Public Policy. 30:117–136.10.1017/S0143814X09990201
  • Staroňová K. 2007. The quality of impact assessment in Slovakia. In: George C, Kirkpatrick CH, editor. Impact assessment and sustainable development. European practice experience. Cheltenham: Edward Elgar; p. 230–254.
  • Turnpenny J, Russel D, Rayner T. 2013. The complexity of evidence for sustainable development policy: analysing the boundary work of the UK Parliamentary Environmental Audit Committee. Trans Inst Br Geogr. 38:586–598.10.1111/j.1475-5661.2012.00549.x
  • Turnpenny J, Nilsson M, Russel D, Jordan A, Hertin J, Nykvist B. 2008. Why is integrating policy assessment so hard? A comparative analysis of the institutional capacities and constraints. J Environ Planning Manage. 51:759–775.10.1080/09640560802423541
  • Zvijáková L, Zeleňáková M, Purcz P. 2014. Evaluation of environmental impact assessment effectiveness in Slovakia. Impact Assess Project Appraisal. 32:150–161.10.1080/14615517.2014.893124

Interviews

  • Interview 1 – Gurská, D, Ministry of Economy, Economic Analyses Unit, supervisory body for economic Impacts, interview conducted by the author, 10 April 2014.
  • Interview 2 – Fašungová, L, Ministry of Labour, Social Affairs and Family; Head of Analytical Centre; supervisory body for social Impacts; interview conducted by the author; 30 April 2014.
  • Interview 3 – Cirák, A, Ministry of Finance, Head of State Budget Unit, supervisory body for fiscal impacts, interview conducted by the author, 30 June 2014.
  • Interview 4 – Krajčovičová, G, Ministry of Finance, E-government Unit, supervisory body for e-government, interview conducted by the author, 26 June 2014.
  • Interview 5 – Nižňanský, G, Ministry of Environment, Head of Environmental Impacts, supervisory body for environmental impacts, interview conducted by the author, 20 June 2014.

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.