Measuring the formal independence of regulatory agencies

Abstract

While the literature on delegation has discussed at length the benefits of creating independent regulatory agencies (IRAs), not much attention has been paid to the conceptualization and operationalization of agency independence. In this study, we argue that existing attempts to operationalize the formal political independence of IRAs suffer from a number of conceptual and methodological flaws. To address these, we define what we understand by independence, and in particular formal independence from politics. Using new data gathered from 175 IRAs worldwide, we model formal independence as a latent trait. We find that some items commonly used to measure independence – notably, the method used to appoint agency executives and the scope of the agency's competences – are unrelated to formal independence. We close by showing that our revised measure partially changes conclusions about the determinants and consequences of formal independence.

Keywords:

• Formal independence
• item response theory
• measurement
• regulatory agencies

ACKNOWLEDGEMENTS

The authors are grateful to Fabrizio Gilardi and Martino Maggetti for their comments and for making their data available, and to Elias Dinas, Victor Lapuente, and the participants at the ‘Coordination and Control’ workshop held in Florence in October 2009 for their feedback. They also wish to thank the JEPP referees for their helpful comments

Notes

This approach can also be challenged: our theory might suggest that some items ought to be included regardless of whether they are empirically useful; or that some items ought to have a certain score based on our prior knowledge. But we believe that our prior knowledge about independence is not sufficiently strong to permit these kinds of judgements.

For competition, the International Competition Network; for financial markets, the International Organization of Securities Commissions; for telecommunications, the International Telecommunication Union; for energy, the International Energy Regulation Network; for pharmaceuticals, the International Conference of Drug Regulatory Authorities; for food safety, the Global forum of food safety regulators; for the environment, the European Union Network for the Implementation and Enforcement of Environmental Law.

We sent the surveys to all 706 contact points on the membership lists of the 502 peak organizations. These lists varied considerably in detail, and we often were not able to say which post within the organization our respondents held. Nevertheless, the fact that initial respondents forwarded on our questionnaire to other individuals within the organization, and the fact that we received hardly any duplicate surveys – even though we sent the survey to more contact points than organizations – suggests that some co-ordination took place within the organizations.

We have assumed ordinal responses. Missing responses resulted from inapplicable questions (questions concerning the board for single-headed authorities) and from non-response. The level of agreement would have been higher had it not been for our cautious interpretation of country-specific statutory provisions. We were often not able to determine whether nominations were binding, and hence opted for the response category ‘No specific provisions’, while the respondents could usually give a definite answer to this question.

The survey is not a random sample of the population; real GDP per capita was a statistically significant predictor of responding to the survey. Our results about the structure of formal independence may not therefore be generalizable to least developed or developing countries. Response rate was not, however, significantly affected by the language of the survey, or the type of legal system. Our results can therefore be generalized across legal systems. Descriptions of each item and associated response categories, as well as response frequencies, can be found at http://chrishanretty.co.uk/.

These results obtained after running MCMCpack's MCMCordfactanal routine for 1 million iterations after a 100,000 iteration burnin, thinned every 200 iterations. Inspection of trace plots for the α, β and γ parameters showed no problems with convergence. Similar discrimination parameters were obtained using marginal maximum likelihood as implemented in the ltm package (Rizopoulos 2006), and through normal factor analysis imputing missing values with the item mean, and taking the first unrotated factor.

In neither case is the correlation significant, which is mainly a consequence of the low number of cases (N = 16).