Regional Innovation Systems: How to Assess Performance

Abstract

Zabala-Iturriagagoitia J. M., Voigt P., Gutiérrez-Gracia A. and Jiménez-Sáez F. (2007) Regional innovation systems: how to assess performance, Regional Studies 41, 661–672. This paper applies a Data Envelopment Analysis (DEA) methodology to the evaluation of regional innovation system performance based on information provided by the European Innovation Scoreboard (EIS) for 2002 and 2003. Those European regions ranked in the EIS as showing better performance in high-technology areas are ranked somewhat differently according to DEA. The results of the present study show that the higher the technological level of a region, the greater is the need for system coordination. Where this is lacking there is a loss of performance efficiency compared with other similar regions. Policy-making in relation to Regional Innovation Systems (RIS) has in the past depended on systemic analysis. Here, a methodology is proposed that combines quantitative and qualitative analyses to enrich the knowledge base for future policy decision-making.

Zabala-Iturriagagoitia J. M., Voigt P., Gutiérrez-Gracia A. et Jiménez-Sáez F. (2007) Les systèmes d'innovation régionaux: comment évaluer la performance, Regional Studies 41, 661–672. A partir des données pour 2002 et 2003 fournies par le European Innovation Scoreboard (EIS), cet article applique la méthodologie de la Data Envelopment Analysis (DEA) à l'évaluation de la performance des systèmes d'innovation régionaux. Il en résulte que le classement des régions d'Europe dont le rang selon l'EIS laisse voir une meilleure performance dans les secteurs à la pointe de la technologie, s'avère différente selon la DEA. Les résultats de cette étude montrent que plus une région est à la pointe de la technologie, plus les systèmes devraient être coordonnés. A défaut de cette coordination, la performance manque d'efficacité par rapport à d'autres régions similaires. Dans le passé, la mise au point de la politique pour ce qui est des Regional Innovation Systems (RIS – des systèmes d'innovation régionaux) dépendait de l'analyse du système. On propose ici une méthodologie qui associe des analyses à la fois quantitatives et qualitatives afin d'enrichir la base de connaissance quant à la future prise de décision.

Systèmes d'innovation régionaux (RIS); Efficacité; Data Envelopment Analysis (DEA)

Zabala-ITURRAGAGOITIA J. M., Voigt P., Gutiérrez-Gracia A. und Jiménez-Sáez F. (2007) Regionale Innovationssysteme: Methoden zur Bewertung der Leistung, Regional Studies 41, 661–672. In diesem Beitrag wird die Methodologie der Data Envelopment Analysis (DEA) anhand der Informationen des Europäischen Innovationsanzeigers (EIS) für 2002 und 2003 zur Bewertung der Leistung von regionalen Innovationssystemen eingesetzt. Wir fanden heraus, dass die europäischen Regionen, die im EIS für Bereiche der Hochtechnologie als leistungsfähiger eingestuft wurden, in der DEA etwas anders abschneiden. Aus den Ergebnissen unserer Studie geht hervor, dass bei einem höheren technologischen Niveau einer Region auch der Bedarf an Systemkoordination wächst. Wenn diese Koordination fehlt, geht im Vergleich zu anderen, ähnlichen Regionen Leistungseffizienz verloren. Politische Entscheidungen im Zusammenhang mit regionalen Innovationssystemen hingen bisher von Systemanalysen ab. Hier schlagen wir eine Methodologie vor, in der quantitative mit qualitativen Analysen kombiniert werden, um den Wissensschatz für künftige politische Entscheidungen zu bereichern.

Regionale Innovationssysteme; Effizienz; Data Envelopment Analysis (DEA)

Zabala-Iturriagagoitia J. M., Voigt P., Gutiérrez-Gracia A. y Jiménez-Sáez F. (2007) Sistemas regionales de innovación: Cómo evaluar el desempeño, Regional Studies 41, 661–672. En este ensayo aplicamos la metodología del análisis envolvente de datos (AED) para evaluar el desempeño de los sistemas regional de innovación basándonos en información proporcionada por los indicadores de la innovación europea, conocido como European Innovation Scoreboard (EIS) para el 2002 y el 2003. Observamos que las regiones europeas que según el EIS muestran un mejor desempeño en áreas de alta tecnología, se clasifican de modo diferente en el AED. Los resultados de nuestro estudio indican que cuanto mayor es el nivel tecnológico de una región, más necesario es coordinar los sistemas. Cuando esta coordinación no existe ocurre una pérdida de la eficiencia en el rendimiento comparado con otras regiones similares. Antes la elaboración de políticas con relación a los Sistemas Regional de Innovación (RIS) dependía de análisis sistemáticos. Aquí proponemos una metodología que combina análisis cuantitativos y cualitativos para enriquecer la base de conocimiento que sirva para tomar decisiones políticas en el futuro.

Sistemas Regional de Innovación (RIS); Eficiencia; Análisis envolvente de datos (AED)

Keywords:

• Regional Innovation Systems (RIS)
• Efficiency
• Data Envelopment Analysis (DEA)

Acknowledgements

Jon Mikel Zabala-Iturriagagoitia's work was funded by the Programme for the Researchers Formation, Department of Education, Universities and Research of the Basque Country.

Notes

1. In the Policy Evaluation literature it is commonly accepted that the effects of any policy cannot be reducible to a single criterion, so the use of both quantitative and qualitative measures is indispensable (Georghiou, 1998; Kuhlmann, 2003).

2. ‘The Systems of Innovation literature takes an ambiguous stand on efficiency’ (Niosi, 2002, p. 293). Thus, ‘we would like to propose that the most relevant performance indicators on … IS … should reflect the efficiency and effectiveness in producing, diffusing and exploiting economically useful knowledge. Such indicators are not well developed today’ (Lundvall, 1992, p. 6). To conclude, ‘aggregate statistics … may reveal some types of efficiency or effectiveness … it thus may be necessary to desegregate statistics, and to build new ones, to understand some observed yet unexplained x-inefficiency of the system as a whole’ (Niosi, 2002, p. 298).

3. Conducting a European-wide comparison at a regional level always involves more or less substantial data problems, e.g. the lack of suitable indicators due to different definitions, short time-series, differences in the criteria applied by different statistics offices, etc. Hence, the present paper differs among three different levels of analysis in this emergent research path. This first step aims to demonstrate the possibilities of this approach in the context of Europe. In a second stage, the study could be replicated for each country to allow institutional aspects to be considered. A third step would involve examining the evolution of efficiency scores, region by region. The time-series needed for these studies will necessarily have to be longer, but the increasing uniformity in each territory as one goes down in the level of analysis will provide much deeper qualitative information for their evaluation.

4. Human resources for innovation (five indicators): New S&E graduates (percentage of 20–29 age class); Population with tertiary education (percentage of 25–64 age class); Participation in life-long learning (percentage of 25–64 age class); Employment in medium-to-high and high-tech manufacturing (percentage of total workforce); and Employment in high-tech services (percentage of total workforce). Creation of knowledge (four indicators): Public R&D expenditures (percentage of GDP); Business expenditure on R&D (percentage of GDP); EPO high-tech patent applications (per million population); and USPTO high-tech patent applications (per million population). Transmission and application of knowledge (three indicators): SMEs innovating in-house (percentage of manufacturing SMEs); Manufacturing SMEs involved in innovation cooperation; and Innovation expenditures (percentage of total manufacturing turnover). Innovation finance, outputs and markets (six indicators): High-tech venture capital investment (percentage of GDP); New capital raised on stock markets (percentage of GDP); New to market products (percentage of sales by manufacturing firms); Home internet access (percentage of all households); ICT expenditures (percentage of GDP); and percentage of manufacturing value-added from high-technology.

5. The seven indicators that constitute the EIS indices for 2002 and 2003 are: Population with tertiary education; Participation in life-long learning; Employment in medium-to-high and high-tech manufacturing; Employment in high-tech services; Public R&D expenditures; Business expenditure on R&D; and EPO high-tech patent applications.

6. The fact that any unit's performance can be obtained as the convex combination of other DMUs – providing virtual units – does not involve any lack of judgement in the analysis. In fact, policy-makers play a direct role in the amount of resources being employed within each subsystem and affect the role of the institutions with the definition and implementation of their regional innovation policies.

7. In the efficiency-related literature concern has been expressed about the convexity restriction and its utility, although there is no consensus to date (Cherchye et al., 1999). The Free Disposal Hull (FDH) (Deprins et al., 1984) could be another suitable alternative to test the role of convexity in this context. The FDH estimator relies on the free disposal assumption of the production set, but not, as the DEA does, on their convexity. Hence, FDH is a more general estimator than DEA (Park et al., 2000).

8. According to the Nomenclature of Territorial Units for Statistics (NUTS) adopted by the European Union and EUROSTAT, the administrative division corresponding to NUTS2 are the units considered as regions. Where data were missing the country average was used and/or inter-temporal constant scores were assumed for a certain region.

9. The 49% variation in per capita regional income can be explained by differences in innovative performance – measured by its RRSII – for 2002 and 2003 (European Innovation Scoreboard, 2002, 2003).

10. Two models were estimated. In the first, both patents and GDP per capita were considered as the desired outputs of any RIS. In the second, patents were considered to be an input rather than an output (all things being equal). The results obtained from both models were, surprisingly, quite similar and significant (the correlation between the models was 65.4% in 2002 and 63.8% in 2003).

11. The patents granted in ‘t’ can be the result (output) of the efforts previously made in time ‘t − n’. In turn, from ‘t’ on, once the patents are already granted, they could be considered as an input for all regions/sectors. However, the time period in the database is not long enough for this assumption to apply. Thus, patents are considered as an input for innovative activities in European regions due to the fact that most patents are generated by a very few regions, but the benefits spill over to all the others (Coe and Helpman, 1995; Georghiou et al., 2003). Nevertheless, this temporal issue is estimated to be a relevant point that might produce an interesting outcome regarding the appropriability of innovation. This could have implications for policy-making.

12. The procedure was performed using XploRe.

13. A further step in this analysis might be to study regions with a high degree of homogeneity (i.e. the Nordic Countries, the Mediterranean area), whose institutions play similar roles, and where the technological level of firms, the number of universities, etc., are similar.

14. If there was strong evidence of national clusters (e.g. due to major differences in RIS, legal frameworks, institutional settings, technological barriers, administrational restrictions, etc.), the proposed second and third levels of aggregation would be more appropriate.

15. RRSII/TE respective rankings: 42nd/110th for 2002 and 36th/124th for 2003.

16. RRSII/TE respective rankings in Europe: 36th/85th and 45th/62nd (Navarre), and 50th/55th and 47th/46th (Basque Country) for 2002 and 2003, respectively.

17. Balearic Islands: RRSII position: 134th/158th, and TE scores: 0.87 (28th) and 1.0 (10th), respectively. Castilla la Mancha: 138th/163rd (RRSII ranks), and TE scores: 0.89 (25th) and 0.98 (27th) for 2002 and 2003, respectively.

18. For an example of the application of the Open Method of Co-ordination in education policy, see Gornitzka (2005).

19. According to the methodology, any ‘under-use’ of inputs will only occur in particular cases where achieving a certain amount of output with less input might be considered as a higher efficient input/output relation and, therefore, would shift the frontier.

20. Since the study aimed at a European-wide comparison and testing the availability of an efficiency approach in this framework, this task cannot be presented in detail. However, in this context, the proposed second and third levels of aggregation would be more appropriate, allowing decision-makers and stakeholders to reorient the resources being used in their RIS.

21. Due to the enormous database that would be needed for a European-wide analysis of these issues, the authors would intend to conduct these future analyses at national level (probably based on Spain) when the second level of the analysis has been accomplished.