Standards as catalyst for national innovation and performance – a capability assessment framework for latecomer countries

Abstract

This paper develops a framework for assessing national standards capability. The framework draws on total quality management (TQM) models as its theoretical basis and was developed through an expert group panel, interviews, and focus group workshops. To verify the framework, pilot implementation was conducted in four countries. The results show that the proposed framework is useful for measuring the strengths and weaknesses of overall capability per pillar and category in a national standards system. The paper contributes to the expansion of the TQM model in assessing national level capability by more effective evaluation and systematic development of national standards systems for developing countries.

Keywords:

• developing countries
• national standards capability
• NSCAF
• standardisation
• conformity assessment
• metrology
• TQM

1. Introduction

The objective of this research was to develop a framework to evaluate national capability for a standards system. To date, there has been no single developed country without a national standards system for setting and implementing national standards, because such a system is essential to the socio-technical development of a nation. Considering the critical roles this national standards system can play, its importance has been often neglected or discussed only as part of a quality system or a trade system.

However, changes in global trade flow in the past couple of decades have gradually increased the recognition of the important role of standards and quality in effective economic development (Guasch, Racine, Sánchez, & Diop, 2007). As Choi, Lee, and Sung (2011) point out, there are growing public policies and academic literature that perceive standardisation as a catalyst for national or company innovation by facilitating access to markets and enabling interoperability between new and existing technologies, products, services, and processes (Blind, 2009; DIUS, 2009; Farrell & Saloner, 1985; Galvin & Rice, 2008; Kano, 2000; Swann & Britain, 2000). For these reasons, emerging late-comer countries, like Korea and China, have considered standards a strategic mechanism for their transition from technology-based rule users or followers to rule generators in the international market (Choung, Hwang, & Choi, 2000; Choung, Ji, & Hameed, 2011; Lee & Oh, 2008).

One of the key concerns in the area of standards is the impact of standards on trade, including technical barriers or mutual recognition of conformity assessment results (Chen & Novy, 2012; Heckelei & Swinnen, 2012; Swinnen & Vandemoortele, 2012). That influence should not be limited to specific sectors, however, but instead linked to practically almost all sectors of an economy, from food and agriculture (Henson, Masakure, & Cranfield, 2011; Maertens, Minten, & Swinnen, 2012; Neilson & Pritchard, 2007; Ponte, Raakjær, & Campling, 2007; Ruben & Fort, 2012) to automotive, electronics, environment, and services (Brenton, Edwards-Jones, & Jensen, 2009; Eden, Bear, & Walker, 2008; Geng, Dong, Xue, & Fu, 2012; Rich & Perry, 2011; Shiroyama, 2007).

The organisations for standardisation, metrology, and accreditation make up a country's national quality infrastructure (Frota et al., 2010). National standards' related activities, therefore, should be considered an indispensable component of national quality and a technological system and also a clear basis for economic competitiveness, as well as being the appropriate evaluation mechanism that is needed for successful national development.

2. Literature review on national standards capability

There is to date only limited academic or empirical research on the development of an evaluation tool to determine national standards capability. Even though some international organisations and developed countries have provided assistance programmes for developing countries to apply and build capacity in national standards activities, few specific tools are as yet available to measure national standards capability overall. Although no tool has been developed to assess overall national standards capability and cover all standards-related sectors, there do exist a few meaningful conceptual frameworks or approaches that can be noted here.

The World Bank and the United Nations Industrial Development Organization (UNIDO) have dealt with certain components of a national standards system in their policy reports and capacity-building programmes and provided positive guidelines for developing the major components of a national standards system infrastructure (Guasch et al., 2007; UNIDO, 2003, 2010). However, neither organisation has tried to create an actual specific framework that can evaluate national standards capability wherein national standards activities are defined for the purpose of trade capacity building. UNIDO's standards, metrology, testing, and quality, a so-called Standards, Metrology, Testing and Quality (SMTQ) program, has not developed any assessment framework, but rather raised the competitive capability so as to conform to market requirements and thus connect to the markets of developing countries. The World Bank manual considers standards and quality systems critical for a national competitive edge (Guasch et al., 2007). The manual contains very useful and comprehensive descriptions of concepts, components, and interactive relations of national standards infrastructure, but it does not develop an assessment framework for a national standards system.

The International Organization for Standardization (ISO) started the Institutional Strengthening (INS) project in 2011 based on the ISO Action Plan for Developing Countries 2011–2015 (ISO, 2010, 2011). The ISO's INS project objective was not to develop an evaluation tool, but instead assist the developing countries in strengthening their own institutional capacity. The ISO's INS project is still in the very early development stages, and the outcome is planned so as to improve the processes of developing countries and formulate meaningful and pragmatic business plans that will drive their organisations forward to greater achievements, especially in terms of their involvement in international standardisation. The ISO's INS project output plan addresses good practice and a manual for the formulation of a national strategy; marketing and communication; an information and communication technology (ICT) infrastructure; an enhanced role in national quality infrastructure building; and stakeholder involvement (ISO, 2011).

The International Telecommunication Union (ITU) and the Korea Communication Commission (KCC) developed the Tool for Assessing Standards Capability (TASC) in the ICT sector. The TASC tool was developed as part of the ITU Bridging the Standardization Gap (BSG) program (ITU, 2009). One specific objective of this ITU tool is to identify primary gaps and provide recommendations for how to improve standards development, its implementation, and the usage capacities of developing countries for ICT standards. The TASC tool contains four evaluation categories – standards development capacity, standardisation of human resources, government standards policy, national standards use, and adoption. The result of the TASC evaluation is described as being one of four different capacity levels, ranging from ‘low’ Level 1, ‘basic’ Level 2, and ‘intermediate’ Level 3, to ‘advanced’ Level 4. The TASC tool is important, as it is the very first tool available to assess national standards capability. Some of its limitations include first of all that the TASC tool is not covering the overall national system but is ICT sector-specific, and second, that it was developed without a theoretical model.

This absence of an evaluation framework for a national standards system and its capability or performance is problematic because there exist currently no systematic mechanisms to measure and improve a national standards system. There is thus a need to develop a more comprehensive framework to describe and assess actual national standards capability. This assessment framework should cover all the key dimensions of a national standards system, such as testing, certification, accreditation, and metrology; and it should also consider various existing approaches and aspects offered in the above approaches, such as institutions, human resources development, stakeholders, and the infrastructure.

Within this context, total quality management (TQM) models fit best as a national standards system is the foundation for a national quality system. TQM is an integrative philosophy of management used to continuously improve the quality of products and processes (Ahire, 1997). Dahlgaard, Chen, Jang, Banegas, and Dahlgaard-Park (2013) reviewed the assessment criteria and limitations of TQM, and further emphasised that TQM is a very flexible model for adaption and its core dimensions should be understood and respected. As such, the core value or the criteria of TQM models has evolved since the mid-1980s. The 25 years of evolution of TQM and related quality studies have been well summarised by Dahlgaard-Park, Chen, Jang, and Dahlgaard (2013). During the last two decades, TQM has been also expanded to non-manufacturing service sectors including higher education, health care, customer satisfaction, tourism, and recently, innovation (van Iwaarden & van der Valk, 2013; Wang, 2012; Xiaorong, Bojian, & Huili, 2012). Asif and Searcy (2014) argued that performance excellence in higher education institutions requires the management of the capabilities related to research, programme design and delivery, and service performance. Chin and Tsai (2012) established a service quality evaluation model for luxurious restaurants in international hotel chains to guarantee the success of the restaurants. Fotiadis and Vassiliadis (2013) explored the effects of new facilities on patients’ perception of the quality of services in the general hospitals in Greece. Recent TQM literature discusses the linkages between: the TQM model and innovation; quality management and innovation; organisational innovation capability and product platform development; and the impact of organisational learning on innovation performance (Chien, Lin, & Lien, in press; Dadfar, Dahlgaard, Brege, & Alamirhoor, 2013; Leavengood, Anderson, & Daim, in press). Also the TQM models have been well developed to evaluate an organisational performance quantitatively. Therefore, as specified in Section 4, TQM criteria are selected as a basis to develop assessment criteria of national standards system and its innovation capability.

3. Methodology

This research used qualitative approaches to develop an assessment framework for national standards capability. Qualitative research is a form of scientific inquiry that spans different disciplines, fields, and subject matter and includes many varied approaches (Denzin & Lincoln, 2005). Qualitative methods are frequently chosen in the certain specific research contexts, which also exactly match the research scope and goal of this current study, namely, to investigate complex phenomena that are difficult to measure quantitatively and generate the kind of data that is necessary for a more comprehensive understanding of a problem, gain insights into potential causal mechanisms, develop sound quantitative measurement processes or instruments, and/or study special populations (Curry, Nembhard, & Bradley, 2009). This paper thus applies a mixture of individual interviews, focus group workshops, and pilot implementation of qualitative study methods in order to investigate the complex phenomenon of a national standards system and identify mechanisms to assess its capability as demonstrated in Figure 1.

Figure 1. Outline of the research method.

First, a group of eight experts were organised as a research team and panel. These eight experts had professional experience that averaged around 19 years, in one or multiple aspects of standards-related areas, including national and international standardisation, conformity assessment for products and management systems, scientific and legal metrology, regulations and trade, and quality management systems. This expert group served as a development team to study the theoretical groundwork and define draft characteristics and create the skeleton framework to prepare and conduct interviews, workshops, and pilot implementation for verification of the final draft framework. In the process, the TQM model was chosen as a theoretical basis.

Second, interviews with individual experts and focus group workshops were held between March 2011 and August 2012. Participating experts came from policy makers, laboratories, industry and academia. The focus group workshops included sessions with experts from four countries that had participated in pilot implementation, thus giving the framework cross-national characteristics.

Third, a pilot implementation was conducted in four countries from August 2011 to October 2012 to validate and upgrade the draft framework. The trial assessments of these four countries’ activities were held in conjunction with focus groups in workshops, in particular with the experts from the assessed countries. This interactive design enabled the experts to participate in focus group workshops and then discuss and provide direct feedback to the framework development team so as to improve the quality of the final framework.

4. Results: the National Standards Capability Assessment Framework

4.1. Defining the three key pillars of national standards system: SCaM

Prior studies have often referred to a standards-related infrastructure as having a national quality infrastructure as summarised in Table 1 (Guasch et al., 2007; Sanetra & Marbán, 2007). These prior studies used a slightly different scope or definition for the components of national standards. For instance, a national standards system sometimes was referred to differently as SMTQ (UNIDO, 2010) or Standardisation, quality insurance and conformity assessment, accreditation, metrology (SQAM) services (Dinu, 2008). It is noted as well that standardisation and metrology were commonly included in the scope, but conformity-related terms were often differently described. These conformity-related terms include testing, inspection, certification, calibration, accreditation, and conformity assessment.

Table 1. Defining the key components of national standards system and a comparison to prior studies.

Description of system	Components of national standards infrastructure	Source: Authors (Year)
▸ This study: national standards system	• SCaM	This study
National/national quality infrastructure or system	• Standardisation, metrology, testing, certification, and accreditation	Sanetra and Marbán (2007)
	• Metrology, standards, testing, and quality (MSTQ)	Frota et al. (2010)
	• Standardisation, certification, testing, inspection, calibration, and accreditation; Metrology	Guasch et al. (2007)
	• SCaM	ISO, UNIDO (2009)
National/national standards infrastructure	• Scientific level (measurement science)	Park, Bahng, and Choi (2010)
	• Procedure level (conformity assessment procedures)
	• Industry level (standards and technical regulations)
SMTQ infrastructure	• SMTQ	UNIDO (2010) and UNIDO (2003)
SQAM services	• SQAM	Dinu (2008) and DTI/DFID (2003)

However, considering that 157 nations are members of the World Trade Organization (WTO) and 131 nations provided the Agreement on Technical Barriers to Trade (TBT) inquire-point services as of August 2012, the definition of conformity assessment by the WTO can be considered one of the most widely referenced. WTO (1995) defines ‘conformity assessment’ as follows: Conformity assessment procedures include, inter alia, procedures for sampling, testing and inspection; evaluation, verification and assurance of conformity; registration, accreditation and approval as well as their combinations. ISO (2006) and ISO, UNIDO (2009) use almost the same definition for the term ‘conformity assessment’.

As presented in Figure 2, drawing on these definitions of the WTO (1995) and ISO, UNIDO (2009), this paper considers conformity assessment as including overarching testing, inspection, certification, and accreditation of laboratories and further defines the three key pillars of a national standards system as Standardisation, Conformity Assessment, and Metrology (SCaM).

Figure 2. SCaM: The three key pillars of a national standards system.

ScaM – related system and their activities constitute the national standards system of a country interdependently.

First, the Standardisation pillar includes the developing, adopting, and disseminating standards for products, processes, and management systems. Second, the Conformity assessment pillar includes activities and procedures to fulfil specific requirements of standards, such as testing, inspection, certification, and their accreditation. Third, the Metrology pillar includes the process for the scientific and legal metrology system that can ensure that measurements are made with appropriate accuracy and reliability both domestically and internationally (ISO, UNIDO, 2009). A national standards system or infrastructure may also serve as the foundation for a national quality system and, further, a national technology innovation system.

4.2. Developing the seven assessment categories using TQM models

After characterising the key pillars of a national standards system, the first task is how to develop a framework that can evaluate national capability, considering the complicated organisational dynamics of the three SCaM pillars. Different models were discussed for consideration, including Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis (Humphrey, 2005), the diamond model (Porter, 1990), and TQM models. Among these, the TQM models were selected as the basis for an assessment framework for specific reasons.

First, TQM models fit best when measuring and describing national standards capacity, as that capability depends on the management system of standards activities of a nation. Second, a national standards system is the foundation for a national quality system, as previously illustrated in Table 1. It is the primary target of TQM, and therefore, TQM criteria matches well in its nature to identify the current status and improve the national standards capability as the basis of a national quality system. Third, the TQM models have been well developed to evaluate an organisational performance quantitatively with scores that are useful for describing national differences in standards system development.

The TQM models, often called performance (business) excellence models, have been used in different regions and countries. Such as the Australian Business Excellence Framework, European Foundation for Quality Management (EFQM) Excellence Model, Korean National Quality Awards, Singapore Quality Award Criteria, and the Baldrige Criteria for Performance Excellence in the USA. In Table 2, the selected assessment criteria for the TQM models in the USA and Europe and the criteria for a national standards system developed by ITU and ISO are compared to develop precise assessment categories for a national standards system.

Table 2. Seven assessment categories of the NACAF.

Model (sources)	Criteria	Comparison
▸ This study NSCAF	▸ This study	Based on TQM model, focus group interviews (national standards system)
	Laws, systems, and institutions
	Strategies and implementation plan, and Stakeholders
	Infrastructure, Human resources, Process and Outcome
TQM: Baldrige Criteria in the USA (NIST, 2011)	Leadership, Strategic planning, and Customer focus	TQM models used in various sectors/countries (for measuring and improving organisational performance)
	Measurement, analysis, and knowledge management
	Workforce focus, Operation focus, and Results
TQM: EFQM Criteria in Europe (Rusjan, 2005)	Leadership, Strategy, People, Partnerships and resources, Processes, products, and services, Customer results, People results, Society results, and Key results
National ICT Standards System Evaluation (ITU, 2009)	Standards development capacity	Developed by an experts group (ICT sector standards system)
	Human resources
	Policy and Standards use and adoption
ISO Institutional (INS) project (ISO, 2010)	National policy and laws, Scope of National Standards Body (NSB), NSB problems and challenges, and NSB operating environment	Developed by an experts group (institutional aspects)
	Other technical assistance activities and other factors

Note: The seven criteria of NSCAF were developed based on TQM criteria such as Baldrige or ETQM, which are more comprehensive than existing components developed by ISO and ITU. NSB, national standards body.

There are many theories about what constitutes TQM and the scope of the criteria needed to implement it. As Alonso-Almeida and Fuentes-Frías (2012) describe TQM, researchers can likely use the most heterogeneous element as a guide and for evaluation, depending on geographical place, even though various models are the viable form to achieve, and under these criteria do offer excellence in quality topics. Reviewing the existing approaches to a national standards system and the various evaluation criteria of TQM, seven categories are distilled to assess a national standards system. The definitions of these seven categories are offered in Table 3.

Table 3. Seven assessment categories of the NSCAF.

Seven categories	Definitions
Laws, systems, and institutions	• The nation should prepare laws and systems related to standards, establish related institutions, and secure budgets for standards and related laws and systems for each pillar, Institutions to implement related laws and systems, their size, and budgets
Strategies and implementation plans	• The nation should establish and implement strategies and implementation plans for developing national standards. This effort means responding to changes in domestic and overseas standards environments to align standards strategies with national development strategies, establish strategies for developing national standards, align standards strategies with national development strategies, and draw up implementation plans for a standards strategy
Stakeholders	• The nation should clearly identify the stakeholders of national standards and their needs and then reflect these needs in standards: Define stakeholders, identify and analyse stakeholders, reflect their needs, and arrange for feedback
Infrastructure	• The nation should secure the infrastructure, including facilities, equipment, and information technology for national standards activity and provide access to the infrastructure for all stakeholders: Secure necessary facilities, equipment, and information technology; provide easy access to necessary facilities, equipment, and an IT system as well as their usage; and address human resources
Human resources	• The nation should secure and manage human resources for national standards and create and operate mid- to long-term programmes for nurturing human resources: Nurture human resources (training, qualifications, and criteria), a human resources management system, and the competency of human resources (scale and level)
Process	• The nation should establish and implement processes for effective and efficient operation of a standards system: Process for establishing an operations plan, processes for operation, and processes for assessment and improvement
Outcome	• The nation should develop outcome indicators, set objectives, and manage them. Have measurable outcome indicators for the other six categories

Note: The detailed definitions of the seven assessment criteria of NSCAF are provided.

4.3. Combining three pillars and seven evaluation categories: The NSCAF Framework

The three SCaM pillars of a national standards system – SCaM – are merged into seven assessment categories to develop the assessment criteria for the full framework. This national standards capability assessment framework (NSCAF) is conceptualised in Figure 3. Therefore, this combination of three SCaM pillars and 7 categories constitutes 21 assessment categories in the NSCAF. These 21 descriptions are also extended to assessment items and a questionnaire for actual assessment of the framework.

Figure 3. Concepts of the NSCAF.

The NSCAF is a combination of three SCaM pillars and seven assessment criteria.

4.4. Scoring and interpreting the results

The total score for the NSCAF evaluation is 1000 points like the Baldrige or EFQM Criteria. Based on the research interviews, focus group workshops and pilot implementation, 400:300:300 are allocated for the three SCaM pillars – 400 sub-total points are allocated for the standardisation pillar, 300 points for the conformity assessment pillar, and 300 points for the metrology pillar.

The underlying philosophy for this score allocation is that the three SCaM pillars should be balanced and interrelated, and the allocation is more of analytical weight. As standardisation is the core aspect of a national standards system, it was discussed and agreed by participating experts in the focus group workshops that standardisation requires more scores than the other two pillars. Although conformity assessment covers a wide range of activities in standardisation, metrology, especially in the developing countries, is a very important area to be developed by any national standards system. In that sense, conformity assessment and metrology are weighted equally when allocating scoring points. Also, it was discussed and noted that there may be a possibility or a need to have to evaluate each pillar separately considering the purpose of the assessment.

Because each pillar has different characteristics, NSCAF gives different scores for the assessment criteria to each pillar. For instance, laws, systems, and institutions are more important and complicated in terms of standardisation and conformity assessment pillars than the metrology pillar; therefore, these categories receive higher scores. An infrastructure that includes hardware facilities and equipment is more important for conformity assessment and metrology pillars than for the standardisation pillar; outcome category is equally allocated 20% of the sub-total scores for each pillar. The scoring for each pillar and category and its percentage share of the (sub-) total score for NSCAF is described in Table 4.

Table 4. Scoring for NSCAF.

Pillars	Total		I. Standardisation		II. Conformityassessment		III. Metrology
Category	Score	%	Score	%	Score	%	Score	%
Laws, systems, and institution	150	15.0	70	17.5	50	16.7	30	10.0%
Strategies, implementation, and plans	110	11.0	50	12.5	30	10.0	30	10.0
Stakeholders	90	9.0	40	10.0	20	6.7	30	10.0
Infrastructure	155	15.5	45	11.3	50	16.7	60	20.0
Human resources	180	18.0	70	17.5	50	16.7	60	20.0
Process	115	11.5	45	11.3	40	13.3	30	10.0
Outcome	200	20.0	80	20.0	60	20.0	60	20.0
Total	1000	100.0	400	100.0	300	100.0	300	100.0

The scoring items for the NSCAF assessment criteria are classified by the kind of information and data to furnish relative scores for the two evaluation groups – process and outcome, as done in the Baldrige Performance Criteria (NIST, 2011). First, the check sheet for the ‘process’ group is used in evaluating category –, and their assessment items are evaluated by using combinations of X-axis for system-building levels and Y-axis for implementation levels of national standards activities. Second, the check-sheet ‘outcome’ group is used when evaluating category , and assessment items are assessed using the two-dimensional combinations of X-axis for outcome levels and the Y-axis for outcome tendency.

The assessment result, per the SCaM pillars, or in total, can categorise the assessed country as being in one of the five stages of national standards capability. Some least-developed countries with very limited capability may fall into the first, (i) introductory stage, while the majority of developing countries will likely fall into the two middle stages, namely, (ii) the establishing stage, and (iii) the growing stages. Many of the newly developed countries and also the developed countries may fall into the two highest stages, namely, (IV) maturity and (v) innovation as presented in Figure 4.

Figure 4. Five stages of countries using NSCAF.

5. Results: validation by pilot implementation of four countries

5.1. Profile of four countries

This section briefly presents the overview and key results of the pilot implementation. Pilot implementation was conducted in four countries in order to improve and verify the NSCAF. The feedback from the pilot implementation was already incorporated into the final NSCAF framework in the previous section in the interest of simplicity here. The participating countries could not be simply selected, however, to suit the convenience of the framework development team. The participating country had to be willing to conduct weeks of pre-assessment and have the resources to hold a few days of face-to-face meetings for actual assessment. That endeavour usually requires a hard decision made by senior executives of the participating institutions. During the pilot implementation period, all four countries were able to participate.

Table 5 highlights the basic data for the four countries that participated in the pilot implementation. A country is a least developed country, by definition of the United Nations (UN-OHRLLS, 2012) – low gross domestic product (GDP), low income, and low exports. B country and C country are both developing countries, but their socio-economic statistics make some differences. While B country has higher GDP per capita than C country, C country has a higher population, GDP, and export volumes than B country. D country is a newly industrialised nation with high GDP and high income. Both B and D countries have high level exports of goods as a present of GDP. Pilot assessments were conducted by a self-assessment by the assessed country and then by a face-to-face main assessment by the framework development experts panel.

Table 5. Profile of four participating countries involved in pilot implementation.^a

Class	GDP (nominal)	Population	GDP per capita (nominal)	Export of goods	Pilot implementationperiod (Year/Month)
Countries^b	(Billion US$)	(Million)	(US$)	(Billion US$)	Self-assessment	Main assessment
A country	13 B	15 M	900	5 B	2011/08–10	2011/11
B country	67 B	15 M	4569	22 B	2012/07–09	2012/10
C country	124 B	92 M	1411	96 B	2012/01–03	2012/04
D country	1116 B	49 M	22,424	557 B	2012/01–02	2012/03

^aThe original data for GDP nominal (http://data.worldbank.org, accessed on 3 November 2012), population, and export of goods (https://www.cia.gov/library/publications/the-world-factbook/index.html were obtained on 5 November 2012.

^bBy agreement with the governments of the four countries studied, the country name is not disclosed in this paper.

5.2. Assessment results for the three SCaM pillars

The results of the NSCAF assessment can be presented in different forms based on the multiple purposes of the analysis. Figure 5 shows the assessment results of the pilot implementation by total scores and the sub-total of each SCaM pillar. These scores indicate that the capabilities of the four countries are now at different development stages, in total or within each pillar.

Figure 5. Pilot implementation results based on three SCaM pillars.

The ranking of overall scores of the four countries do not always match the ranking of each pillar.

In total, the results of this study indicate that A country with 373 score points was positioned at the establishing stage; B country with 541 points and C country with 563 points were similarly at the same level or the growing stage; D country with 737 points was at the mature stage. Generally, these results match the economic development levels described in Table 5. However, it is also notable that B and C countries showed the same stage of overall national standards capability even though the GDP per capita for B country was almost three times that of C country (around a 3000 US$ gap). This analysis also applies to the comparison for A and C countries, which indicates a different stage of capabilities although their GDP difference per capita is not that large (around a 500 US$ gap). By reasonable assumption then, this variation is related, at least in part, to the institutional development history and the gap in ‘GDP:Export ratio’ for the countries. C country had the highest ‘GDP:Export ratio’ (78%) compared to the three other countries and A (33%), B (42%), and D (50%) countries; also C country established its national standards law and institutions much earlier than the other two countries.

Specifically, these results suggest that each of the three SCaM pillars were developed with a different balance for the four countries. A country was assessed to have better capability in the metrology pillar than Country B, while its overall capability, especially for conformity assessment and metrology pillars, was quite far behind B country. Among A, B, and C countries, B country had the most superior capability in the conformity assessment pillar, while C country had the highest capability in the standardisation and metrology pillars. D country was assessed to have superior capability than the other three countries for all SCaM pillars, and especially for conformity assessment. These results suggest that the three SCaM pillars in any nation may be developed having a different balance due to different socio-economic, technical and cultural scenarios.

5.3. Results for the seven assessment categories

The results of the NSCAF assessment can be presented as seven categories to show which capability category in a country is well-developed or under-developed. Figure 6 illustrates the standards capabilities for the four countries under review by seven pillars, in a heptangular radar chart.

Figure 6. Pilot implementation results using the seven categories. (a) Overall SCaM capability. (b) Standardisation capability. (c) Assessment of conformity capability. (d) Metrology capability.

Figure 6(a) demonstrates their overall capability using seven assessment categories for all the SCaM pillars. A country had the least capability in all assessment categories among the four countries, but it is also equipped with a relatively good infrastructure that is much closer to B and C countries. D country had superior capabilities than the other countries in general, and its human resources capability and outcome capability are relatively low compared to the other five categories. B and C countries presented as quite close in overall capability, with some ups and downs; in most pillars – it was noted that B country lacked in process management, while C country lacked in stakeholder engagement in all SCaM pillars.

Figure 6(b) demonstrates the capability for the standardisation pillar using seven assessment categories. A country had the least capability in all categories of the four countries, and in particular limited capabilities in the second category for developing strategies and implementation plan. D country had superior capabilities than the other countries in general. However, it showed a similar level of capabilities in certain categories with B or C country, for example, its first category of laws, systems, and institutions and its sixth category of process management. B and C countries showed almost the same capabilities although there were certain small gaps per assessment category. Particularly, B country had a superior infrastructure while C country had superior human resources.

Figure 6(c) demonstrates capability in the conformity assessment pillar using seven assessment categories. In all categories for the four countries, A country had inferior capability, and D country had superior capability compared to the other countries. The overall capability of B and C countries was quite comparable, but B country clearly demonstrated a higher capability in stakeholder engagement and human resources development, while C country was stronger in infrastructure development.

Figure 6(d) demonstrates capability in metrology pillar using seven assessment categories. D country clearly demonstrated larger capability in all categories than did the other three countries. The comparisons of those three countries produced a somewhat more complex picture – A country was assessed as most capable in terms of infrastructure and outcome; B country was in laws/systems/institutions, strategies and implementation plan, stakeholder engagement; and C was in process management, human resource development, and infrastructure.

6. Summary and conclusions

No truly available framework has yet been offered for assessing a national standards system. The proposed NSCAF in this paper can serve as a foundation model for future theoretical development and more pragmatic application for national development in the area of standards. In particular, the pilot implementation undertaken here demonstrates that the NSCAF is useful for recognising overall development status and distinguishing whether a standards system is developed in a balanced manner that addresses all three SCaM pillars and the seven categories.

The results of NSCAF can describe detailed strengths and weaknesses of countries using either absolute points or by a comparison with other countries per seven categories. Overall, the results of the research indicate that the development stage of a national standards system may fit an industrial development stage, but the detailed development of national standards systems will vary as a whole or for more than three SCaM pillars for the assessed countries, depending on their individual industrial development and trade characteristics. The feedbacks on the NSCAF from pilot implementation were already incorporated into the framework described in the previous section.

Two years of development and pilot implementation in a limited number of countries for the NSCAF, however, may not be enough to develop a fully mature framework. There is still room to improve. Some issues can be overcome by revising the framework itself, while others may need better implementation of the best know-how for assessment teams and evaluators as has been done for most TQM models. Regardless of this limitation, the NSCAF can be considered as a promising theoretical framework that be used to describe complex components and the dynamic interrelationships of a national standards system. In practice, the NSCAF can be beneficial not only for individual developing countries, but also for international organisations or developed countries who seek to provide capacity building for the developing countries. Also, the NSCAF can be useful for TQM scholars as it adapts TQM model to assess national standards capability and its innovation performance.

Future research may include undertaking in-depth case studies of one or multiple countries. The framework could be further stabilised by wide-ranging assessment implementation in countries that have different population sizes, economies, import and export volumes, industrial structures, and overall industrial development statuses. The NSCAF assessment could be further developed to be used for comparison purposes for countries not only in a horizontal study of a specific year, but also in a longitudinal study over several years so the development trajectory of a national system can be fully developed and then analysed and studied precisely.

Acknowledgments

This research, developing National Standards Capability Assessment Framework (NSCAF), has been part of the International Standards Cooperation Program (ISCP) project, which is funded by the Korean Agency for Technology and Standards (KATS) and operated by the Korean Standards Association (KSA). Special thanks should be given to Dr Kap-Dong Park of the University of Science and Technology (UST) and Mr Byung-soo Yoon of Korean Association of Standards and Testing Organizations (KASTO) for their contributions in developing assessment criteria for the metrology section of the assessment framework; and to Mr Jong-Yoon Jun of KATS and Ms Sunghyun Park of KSA for their support in enabling the authors to conduct this study.