Adaptable Security Maturity Assessment and Standardization for Digital SMEs

ABSTRACT

Small and Medium-sized Enterprises (SMEs) constitute a very large part of every country’s economy and play an essential role in economic growth and social development. SMEs are frequent targets of cyberattacks. Unlike large enterprises, SMEs generally have limited capabilities regarding cybersecurity practices. Assessment and improvement of cybersecurity capabilities are crucial for SMEs to survive and sustain their operations. Despite the availability of maturity assessment models and standards to assess and improve cybersecurity capabilities, SMEs’ specific requirements and roles in the digital ecosystem are often neglected. This paper presents high-level SME requirements regarding cybersecurity maturity assessment and standardization and translates them into an Adaptable Security Maturity Assessment and Standardization (ASMAS) framework to address this gap. The framework is demonstrated by a web-based software prototype. In the evaluation study conducted with SMEs, we obtained positive results for perceived usefulness, perceived ease of use of the framework, and intention to use it.

KEYWORDS:

• Cybersecurity
• information security
• SME
• maturity
• standardization

1. Introduction

Information security and cybersecurity deal with ensuring confidentiality (C), integrity (I), and availability (A) of information. Other information properties, such as authenticity and reliability, can also be involved. According to the ISO/IEC 27032 standard, cybersecurity is preserving those properties in cyberspace. In contrast, information security is not limited to cyberspace and is preserving the CIA in general.¹ Having made this distinction, our focus in this paper is all-encompassing. Thus, we investigate cybersecurity and information security in conjunction and refer to them as security.

According to the World Bank, small and medium-sized enterprises (SMEs) represent 90% of businesses and more than 50% of employment worldwide.² As SMEs are the backbone of every country or region’s economy and digitalization is no longer optional, their resilience to malicious attacks and dependability are increasingly important. SMEs often share a business ecosystem by providing services to large enterprises. An OECD report on the digital transformation of SMEs states that during the ongoing pandemic, SMEs are increasingly using online platforms.³ The prior security challenges for SMEs remain but are amplified with the surge of teleworking and the need for operating remotely.^3,4 Malicious actors exploit these difficult times for their objectives, and governments issue alerts for individuals and organizations to warn them and increase awareness.⁵ Having weak security practices has a twofold effect on SMEs. On the one hand, it can create a barrier for them to engage with large businesses. On the other hand, it can make them targets for attacks as a gateway to penetrate their alliances. SMEs can build trust in their existing or target business ecosystem by establishing good security practices. By doing so, they will have the advantage of pursuing new engagement opportunities.³ “Digital technology and security” is identified as one of the dimensions of digitally enabled growth in SMEs.⁶

The European Digital SME Alliance represents about twenty thousand digital SMEs in Europe. Their position paper on Covid-19 economic recovery proposes the key areas in which quick actions are needed for recovery, one of which is cybersecurity & standards. They emphasize the heterogeneity of SMEs, thus the need for tailored and practical solutions and ensuring SMEs’ access to and awareness of standards.⁷ Both security maturity assessments and security standardization help organizations improve their security capabilities and processes.^8,9 We refer to standardization as adopting existing standards (international, regional, or national). Despite the challenges faced by SMEs, research on cybersecurity considering SMEs has been scant in the literature.¹⁰

Originating from the software engineering domain, organizations have used maturity models to assess and improve their capabilities for a couple of decades.¹¹ Maturity modeling has attracted researchers’ attention in various domains.¹² Information security and cybersecurity domains were no exception.^13–15

Cybersecurity or information security needs, goals, and requirements depend on the organizational context.¹⁶ Therefore, the adaptivity of security solutions, including maturity models and standards, to varying organizational contexts is essential. In security maturity assessment literature, researchers have investigated the organizational context and adaptivity of the assessment models from various angles. An information security maturity assessment and process improvement tool has been proposed for SMEs.¹⁷ This tool is in the form of a standards-based questionnaire conducted by the researchers. However, there is no information in the published work about how the questionnaire is adapted to different organizational contexts.¹⁷ Mijnhardt et al.¹⁸ approach organizational characteristics in two ways. First, as some indicators such as the number of employees and revenue. Second, as the nature of business processes such as outsourcing of or complexity in information technologies. Yigit Ozkan and Spruit¹⁹ have investigated the design requirements for an information security maturity model adaptable to SMEs by focusing on internal characteristics of SMEs, such as lack of organizational capabilities, short-term vision, and orientation.

In cybersecurity standardization literature, gaps and needs for SMEs have been identified in a research agenda that points out the adaptivity issues within several research questions.¹⁰ Barlette and Fomin²⁰ state the need for future research on the creation and adoption of simplified security methods or standards dedicated to SMEs. Manso et al.²¹ recommend that “standards applicable by SMEs should incorporate maturity levels with different sets of requirements to facilitate a phased implementation” to facilitate the implementation of security standards by SMEs. Inspired by these research gaps in the literature regarding adaptivity issues in security maturity assessment and standardization, and adoption challenges of SMEs, we state our research objective as “To integrate security maturity assessment and standardization in an adaptive instrument to support concurrent implementation efforts of digital SMEs.”

To address this research objective, we employ a Design Science Research approach. We investigate the needs, goals, and requirements of SMEs and propose a design artifact as a solution. Our design artifact is the Adaptable Security Maturity Assessment and Standardization (ASMAS) framework for digital SMEs. The framework builds on a set of high-level SME requirements, is adaptable to the different roles SMEs take in digital ecosystems, and embeds security risk management and standardization concepts. The artifact is developed to support SMEs in establishing, improving and demonstrating security maturity and standardization concurrently. The framework may also guide researchers and practitioners in the development of security maturity assessment models for SMEs.

The remainder of the paper is structured as follows. Section 2 provides the relevant background information. In Section 3, we explain our research approach and methodology. Section 4 introduces the high-level SME requirements regarding security maturity assessment. Section 5 presents the ASMAS framework and its aspects that address the high-level requirements. In Section 6, we describe the software prototype. Section 7 presents the evaluation of the framework and the evaluation results. In Section 8, we discuss the results and the limitations. Finally, conclusions are drawn, and potential areas for future research are proposed.

2. Background

2.1. SME characteristics and categories

Several researchers investigated how SME characteristics are different from larger companies.^22–24 The implications of SME characteristics on information security have been investigated at single SME and cluster levels.^18,25,26 Furthermore, the effect of SME characteristics on the design of information security maturity models has been investigated, and the design requirements to be considered have been reported.¹⁹ The question of what approach to take for categorizing SMEs according to their security requirements has been a pertinent one. The European Digital SME Alliance has proposed SME categories with respect to SMEs’ role in the digital ecosystem in their position paper on the European Union Cybersecurity Act and the role of standards for SMEs.²⁷ Table 1 presents these categories.

Table 1. SME categories according to their roles in the digital ecosystem.^27.

SME Category	Description
Digital enablers	SMEs that are active in developing and providing cybersecurity solutions.
Digitally based	SMEs that are highly dependent on digital solutions for their business.
Digitally dependent	SMEs that depend on digital solutions as end-users.
Start-ups	SMEs that neglect or are not well aware of cybersecurity and require specific measures and incentives to adopt cybersecurity solutions.

The European Digital SME Alliance focuses on two challenges of SMEs: cybersecurity and standardization that are to be addressed by distinguishing the SME categories.²⁷ The categorization in Table 1 takes into account the different security requirements of digital SMEs that originate from their various roles in the digital ecosystem. As our research aim is to support digital SMEs in their security and standardization efforts, in this research, we opted to use these categories presented in Table 1. SMEs’ security and standardization requirements depend on and are shaped by their roles in the digital ecosystem. The Start-ups category in this categorization does not refer to the stage of the operation of a company, as described in Table 1, these SMEs are the ones that neglect or are not well aware of cybersecurity.

2.2. Security standardization and SMEs

Despite SMEs’ challenges in security standardization,²⁷ there are no information security or cybersecurity standards available specifically for SMEs.¹⁰ Barlette and Fomin²⁰ state that few information security standards are theoretically suitable for SMEs, but given the cost, the skills needed, and the language issues, it can be assumed that there is no method that can help SMEs to improve their security. This hasn’t changed in time; however, there are guidelines, technical reports, and frameworks that can help SMEs in security standardization.^28–30 Security standardization produces opportunities but also presents challenges for SMEs. The Digital SME alliance in corporation with Small Business Standards has published an SME Guide³¹ for the implementation of ISO/IEC 27001 for establishing an information security management system. There are initiatives of public and private sector actors in different countries to help SMEs with cybersecurity. These initiatives are in the form of guidelines, frameworks and certification schemes. Some examples are as follows: Cyber Essentials from the UK,³² The Center for Cyber Security Belgium SME Guide from Belgium,³³ Center for Internet Security Controls from USA,³⁴ and ETSI (global),³⁵ NIST Small Business Information Security from USA,³⁶ and Finnish Cyber Security Certificate from Finland.³⁷

2.3. Security maturity assessment

Maturity models in different domains have been developed and used since they became popular after the introduction of the Capability Maturity Model of the Software Engineering Institute of Carnegie Mellon University.¹¹ There is abundant research related to security maturity modeling.^8,14,38

The SME characteristics that influence their security maturity proposed by Mijnhardt et al.¹⁸ are indicators (e.g., number of employees, revenue) to distinguish between a wide variety of different organizations. Following a similar categorization, Sánchez et al.³⁹ proposed a maturity model for information security management within SMEs. In another study, researchers investigated how to address internal SME characteristics for designing information security maturity models.¹⁹ Benz and Chatterjee⁴⁰ proposed a cybersecurity assessment tool specifically for SMEs. The tool is based on the National Institute of Standards and Technology’s (NIST) Cybersecurity Framework (CSF),⁴¹, and it uses a subset of activities of the NIST CSF that are chosen as applicable to or feasible for SMEs. Although the proposed tool provides SMEs with a tool for assessing their cybersecurity capabilities concerning (a subset of) a reference model, it does not take the SMEs’ different roles in the digital ecosystem and their different needs that emerge out of their roles.⁴⁰ NIST CSF itself includes tiers that “describe an increasing degree of rigor and sophistication in cybersecurity risk management practices” which may guide SMEs to start with informal and reactive approaches and progress toward more risk-informed approaches.⁴¹

Although research has been carried out on security maturity models targeting SMEs, or the adaptability of existing models to SMEs, no studies have considered the different roles SMEs take in the digital ecosystem.

3. Research methodology

We followed the Design Science Research (DSR) methodology consisting of the following steps: identify problem and motivate, define objectives of a solution, design and development, demonstration, evaluation and communication.⁴² Accordingly, our research includes realizing a problem situation, identifying high-level SME requirements (objectives of a solution), developing the framework, demonstrating and evaluating the use of the framework with a prototype, and communicating the research results. Our research process is presented in Figure 1.

Figure 1. The research process.⁴²

As discussed in the introduction and background sections, we reviewed the literature to identify the problem and motivate our research. The problem identification and motivation step provided the input for setting the high-level requirements for security maturity modeling and standardization for SMEs. These requirements serve as objectives for the design and development step to be addressed by the framework (i.e., design artifact) aspects and components. The framework was developed by assembling the aspects and components to address the requirements.

To evaluate our framework, we adapted the “Prototyping” evaluation pattern for DSR artifacts, and hence we provide an implementation of the solution through a prototype.⁴³ We created a knowledge base in the prototype to demonstrate the aspects included in the framework. Pries-Heje et al.⁴⁴ investigate the strategies for DSR evaluation and propose a framework that encompasses both ex-ante and ex-post orientations in naturalistic or artificial settings for DSR evaluation. We adapted evaluation constructs from the Technology Acceptance Model (TAM)⁴⁵ to evaluate our framework with SMEs for predicting how likely our design artifact is to be adopted in practice.⁴⁶ Our evaluation study was conducted with practitioners from real SMEs after the artifact was constructed; therefore, our evaluation is ex-post. We evaluated the ASMAS framework by conducting interviews with SMEs, which is accompanied by an evaluation form.

4. High-level requirements

In the DSR, understanding the problem space is crucial to propose useful artifacts to real-world problems.⁴⁶ There are four key concepts to understand and define the problem space: needs, goals, requirements, and stakeholders.⁴⁷ The stakeholders in our research are SMEs. On the one hand, being non-homogeneous, SMEs have different needs regarding cybersecurity and standardization that depend on their organizational context (i.e., their role in the digital ecosystem). On the other hand, SMEs’ goal (intended outcome) is to secure their organization against cyber threats. We derived a set of high-level requirements (HLRs) regarding security maturity modeling and security standardization for SMEs to address this goal.

4.1. HLR1- Easy to use, self-assessment, do-it-yourself

Assessment and improvement planning should be easily realized by SMEs, requiring minimal extra resources.

Rationale: SMEs lack the resources (time, money, and expertise) to establish security capabilities and security standardization.^22,48 The existing security maturity models and standards are costly and complex. Lack of financial resources is a barrier for SMEs to get external support to help with cybersecurity.⁴⁹ Self-assessment is a means by which an organization assesses compliance to a selected reference model or module without requiring a formal method.⁵⁰ Ritchie and Dale⁵¹ summarize the benefits of self-assessment from a quality management perspective. Most of the benefits are also applicable from a security management perspective. The following are critical for SMEs with a security perspective: self-assessment helps keep costs down, raises understanding and awareness of security, and develops a holistic approach to security. The Cyberwatching.eu project surveyed cybersecurity standardization gaps. Their white paper summarizes the findings and recommendations and states the need to explore self-assessment and other low-cost solutions.⁵²

4.2. HLR2 – Situational awareness

The assessment model should provide customized guidance and implementation plan according to SME categories.

Rationale: Digitalization brings security as a critical element in business model scaling, and it is both a necessity and an enabler for SMEs.⁵³ Depending on the digitalization level, SMEs’ needs for cybersecurity dramatically differ. None of the available security models addresses this situational aspect. In design science research, artifact mutability ‒ the adaptability of DSR artifacts ‒ is proposed as one of the components of design theories.⁵⁴ The design of adaptable artifacts is referred to as situational artifact construction (SAC). SAC allows the researcher to develop artifacts which are adaptable to different design problems within a problem class, and to understand the relevant design situations within this class.⁵⁵ As the costs for adapting a more generic solution artifact to a specific design problem are higher than those for adapting the more specific solution artifact, developing situational artifacts reduces the cost of adaptation.⁵⁵ Understanding the organization’s context is the primary step when establishing information security or cybersecurity.¹⁶ SMEs are not homogenous, and they differ in their requirements for security. The difference between SMEs can be characterized by their role in the digital ecosystem.^27,53

4.3. HLR3 – Support for standardization and standards-transparency

The framework should support the ability to adhere to related standards on security. The relation between security capabilities and standards should be transparent. The framework should provide a progressive path for adopting standards and frameworks to help SMEs follow a step-by-step approach.

Rationale: In standard development, SMEs are often neglected and require financial support, access to technical expertise, and assistance to be active stakeholders.⁵⁶ Maturity models have the basic design principle of “Definition of central constructs related to the application domain.”⁵⁷ If the application domain has achieved a level of maturity to have published standards by standards developing organizations, these standards can be used as sources for defining domain-specific constructs of the maturity model.⁵⁸ A maturity assessment model having constructs based on standards in the application domain can help organizations in both their maturity improvement and standardization efforts simultaneously by integrating maturity assessment and standardization in the same tool (design artifact). Organizations using the maturity assessment model to improve their cybersecurity would be able to adhere to (or adopt) the standards in the application domain. While establishing security capabilities, SMEs can improve their adherence to security standards. This can be accomplished by using standards as the primary source for maturity model capabilities. Consequently, we derive the following HLR:

4.4. HLR4 – Provide security awareness

The model should help increase security awareness concerning the assessed capabilities by considering SME categories.

Rationale: SMEs’ awareness of security and related standards is low. This partly stems from the lack of resources and the security domain’s perceived complexity.⁵⁹ SMEs’ level of security awareness might differ according to their role in the digital ecosystem. By considering human actors as part of the solution rather than the problem regarding security,⁶⁰ awareness of the organizations’ employees and managers has critical importance.

4.5. HLR5 – Maintainability and adaptability by design

New standards, threats, risks, and capabilities should easily be included in the assessment model.

Rationale: Given the ever-changing and dynamic nature of security threats and risks, it is crucial to incorporate emerging security capabilities and standards. This will ensure the maintainability of the assessment model and support it to be future-proof. In design science research, this phenomenon is referred to as “mutability-in-use,” a strategy that takes into account the future needs that may emerge when the artifact is in use.⁶¹

In the following section, we propose an adaptable security maturity assessment and standardization framework to address the HLRs.

5. The Adaptable Security Maturity Assessment and Standardization (ASMAS) framework

The ASMAS framework integrates five aspects: organization, standardization, risk management, assessment and measurement, and improvement. Figure 2 illustrates the meta-model of the ASMAS framework.

Figure 2. The meta-model of the Adaptable Security Maturity Assessment and Standardization (ASMAS) Framework.

In the following subsections, we elaborate on why the aspects are needed to address the high-level requirements presented in Section 4 and how the aspects work together. The framework addresses the SME categories shown in Table 1. The Organization aspect presented in Section 5.1 elaborates on how this is accomplished. A mapping of the high-level requirements and the aspects that address them is presented in Section 5.6.

5.1. Organization aspect

The Organization aspect is included in the framework to mainly make it situation-aware and enable self-assessment by organizations. The Organization aspect comprises the SME categories and Implementation Groups (IGs). IGs are composed of Control Categories. Each control category (1 to n) has several controls populated from standards and frameworks. IGs are constructed according to SME categories and provide SMEs in each category with a tailored perspective on the controls and capabilities relevant to their organizational profile. The Organization aspect enables SMEs to explore and use the framework according to their IG. Figure 3 illustrates the Control Categories, IGs, and the associations between the SME categories and the IGs.

Figure 3. Control categories, implementation groups, and their associations with the SME categories.

The Organization aspect addresses the “self-assessment” HLR(#1) and the “situational awareness” HLR(#2) by using a designated implementation group per SME category. The Organization aspect introduces configuration parameters for artifact mutability.⁶²

5.2. Standardization aspect

The standardization aspect is included in the framework to provide organizations with support in their standard adherence and awareness processes. The Standardization aspect comprises standards and frameworks. Standards and frameworks are the sources of security controls in the Risk Management aspect. Controls are the sources for capabilities and assessment questions in the Assessment and Measurement aspect. The Standardization aspect addresses HLR(#3) (support for standardization and standards-transparency), and HLR(#5) (maintainability and adaptability by design) by facilitating the inclusion of new standards and frameworks (mutability-in-use).

Overlapping controls in the standards and frameworks should be taken into account to eliminate duplicate end-user efforts. This is accomplished by deriving the capabilities (see Section 5.4) by considering all the controls from relevant standards and frameworks for each control category.

The Standardization aspect introduces best practices from standards and enables the integration of maturity assessment and standardization in one framework.

5.3. Risk management aspect

The Risk Management (RM) aspect is included in the framework to provide organizations with cybersecurity awareness based on threats, risks, and associated controls from the standards and frameworks. The RM) aspect introduces the core concepts of security risk management and comprises threats, risks, and controls. Ontologies and taxonomies can be used as sources of threats and risks. Control categories and controls are to be derived from standards and frameworks. Control categories are considered as a group of controls that have common objectives. The RM aspect enables SMEs to explore and use the framework according to their threats and risks. The RM aspect addresses “the situational awareness” HLR (#2) (by the association of implementation groups and controls). As the RM aspect incorporates risks and threats, it addresses the “provide security awareness” HLR (#4). The RM aspect also addresses “the maintainability and adaptability by design” HLR (#5) by enabling the inclusion of emerging threats, risks, and controls (mutability-in-use).

For specific SMEs, we presume that a security risk analysis will yield more appropriate controls than those assigned per SME category through implementation groups. The concern is to provide SMEs with a quick start to security that can be improved upon. The RM aspect in the framework is for supporting SMEs to have a risk-based view on the controls.

5.4. Assessment and measurement aspect

The Assessment and Measurement (A&M) aspect is included in the framework to provide organizations with a means for self-assessing their security capabilities. One of the purposes of using a security maturity model is the assessment of existing capabilities and measuring performance.^8,57,63,64 This purpose of use is referred to as “descriptive purpose of use.”⁵⁷ The A&M aspect comprises the following: capabilities, assessment questions, measurement mechanism, focus areas, response scale, and maturity levels. To assess the current maturity level of a functional domain, measures must be defined for each of the capabilities. This can be done by formulating assessment questions for each capability. Formulation of the questions is usually based on the descriptions of the capabilities, experience, and practices.⁶⁵

A measurement mechanism is required for measuring the current level of an organization’s capabilities. Although a two-level scale (Implemented‒Not-implemented) might be used, a four-level scale will give higher precision.⁶⁶ Focus areas are the areas that have to be developed to achieve maturity in a functional domain.⁶⁵ In our framework, control categories in the Standardization aspect correspond to focus areas in the A&M aspect. Given the answers to the assessment questions, a maturity level per control category (i.e. focus area) can be calculated according to the desired measurement mechanism. The function calculatematurityLevel() function in Figure 2 is to be developed to achieve this purpose. Together with the Organization aspect, the A&M aspect addresses the self-assessment HLR(#1) and situational awareness HLR(#2). SMEs will face the applicable assessment questions associated with their organizational profile (mutability-in-design). Measurement of the maturity is based on the answers provided to the assessment questions.

5.5. Improvement aspect

The improvement aspect is included in the framework to provide organizations help with approaching security and standardization progressively. The Improvement aspect is tied to the capability assessment questions and triggered by the assessment results. Another purpose of using maturity models is to improve capabilities to the desired level on the scale. This purpose of use is referred to as “prescriptive purpose of use.”⁵⁷ The Improvement aspect comprises improvement actions. When the SME performs a self-assessment using the assessment questionnaire, based on the given answers, the capabilities that are not currently fully implemented can be used to formulate a customized improvement plan that also facilitates standardization efforts.

The standards transparency HLR(#3) is addressed by the Improvement aspect, which ensures that SMEs have a quick reference for the capabilities (that are derived from standards and frameworks), increase their standards awareness and adherence to standards. Together with the Organization and Risk Management aspect, the Improvement aspect addresses the situational awareness HLR(#2). Personalized improvement plans can be prepared by selecting controls that are designated according to the SME categories. Both the Organization aspect (by implementation groups) and the Risk Management aspect (by the association of implementation groups and controls) play a role in accomplishing this. As part of the Improvement aspect, providing security awareness addresses another HLR(#4).

5.6. Mapping of high-level requirements and aspects

Table 2 shows the mapping of the high-level requirements and the aspects that address the corresponding requirement. The rationale for these mappings is discussed in Section 5.1–5.5.

Table 2. Mapping of the high-level requirements (HLRs) and the framework aspects.(O: Organizational, S: Standardization, RM: Risk Management, A&M: Assessment and Measurement, I: Improvement).

#	Requirement	O	S	RM	A&M	I
HLR1	Easy to use, self-assessment, do-it-yourself	✓			✓
HLR2	Situational awareness	✓		✓		✓
HLR3	Support for standardization and standards-transparency		✓			✓
HLR4	Provide cybersecurity awareness			✓		✓
HLR5	Maintainability and adaptability by design		✓	✓

The ASMAS framework addresses all high-level requirements by integrating the aspects that are described above and visually presented in the meta-model (Figure 2). Given the challenges faced by SMEs, the ASMAS framework proposes a “one-stop-shop” for SMEs to start with security and standardization.

6. The software prototype and the knowledge base

A prototype is implemented to demonstrate the ASMAS framework as a web-based application using a low-code software development tool ‒ Mendix Studio Pro 9.0.2 platform ‒ (free version).⁶⁷ The screenshots of the prototype are presented in Appendix 1 and illustrations based on scenarios are presented in Appendix 2. Figure 4 presents the conceptual model of the prototype.

Figure 4. The conceptual model of the ASMAS framework prototype.

As shown in Figure 4, the prototype includes a knowledge base of the framework components (e.g., control categories, controls, standards, risks, threats). The knowledge base was populated from five standards and frameworks (control sources).²⁸ These standards and frameworks are as follows: Cyber Essentials from the UK,³² The Center for Cyber Security Belgium SME Guide from Belgium,³³ Center for Internet Security Controls from USA,³⁴ and ETSI (global),³⁵ NIST Small Business Information Security from USA,³⁶ and ISO/IEC 27002:2013.⁶⁸ We analyzed these standards and frameworks and put forward a union set of control categories (see Section 5.3). We used the SME categories presented in Table 1 to identify the SME profiles (Implementation Groups) in the prototype. We populated the knowledge base with the threat taxonomy of ENISA (European Union Agency for Cybersecurity),⁶⁹ and with the taxonomy of operational cybersecurity risks from the Carnegie Mellon University, USA.⁷⁰

The knowledge base’s content implemented in the prototype is summarized in Tables 3 and 4. Appendix 3 presents the illustration of the controls and capabilities for a control category.

Table 3. The summary of control-related content of the knowledge base.

Standard/ Framework	# of Controls
Cyber Essentials	5
The Center for Cyber Security Belgium SME Guide	13
Center for Internet Security Controls	20
NIST Small Business Information Security	33
ISO/IEC 27002	123
Total # of Controls	194
Total # of Control Categories	18

Table 4. The summary of capability, risk, and threat-related content of the knowledge base.

Data	Total #
Capabilities	251
Risk Classes	4
Risk Sub-Classes	13
Risks	57
Threat Categories	8

In the prototype, the process starts with the SME manager identifying their organization’s category according to Table 1. Subsequently, the prototype assigns an implementation group for the SME, and the SME manager can view the recommended set of controls for their category. The prototype incorporates considerably simple frameworks such as Cyber Essentials, as well as a comprehensive standard such as ISO/IEC 27002. The inclusion of such diverse sources of controls and unified control categories enables digital SMEs to be able to see and compare the controls that standards and frameworks offer. A digital SME, having implemented Cyber Essentials, can see what additional controls are included in ISO/IEC 27002. SMEs can also focus on one control category that they consider would be useful in their business context and plan for acquiring the capabilities associated with that control category. As the prototype has been designed in a way that the appropriate set of control categories are assigned to SME profiles, the cost of adopting the capabilities can be reduced.

Table 5 presents the functions implemented in the software prototype.

Table 5. The functions in the ASMAS framework prototype.

Function	Description
Definitions	This function is used to populate the knowledge base. Using this function, it is possible to add new entities and items (e.g., new controls, new control categories, new risks) to the knowledge base (mutability-in-use).
Company	This function is used to define the company names for demonstration purposes. Using the “Company-Capability” sub-function, it is possible to view the controls assigned to each company.
Views	This function has three sub-functions. The “SME Category-Control” is used to query the controls associated with an SME category. The “Threat view on Controls” enables the user to query all the associated controls with a threat category. Using the “Risk view on Controls,” the user can query the related controls by choosing a risk class and a risk sub-class.
Search	This function is used for querying the controls. The user can select a control category and a control source and query the associated controls. If no control category or control source is chosen, then all the controls in the knowledge base are listed.
Assessment	This function is for performing capability assessments and viewing the results. The user can enter the implementation status for the capabilities. The capability order and capability level are displayed while performing the assessment.

7. Evaluation of the framework

In this section, we present the evaluation of the ASMAS framework by instantiating and demonstrating the framework using the software prototype and the evaluation results. We followed purposive sampling to find the participants for our evaluation study that fit our research context. We aimed to find SMEs that are related to cybersecurity maturity. To acquire SMEs for our evaluation study, we reached out to the SME participants in the EU Horizon2020 SMESEC project⁷¹ and the EU Horizon2020 GEIGER project.⁷² The focus of these two projects is cybersecurity for SMEs. We conducted evaluation sessions with SMEs that responded positively to our request.

We have conducted seven evaluation sessions with six SMEs. Each session consisted of an approximately one-hour online interview. There were two experts from one SME who were willing to participate in the evaluation study. In each session, one of the researchers presented the conceptual framework and then demonstrated the framework using the software prototype. This was followed by a discussion. At the end of each interview session, we sent out evaluation forms to the participants to obtain their feedback on the utility of the proposed framework.

To understand and predict the acceptance of our design artifact, we have focused on the constructs perceived usefulness, ease of use, and intention to use in line with the Technology Acceptance Model (TAM) core constructs.^45,46 Perceived usefulness refers to the user’s perception concerning how the design artifact enables the user to enhance their performance in a given context. Perceived ease of use entails the user’s perception concerning the degree to which use of the artifact would not require physical or mental effort. Intention to use explains user acceptance of the design artifact.⁴⁵ The perceived ease of use and perceived usefulness affect the intention to use the design artifact. External variables (e.g., training, background experience, risks, and opportunities) also affect the intention to use. Intention to use can be used to explain what utility is created by means of the use of the artifact. We used these three constructs to guide the further design of our evaluation interviews and questionnaires. Table 6 presents the set of questions in the evaluation form to evaluate each construct. We used three statements per evaluation construct to evaluate the framework resulting in nine statements.

Table 6. Set of questions used to evaluate the utility of the ASMAS framework.

Evaluation Construct	#	Statement
Perceived usefulness	1	I think this framework contributes to supporting SMEs to assess and improve their information security and cybersecurity.
	2	I think this framework contributes to helping SMEs increase awareness of information security and cybersecurity security.
	3	I think this framework contributes to helping SMEs increase awareness of information security and cybersecurity security standardization.
Perceived ease of use	4	I think this framework is easy to use to assess and improve my company’s information security and cybersecurity.
	5	I think this framework is easy to use to improve my company’s awareness of information security and cybersecurity security.
	6	I think this framework is easy to use to improve my company’s awareness of information security and cybersecurity security standardization.
Intention to Use	7	I would use this framework to assess and improve my company’s information security and cybersecurity.
	8	I would use this framework to improve my company’s awareness of information security and cybersecurity security.
	9	I would use this framework to improve my company’s awareness of information security and cybersecurity security standardization.

In the evaluation form, we used a 5-point Likert scale to understand the level of agreement of the interviewee concerning each statement, for which 1 represents ‘strongly disagree’ and 5 represents ‘strongly agree.’ We also gathered the characteristics of the SMEs and the interviewees via the evaluation form. These characteristics are presented in Table 7.

Table 7. Evaluation study participants’ characteristics

SME #	Country	Category^Citation27	# of Employees	Interviewee’s # of Years Security Experience	Interviewee’s Role**	SME Size^Citation73
1	United Kingdom	Digital Enabler	<10	23	CEO	Micro
2	Egypt	Digitally Based	<10	2	CEO	Micro
3*	Netherlands	Digital Enabler	<250	4	Security Specialist	Medium
3*	Netherlands	Digital Enabler	<250	6	Business Security Consultant	Medium
4	Switzerland	Digitally Based	<10	1	CEO	Micro
5	Switzerland	Digitally Dependent	<10	20	CISO	Micro
6	Italy	Digitally Based	<50	5	CTO	Small

* two separate evaluation sessions have been conducted with SME #3. ** CEO: Chief Executive Officer, CISO: Chief Information Security Officer, CTO: Chief Technology Officer

7.1. Evaluation results

The results of the evaluation are presented in Table 8. We elaborate on the findings per evaluation criteria and present quotes from the participants in the following paragraphs.³²

Table 8. Responses to the evaluation form statements.

Evaluation Construct	Statement	Strongly disagree (1)	Disagree (2)	Neutral (3)	Agree (4)	Strongly agree (5)	Average Score
Perceived usefulness	1	0%	0%	14,3%	28,6%	57,1%	4.43
	2	0%	0%	28,6%	42,9%	28,6%	4.00
	3	0%	0%	14,3%	28,6%	57,1%	4.43
	Average score per construct						4.29
Perceived ease of use	4	0%	0%	14,3%	28,6%	57,1%	4.14
	5	0%	0%	28,6%	42,9%	28,6%	4.00
	6	0%	0%	28,6%	14,3%	57,1%	4.29
	Average score per construct						4.14
Intention to Use	7	0%	14,3%	14,3%	42,9%	28,6%	3.57
	8	0%	14,3%	28,6%	28,6%	28,6%	3.43
	9	0%	14,3%	14,3%	14,3%	57,1%	3.86
	Average score per construct						3.62

We also included optional open-ended questions in our evaluation form to gather more insights on the perception of the framework. Not every SME preferred to answer those questions. We also present and discuss the answers we gathered for the open-ended questions in the following paragraphs.

7.1.1. Perceived usefulness

Regarding the perceived usefulness, the majority of the participants considered the ASMAS framework as useful, given the high responses associated with the corresponding statements (statements 1, 2, and 3).

The answers to the open-ended questions support the findings as follows.

“The framework can also be used to support standard compliance efforts.” [Participant 1]

“Provides order, structure, guidance, helps SMEs get started, extensible, can be used or adopted by practitioners” [Participant 3]

“Having a single place where all the major standards/frameworks are collected without having to look for them in many different places is certainly a strong point. Prioritization of the different gaps is also a strong point.” [Participant 7]

7.1.2. Perceived ease of use

Most of the participants considered the ASMAS framework easy to use, given the high responses associated with the corresponding statements (statements 4, 5, and 6). The answers to the open-ended questions support the findings as follows.

“Easy to use, particularly if a minimal level of proficiency is already present. Great for figuring out different standards and requirements.” [Participant 4]

“Despite being easier to use than the standards that are incorporated, it may still be useful to provide a tutorial/”getting started” for SMEs in order to get them started using the framework.” [Participant 4]

“Although it provides a very good introduction into the topic, SME might be overwhelmed by the number of questions and tasks.” [Participant 5]

7.1.3. Intention to use

The participants had diverse responses associated with the corresponding statements (statements 7, 8, and 9) regarding the intention to use the framework. The answers to the open-ended questions support the findings as follows.

“It’s really interesting and definitely, I would use it if it’s available.” [Participant 2]

“I would consider using this tool, but now am getting more familiar with ISO 27001 every day and may not be in the target audience anymore.” [Participant 4]

One SME disagreed with all of the statements that were designed to evaluate the intention to use the framework. This SME defines its business as follows: “Provider of a SaaS Solution for Internal Control, Audit Management, ISMS, Governance Risk and Compliance.” As their security maturity is relatively high and they provide security services consultancy, their intention to use the framework is low.

7.1.4. Other feedback and discussions

“The framework could also support subcategories of SMEs under the given categorization” [Participant 1]. We discussed this feedback with the participant; it refers to more tailoring for SME categorization. For example, subcategories might be introduced according to other characteristics of an SME category.

“We have the experience that even if SMEs want to take care of the topic, business needs always have priority, and since they are short on resources, this topic often gets the least attention. Not good, but the reality.” [Participant 5]. This feedback is relatively straightforward, and supports the high-level requirements that we pointed out in Section 4.

“It would be nice if the framework could give an indication of the initial status of the SME (good – sufficient – bad, for example, or a scoring system) and update this each time a new assessment is performed. SMEs might not be able to cope with all the gaps identified by the framework due to lack of time so maybe you could reduce the gaps to just the ones which are more important and leave the less important ones as simple ‘suggestions.’” [Participant 7]. This feedback is about complexity as the participant suggests categorizing the capabilities as mandatory/suggestion. The implementation of the capability levels from A-D in the prototype was designed to help reduce the implementation complexity for SMEs. This can be interpreted as level A capabilities being mandatory to implement and followed by B, C, and D level capabilities.

8. Discussion and Limitations

To summarize, we demonstrated the framework aspects and the high-level requirements by using a software prototype. Concerning validity, the prototype addresses HLR2, HLR3, HLR4, and HLR5. The HLR1 (Easy to use, self-assessment, do-it-yourself) is partially addressed by the Assessment and Measurement aspect, and the Organization aspect in the prototype as the SMEs are exposed to the components (i.e., controls, control categories, and capabilities) of the framework according to their profile that makes the framework easier to use. The demonstration of the perceived ease of use as part of the HLR1 requires SMEs’ involvement as the framework’s end-users. Concerning the technical feasibility, the development of the software prototype shows that the framework can be operationalized. The answers to the open-ended questions and the feedback gathered during the interviews support the high-level requirements presented in Section 4. As discussed in Section 4, designing situational artifacts reduces the cost of adaptation. As the ASMAS framework proposes the set of control groups and capabilities dedicated to SME profiles (i.e., based on their roles in the digital ecosystem), SMEs have the opportunity to have the advantage of cost reduction benefits by implementing only the appropriate controls and capabilities concerning their profiles.

We explained the selection process of the evaluation study participants in Section 7. Due to the high resource intensiveness of the evaluation sessions both on the researchers’ side and the interviewees’ side and the difficulties in finding other SMEs, we had to limit the number of evaluation sessions to 7. Our evaluation study has been conducted through in-depth interviews. Each session has taken at least 1 hour in addition to preparations. During the sessions, we introduced the framework, demonstrated the framework via the prototype, and answered the questions posed by the interviewees. The evaluation sessions are accompanied by an evaluation form sent to the interviewees. We conducted evaluation sessions with SMEs that responded positively to our request. Considering the design of the evaluation study (in-depth interviews), the number of SMEs who participated in the evaluation study is deemed reasonable. More interviews, especially at least one interview with a start-up SME, can be considered future work.

The ASMAS framework can support current initiatives for adopting standards and frameworks (e.g., Cyber Essentials,³² The Center for Cyber Security Belgium SME Guide,³³ Center for Internet Security Controls,³⁴ ETSI (global),³⁵ and NIST Small Business Information Security). A scenario illustrating how this can be achieved is given in Appendix 2, Scenario 1. The example scenario is illustrated based on The Center for Cyber Security Belgium SME Guide, but the same approach is applicable to the other standards and frameworks. The framework also enables the adoption of more than one standard and framework simultaneously with the help of a unified set of control categories. It is worth mentioning that adopting or adhering to more than one standard or framework can be a business (e.g., expanding business in other countries) or contractual requirement. In comparison to current initiatives mentioned in the paper at hand, the ASMAS framework does not offer new controls for cybersecurity. The ASMAS framework aims for putting applicable initiatives together within a unified set of control categories and enhancing their use with a situational assessment perspective that incorporated risk and threat information.

Although further research could be conducted to validate the high-level requirements, the evaluation results of the design artifact that addresses the HLRs support their validity implicitly. In addition, the quotes from the interviewees support the validity of the HLRs.

9. Conclusion

Emerging cyber threats, standards, and regulatory requirements place organizations under pressure to implement security measures and provide assurance to regulators promptly. Organizations can establish and improve their security and compliance using maturity assessments and standards. We have formulated our research objective to support organizations to accomplish this with an adaptive instrument. We have proposed an Adaptable Security Maturity Assessment and Standardization (ASMAS) framework to address our research objective. We have presented the high-level requirements of such a framework by adopting an SME perspective. Since SMEs are not homogenous, and their cybersecurity needs differ, we augmented our framework by incorporating SMEs’ roles in the digital ecosystem. We then presented the five aspects of the framework: Organization, Standardization, Risk Management, Assessment and Measurement, and Improvement that facilitate a novel approach to security maturity assessment and standardization through a meta-model and a software prototype. We pointed out how the aspects support the five high-level requirements, and we integrated the aspects to construct a software prototype that demonstrates the validity and applicability of the framework.

We conducted seven evaluation interviews with six SMEs from five countries. We used the evaluation constructs based on the Technology Acceptance Model to explain and predict the utility of the ASMAS framework. The evaluations using a Likert scale (1–5) resulted in average scores of 4.29 for perceived usefulness, 4.14 for perceived ease of use, and 3.62 for intention to use evaluation constructs. When we specifically looked into the perceived usefulness of the framework for increasing the awareness of cybersecurity standardization, 85.7% of the interviewees responded positively (agree and strongly agree). The same result has been achieved for the perceived usefulness of the framework for supporting SMEs to assess and improve their information security and cybersecurity. These results show that SMEs can benefit from the approach of integrating cybersecurity maturity assessment and standardization in the same framework.

This holistic and integrated approach of a multi-aspect security assessment framework that facilitates both standardization and risk management adopting an SME perspective, is the first of its kind. Given the challenges faced by SMEs with limited resources, we believe our holistic approach can facilitate and consolidate SMEs’ security assessment and standardization efforts.

By conducting maturity assessments based on the ASMAS framework, SMEs can assess their security capabilities, identify areas of improvement and increase adherence to standards. SMEs can also use the high-level requirements and the framework to evaluate any maturity model proposed for their use.

A real-world implementation of the framework should include security functions (e.g., users, roles, authorizations). Future research can implement these security functions, and a more naturalistic evaluation can be carried out in a real organizational context. The findings from the evaluation study show that regarding the utility of the framework, generally positive results have been obtained. This research has shown that security maturity assessment and standardization frameworks such as ASMAS provide a much-needed and feasible foundation for a more secure future for SMEs, guided by established best practices.

Acknowledgments

This work was made possible with funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 740787 (SMESEC). The opinions expressed and arguments employed herein do not necessarily reflect the official views of the funding body.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Horizon 2020 Framework Programme [740787].

Appendix 1:

Screenshots of the Prototype

Home Screen:

On the Home screen, we see an introduction text that explains the standards, frameworks, and taxonomies included in the knowledge base of the prototype. In the navigation menu on the left side of the screen, there are menu items that correspond to the functions implemented in the prototype.

Definitions: SME Category

Using this function, the SME categories and the definitions of the SME categories can be defined. As can be seen here, it is always possible to add more categories or change/delete the existing ones. This is ensured by maintainability by design high-level requirement.

Definitions: Controls

Using this function, controls from standards and frameworks can be defined. Control Source is the name of the standard or the framework.

Definitions: Control Categories

Using this function, control categories from standards and frameworks can be defined. Every control belongs to a control category.

Definitions: Capabilities

Using this function, capabilities per control category can be defined. Capabilities are derived from the implementation guidance presented in standards and frameworks. Capabilities belong to a Level characterized by a letter (i.e., A, B, C, D, E) and an implementation order characterized by Capability Order. The Capability Order is the implementation order of the capabilities associated with a control category.

Maturity Assessment

With this function, organizations can assess their cybersecurity maturity by evaluating the capabilities that are applicable to their organizational profile. The assessment is done by giving ratings to the implementation status of the capabilities. Implementation status can be “Fully Implemented,” “Largely Implemented,” “Partially Implemented” or “Not Implemented.” A capability can also be “Not applicable” to an organization. In the screenshot, we also see the information about the assessment (i.e., Assessment date, assessor, company name, and the category). Several assessments can be performed and saved in the prototype. It is to be noted that SME#3 is a Digitally Dependent SME, therefore, only a subset of capabilities (115 out of 251) are shown in the assessment. In case of a Start-up SME, the number of capabilities that are shown to the user reduces to 79 out of 251.

Maturity Assessment Results

In this screen, organizations can review their previously done assessment’s results. The pie chart illustrates the implementation status percentages. Although the current implementation in the prototype presents the whole assessment in one pie chart, a pie chart per control category can also be prepared.

Appendix 2:

Example Scenarios

Scenario 1: The framework can support current initiatives for adopting standards and frameworks

The Center for Cybersecurity Belgium SME Guide example:

SMEs can see the controls that come from the Center for Cybersecurity Belgium SME Guide associated with their Digital SME profile. The figure below shows 8 controls associated with the Digitally Dependent SME profile.

SMEs can select a risk class and sub-class and see the controls that come from the Center for Cybersecurity Belgium SME Guide. In the figure below, the controls associated with the “Actions of people” class, and “Deliberate” risk sub-class can be seen.

SMEs can select a threat and see the controls that come from the Center for Cybersecurity Belgium SME Guide. In the figure below, the controls associated with the “Failure/Malfunction” threat can be seen.

Scenario 2: The framework can support capability assessment tailored to SME Profile

SMEs can create an assessment for assessing the capabilities associated with their Digital SME profile. The figure below shows an assessment that is created for SME#4 which is a Start-up SME. As can be seen from the figure, the number of capabilities shown in the assessment is 79 out of 251.

SMEs can carry out an assessment of the capabilities associated with their Digital SME profile. The figure below shows the assessment partially completed by SME#4.

SMEs view the results of the assessments that they have carried out. The figure below shows the results of an assessment that has partially been completed by SME#4.

Appendix 3:

Illustration of the Controls and Capabilities for a Control Category

In the prototype, we investigated the standards and frameworks and proposed a unified set of 18 control categories. Each control category (e.g., Access Control) is populated with controls from different standards and frameworks. As expected, there are similar or duplicate controls in each control category. From a capability assessment and improvement point of view, we investigated the controls in the standards and frameworks and proposed a number of capabilities to achieve the objective of the control category. Each capability is associated with a level that provides a progressive approach for improvement planning and an order to provide guidance for implementation planning. The following figure presents the controls from the standards and frameworks for the Access Control control category. There are 24 controls in the Access Control control category.

The investigation of these controls resulted in 17 unique capabilities, as presented in the following figure. Each of the capabilities is associated with a letter representing the capability level (A, B, C, D) and an order (1 to 17). Deriving the capabilities by investigating all the controls in a control category eliminates the duplicate controls and results in a unified set of capabilities to achieve the objectives of the control category.