The impact of enterprise architecture management on information systems architecture complexity

ABSTRACT

Significant investments in information systems (IS) over the past decades have led to increasingly complex IS architectures in organisations, which are difficult to understand, operate, and maintain. We investigate this development and associated challenges through a conceptual model that distinguishes four constituent elements of IS architecture complexity by differentiating technological from organisational aspects and structural from dynamic aspects. Building on this conceptualisation, we hypothesise relations between these four IS architecture complexity constructs and investigate their impact on architectural outcomes (i.e., efficiency, flexibility, transparency, and predictability). Using survey data from 249 IS managers, we test our model through a partial least squares (PLS) approach to structural equation modelling (SEM). We find that organisational complexity drives technological complexity and that structural complexity drives dynamic complexity. We also demonstrate that increasing IS architecture complexity has a significant negative impact on efficiency, flexibility, transparency, and predictability. Finally, we show that enterprise architecture management (EAM) helps to offset these negative effects by acting as a moderator in the relation between organisational and technological IS architecture complexity. Thus, organisations without adequate EAM are likely to face large increases in technological complexity due to increasing organisational complexity, whereas organisations with adequate EAM exhibit no such relation.

KEYWORDS:

• Information systems architecture
• complexity
• enterprise architecture management

1. Introduction

Technological advances over the last decade have led to the development of large-scale interconnected information systems (IS) architectures, upon which organisations rely to conduct their daily business operations (Henfridsson & Bygstad, 2013; Legner et al., 2017). Recent implications of IS, such as sophisticated digital service and product offerings have further fuelled this growth through significant IS investments (McKelvey et al., 2016). Examples range from public electronic health records (S. Hansen & Baroody, 2019) and sophisticated cross-organisational finance platforms (Gomber et al., 2018) to specific retailers that open up new digital sales and distribution channels (R. Hansen & Kien, 2015). We investigate this development and associated challenges from an architectural perspective that considers the complete ensemble of all IS components in an organisation. Consequently, the term IS architecture refers to the entire set of all fundamental IS components and their interdependencies, as well as organisational processes and management efforts to keep local IS investments in line with long-term, enterprise-wide objectives (Haki et al., 2020; Zachman, 1987).

On the one hand, the increasing prevalence of IS has been theorised as a competitive advantage (Mata et al., 1995) and large-scale interconnected IS architectures are commonly associated with productivity increases (Melville et al., 2004). On the other hand, the inherent complexity of such rapidly growing and constantly changing IS architectures also causes issues in their development and maintenance (Dwivedi et al., 2014; Haki et al., 2020). For example, IS components generally do not act in isolation, but are interdependent with other IS components (Bernus & Schmidt, 2006). Therefore, any changes (e.g., in reaction to extended customer interactions or new business requirements) to a single IS component may have unintended effects on multiple related IS components (Mocker, 2009). Since IS development is usually carried out simultaneously in several different IS projects, this leads to potentially inconsistent or redundant applications, software systems, and IT infrastructure components (Hanseth & Lyytinen, 2010). In effect, increasing complexity causes the overall IS architecture to become difficult to maintain and organisations struggle to flexibly respond to required or desired changes (Schmidt & Buxmann, 2011).

In response to these challenges, researchers have investigated how to better restrain and control the development of IS architectures (Cram et al., 2016; Wiener et al., 2016). In practice, large organisations commonly employ enterprise architecture management (EAM), referring to enterprise-wide IS standardisation and harmonisation efforts to avoid unnecessary redundancies and inconsistencies (Aier & Winter, 2009; Ross et al., 2006). EAM activities thereby allow to align local short-term, project-related IS investments with long-term, organisation-wide objectives (Sidorova & Kappelman, 2011). In the following, we use the term architectural outcomes to refer to these long-term, organisation-wide EAM objectives, comprising efficiency, i.e., the ability to provide all necessary IS capabilities with minimal resources (Lange et al., 2016; Schmidt & Buxmann, 2011), flexibility, i.e., the ability to quickly adapt an organisation’s IS to changing conditions or objectives (Amarilli et al., 2016; Li & Madnick, 2015), transparency, i.e., the ability to understand how an organisation’s IS operate (Attewell, 1992), and predictability, i.e., the ability to predict the effects of changes on IS (Geraldi, 2009; Renn et al., 2011).

Conceptually, our research builds on the work of Xia and Lee (2005) on information systems development project (ISDP) complexity, who distinguish structural and dynamic, as well as technological and organisational aspects. This distinction provides a useful foundation, which, in line with the EAM perspective, goes beyond a purely technical view on IS architecture complexity to also include organisational aspects.

In this context, the purpose of the paper at hand is threefold. First, we transfer Xia and Lee's (2005) model to IS architecture and extend it by articulating the relations between the different building blocks of IS architecture complexity. Second, we analyse the effect of IS architecture complexity on architectural outcomes (i.e., efficiency, flexibility, transparency, and predictability). Third, we systematically specify the role of EAM in moderating the relation between IS architecture complexity and architectural outcomes. Overall, we pose the following research question:

RQ: What is the relation between IS architecture complexity and architectural outcomes, and how is this relation affected by EAM?

Methodologically, we first conduct a series of sequential focus group sessions to hypothesise relations between the identified constructs, based on the existing conceptualisation of Xia and Lee (2005), and to develop scale items for a quantitative survey instrument. Subsequently, we collect 249 survey responses from IS managers to test our hypotheses through a partial least squares (PLS) approach to structural equation modelling (SEM; Jr., J. Hair et al., 2014).

Our results clarify the links between the organisational and technological as well as the structural and dynamic aspects of IS architecture complexity. Furthermore, we use this conceptualisation to analyse the impact of IS architecture complexity on related architectural outcomes, which complements extant empirical studies (e.g., Cong & Romero, 2013; Schilling et al., 2017). Finally, we explicate the impact of EAM on the relation between IS architecture complexity and architectural outcomes by demonstrating a highly significant moderation effect of EAM on the relation between the organisational and technological components IS architecture complexity. Specifically, we find that EAM helps to limit the extent to which increases in structural organisational complexity lead to increases in structural technological complexity, thereby reducing the overall negative effects (in terms of architectural outcomes) of increasing IS architecture complexity in large organisations.

2. Background and theoretical development of hypotheses

IS architecture complexity is a multifaceted construct, which has been reflected in a number of research streams from both theoretical and empirical perspectives. Stated concisely, the theoretical research stream acknowledges complexity as an inherent property of many modern IS and discusses how theoretical perspectives, such as complex adaptive systems (CAS), can be applied to better understand this complexity (Benbya & McKelvey, 2006; Haki et al., 2020; Merali, 2006; Sharma & Yetton, 2007). Furthermore, the ever-increasing need to address practical issues arising from increasingly complex IS architecture in large organisations has led to another stream of empirical research that investigates specific challenges. Empirical studies include research on application portfolio complexity (Aleatrati Khosroshahi et al., 2016; Mocker, 2009), enterprise architecture complexity (Lakhrouit & Baïna, 2016; Schütz et al., 2013), IS program complexity (Piccinini et al., 2014), and dynamic effects of IS architecture complexity (Schilling et al., 2017). We aim to consolidate these insights, by considering both technological and organisational aspects of IS architecture complexity, while simultaneously evaluating the potential of EAM to reduce technological complexity.

Integrating this corpus of extant research requires a high-level organising frame of reference that allows to conceptualise, develop, and differentiate key aspects of IS architecture complexity. We opt for the framework of Xia and Lee (2005), as it is empirically well-grounded and sufficiently general to be relevant to our context. Regarding the latter point, we note, however, that this framework was originally developed to measure information systems development project (ISDP) complexity, a construct that is different from our research context (i.e., single IS development projects vs enterprise-wide architectural considerations). Thus, construct development and validation requires a careful adaptation procedure to ensure that the phrasing of related measurement items matches the IS architecture context (MacKenzie et al., 2011), for which we collect foundational qualitative data from focus groups in the first phase of our research design.

Xia and Lee (2005) distinguish four different elements of ISDP complexity along two dimensions (see, Figure 1). First, structural complexity (capturing the size, multiplicity, diversity and interdependencies of related components) is differentiated from dynamic complexity (capturing the rate and pattern of changes). Second, technological complexity (complexity in the technological setting, covering applications, users, interfaces, and technologies, as well as related interdependencies and coordination activities) is differentiated from organisational complexity (complexity in the organisational setting, covering business processes, workflows, organisational roles and structures, as well as related interdependencies and coordination activities). Combining these two dimensions results in four components of IS architecture complexity (Xia & Lee, 2005, pp. 55–56): Structural organisational complexity (SORG), Dynamic organisational complexity (DORG), Structural technological (IT) complexity (SIT), and Dynamic technological (IT) complexity (DIT).

Figure 1. Conceptualisation of IS architecture complexity, based on .Xia and Lee (2005)

In the following, we elaborate on these distinctions in the context of IS architecture complexity. We also discuss the role and impact of EAM on IS architecture complexity as well as architectural outcomes that are impeded by high levels of IS architecture complexity.

2.1. Structural and dynamic aspects of IS architecture complexity

We distinguish between complexity arising from the structural setup of an IS architecture and complexity that is due to dynamic aspects (Schilling et al., 2017; Schneider et al., 2014; Xia & Lee, 2005).

2.1.1. Structural complexity

Structural complexity refers to complexity that can be evaluated through structural properties of the IS architecture, essentially capturing its scope, size, and variety, as well as the interdependencies among its constituent components (Xia & Lee, 2005). All these aspects (scope/size, variety, and interdependencies) in combination lead to structurally complex IS architectures.

Regarding the scope and size of IS architectures, structurally complex systems generally involve a large number of interacting components (Benbya & McKelvey, 2006). Thus, related studies on IS architecture complexity commonly include measures that capture the size and scope of the entire set of IS in organisations (Aleatrati Khosroshahi et al., 2016; Mocker, 2009; Piccinini et al., 2014; Xia & Lee, 2005).

However, a large set of unconnected IS components alone does not imply a structurally complex IS architecture. Instead, structural complexity of IS architecture arises from multiple interdependencies among a large set of diverse, individual components, which create a tightly interrelated and entangled ensemble (Benbya & McKelvey, 2006; Mocker, 2009; Schneider et al., 2014). Therefore, it is necessary to evaluate interactions among subgroups of components (Tait & Vessey, 1988), the overall level of integration within the constituent components of IS architecture (Aleatrati Khosroshahi et al., 2016; Ribbers & Schoo, 2002), and the alignment between the different technical and organisational components (Wood, 1986). For example, Mocker (2009) investigates interfaces between IS components and functional overlaps to measure interdependency-related application portfolio complexity.

Finally, the structural complexity of IS architecture is compounded by the diversity of its constituent components. In this context, diversity is a multidimensional construct, comprising variety (i.e., a large number of different component types), separation (i.e., different component types are clearly distinguishable), and disparity (i.e., component types are evenly distributed; Baccarini, 1996; Ribbers & Schoo, 2002).

2.1.2. Dynamic complexity

Dynamic complexity captures complexity due to the temporal dynamics of the IS architecture. Frequent technological and organisational changes require that organisations continuously adapt their IS architecture, often causing unintended consequences through complex interactions over time (Beese et al., 2015; Haki et al., 2020; Schilling et al., 2017). Therefore, studies on IS architecture complexity commonly include constructs that measure dynamism (referring to the overall rate of change, i.e., how often something changes; Meyer & Curley, 1991) and variability (referring to the extent of changes, i.e., how large changes are; Ribbers & Schoo, 2002). Additionally, several studies emphasise the role of uncertainty, instability, unpredictability (Gerow et al., 2014), and ambiguity (McKeen et al., 1994) during IS development.

2.1.3. Relation between structural and dynamic complexity

In their research on ISDP complexity, Xia and Lee (2005) find structural and dynamic complexity to be distinct yet related constructs (cf., Table 5, Xia & Lee, 2005, p. 70). If an organisation is structurally complex, it is very difficult to predict and control the dynamic effects of changes over time in IS development (Xia & Lee, 2005). Similarly, we expect the structural complexity of IS architectures to contribute to its dynamic complexity, since it becomes increasingly difficult to adequately prepare for and react to unforeseen organisational changes or shifts in technology trends (Gerow et al., 2014). We therefore hypothesise a relation between static and dynamic complexity in the context of IS architecture complexity: if the static structure of an IS architecture is already complex, this will impact the complexity of its dynamic behaviour over time:

H1. Structural complexity is positively associated with dynamic complexity.

Table 1. Overview of focus group workshops.

Date	Participants	Goal
June 2014	16	Develop an understanding of IS architecture complexity and its role in organisations.
October 2014	13	Identify drivers of complexity, as well as common goals of complexity management.
February 2015	13	Discuss relations between the constructs (i.e., hypotheses).
May 2015	8	Validate a preliminary version of the conceptual model and the questionnaire.

Table 2. Demographics of survey respondents (249 total responses).

Industry (233 responses)	Percent			Company size (237 responses)	Percent
Utilities	6.4 %			<50	2.9 %
Financial services	30.0 %			50–249	8.0 %
Health care	3.4 %			250–999	10.2 %
Retail	4.7 %			1000–4999	27.8 %
Information and communication	13.7 %			>5000	51.1 %
Government	5.2 %			Length of employment (212 responses)
Manufacturing and processing	8.6 %			< 2 years	5.7 %
Insurance	17.2 %			2–5 years	27.4 %
Transport and logistics	6.4 %			6–10 years	25.0 %
Others	4.3 %			> 10 years	42.0 %

Table 3. Overview of constructs and scales (R² values of the final model).

Construct	No. of items	Mean	Standard deviation	Composite reliability	Cronbach’s α	AVE	R^2
SORG	7	4.022	0.889	0.810	0.734	0.383	-
DORG	5	3.089	0.967	0.793	0.677	0.440	0.270
SIT	5	3.529	0.971	0.851	0.780	0.538	0.370
DIT	4	3.254	1.018	0.861	0.785	0.608	0.595
EAM	5	3.168	1.021	0.850	0.788	0.534	-
AO (2^nd order)	15	2.974	1.040	0.908	0.889	0.418	0.309
Efficiency	3	2.809	0.876	0.811	0.654	0.590	0.553
Flexibility	4	3.079	0.950	0.867	0.794	0.623	0.730
Transparency	4	2.827	0.937	0.869	0.798	0.624	0.767
Predictability	3	3.192	0.895	0.842	0.726	0.641	0.618

Table 4. Total effects in the final model (***: p < 0.01).

	DORG	SIT	DIT	AO	EFF	FLEX	TRSP	PRED
SORG	0.520***	0.346***	0.338***	−0.204***	−0.152***	−0.174***	−0.179***	−0.160***
DORG		0.281***	0.457***	−0.208***	−0.155***	−0.178***	−0.182***	−0.163***
EAM		−0.402***	−0.410***	0.241***	0.179***	0.206***	0.211***	0.189***
SIT			0.502***	−0.480***	−0.357***	−0.410***	−0.421***	−0.378***
DIT				−0.231***	−0.172***	−0.197***	−0.202***	−0.181***
AO					0.744***	0.854***	0.876***	0.786***

Table 5. Overview of hypotheses testing.

ID	Statement	Related paths in Figure 5	Result
H1	Structural complexity is positively associated with dynamic complexity.	SORG→ DORG: 0.52*** SIT→ DIT:0.50*** SORG→ DIT:(not significant)	Mostly supported^a
H2	Organisational complexity is positively associated with technological complexity.	SORG→ SIT:0.20*** DORG→ DIT:0.32*** SORG→ DIT:(not significant) DORG→ SIT:0.28***	Mostly supported^a
H3	EAM moderates the relations between organisational complexity and technological complexity.	EAM→ SIT:-0.40*** EAM→ DIT:-0.21*** EAM×SORG → SIT: −0.21***	Supported
H4	Technological complexity is negatively associated with architectural outcomes.	SIT→ AO:-0.37*** DIT→ AO:-0.23***	Supported

^aExcept for the insignificant SORG→DIT relation

2.2. Organisational and technological aspects of IS architecture complexity

IS architectures are recognised as complex socio-technical systems, in which organisational components interact with technological components to process information (Alter, 2002; Haki et al., 2020; Lyytinen & Newman, 2008). Consequently, we distinguish organisational complexity from technological complexity in our conceptualisation (Lyytinen & Newman, 2008; Xia & Lee, 2005).

2.2.1. Technological complexity

Technological complexity refers to the complexity of the technological setting (Xia & Lee, 2005). This includes, for example, technology platforms (Hanseth & Ciborra, 2007), system design and integration techniques (France & Rumpe, 2007), computing languages (Meyer & Curley, 1991), and development methodologies (McKeen et al., 1994; Meyer & Curley, 1991). Studies on IS architecture complexity often also consider the age of technological IS components (Mocker, 2009), arguing that complexity is primarily an issue for outdated IS architectures with many legacy systems and obsolete components (Beetz & Kolbe, 2011; Kroenke et al., 2013).

2.2.2. Organisational complexity

Organisational complexity captures the complexity of the organisational setting (Xia & Lee, 2005). Since the technological components of an IS architecture are intractably linked to the surrounding organisational setting (Alter, 2002), the complexity of this organisational setting is thus reflected in the IS architecture itself. Organisational components that contribute to IS architecture complexity include, for example, organisational hierarchies and team compositions (Barki et al., 2001; Bosch-Rekveldt et al., 2011), business processes and workflows (Scott & Vessey, 2002), and the intricacy of IT supported business tasks (Petter et al., 2008).

2.2.3. Relation between organisational and technological complexity

In large organisations, IS managers and architects are continuously trying to align organisational and technological objectives (Brosius et al., 2018; Haki & Legner, 2013). In practice, this alignment usually takes place by making technological changes in reaction to organisational changes (Ramasubbu et al., 2014; Winter & Schelp, 2008). That is, the organisation first agrees on a target business architecture (e.g., new business processes, workflows, and organisational structures) and then the technological aspects of the IS architecture (e.g., new IT infrastructure and tools) are adapted adequately support these organisational requirements (Ross et al., 2006). This relation is further backed by empirical evidence that confirms Conway’s law (Conway, 1968) to still hold true in many modern organisations (Colfer & Baldwin, 2016), essentially arguing that the IT artefacts developed by an organisation tend to mirror the structure of the organisation itself (Constantinides et al., 2018). Consequently, we hypothesise a positive association between organisational complexity and technological complexity.

H2. Organisational complexity is positively associated with technological complexity.

2.3. The role of enterprise architecture management

In response to increasing IS architecture complexity, many organisations develop and maintain architectural plans, which reflect the organisation’s business and IT landscape in its current and targeted future states (Lange et al., 2016; Niemann, 2006). EAM, referring to associated management activities, aims to address associated challenges of IS architecture complexity by providing a holistic view on an organisation’s IS architecture (Ross et al., 2006; Schmidt & Buxmann, 2011) and by purposefully guiding its development towards enterprise-wide objectives (Aier et al., 2011; Boh & Yellin, 2006). In practice, EAM is commonly implemented through the aforementioned architecture plans (Lange et al., 2016) and architecture principles (Greefhorst & Proper, 2011), as well as related organisational roles and responsibilities for monitoring and enforcing these plans and principles through compliance assessments (Boh & Yellin, 2006; Cram et al., 2016; Foorthuis, 2012). More recently, there have also been increasing efforts to establish a more agile approach to EAM, aiming at a more direct interaction with all involved stakeholders while continuously revising and communicating the central architectural artefacts (Buckl et al., 2011; Schilling et al., 2018; Winter, 2014).

While acknowledging research that highlights adoption issues in EAM initiatives (Ross & Quaadgras, 2012; Simon et al., 2014), it is generally expected that a higher level of EAM maturity in an organisation will lead to improved architectural outcomes (Lange et al., 2016; Ross, 2003; Simonsson et al., 2010, 2008). Consequently, when studying the relation between organisational complexity and technological complexity, we expect that the observed effects will depend on the presence of a mature EAM function with well-established roles, responsibilities, and processes, and which has a high reach and impact in the overall organisation (Boh & Yellin, 2006; Lange et al., 2016; Schilling et al., 2018). Therefore, we hypothesise

H3. EAM moderates the relations between organisational complexity and technological complexity.

2.4. Architectural outcomes

Ultimately, we are interested in understanding the relative impact of IS architecture complexity in terms of architectural outcomes (Lange et al., 2016; Schmidt & Buxmann, 2011), referring to desirable, organisation-wide objectives, which may be negatively impacted by high levels of IS architecture complexity. Several empirical studies discuss the effects of IS architecture complexity and highlight negative consequences, such as increased coordination efforts, higher failure rates of large implementation projects, and diminished flexibility (e.g., Beetz & Kolbe, 2011; Hanseth & Bygstad, 2012; Widjaja et al., 2012). EAM aims to prevent or mitigate these negative consequences by working towards desirable architectural outcomes, resulting from a systematically governed IS development in line with long-term, enterprise-wide objectives.

Common architectural outcomes include efficiency, flexibility, transparency, and predictability. Efficiency refers to the ability of the organisation to only use minimal resources while still ensuring that all necessary IS capabilities are provided with sufficient quality (Lange et al., 2016; Schmidt & Buxmann, 2011). Flexibility captures how quickly an organisation can adapt their IS architecture in reaction to novel or changing objectives and conditions (Amarilli et al., 2016; Li & Madnick, 2015). Transparency relates to the ability to track and understand the detailed operation of the organisation’s IS architecture (Attewell, 1992). Finally, predictability refers to the organisation’s ability to predict the overall impact of changes and adjustments made to the IS architecture (Geraldi, 2009; Renn et al., 2011).

In general, evidence points towards a negative relation between the technological complexity of IS architecture and architectural outcomes, i.e., for a group of otherwise similar organisations, those with a technologically more complex IS architecture are expected to score worse in terms of architectural outcomes. Both Mocker (2009) and Aleatrati Khosroshahi et al. (2016) confirm a negative correlation between several complexity indicators (e.g., a large amount of application interfaces or redundant IT applications) and architectural outcomes, demonstrating that more complex IS architectures tend to require higher operation and maintenance costs. Furthermore, high levels of IS architecture complexity prevent people from recognising optimal ways to adapt IS to cope with new requirements, thereby decreasing transparency (Arteta & Giachetti, 2004; Beese et al., 2015). This also reduces the capacity of the IT function to adequately adapt to changing demands and opportunities, leading to low levels of flexibility (Tiwana & Konsynski, 2010). Similarly, dynamic complexity has been linked to increasing difficulties to predict IS behaviour (Xue et al., 2011). Overall, we thus hypothesise

H4. Technological complexity is negatively associated with architectural outcomes.

Figure 2 depicts the overall research model and hypotheses.

Figure 2. Research model and hypotheses.

3. Research method

For testing our theoretically derived research model (see, Figure 2), we first developed a survey by adapting existing scales to the context of IS architecture complexity (survey development), which was then used to quantitatively test our hypotheses (quantitative evaluation).

Survey development: We followed the recommendations of MacKenzie et al. (2011) for the measurement adaptation and development process. Building on our theoretical foundation, we first conducted a sequential series of focus group workshops to support the conception of a questionnaire (Freitas et al., 1998). We opted for focus groups as a means for collecting data from multiple participants, since this allowed us to evaluate the relevance of theoretically derived constructs from multiple perspectives and to discuss subtle nuances in expression and meaning in the formulation of specific measurement items (Freitas et al., 1998; Stewart et al., 2007).

Specifically, we conducted a series of four two-day focus group workshops in 2014 and 2015 (see, Table 1) with senior enterprise architects and IS managers from ten large European companies from the banking, insurance, logistics, and utilities sectors. We selected participants based on their knowledge, their experience within the firm, and their ability to contribute to the topic. Each workshop was prepared and moderated by two of the authors, following the guidelines of Stewart et al. (2007), while two other researchers took notes for subsequent analysis. In between the workshops, we evaluated these notes and additional material (e.g., slide-sets, memos, and meeting notes from participating companies). During each workshop, participants were grouped into three smaller subgroups (two for the last workshop) for discussions, taking care to specifically grouped participants from different industries and different companies together to ensure a more general view on the topic.

In general, the moderators first introduced the topic during each workshop, but then had almost no involvement during the actual sub-group discussions in order to avoid introducing any researcher bias. The aim of the first workshop was to develop a mutual understanding of IS architecture complexity in large organisations. Based on this understanding, participants then discussed their efforts to better manage IS architecture complexity and associated challenges during the second workshop. In the third workshop, participants grouped and positioned the previously identified constructs in relation to each other and discussed interdependencies. The final workshop was used to review and validate a preliminary version of the conceptual model, including a discussion of the specific formulations in the questionnaire (Freitas et al., 1998). This was the only focus group workshop during which we initially introduced a concrete artefact (the questionnaire constructs and formulations) into the discussion and more strongly moderated the discussion to ensure that all items are reviewed (Freitas et al., 1998).

Following MacKenzie et al. (2011), we then assessed the content validity of the items and the resultant measurement scales through a sorting exercise and a pilot test with twelve IS researchers. For the sorting procedure, participants were asked to group the proposed items into conceptually matching categories or assign them to a “not fitting” pile. Furthermore, participants should point out any potential issues with the items (e.g., ambiguous or badly phrased) or the constructs (e.g., missing aspects). A combined analysis of the focus group data, the results from the sorting procedure, and the statistics from the pilot test then led to the final set of measurement items that were included in the survey (see Appendix A).

Quantitative evaluation: We then collected a first data set from 106 respondents via a paper-based survey handed out to 123 participants of a European IS practitioner conference in July 2015 (86% response rate), using a five point Likert scale [(1) Strongly disagree; (2) Disagree; (3) Neither agree nor disagree; (4) Agree; (5) Strongly agree] to measure participant agreement with several statements about complexity, EAM, and architectural outcomes in their organisations (see Appendix A). At this conference, IT managers and enterprise architects, mostly from large European companies, presented and discussed current EAM-related developments in their organisations. As suggested by Cycyota and Harrison (2002), the research project was introduced by a well-known and respected community member, resulting in a rather high response rate (Sivo et al., 2006). The survey was handed out after a break between two sessions when the participants entered the conference room, thereby ensuring that each participant was approached at most once. Based on this data set, an initial estimate of predictive power (following Faul et al., 2009) indicated that roughly 240 responses in total were required in order to be statistically significant at the p < .01 level. Thus, we sent an online survey to an additional 571 contacts and collected data from an additional 143 respondents (25% response rate) during July and August 2015, totalling 249 responses from both the paper-based survey and the online survey. For both surveys, we took care that only relevant respondents – experienced IS managers of large organisations – were targeted.

Survey data was analysed according to the PLS-SEM guidelines of Gefen et al. (2011) and Ringle et al. (2012). We chose PLS-SEM over covariance-based SEM approaches to reduce the risk of overfitting during analysis (Gefen et al., 2011). The tests were performed using SmartPLS version 3.2.9. (Ringle et al., 2015). Missing values were handled through mean replacement and bootstrapping was conducted with a sample size of 5000 to assess path estimate significance (Jr., J. Hair et al., 2014).

4. Construct operationalisation and survey development

In the following we report on the results of the survey development phase and discuss the specific formulations that are used in our survey. See Table 7 in Appendix A for the precise wordings of all items, their loadings (“Ld”.) on the respective scale, and the associated t-values; Section 5 subsequently describes the quantitative data that is used to calculate these values.

Structural organisational complexity (SORG): Discussions in the focus groups revealed that structural organisational complexity is mainly driven by the necessity to adapt a large set of interdependent organisational structures to a quickly and continuously changing competitive environment (Winter & Schelp, 2008). Our operationalisation of structural organisational IS architecture complexity therefore comprises items that capture the need to support many diverse and interrelated business tasks as well as related workflows and functionalities (SORG1, SORG2, SORG4, SORG5, SORG7). This is particularly difficult for organisations with many cross-departmental tasks and projects (SORG6), which may lead to inconsistencies and interdependencies that are not easy to untangle. Furthermore, participants highlighted the structural organisational complexity that is introduced through highly diverse user requirements, ranging from differences in expertise and terminology to different understandings of business task requirements (SORG3).

Dynamic organisational complexity (DORG): Focus group participants generally agreed with the conceptualisation of dynamic complexity as “rate and pattern of changes” (Xia & Lee, 2005, p. 56) in the structural setting. Therefore, the operationalisation of dynamic organisational complexity targets the rate and pattern of changes in business requirements (DORG1), business tasks (DORG2), employees’ roles and positions (DORG3), and related workflows and processes (DORG5). Furthermore, participants emphasised the organisational complexity that accompanies functional changes in employed technologies (DORG4), since major system updates often occur through complex organisational processes.

Structural technological (IT) complexity (SIT): In line with related literature (e.g., Aleatrati Khosroshahi et al., 2016; Mocker, 2009), focus group participants affirmed that structural technological complexity results from the number and diversity of technological components (SIT5) as well as their interrelations (SIT4), leading to difficulties with coordination and integration (SIT2). Additionally, many participants voiced particular difficulties resulting from large numbers of system users (SIT1), leading to structurally very complex user role and access management systems that need to meet the users’ requirements while simultaneously complying with extant data protection regulations (SIT3).

Dynamic technological (IT) complexity (DIT): Like its organisational counterpart, the discussion on dynamic technological complexity emphasised the rate and pattern of changes in the individual technological components (DIT3) as well as in the overall IT landscape (DIT4). Moreover, focus group participants highlighted the complexity introduced by uncertainties and unclarities (e.g., through lacking documentation) in this setting, which prevents organisations from getting a clear understanding of current developments. Thus, our conceptualisation of dynamic technological complexity also targets uncertainties in the development of the IT landscape (DIT1) as well as unclear relations between different system components (DIT2).

Enterprise architecture management (EAM): In agreement with empirical research on the use of EAM in organisations (Lange et al., 2016; Niemi & Pekkola, 2017; Winter & Fischer, 2007), discussions focussed on classical EAM artefacts such as principles (EAM2), processes (EAM3), and guidelines (EAM4). Focus group participants also confirmed classical discussions on EAM (Boh & Yellin, 2006) by highlighting the importance of an adequate assignment of roles and responsibilities (EAM1), such as allocating decision rights to committees that reach beyond individual departments and projects (EAM5).

Architectural outcomes (AO): In line with Schmidt and Buxmann (2011), focus group participants clearly stated that the main targeted architectural outcomes are to ensure a sustainable development of the IS architecture by increasing efficiency while simultaneously providing sufficient flexibility. Additionally, the use of EAM in large organisations is also increasingly ratified with transparency and predictability, which are compulsory to meet regulatory requirements and to manage and reduce organisational risks. Discussions concluded that these are four distinct but related constructs (efficiency, flexibility, transparency, and predictability), which, in combination, reflect the most important architectural outcomes. Consequently, we operationalise architectural outcomes as a reflective second-order construct comprising four sub-constructs.

Efficiency (EFF): Conversations in the focus group sessions confirmed that efficiency should essentially capture the ability to provide a high level of performance for comparatively low cost (EFF1), which fundamentally includes the cost-efficient operation and development of IS (EFF2). Moreover, participants agreed that this is commonly achieved by avoiding unnecessary redundancies (EFF3), in line with related empirical studies (e.g., Mocker, 2009).

Flexibility (FLEX): Discussants argued that flexibility primarily refers to an organisation’s ability to react with speed and dexterity to changes, by conducting necessary adjustments in time and quality (FLEX1). This ability is directly related to the potential of the organisation to anticipate and react to foreseeable changes in the future, such as new technological trends or major shifts in the organisation’s business environment (FLEX2). Furthermore, organisations also require effective processes that quickly deal with urgent unforeseen issues, for example, by being able to quickly deploy critical updates (FLEX3). Finally, focus group participants emphasised the importance of being able to quickly adopt new ideas and technologies in their target architectures (FLEX4).

Transparency (TRSP): Deliberations in the focus group sessions showed that transparency, in the context of IS architecture, is on the one hand related to providing transparency to end users (TRSP1) as well as developers and IT managers. This can, for example, manifest in an IT landscape that is easy to explain and communicate (TRSP2), which makes its use and operation quickly accessible to new members of the organisation (TRSP3). On the other hand, transparency is also related to the organisation’s ability to explain unexpected behaviour (TRSP4) to both internal and external stakeholders, such as regulatory agencies.

Predictability (PRED): Finally, focus group participants agreed that predictability should capture both the organisation’s ability to reliably estimate the costs and effects of changes (PRED1) and the ability to reliably maintain and operate the overall IS architecture. Thereby, reliable operation on the one hand refers to few failures and critical errors (PRED2) and on the other hand to few major changes that disrupt daily operations in unforeseen ways (PRED3).

5. Results

We now present the results of our quantitative analysis based on PLS-SEM (Gefen et al., 2011).

5.1. Data analysis and descriptive statistics

Initially, we carefully inspected all 249 collected responses from both the online and the paper-based survey and did not find any unusual response patterns or outliers. Furthermore, at most three items were left unanswered by any given respondent. Thus, we employed the full dataset of 249 responses in our analysis, using mean value replacement to deal with missing values (Gefen et al., 2011).

Table 2 shows the demographics of our collected data (industry, company size, and length of employment). We specifically targeted experienced IS managers in large organisations. Consequently, more than half of our responses come from organisations with more than 5000 employees and two out of three respondents have been with their current company for more than 6 years.

5.2. Measurement model and construct validity tests

Table 3 presents general scale information (number of items, mean, and standard deviation) and statistical quality criteria (composite reliability, Cronbach’s α, average variance extracted (AVE), and R² values) of the final model (see, Figure 3; Gefen et al., 2011). All constructs are measured in reflective mode, since the individual items are indicative of the latent variables, but do not constitute a complete, exhaustive description (Ringle et al., 2012). Similarly, the second order architectural outcomes (AO) construct is of the reflective-reflective type using the indicator reuse approach (Lohmöller, 1989). As recommended by Lohmöller (1989) and Gefen et al. (2011), the constituent first order components have similar numbers of indicators.

Figure 3. Final model with second order constructs.

Regarding construct reliability, the values for the composite reliability (> 0.7) and Cronbach’s α (> 0.6) are within acceptable ranges (Jr., J. Hair et al., 2014). Since the average variance extracted is below 0.5 for the SORG and DORG constructs, we conducted a series of separate factor analyses for all first order constructs (i.e., all constructs except AO) following Gefen and Straub (2005), to ensure unidimensionality. In all cases a single underlying factor was extracted that explains most of the variance and on which the items load evenly. This, in combination with the good values for composite reliability (> 0.7, cf., Jr., J. Hair et al., 2014), provides evidence that the constructs are explained by their indicators rather than by error terms (Gefen & Straub, 2005), and consequently establishes convergent validity. In the final measurement model no indicator has a loading below 0.4 (Jr., J. Hair et al., 2014) and all indicators are highly significant at the 0.01 level (t-value > 2.576), see Appendix A.

Discriminant validity was confirmed using Heterotrait-Monotrait (HTMT) analysis in combination with a cross-loading analysis (see Appendix B). Finally, we tested the predictive relevance of the model with the non-parametric Stone-Geisser test (Jr., J. Hair et al., 2014); also see Appendix B.

5.2.1. Moderation analysis

Based on our theoretical discussion and on our focus group data, we hypothesise (H4) that EAM affects the strength of the relation between organisational complexity (SORG and DORG) and technological complexity (SIT and DIT). We analysed this moderation effect using a product indicator approach with orthogonalisation in SmartPLS, which is suitable for reflective models and recommended for hypothesis testing (Jr., J. Hair et al., 2014).

To avoid misinterpreting interaction effects and confusing simple direct effects with actual moderation effects, we first analysed alternative models, including a baseline model without the moderator variable (Model 1), a model with only direct effects (Model 2), and a model with full moderation effects (Model 3; J. F. Hair et al., 2013; Henseler & Fassott, 2010; Jr., J. Hair et al., 2014). Figure 3 displays these models as well as the final model (Model 4), which we used in subsequent analyses. Figure 3 also shows the path coefficients and significance levels (**: p < 0.05; ***: p < 0.01) for each path, as well as the R² values for all endogenous constructs. Statistically non-significant paths (p > 0.1) are drawn as dashed lines.

Figure 4. Different moderation models.

Looking at Model 1 in Figure 3, we find our hypotheses confirmed in this baseline model, with the single exception of the path from SORG to DIT. Considering the statistically highly significant other paths, the relatively large R² values, and the results from the previous validity tests, this establishes a good starting point for further moderation analysis. Model 2 in Figure 3 then shows that EAM has a strong direct effect on both SIT and DIT, thus requiring us to check all potential moderation effects (i.e., EAM×SORG→SIT, EAM×DORG→SIT, EAM×SORG→DIT, and EAM×DORG→DIT). Model 3 in Figure 3, however, reveals that only the moderation effect EAM×SORG→SIT is significant at the p < 0.01 level, whereas all other moderation effects are not statistically significant (p > 0.1). Furthermore, Model 4 in Figure 3 confirms that this moderation effect (EAM×SORG→SIT) remains significant if the other moderation effects are removed from the model. Consequently, Model 4, including both direct effects (EAM→SIT and EAM→DIT), one moderation effect (EAM×SORG→SIT), and excluding the insignificant direct relation (SORG→DIT, which is fully mediated via the SIT and DORG constructs) is used for subsequent analysis and hypothesis testing.

We then conducted a simple slope analysis for the moderation effect (see, Figure 4) to better understand how the moderator variable EAM affects the relation between SORG and SIT (Aiken et al., 1991). Accordingly, we calculated two additional regressions that represent the relation between SORG and SIT when EAM is below or above average (i.e., when the z-value of EAM is +1 standard deviation or −1 standard deviation). Figure 4 displays these two regressions in addition to the default regression (i.e., EAM at mean), where both axes are scaled to standard deviations of the respective constructs. This reveals a noticeable difference: while there is a clear relation between SORG and SIT on average (slope = 0.198), this relation is significantly more pronounced for low levels of EAM (slope = 0.406) and almost non-existent for high levels of EAM (slope = −0.008). In non-technical terms, this implies that organisations without a mature (in terms of well-established roles, responsibilities and processes) and organisationally well-anchored (in terms of reach and acceptance) EAM function are likely to face large increases in technological complexity as a consequence of increasing organisational complexity. In contrast, organisations with a mature and well-anchored EAM tend to see almost no increase in technological complexity due to increasing organisational complexity.

Figure 5. Simple slope analysis.

5.3. Analysis of the final structural model

Figure 5 shows the final model with second order constructs, including path coefficients (the numbers next to the arrows) and R² values (the numbers at the bottom right of each construct). All relations in Figure 5 are significant at the p < 0.01 (***) level, implying that it is very unlikely to observe these relations by chance. Path coefficients indicate the direct effect that one variable has on another variable; for example, if SORG increases by one standard deviation, then SIT is expected to increase by 0.20 standard deviations. Negative numbers indicate inverse relationships, e.g., if SIT increases, we expect AO to decrease. Finally, the R² values specify the extent to which a certain construct can be explained by the predictor variables (all incoming arrows). Higher R² values correspond to less unexplained variance. For multi-faceted constructs such as AO (Architectural Outcomes, R² = 0.31), we expect that we are only able to explain a fraction of the total variance, since many other factors specific to a single organisation will also influence these constructs.

To ensure that no significant path has been left out in the final model, we compared these results with an analyses of the saturated model (Gerbing & Anderson, 1988); see Appendix D. Next, we performed linear regression analysis tests between connected constructs to ensure that the variance inflation factors (VIF) and Durbin-Watson statistics are within acceptable thresholds (Kutner et al., 2005). Finally, we employed multi-group-analysis (Jr., J. Hair et al., 2014) to confirm that our final model does not differ significantly for different the respondent characteristics from Table 2 within our sample (see Appendix C) as well as to check for common method bias (Podsakoff et al., 2003) and nonresponse bias (Gefen et al., 2011; Sivo et al., 2006) in our survey data.

After confirming the robustness of the final model, we then calculated the total effects (see, Table 4), i.e., the sum of direct and indirect effects between constructs (Jr., J. Hair et al., 2014). The effect sizes in Table 4 thus also reflect additional effects through intermediate constructs, and can therefore be used to investigate relations between constructs that are not directly connected in the final model (cf., Figure 5). Notably, we can see that all IS architecture complexity constructs (SORG, DORG, SIT, DIT) have highly significant negative effects on architectural outcomes (AO), whereas EAM has a highly significant positive effect (column AO in Table 4).

5.3.1. Hypothesis testing

Considering the preceding analysis, we now review our hypotheses. Table 5 provides an overview of the hypotheses along with the related paths in the final model (see, Figure 5).

H1 is mostly supported: there is a positive relation between structural complexity and dynamic complexity. In detail, we find that for organisational complexity there is a significant positive relation between SORG and DORG (0.52***) and similarly for structural complexity there is a significant positive relation between SIT and DIT (0.50***). In contrast, SORG is not directly associated with DIT, but rather constitutes a fully mediated effect. Consequently, this does not mean that there is no effect, but rather that total effect (see, Table 4, SORG→DIT: 0.338***) can be fully explained by the indirect effects through the mediating constructs (i.e., SORG→SIT→DIT and SORG→DORG→DIT).

Similarly, H2 is mostly supported: there is a positive relation between organisational complexity and technological complexity. In detail, we find that for structural complexity there is a significant positive relation between SORG and SIT (0.20***) and similarly for dynamic complexity there is a significant positive relation between DORG and DIT (0.32***). Furthermore, the relation between DORG and SIT is highly significant (0.27***). Similar to our discussion of H1, we note that the relation between SORG and DIT (−0.03) does show a statistically significant total effect (0.311***, see, Table 4).

We find support for H3 and our analysis clarifies the nature of the moderation effect of EAM on the relation between organisational and technical complexity. First, we find a significant direct negative relation between EAM and SIT (−0.40***) as well as EAM and DIT (−0.21***), meaning that adequate EAM reduces technological complexity. Second, we observe a significant moderation effect from EAM on the relation between SORG and SIT (−0.21***). This moderation is confirmed through a simple slope analysis (see, Figure 4), showing that low levels of EAM strengthen the positive relation between SORG and SIT (slope 0.408), whereas high levels of EAM make this relation become insignificant (slope −0.008).

Finally, we find clear support for H4, i.e., there is a negative relation between technological complexity and architectural outcomes. Both the relation between SIT and AO (−0.37***) and the relation between DIT and AO (−0.23***) are statistically significant at the p < 0.01 level.

6. Discussion

This research makes contributions in three areas (see, Table 6): First, we provide a theoretically developed and empirically validated model of IS architecture complexity, primarily building on and extending the work of Xia and Lee (2005). Second, we contribute to the growing body of empirical research that investigates the effects of increasingly complex IS architectures. Third, our analysis reveals and details the moderating effect of EAM on the relation between organisational and technical complexity.

Table 6. Summary of key contributions.

Contribution	Implications for research	Implications for practice
A theoretically developed and empirically validated model of IS architecture complexity	Provide an organising framework that integrates complementary perspectives on IS architecture complexity into a single model that clarifies interrelations.	Enable practitioners to clearly describe and communicate problems and levers for solutions in the context of IS architecture complexity.
An empirical analysis of the effects of IS architecture complexity on architectural outcomes	Complement extant studies by detailing relations between constructs and additionally highlighting the relevance of the organisational and dynamic aspects of IS architecture complexity.	Facilitate the development of suitable complexity management approaches by estimating expected effects.
Clarification of the moderating effect of EAM on the relation between organisational and technological complexity	Bridge research on EAM with research on IS architecture complexity by demonstrating a highly significant moderation effect and explaining its nature.	Support and guide investment decisions for EAM efforts by detailing how EAM contributes to desired architectural outcomes.

6.1. Conceptualising IS architecture complexity

A central part of the theoretical model is the conceptualisation of IS architecture complexity, comprising the different IS architecture complexity constructs (SORG, SIT, DORG, and DIT) as well as their interrelations (see the box labelled “IS Architecture Complexity” in Figure 2). We follow the idea of Xia and Lee (2005) to distinguish organisational from technological complexity and static from dynamic complexity, and we extend the original conceptual model in two ways. First, we change the research context from ISDP complexity to IS architecture complexity. This change of context (ISDP complexity vs IS architecture complexity) is supported by an extensive instrument development phase preceding the quantitative analysis (MacKenzie et al., 2011). Second, we extend the model of Xia and Lee (2005) by hypothesising and testing relations between the different IS architecture complexity constructs. This further complements the original work of Xia and Lee (2005), who find similar statistical quality criteria for the uncorrelated second-order model (i.e., SORG, SIT, DORG, and DIT are all independent constructs) and the fully correlated model (i.e., SORG, SIT, DORG, and DIT are all connected to one another; cf., Table 5 in Xia & Lee, 2005). Our model may thus be considered an extension of Xia and Lee (2005) in the sense that we do not assume that all factors are either completely correlated or completely uncorrelated. Instead, our confirmation of H1 and H2, i.e., the relation between organisational vs technological and structural vs dynamic complexity respectively, explains in more detail how different aspects of IS architecture complexity are related.

Conceptual models, and extensions thereof, make significant contributions to both research and practice. Researchers fundamentally rely upon well-developed and tested conceptual models as organising frameworks, which provide a basis for consistently defining constructs. This allows for a meaningful comparison of the results of different studies and thus enables the cumulative creation of scientific knowledge (Vom Brocke et al., 2009). A robust conceptual model also provides practitioners with a suitable language to describe and communicate problems and potential solutions in the context of IS architecture complexity.

6.2. Effects of IS architecture complexity

Our research builds on and adds to the growing body of knowledge on the effects of IS architecture complexity. In general, the R² value of 0.31 in our final model (see, Figure 5) resembles approximately the values that can be expected from reviewing related survey-based SEM studies. For example, Schilling et al. (2017) use a similar conceptualisation for IS architecture complexity, which in their model leads to a R² value of 0.31 for the IS architecture outcomes construct, comprising flexibility and efficiency (Schilling et al., 2017, see Figure 2, p. 11). Likewise, Schmidt and Buxmann (2011), find that their EAM construct allows them to explain flexibility (R² = 0.40) and efficiency (R² = 0.28). In comparison to both studies, we present a more detailed view on the predictor construct (i.e., the IS architecture complexity part of Figure 2) by detailing the relations between the SORG, DORG, SIT, and DIT constructs, and accordingly explicitly measuring the individual contributions of both SIT (−0.482***) and DIT (−0.234***) on architectural outcomes (see, Table 4). Furthermore, our analysis of total effects in Table 4 also allows to measure the contributions of the preceding constructs on architectural outcomes, i.e., SORG (−0.197***), DORG (−0.211***), and EAM (0.242***).

In comparison with non-survey based quantitative studies on IS architecture complexity (e.g., Aleatrati Khosroshahi et al., 2016; Mocker, 2009) we offer a more complete conceptualisation of the phenomenon. Following Xia and Lee (2005), we also consider dynamic and non-technical aspects of IS architecture complexity. For example, complementary to Aleatrati Khosroshahi et al. (2016) and Mocker (2009), who focus on the structural technological aspects (i.e., SIT in our model), we find that the other DORG, SORG, and DIT constructs also have significant effects (see, Table 4). Thus, our study contributes an extended perspective on IS architecture complexity and related effects on architectural outcomes.

In addition to this theoretical contribution, practitioners may use the estimates from the final model (see, Figure 5 and Table 4) to guide complexity management efforts in their organisation. This helps to develop suitable complexity management approaches, as it facilitates the evaluation of potential consequences of actions.

6.3. The moderating effect of EAM

In their Delphi study on the complexity of IS programs Piccinini et al. (2014, p. 9) conclude:

“[An] interesting overall result of our study is that a new meta-category emerged, i.e., CCD [coordination and control deficiencies], which appears with one component in the final ranking, i.e., ‘unclear or ill-defined program methodology’. This means that if a sound methodology is not in place in an IS program, complexity as perceived by IS program practitioners increases”.

Similarly, early discussions in our focus groups clearly indicated that EAM, or similar coordination and control activities, need to be considered when studying IS architecture complexity, as these are expected to strongly influence the extent to which organisational complexity manifests as technological complexity. Our confirmation of H4 and the total effect of EAM on AO (0.242***, see, Table 4) strongly affirms this expectation. Thus, our work extends previous exploratory studies, such as Piccinini et al. (2014), by clarifying the nature of the moderating effect of EAM on the relation between organisational and technological complexity (see, Figure 3 and Figure 4).

Practitioners may use this insight to justify investment decisions in EAM. At the time of the focus group workshops, all participating companies had launched significant IS architecture complexity management programmes with multi-million-dollar budgets per year. The expected and actual contribution of such programmes is, however, difficult to measure. Our research offers a scientifically tested confirmation that there are noticeable benefits of EAM, offering an effective way to avoid unnecessary technological complexity arising from potentially necessary organisational complexity. Furthermore, we find that the notion of necessary and unnecessary complexity is frequently misconstrued in practice. Practitioner-oriented studies on organisational and technological complexity – including the excellent work of Martin Mocker (Mocker & Boochever, 2020; Mocker & Heck, 2015; Mocker et al., 2016, 2014) – correctly argue that increases in organisational complexity tend to coincide with increases in technological complexity, so that organisations should take great care to (i) avoid unnecessary organisational complexity in general and (ii) try to reduce unnecessary technological complexity that is not corresponding to organisational complexity. Consequently, many complexity reduction initiatives initially focus on relatively simple organisational areas, since they expect these to have a very low level of “necessary organizational complexity”, thereby providing the highest potential to reduce “unnecessary technological complexity”. Our results contradict this idea: While we also confirm a clear positive relation between technological complexity and organisational complexity, we additionally find that the expected impact of EAM to reduce technological complexity becomes higher as the organisational complexity increases. In the most extreme cases – considering only the companies that do EAM really well – we do not find any correlation between organisational complexity and IT complexity; i.e., your organisation may become very complex, but this need not manifest in IT complexity at all. Consequently, it might be beneficial to specifically focus efforts to reduce technological complexity on the most complex areas of the organisation, which may seem counterintuitive at first.

6.4. Generalisability, limitations, and research opportunities

Regarding generalisability, we first note that our data sample mostly covers large European companies and certain industries (e.g., insurance) are overrepresented. We believe, however, that our results will apply to most large organisations that fundamentally rely on IT systems in the execution of their core business processes, since we explicitly test the robustness of the final model via multi-group-analysis against specific methodological biases and against variations in our sample demographics (see Appendix B; Seddon & Scheepers, 2012, 2015). Furthermore, several statistical results in our model are closely in line with other studies (e.g., Aleatrati Khosroshahi et al., 2016; Mocker, 2009; Schilling et al., 2017; Schmidt & Buxmann, 2011) that investigate similar constructs (IS-related complexity) in similar populations (large organisations from common industries), thereby giving additional evidence that the observed relations in our model are likely apply to other large organisations as well (Seddon & Scheepers, 2012). In contrast, we do not expect our results to apply to small organisations (e.g., start-ups, which might not experience complexity in the same way as larger organisations with a long history) and to niche industries, which use fundamentally different IS (e.g., high-security military operations).

Furthermore, unforeseen technological disruptions, such as, for example, fundamentally new ways to build IS architectures in cloud environments could make our results obsolete. One promising avenue for future research therefore lies in investigations that try to evaluate the impact of disruptive technologies on IS architecture complexity and on EAM. A recent informal discussion of our survey results with several practitioners in February 2020 suggested that our findings currently still apply to their organisations. However, they are closely monitoring advances in cloud technologies, artificial intelligence, as well as new DevOps practices as potential solutions to long-standing issues with IS architecture complexity. Newer trends in EAM, such as agile EA and architectural thinking (Buckl et al., 2011; Schilling et al., 2018; Winter, 2014), also warrant further investigations.

Concerning limitations, the combination of focus group data and a quantitative survey-based PLS-SEM analysis enables us to derive interesting results; but it also limits the extent to which we can understand the observed phenomenon in more detail. We therefore point out three important limitations of this research to avoid misinterpretations.

First, the participants in our focus group discussions are rather homogenous (IT affine people from large European companies). Thus, while we took great care to allow for open discussions without pushing participants into specific directions (Freitas et al., 1998), the individual perceptions and group dynamics may lead to focal points in the survey that are not fully representative for other cultures (e.g., Asian or African corporations).

Second, one needs to consider the methodological implications of using PLS-SEM with SmartPLS when analysing complex phenomena that may fundamentally be non-linear. PLS essentially conducts a simultaneous series of linear regressions to minimise an overall error-term in the model. While we find the paths in our model to be statistically significant, these may, in many cases, be only rough approximations of truly non-linear relations. For example, Aleatrati Khosroshahi et al. (2016) observe an exponential relation between IS architecture complexity indicators and outcome measures, based on a large dataset from a single organisation. Consequently, we conducted tests for similar exponential models in R, but did not find a statistically satisfactory relation, most likely due to our analysis being limited by a comparatively (for non-linear analyses) small data set of 249 survey responses.

Third, neither the design of our focus group workshops, nor our survey data is suitable for longitudinal or multi-level analyses. IS architecture complexity is generally assumed to be an emergent phenomenon that arises over time (Beese et al., 2015; Haki et al., 2020) and consequently many of the hypothesised positive or negative relations are expected to differ for different parts of an organisation and in different points in time. Our model does not pick up on such effects but instead averages perceptions across a large set of organisations.

Considering these limitations, another promising avenue for future research are more in-depth qualitative investigations – or quantitative studies with a lot of data – that aim to understand the nature and the precise mechanisms of the statistical relations that we observed. We argue that our analysis demonstrates that, on a high level, there is a significant benefit of more mature EAM, in particular when an organisation’s IT needs to support very complex organisational structures and processes. In reality, however, EAM and IS architecture complexity are very intricate constructs, whose mechanisms need to be understood in detail to be effectively managed in a given organisational context (Haki et al., 2020).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Appendix A:

Survey details

Table 7. List of survey items with loadings (Ld.) and t-values. Items marked with (*) are reverse-coded.

Item-ID		Item	Ld.	t-values
Structural Organisational Complexity	SORG1	The IT systems support many different functionalities.	.654	12.94
	SORG2	Most IT supported tasks are rather extensive and complex.	.745	19.17
	SORG3	The expertise and requirements of system users vary considerably.	.518	7.33
	SORG4	Diverse IT systems are involved in managing and controlling workflows.	.664	10.29
	SORG5	The successful completion of a task is often interdependent with other tasks and processes.	.498	6.15
	SORG6	In your organisation, many interdependencies between organisational units and many cross-departmental tasks and projects exist.	.586	9.63
	SORG7	IT-supported workflows are highly integrated. The successful completion of a work process often depends on processes in other systems.	.629	11.16
Dynamic Organisational Complexity	DORG1	System requirements often are short-lived and change rapidly.	.709	16.39
	DORG2	The number of different tasks that need to be addressed by the IT rises quickly.	.746	21.23
	DORG3	Employees frequently change their roles and positions, and thus also their area of responsibility.	.681	13.35
	DORG4	The employed technologies often still receive functional updates, which frequently lead to large system updates.	.682	14.60
	DORG5*	Most workflows and processes remain stable throughout longer time periods.	.460	5.86
Structural IT Complexity	SIT1	The IT systems support a large number of users.	.572	8.80
	SIT2	Coordination and integration of the different systems is a difficult and not yet adequately solved task.	.820	30.74
	SIT3	Coordination between human users, software and hardware is difficult and often leads to problems.	.795	30.77
	SIT4	Interactions and dependencies between individual system components lead to a complex system landscape and are often the cause of problems.	.723	18.99
	SIT5	The number and variety of systems leads to a complex and hard to comprehend information system landscape.	.731	22.87
Dynamic IT Complexity	DIT1	The development of the IT landscape is characterised by uncertainties, making the impact of individual changes difficult to evaluate.	.796	29.98
	DIT2	The relationships between individual components are unclear, leading to ambiguous results, which are difficult to understand without additional information.	.811	39.75
	DIT3	In the organisation, there exists a large and rapidly changing range of different systems, technologies and requirements.	.720	16.89
	DIT4	The development of the IT landscape is very dynamic, requiring frequent and often difficult to understand adjustments.	.788	25.25
Enterprise Architecture Management	EAM1	Roles and responsibilities are assigned through well-defined processes, which include relevant stakeholders from business and IT.	.743	14.26
	EAM2	The development of the system landscape follows well-defined and established principles.	.806	21.07
	EAM3	IT system requirements are developed through well-defined processes, which include relevant stakeholders from business and IT.	.612	7.87
	EAM4	There are established guidelines on what types of tools and technologies are included in the portfolio of the organisation.	.757	17.86
	EAM5*	Decisions about the use of technologies are made opportunity-driven by individual departments and projects.	.720	16.96
Efficiency	EFF1	The operating and development costs of the IT landscape are low compared to the scope of performance.	.748	15.82
	EFF2	The time and costs required to use, operate and develop the system landscape are comparatively low.	.812	30.40
	EFF3	Redundancies are avoided. There is little overlap between different applications and functional capabilities.	.742	18.73
Flexibility	FLEX1	The IT department is able to react with speed and dexterity. Even larger adjustments are implemented in time and quality.	.853	47.98
	FLEX2	Foreseeable adjustments are well anticipated and can be implemented without much difficulty.	.847	41.00
	FLEX3	The IT department can implement critical updates and bug fixes quickly. There is little time between the decision to change something and the implementation of the change.	.819	35.67
	FLEX4	The company is innovative and technologically advanced. New ideas and technologies are quickly adopted and used in production.	.615	13.42
Transparency	TRSP1	The system landscape is transparent. Processes and results are understandable for the end-users.	.770	23.78
	TRSP2	The development and behaviour of the IT landscape can be explained and communicated easily.	.861	61.80
	TRSP3	Use and operation of the system landscape is easy to learn. New employees become acquainted with the systems quickly and without major problems.	.719	19.51
	TRSP4	The behaviour of the system landscape is comprehensible. The causes of exceptions and errors can be quickly traced.	.803	35.64
Predictability	PRED1	The effects of adjustments and changes are predictable. Costs and expenses for developments can be estimated reliably.	.818	41.29
	PRED2	The IT systems operate reliably. Failures and critical errors occur rarely or not at all.	.836	31.21
	PRED3	The IT system landscape is relatively stable. Major changes occur infrequently and affect only few components.	.745	18.69

Appendix B:

Analysis of discriminant validity and predictive validity

Discriminant validity was tested using Heterotrait-Monotrait (HTMT) analysis (see Table 8) and the results are within the recommended thresholds (< 0.9) of Henseler et al. (2015). One notable exception is the second order AO construct (bold in Table 8), since it shares the same indicators with the underlying lower order constructs (Jr., J. Hair et al., 2014).

Table 8. Heterotrait-monotrait (HTMT) analysis of discriminant validity.

	SORG	DORG	SIT	DIT	EAM	AO	EFF	FLEX	TRSP
DORG	0.673
SIT	0.454	0.536
DIT	0.382	0.707	0.872
EAM	0.229	0.229	0.456	0.471
AO	0.290	0.370	0.627	0.576	0.406
EFF	0.384	0.322	0.509	0.490	0.269	0.988
FLEX	0.143	0.230	0.493	0.417	0.423	1.004	0.770
TRSP	0.267	0.350	0.651	0.564	0.542	1.026	0.740	0.766
PRED	0.284	0.436	0.552	0.655	0.449	0.950	0.608	0.663	0.781

However, a detailed cross loadings analysis shows that each indicator has a stronger correlation with the assigned latent variable than with any other latent variable (see, Table 9), indicating that the model has good discriminant validity. Furthermore, we tested the predictive relevance of the model with the non-parametric Stone-Geisser test by applying a blindfolding procedure with an omission distance of 7 in SmartPLS (Jr., J. Hair et al., 2014). All Q² values (DORG: 0.110, SIT: 0.176, DIT: 0.334, AO: 0.119) are larger than zero, indicating that the model has predictive validity (Götz et al., 2010).

Table 9. Cross loading analysis.

Item-ID	SORG	DORG	SIT	DIT	EAM	EFF	FLEX	TRSP	PRED
SORG1	0.654	0.352	0.189	0.217	−0.047	−0.190	−0.054	−0.156	−0.145
SORG2	0.745	0.482	0.291	0.241	0.045	−0.239	0.008	−0.165	−0.175
SORG3	0.518	0.216	0.174	0.093	−0.087	−0.165	−0.087	−0.130	−0.097
SORG4	0.664	0.357	0.192	0.185	0.009	−0.194	0.011	−0.104	−0.083
SORG5	0.498	0.136	0.177	0.126	0.078	−0.097	−0.081	−0.122	−0.161
SORG6	0.586	0.302	0.197	0.202	0.136	−0.150	0.061	−0.063	−0.074
SORG7	0.629	0.247	0.122	0.118	0.118	−0.138	0.051	−0.062	−0.107
DORG1	0.375	0.709	0.192	0.329	0.003	−0.066	0.020	−0.108	−0.130
DORG2	0.447	0.746	0.296	0.397	0.049	−0.262	−0.145	−0.207	−0.246
DORG3	0.308	0.681	0.323	0.386	−0.070	−0.181	−0.080	−0.109	−0.147
DORG4	0.396	0.682	0.211	0.312	−0.003	−0.077	0.045	−0.098	−0.157
DORG5	0.125	0.460	0.226	0.261	−0.048	−0.124	−0.162	−0.322	−0.297
SIT1	0.483	0.392	0.572	0.353	−0.153	−0.280	−0.184	−0.288	−0.248
SIT2	0.076	0.228	0.820	0.584	−0.421	−0.278	−0.347	−0.433	−0.362
SIT3	0.189	0.210	0.795	0.559	−0.379	−0.287	−0.306	−0.401	−0.301
SIT4	0.330	0.281	0.723	0.450	−0.181	−0.257	−0.294	−0.299	−0.301
SIT5	0.174	0.307	0.731	0.560	−0.259	−0.251	−0.309	−0.455	−0.356
DIT1	0.205	0.389	0.638	0.796	−0.393	−0.144	−0.256	−0.423	−0.387
DIT2	0.083	0.332	0.537	0.811	−0.415	−0.285	−0.371	−0.411	−0.470
DIT3	0.407	0.458	0.435	0.720	−0.177	−0.314	−0.241	−0.293	−0.359
DIT4	0.240	0.440	0.535	0.788	−0.269	−0.180	−0.193	−0.274	−0.354
EAM1	0.118	0.059	−0.210	−0.238	0.743	0.122	0.207	0.270	0.200
EAM2	0.123	0.105	−0.285	−0.263	0.806	0.180	0.316	0.351	0.301
EAM3	0.097	0.028	−0.236	−0.181	0.612	0.203	0.295	0.317	0.255
EAM4	0.104	0.096	−0.234	−0.289	0.757	0.105	0.255	0.386	0.267
EAM5	−0.128	−0.212	−0.391	−0.430	0.720	0.102	0.161	0.255	0.272
EFF1	−0.167	−0.120	−0.205	−0.175	0.136	0.748	0.365	0.329	0.194
EFF2	−0.198	−0.209	−0.342	−0.259	0.140	0.812	0.480	0.467	0.381
EFF3	−0.276	−0.172	−0.283	−0.226	0.152	0.742	0.446	0.431	0.424
FLEX1	−0.044	−0.091	−0.363	−0.283	0.236	0.540	0.853	0.553	0.458
FLEX2	−0.032	−0.148	−0.377	−0.365	0.315	0.476	0.847	0.572	0.503
FLEX3	−0.015	−0.065	−0.281	−0.294	0.262	0.418	0.819	0.461	0.484
FLEX4	0.092	0.062	−0.210	−0.094	0.205	0.332	0.615	0.343	0.230
TRSP1	−0.115	−0.090	−0.442	−0.376	0.479	0.431	0.460	0.770	0.439
TRSP2	−0.080	−0.188	−0.418	−0.387	0.386	0.446	0.501	0.861	0.563
TRSP3	−0.298	−0.262	−0.335	−0.265	0.224	0.438	0.400	0.719	0.401
TRSP4	−0.127	−0.208	−0.440	−0.400	0.252	0.400	0.589	0.803	0.550
PRED1	−0.142	−0.207	−0.428	−0.468	0.352	0.457	0.580	0.632	0.818
PRED2	−0.110	−0.183	−0.307	−0.351	0.274	0.274	0.408	0.446	0.836
PRED3	−0.233	−0.308	−0.269	−0.380	0.209	0.313	0.258	0.367	0.745

Appendix C:

Analysis of control variables and methodological biases

We test our final model (see, Figure 5) against variation of the collected demographic characteristics (company size, industry sector, and employee tenure) in our sample population by using multi-group-analysis (Jr., J. Hair et al., 2014). The idea is to split the original data set into distinct smaller subsets (e.g., one subset with all responses from companies with less than 5000 employees and one subset with all responses from companies with more than 5000 employees). Then, the final PLS-SEM model is calculated for each subset and the output is compared to test for statistically significant differences (i.e., non-overlapping confidence intervals). In our case, we are required to group several smaller categories together (e.g., combining industries with few responses in our sample) to allow for statistically meaningful comparisons. We compare several different subgroups (see the comparisons and p-values in Table 10) and find no significant (p > 0.1) differences. This only implies that there are no significant differences for specific variations within(!) our sample in terms of company size, industry, tenure, and the survey instrument (paper vs online survey). However, certain populations (e.g., very small organisations, Asian/African organisations, or specific niche industries) are still excluded from our analysis.

Table 10. Results (p-values) of the SmartPLS multi-group-analysis.

	Comparison groups (p-values)
Path	Company size (>5000 vs ≤5000 employees)	Industry (Finance vs IT)	Industry (Finance + IT vs all others)	Tenure (>10 years vs ≤10 years)	Survey (Paper vs online)
SORG→DORG	0.225	0.262	0.616	0.138	0.718
SORG→SIT	0.546	0.325	0.660	0.841	0.898
DORG→DIT	0.902	0.517	0.501	0.896	0.243
DORG→SIT	0.459	0.765	0.993	0.372	0.615
EAM→SIT	0.949	0.185	0.432	0.332	0.995
EAM→DIT	0.327	0.286	0.395	0.441	0.125
SIT→AO	0.758	0.359	0.100	0.769	0.774
DIT→AO	0.531	0.158	0.911	0.778	0.965
Moderation	0.797	0.565	0.896	0.900	0.930

Additionally, we argue that this multi-group-analysis (Jr., J. Hair et al., 2014) can be employed to test for inflated estimates due to common method bias (Podsakoff et al., 2003). To this end, we compared the responses from the online and the paper-based survey and found no significant differences (Jr., J. Hair et al., 2014; Rigdon et al., 2010). In combination with the high response rate (88%) of the paper-based survey, this also allows us to rule out nonresponse bias (Gefen et al., 2011; Sivo et al., 2006).

Appendix D:

Saturated model analysis

To ensure that no significant path has been left out in the final model, we compared the results from our final model (see, Table 4 and Figure 5) with the results obtained from the saturated model (Gerbing & Anderson, 1988), in which there are paths among all pairs of latent variables (see, Table 11 of Appendix B). We find that all significant paths in our theoretical model (Figure 2) remain significant in the saturated model (Gefen et al., 2011). Furthermore, we checked that adding paths from the saturated model would not significantly increase the observed effect sizes (Gefen et al., 2011).

Table 11. Path coefficients of the saturated model (***: p < 0.01, *: p < 0.1).

	DORG	SIT	DIT	AO
SORG	0.514***	−0.201***	−0.032	−0.050
DORG		−0.280***	−0.345***	−0.034
SIT			−0.494***	−0.304***
DIT				−0.150*
EAM		−0.398***	−0.207***	−0.242***