On the stress potential of an organisational climate of innovation: a survey study in Germany

ABSTRACT

Technological innovation has become critical for market survival for many companies in the era of digital transformation. However, the organisational perspective to constantly stay up-to-date with technological innovations can become strenuous on an individual level due to the need to constantly adapt to the demands of a permanently changing technological environment. Based on a large-scale online survey of employees in Germany (N = 1,115), we investigated the positive and negative effects of an organisational climate of innovation from a digital stress perspective. Our results indicate that an organisational climate of innovation creates uncertainty in individuals as it supports technological change but can also help to reduce the perception of technological unreliability as it fosters an environment of mutual support. This research, therefore, adds to the idea of a pronounced innovation climate as a double-edged sword and shows that permanently pushing technological innovation should also be viewed in the light of its potential side effects. It follows that innovation initiatives should balance organisational and individual requirements to stay, or become, more competitive, thereby explicitly considering the side effects that too much and/or too fast innovation may have on users.

KEYWORDS:

• Innovation climate
• digital stress
• uncertainty
• unreliability
• user satisfaction
• job satisfaction

1. Introduction

Being able to handle changes has become a key challenge for many companies, with information and communication technologies (ICT) being a major cause of the constant struggle to innovate (Sambamurthy, Bharadwaj, and Grover 2003). Not only do products or services that are provided by companies undergo constant changes, but so do the ICT that are used to support their creation. In this context, it has been argued that an information system ‘that suppresses an organization’s ability to be agile cannot be tolerated as it may lead to the demise of the organization’ (Seo and La Paz 2008, 139). The idea of not only keeping pace, but actively changing the organisational technology landscape to be prepared for the innovation affordances that a rapidly changing business environment demands has become critical (García-Morales, Ruiz-Moreno, and Llorens-Montes 2007).

Companies and their members consequently face an ever-increasing number of technological innovations (i.e. technologies or practices that they use for the first time, Klein, Conn, and Sorra 2001). This was, for example, illustrated by Meyer et al. (2017) who investigated the technology use of a group of heavy ICT users (i.e. software developers). They found that their 20 participants collectively used 331 different applications over the study period (11 work days on average), with an average of 16 different applications per day. Such overwhelming use, among other reasons, may explain why Brynjolfsson (1993), almost three decades ago, prominently discussed the general notion that the use of ICT is not a straight way to success for organisations. He presented his work on the ‘productivity paradox of IT’, which illustrated that the greater diffusion of IT (in the United States during the 1970s and 1980s) was not accompanied by a comparable increase in productivity. In the meantime, we also learned that it is not only possible that the use of ICT may not lead to the awaited benefits on the societal and organisational levels, but that it may also lead to negative effects, in particular on the individual level.

For such negative consequences of the use of digital technologies at work, Brod (1982) coined the term ‘technostress’ and tellingly defined this phenomenon as (754): ‘ … a condition resulting from the inability of an individual or organization to adapt to the introduction and operation of new technology’. In line with this definition of technostress (or ‘digital stress’ as we will refer to it in the context of this study), early studies on ICT’s downsides have focused on the disruptive changes caused by technology implementations and their potential adverse consequences (e.g. Wastell and Newman 1993; Korunka, Weiss, and Karetta 1993; Smith and Carayon 1995). The increased diffusion of ICT has also led to increased research interest into the impact of digital stress on individual and organisational well-being (e.g. Riedl 2013; Fischer and Riedl 2017). The current wave of digitalisation (Legner et al. 2017) and specific technological trends (e.g. ubiquitous computing, Cascio and Montealegre 2016) indicate that the disruptive potential of technological change in the workplace will increase even further in the future.

Importantly, the social context in the organisation and in particular organisational culture and climate are crucial for an organisation and its members to keep up with the demands of being consistently innovative (Glisson 2015). Managers play a key role in this context, as they can support the creation of an organisational climate of innovation (Watts and Henderson 2006), which is characterised by openness to new ideas and mutual support (e.g. Ekvall 1996). Such an organisational climate can not only improve organisational creativity and innovation, but also positively affects individual well-being (e.g. Rasulzada and Dackert 2009).

Yet, the managerial-level view of technological change as being solely positive should be contested. Dwivedi et al. (2015) highlighted that perceptions related to information systems can differ between groups and individuals (particularly between the manager and user views). Therefore, although it seems that technological innovation, characterised by frequent updates, upgrades and reengineering of organisational ICT can lead to substantial positive organisational effects (e.g. Melville, Kraemer, and Gurbaxani 2004), it is not certain that this is also true on the individual user level. Vaniea and Rashidi (2016), for example, investigated how users experience the process of a software update and found that an update (i.e. a change in the technological environment) is linked to perceived user risk (e.g. lack of usefulness of new features, new bugs, unreliability of the system after the update) and may, therefore, involve substantial perceived uncertainty.

Rafferty and Griffin (2006) also found that the frequency of change is an important characteristic of organisational change that leads to individual uncertainty perceptions related to the new technologies. This finding is of particular importance in the context of technological change, as the current digital transformation wave further accelerates the rate of technological innovation. In line with this development, a recent survey (N = 682 respondents, with 48% being members of the senior management or even higher in the organisational hierarchy, Harvard Business Review Analytic Services 2016) found that a lack of risk tolerance and the inability to experiment are amongst the five most important barriers to the successful use of digital technologies. More than 30% of the respondents in this survey also indicated that the individual ability to adapt to change is the most important skill sought in employees. The organisational pressure to innovate is, therefore, translated into a demand on the individual to constantly keep up with technological change and to cope with the resulting uncertainty.

In the study reported in the present paper, we are therefore interested in pursuing the idea that an organisational climate that encourages and supports the implementation of technological innovations (Klein and Sorra 1996) and therefore entails organisational disruption can also have negative side effects. In particular, we want to answer the following research question:

“Can an organizational climate of innovation lead to individual stress perceptions related to the implementation of new technology?”

The remainder of this paper is structured as follows: In Section 2, we present the theoretical framework for this study, introduce its main constructs and how they are related to each other. In Section 3, we present the methods that were used to test its propositions. In Section 4, we then outline the results and discuss their implications for research and practice. Finally, in Section 5, we summarise the findings of our study and highlight limitations and avenues for future research.

2. Theoretical framework

In their discussion on the two-faced nature of a creative environment, Isaksen and Ekvall (2010) supported the notion that innovation may not only be perceived as beneficial, creating positive forms of tension (e.g. constructive debates that lead to new ideas), but also as harmful when individual interests are at stake and conflict arises. One potential avenue to explain why an organisational climate of innovation can be in conflict with individual interests is through the uncertainty that can accompany organisational changes. Edwards (2000), for example, argues (451): ‘Change in the repertoire (partial or total) through innovation is not guaranteed rather, it is inherently uncertain. It represents an attempt to alter the organization-specific practices previously embedded over time in its operating procedures’. This effect is also reported by Rafferty and Griffin (2006) who found that perceived uncertainty mediates the relationship between organisational change and important job outcomes (i.e. reduced job satisfaction and increased turnover intention). The structure of the research model utilised by Rafferty and Griffin (2006), which includes organisational change leading to the perceptions of stressors (i.e. uncertainty in their case) that then influence organisational outcomes is also common in research on digital stress. Research on digital stress does not only focus on the effects of ICT on individuals in isolation but has rather followed the recommendations by Orlikowski and Barley (2001) to combine the strengths of research on information technology and research on organisations. Accordingly, organisational stress theories are the basis for most of the research into digital stress, which is particularly evident in the structure of the research models that are applied in many of these investigations. This structure includes three key components: (1) situation, (2) stressors, and (3) effects or outcomes. As prominent studies on digital stress have adopted this structure before (e.g. Ragu-Nathan et al. 2008; Ayyagari, Grover, and Purvis 2011; Adam et al. 2017), we also use it as the basis for this investigation.

2.1. Situation

The situational characteristic that serves as the focus of this study is an organisational climate that supports technological change. For technological innovation, a distinction can be made between the use of a novel technology and its creation (i.e. idea utilisation as opposed to idea generation, Kim and Chung 2017) with the focus here being on the implementation of a new technology (i.e. its continued use, Klein and Sorra 1996) rather than the initial decision to adopt it in the organisation. As outlined by Coccia and colleagues (e.g. Coccia 2017, 2019a, 2019b; Coccia and Watts 2020), technological innovation is an evolutionary process that requires the ability to consistently solve problems (e.g. cope with unexpected behaviours of new technology). In such a process human affect can play an important role, with behavioural consequences. This has, for example, been indicated by Brinks (2020) who argues that certain situations (e.g. use of specific technologies) will be repeated if they lead to positive affective reactions in individuals or, conversely, not continued if they lead to negative effects, such as stress.

Cowan, Sanditov, and Weehuizen (2011) in this context highlight that (179): ‘Ultimately, what matters is not whether an innovation in itself is stress-inducing or stress-reducing, but rather the conditions under which it is so’. This claim has been supported frequently by studies that showed that an organisational climate that encourages learning and interaction among its members supports the innovation process (e.g. Albrecht and Hall 1991; Andersson, Moen, and Brett 2020; Ramirez Heller, Berger, and Brodbeck 2014). Motivated by this empirical finding, we focus on the potential relationship between an organisational climate that supports the implementation of technological innovations within organisations and individual perceptions of stress related to these technologies. Innovation is therefore treated as an independent variable in our theoretical framework (Janssen 2004) and we assume that technological innovation will not be sustainable within an organisational climate that leads to increased stress perceptions.

To further define the concept of an organisational climate, we have to distinguish it from the related concept of an organisational culture. For this purpose, we use the distinction made by Denison (1996) and differentiate culture and climate as follows:

[Culture is the] deep structure of organizations, which is rooted in the values, beliefs, and assumptions held by organizational members [… while climate is] rooted in the organization’s value system, but tends to present […] social environments in relatively static terms, describing them in terms of a fixed (and broadly applicable) set of dimensions. (624)

Hence, while culture is a concept that is more intangible and hard to grasp for individuals outside of an organisation, organisational climate is more observable and rooted in an organisation’s culture.

For organisational climate, several conceptualizations have been proposed, such as the four dimensions model of organisational climate by James and James (1989) (role stress and lack of harmony; job challenge and autonomy; leadership facilitation and support; work group cooperation, friendliness, and warmth). However, generalised climate conceptualizations such as the one by James and James have also been criticised. Schneider (2000), for example, argued that the dimensionality of organisational climate should depend on the specific phenomenon (topic) of an investigation. It follows that broad dimensions are typically less useful than more specific ones. Based on this rationale, we draw upon the classification of organisational climates by Patterson et al. (2005). In this classification, four types of primary climates are distinguished based on whether they have a more external or internal orientation and whether they have a more flexible or controlled orientation. One of the quadrants in this two-by-two classification is referred to as the ‘open systems model’ and is essentially a climate that is open to external impulses (e.g. new technological trends) and flexible in its structure (e.g. constantly testing new ideas and having dynamic procedures).

These facets can also be found in conceptualizations of an organisational climate of innovation that focus on technology implementation. For example, Klein, Conn, and Sorra (2001) highlight that an ‘implementation climate’ is characterised by (813): ‘ … employees perceiv[ing] that innovation implementation is a major organizational priority – promoted, supported, and rewarded by the organization … ’. A comparable definition has also been adopted in the context of research on digital stress by Tarafdar, Tu, and Ragu-Nathan (2010) who proposed that an organisational climate that is supportive of innovation (315) ‘ … encourage[s] communication, encourage[s] new ideas, and promote[s] supportive relationships among employees … ’. Due to a comparable focus on digital stress and the fit with the ‘open systems model’ (i.e. utilisation of new ideas and organisational structures to support their implementation), we adopt the definition by Tarafdar, Tu, and Ragu-Nathan (2010) for the purpose of this study.

2.2. Stressors

The Transactional Theory of Stress by Lazarus and Folkman (1984) has become one of the most established theories in the context of digital stress (Fischer and Riedl 2015). In their review of prominent theories of organisational stress, Sonnentag and Frese (2013) highlight that the Transactional Theory conceptualises stress as a constant exchange between the individual (e.g. with certain desires and available resources) and the environment (e.g. with certain situational demands such as work-related tasks). If this exchange between the individual and situational demands is out of balance (e.g. too much work than can be handled by the employee), detrimental effects for individual well-being and job-related performance can be the consequence.

Importantly, situational appraisal plays a key role in this exchange. More specifically, while the individual constantly processes environmental stimuli (e.g. visual stimuli and sounds), only few of these stimuli are processed consciously and constitute situational demands (e.g. job tasks that currently require attention). These situational demands (e.g. workload) do not always lead to negative consequences (e.g. work-related exhaustion), which can be explained by potential differences in individual appraisal. While one individual can consider certain situational demands as a challenge, another one can perceive them as a threat (e.g. due to different desires, such as one individual desiring skill development and new tasks while another desires a structured job with high levels of routine) (Cummings and Cooper 1998). Due to these differences in appraisal, we should consider that even situational characteristics that are often regarded as mainly beneficial to individuals such as an organisational climate of innovation could be a source of distress.

Sources of distress (i.e. stressors) in the context of digital technologies are often operationalised through the five ‘technostress creators’ introduced by Ragu-Nathan et al. (2008). These stressors include (Tarafdar et al. 2011, 117): overload (‘too much’), invasion (‘always connected’), complexity (‘difficult’), insecurity (‘uncomfortable’), and uncertainty (‘too often and unfamiliar’). Amongst these stressors, we already highlighted uncertainty as a potential consequence of disruptive changes in organisations at several points before and will therefore focus on this specific stressor as a potential negative effect of an organisational climate of innovation. In previous research on digital stress, Ayyagari, Grover, and Purvis (2011) linked characteristics of the technological environment such as perceived uncertainty to general work-related stressors (e.g. role ambiguity). Further, Ragu-Nathan et al. (2008) present a conceptualisation of uncertainty that is specific to ICT. While they describe that uncertainty felt in relation to rapid changes of ICT in the organisation ‘ … is similar to role ambiguity in that both describe situations involving ambiguity about expectations and outcomes … ’ (430), they particularly highlight that ‘ … ICT changes and upgrades unsettle users and create uncertainty so that they must constantly learn and educate themselves about new ICTs’ (427). From a user perspective, it is not certain though that effort invested into keeping up with new technological developments will be worthwhile; moreover, education effort also takes away time from actual work tasks (Ayyagari, Grover, and Purvis 2011).

Although the five technostress creators are established stressors in the context of research on digital stress (e.g. Sarabadani, Carter, and Compeau 2018), previous research has also highlighted that there is a potential for many more stressors related to ICT to be part of future investigations (Fischer, Pehböck, and Riedl 2019). For example, in more recent research, this list has been extended by further stressors such as unreliability (a sense that ICT does not perform as expected, Adam et al. 2017; Fischer and Riedl 2015; Riedl et al. 2012). Reliability is a core characteristic of ICT as it encapsulates the need for a system to be ‘dependable’ (DeLone and McLean 2003, 18) in general and in particular ‘ … to perform promised service dependably and accurately … ’ (Jiang, Klein, and Carr 2002, 146). A lack of reliability (unreliability) can have grave consequences. For example, a downtime of Amazon’s website on ‘Prime Day’ in 2018 caused up to $ 100 million in lost sales (Wolfe 2018). Technological change is often accompanied by such undesirable effects (Benamati and Lederer 2001b), with minor hassles (e.g. slow response times or unexpected system behaviours) being a well-known problem (e.g. Riedl and Fischer 2018). As we conceptualised an organisational climate of innovation as having two main facets (i.e. implementation of new ideas and mutual support), we also chose to focus on two ICT-related stressors, which are expected to each have a unique relationship with these facets. Next to perceived uncertainty, we chose perceived unreliability as the second stressor as it is also among the most prevalent ICT-related stressors (e.g. Riedl 2013).

2.3. Outcomes

In their review, Sarabadani, Carter, and Compeau (2018) summarised important outcome variables that have been frequently studied in the context of digital stress, which include user satisfaction, job satisfaction, performance, productivity, and commitment. They found that all of these outcome variables are negatively affected by the aforementioned set of technostress creators. Amongst these outcome variables, we decided to focus on user satisfaction, as it is an outcome variable specific to the context of technology use in organisations and job satisfaction as a more general outcome that is indicative of how employees feel about their work in general.

As shown by DeLone and McLean (1992) in their review of measures of information systems success, user satisfaction is amongst the most important variables to determine, whether new technology has been successfully implemented. This circumstance is further highlighted in the theoretical model presented by DeLone and McLean (2003) which posits that user satisfaction positively influences the actual use of technology, a relationship that has subsequently been validated in numerous studies (Petter, DeLone, and McLean 2008). In line with Bhattacherjee (2001), we broadly define user satisfaction as (359): ‘[u]ser’s affect with (feelings about) prior [ICT] use.’ Such a wide definition is needed due to the practice to operationalise user satisfaction as a form of enjoyment and satisfaction with the use of a specific type of technology, often involving particular characteristics of the specific technology (e.g. hard- and software features, quality of information, or the provided outputs, DeLone and McLean 1992).

As ICT provided in the work environment are part of the resources that are required to successfully fulfil job demands, we argue that technological changes will also have consequences for job satisfaction. This has, for example, been found by Spector and Jex (1998) who report that organisational constraints (e.g. interruptions and faulty equipment) negatively correlated with job satisfaction. Hence, while an organisational climate of innovation may not have a negative effect on job satisfaction in itself, its potential by-products in the form of uncertainty and unreliability could be mediators that adversely affect the ‘ … pleasurable or positive emotional state resulting from the appraisal of one’s job or job experiences’ (Locke 1976, 1300).

In Table 1 below, we summarise the five main constructs that we focus on in this investigation and the working definitions that we base our understanding of these constructs on.

Table 1. Construct definitions for main components in the research model.

Construct	Definition	Source
1. Situation:
Organisational climate of innovation	‘ … [a climate that] provide[s] support for innovation, encourage[s] communication, encourage[s] new ideas, and promote[s] supportive relationships among employees … ’	Tarafdar, Tu, and Ragu-Nathan (2010, 315)
2. Stressors:
Perceived uncertainty	‘users feel unsettled by continual upgrades and accompanying software and hardware changes’	Tarafdar et al. (2011, 117)
Perceived unreliability	‘Users face system malfunctions and other IT hassles’	Fischer and Riedl (2015, 1462)
3. Outcomes:
User satisfaction	‘Users’ affect with (feelings about) prior [ICT] use.’	Bhattacherjee (2001, 359)
Job satisfaction	‘ … a pleasurable or positive emotional state resulting from the appraisal of one’s job or job .’	Locke (1976, 1300)

2.4. Hypothesis development

The first relationship deals with the relationship between situational characteristics, in our study an organisational climate of innovation, and the two stressors, namely perceived uncertainty and perceived unreliability.

In line with the notion supported by Orlikowski and Barley (2001) that organisational influences (e.g. organisational climate) can act as an enabling force, but also as a constraining force, we argue that the two facets of an organisational climate of innovation (i.e. implementation of new ideas and mutual support) hint at its potential dual nature related to stress appraisal. In previous research on digital stress, these two aspects have mostly been investigated in isolation and the research focus has therefore also been mostly on either its beneficial or harmful nature (we highlight this circumstance in Appendix 1). In this study, focus on both sides based on the following evidence.

In a survey study by Ayyagari, Grover, and Purvis (2011) it was found that among the characteristics of an organisation’s technological environment, its pace of change is most strongly related to role ambiguity perceived by individuals (role ambiguity is defined as a ‘[lack of] certainty about duties, authority, allocation of time, and relationships with others; the clarity or existence of guides, directives, policies; and the ability to predict sanctions as outcomes of behaviour’, Rizzo, House, and Lirtzman 1970, 156). Agility and a high pace of change are promoted in organisations with a climate of innovation (Crocitto and Youssef 2003); yet, such a climate may also cause individual uncertainty related to organisational ICT. The underlying rationale of this relationship is that the less constant an individual’s environment, the higher the perceived uncertainty. Evidence in brain research (e.g. Critchley, Mathias, and Dolan 2001) and in IS research (e.g. Pavlou, Liang, and Xue 2007) supports this rationale.

While a climate of innovation can foster change and therefore uncertainty (i.e. through new, disruptive ideas that are then implemented, such as new ICT), it also provides individuals with an environment in which resources are available to deal with the affordances of new ICT (i.e. through mutual support, for example, when new ICT do not function as expected). For example, King et al. (2007) found that individuals showed a better work performance in an organisational climate of innovation. Specifically, they found that individuals were able to better deal with the distress caused by work demands (e.g. the feeling of being overwhelmed) in such an organisational climate. In the technology context, Day et al. (2012) found that the provision of organisational support to handle ICT hassles (e.g. personal assistance to deal with technology malfunctions) reduced the likelihood of individuals showing signs of reduced well-being (e.g. reports of headache or symptoms of burnout). It therefore becomes clear that an organisational climate of innovation can both tax individual resources by causing change, and hence uncertainty, but that it also provides individuals with resources (e.g. mutual support by members of the organisation) to cope with stress. Related to our two stressors, we therefore hypothesise that while an organisational climate of innovation will increase perceived uncertainty, it also offers the resources to reduce perceived unreliability of workplace technology:

H1a: An organizational climate of innovation positively affects perceived uncertainty.

H1b: An organizational climate of innovation negatively affects perceived unreliability.

For the relationship between the two stressors and the two outcome variables (user satisfaction and job satisfaction), we partly replicate previous research findings. Hence, in line with the results of previous research on digital stress (e.g. Tarafdar, Tu, and Ragu-Nathan 2010; Ragu-Nathan et al. 2008), and in particular based on the review results by Sarabadani, Carter, and Compeau (2018), we expect that both stressors will be negatively related to our outcome variables. While these hypotheses are in line with previous findings, we emphasise that unlike previous research investigating the combined impact of five ‘Technostress Creators’, we look at the individual impact of individual stressors. Moreover, replication of existing research findings is a significantly neglected topic in research in general (Camerer et al. 2016, 2018) and specifically in IS research (Dennis and Valacich 2014). In addition, while we expect negative effects of both stressors on our outcome variables, perceived unreliability has, to our knowledge, not been part of any previous survey studies that looked at its effects on user satisfaction or job satisfaction.

H2a: Perceived uncertainty negatively affects user satisfaction.

H2b: Perceived uncertainty negatively affects job satisfaction.

H2c: Perceived unreliability negatively affects user satisfaction.

H2d: Perceived unreliability negatively affects job satisfaction.

Next to the relationship between situational characteristics and stressors (H1) and the relationship between stressors and outcomes (H2), we also expect further relationships between the included variables that we discuss briefly, as we do not include specific hypotheses for them. While not the specific focus of the present study, these relationships are included to create a more complete research model, one that is in line with the findings of previous research and therefore more closely resembles reality. First, next to the mediating effect of an organisational climate of innovation on our outcome variables through its impact on the two included stressors, we can also assume a direct relationship. Such a relationship can also be supported by the findings of previous studies in the context of the impact of organisational climate and related practices on individual satisfaction with work and performance at work (e.g. Eisenberger et al. 1997; Johnson and McIntye 1998; Lee et al. 2014; Delaney and Huselid 1996; Gelade and Ivery 2003; Ramirez Heller, Berger, and Brodbeck 2014).

Second, next to the direct link between situation and outcomes, we can also assume relationships between our two stressors and our two outcome variables. For our included stressors this may be indicated by the previous use of higher-order constructs in the context of research on digital stress (e.g. the ‘Technostress Creators’ by Ragu-Nathan et al. 2008, which is a second-order construct with five reflective dimensions). Yet, we still keep these stressors separate (i.e. without using a higher-order construct) as previous research into the separate effects of ICT-related stressors has shown that there is a possibility that non-consistent effect patterns emerge (e.g. in the study by Ahmad, Amin, and Ismail 2014, out of the five Technostress Creators dimensions, only two where significantly related to organisational commitment).

For the directionality of this relationship, there is only sparse evidence in the context of research on digital stress, with Shu, Tu, and Wang (2011) highlighting that these two stressors often accompany each other when new systems are introduced but they do not discuss whether the perception of these stressors could influence each other. We find some evidence for a potential directionality in a recent survey by Morris, Becker, and Parkin (2019) though, who found that the process of changing a system causes uncertainty, particularly as users are expecting problems with their device if an update takes place.

For user satisfaction and job satisfaction, evidence for a potential connection is not as clear. While satisfaction with certain characteristics of ICT has been found to be a correlate of job satisfaction before (e.g. satisfaction with information provided by ICT or system usefulness, Ang and Soh 1997), this correlation has not been stable throughout previous research. For example, Joshi and Rai (2000) found no direct relationship between the perceived quality of the information provided by ICT at work (a potential indicator of user satisfaction, Petter, DeLone, and McLean 2008; DeLone and McLean 2003) and job satisfaction. Rather, they found that the relationship was fully mediated by role stressors (i.e. role ambiguity and role conflict).

Based on these initial arguments and findings, we tentatively expect that (i) perceived uncertainty will be positively related to perceived unreliability and that (ii) user satisfaction will be positively related to job satisfaction. Yet, the evidence regarding these relationships is only sparse and we therefore retain the option to exclude these relationships should they impact the quality of the tested model in a negative way (e.g. in terms of model fit).

In Figures 1 and 2, we present our research model, which illustrates how the main components of our investigation are related to each other. Hypothesised relationships are indicated by solid lines while additional relationships that are tentatively explored are indicated by dashed lines (i.e. they may be removed from the model should they reduce its explanatory power or quality in terms of model fit).

Figure 1. Research model. Solid lines between constructs indicate hypothesised relationships while dashed lines indicate relationships that can be assumed based on previous research but may be removed for parsimony reasons.

Figure 2. Overview of study results. Model fit: χ² = 572.07; d.f. = 274; χ²/d.f. = 2.09 (< 5; Wheaton et al. 1977); CFI = .960 (>.95; Hu and Bentler 1999); RMSEA = .044 (<.06 / <.08; Hu and Bentler 1999; Hooper, Coughlan, and Mullen 2008), SRMR = .048 (<.05 / <.08; Hu and Bentler 1999; Hooper, Coughlan, and Mullen 2008); GFI = .925 (>.90; Hooper, Coughlan, and Mullen 2008); AGFI = .904 (>.90; Hooper, Coughlan, and Mullen 2008); NFI = .927 (>.90 / >.95; Bentler and Bonett 1980; Hu and Bentler 1999); TLI (NNFI) = .953 (>.80 / >.95; Hu and Bentler 1999; Hooper, Coughlan, and Mullen 2008).

In addition to the main constructs of our study, we also included age, gender, level of education, and computer self-efficacy as control variables. Age, gender and education are variables that are routinely reported as sample characteristics in the context of research on ICT-related stress and have also been found to be related to this type of stress perception (e.g. Tams, Thatcher, and Grover 2018; Riedl et al. 2013; Ragu-Nathan et al. 2008). With regard to self-efficacy in general, it has been argued that people tend to avoid situations for which they may not have the capabilities to deal with. However, if they have sufficient self-efficacy, they will face the struggle and try to push through such situations until they succeed (Igbaria and Iivari 1995). In line with this idea, in social cognitive theory, self-efficacy is defined as ‘confidence in one’s ability to cope with a stressful situation’ (Wood and Bandura 1989, 588), which was extended by Compeau and Higgins (1995), who define computer self-efficacy as the ‘ … judgment of one’s capability to use a computer’ (192). Because computer self-efficacy may have a positive impact on both user satisfaction (e.g. Adam Mahmood et al. 2000) and job satisfaction (e.g. Henry and Stone 1995), we included it as a control in the present study.

3. Methodology

In this section, we discuss the survey study and the data analysis procedures that were used to test our research model.

3.1. Data collection

We used a survey design to test our research model. According to the findings of a review on the data collection methods used in previous research on digital stress by Fischer and Riedl (2017), self-report measures are by far the most prominent type of measurement method. This circumstance can, for example, be justified from a theoretical point of view, as a survey approach or self-reports in general are in line with the Transactional Theory of Stress by Lazarus and Folkman (1984), because individual perceptions and their appraisal are pivotal for the emergence of stress. Therefore, we collected our data through an online survey that was distributed by a market research company (i.e. Bilendi; see https://www.bilendi.co.uk/). To make it clear for our respondents that we are interested in the organisational context, we provided an introductory text that explained that we focus on ICT at work. Further, we also provided a list of technologies that we consider as ICT in line with Ayyagari, Grover, and Purvis (2011, A2) (e.g. communication technologies such as e-mail or generic application technologies such as word processing software).

We aimed for a sample that is representative of the German employed population (i.e. average age close to 44 years, Destatis - Statistisches Bundesamt 2018; with approximately equal gender distribution, Destatis - Statistisches Bundesamt 2019). We gathered a total of N = 1,392 completed surveys of which 265 were removed due to a significant amount of missing data (i.e. more than 10% of answers were ‘Don’t know’); a further 6 questionnaires were removed because of low engagement (i.e. standard deviation of answers below .5) (e.g. DeSimone, Harms, and DeSimone 2015; Meade and Craig 2012), and an additional 6 questionnaires were removed as they did not completely indicate their demographic information (i.e. age, gender, or highest level of education). Hence, our final sample is N = 1,115 completed questionnaires.

We used established measures for construct measurement, except for perceived unreliability. We mainly used a 7-point Likert scale for measurement (1 – strongly disagree to 7 – strongly agree) and the survey was administered in German. For innovation climate we used the 5-item scale by Tarafdar, Tu, and Ragu-Nathan (2010) (sample item: ‘We have a very open communications environment.’). For user satisfaction we used the 4-item scale by Bhattacherjee (2001) (e.g. ‘How do you feel about your overall experience of utilising ICT in connection with your work tasks?). For job satisfaction we used the 3-item scale by Ragu-Nathan et al. (2008) (e.g. ‘I like doing the things I do at work.’). For perceived uncertainty we used the 4-item scale by Ragu-Nathan et al. (2008) (e.g. ‘There are constant changes in computer software in our organization.’). For computer self-efficacy we used the 10-item scale by Compeau and Higgins (1995) (e.g. ‘I could complete my tasks using new ICTs if there was no one around to tell me what to do as I go.’). The three remaining controls were measured as follows: age as metric variable in years, gender as nominal variable (male, female), and education as ordinal variable (no degree, compulsory education, vocational training, university-entrance diploma, university degree, other).

At the time of data collection, there was no existing scale for perceived unreliability available and we therefore developed a measurement instrument. The items were based on the indicators used to measure reliability in a seminal paper by Ayyagari, Grover, and Purvis (2011) and centred around the idea of ICT in the workplace being ‘too unstable’ (Fischer and Riedl 2015; Adam et al. 2017) (e.g. ‘I think that I lose too much time due to technical malfunctions.’). For the development of the scale for perceived unreliability, initially formulated items were circulated at the authors’ research institution and refined based on the feedback (e.g. unclear formulations were changed). We then included the preliminary measure in a survey that took place at this institution in the beginning of fall 2017, which resulted in N = 198 complete questionnaires mostly from students and faculty. The reliability score (coefficient alpha) of this initial instrument was .756 and we decided to retain all four items for this study, as we aimed for at least three strong items during exploratory factor analysis (EFA) (Costello and Osborne 2005; Reio and Shuck 2015). In line with our introduction of the unreliability construct, these items mainly reflect the impact of constant hassles rather than singular disruptive events and since the time of data collection, comparable instruments have been used by Adam et al. (2017). An overview of our measurement instruments and initial results of our reliability and validity assessments, which are presented in more detail in the next section, can be found in Appendix 2.

For our analyses, the complete sample was randomly split up into two sub-samples, one for the exploratory factor analysis (EFA) (N = 564) and one for the confirmatory factor analysis (CFA) (N = 551) and the causal modelling in accordance with comparable studies such as Ragu-Nathan et al. (2008). After splitting our sample, we used Levene tests of all included indicators to test whether these two samples were statistically different. As none of these tests were statistically significant at the .05 level, the sub-samples are comparable (see Table 2 for an overview of the main characteristics for both samples; further descriptive statistics for both sub-samples, including indicator means, standard deviations, skewness and kurtosis, can be found in Appendix 2). Further, we achieve an item-to-subject ratio of 19:1 (N = 564 / 30 items) for our EFA, which is above the recommended thresholds of 10:1 and close to 20:1 (Costello and Osborne 2005).

Table 2. Overview of samples used for EFA, CFA and SEM.

	Sample for EFA, N = 564	Sample for CFA and SEM, N = 551
Age	Avg.: 44.44 (Std. dev.: 11.82)	Avg.: 44.72 (Std. dev.: 11.67)
Gender	Female: 300 (53.2%)	Female: 284 (51.5%)
	Male: 264 (46.8%)	Male: 267 (48.5%)
Highest Level of Education	No degree: 2	No degree: 1
	Compulsory Education: 17	Compulsory Education: 14
	Vocational Training: 233	Vocational Training: 233
	University-Entrance Diploma: 120	University-Entrance Diploma: 112
	University Degree: 180	University Degree: 179
	Other: 12	Other: 12

3.2. Reliability and Validity Assessment

We conducted an exploratory factor analysis (EFA) with all items in our latent constructs in SPSS 25. First, we tested the normality assumption for all our indicators using Shapiro-Wilks tests and found that univariate normality cannot be assumed. Hence, instead of maximum likelihood we used principal axis factoring as our extraction method (Costello and Osborne 2005). For the rotation we used promax, as oblique methods are generally recommended for the social and behavioural sciences where correlations amongst constructs should always be expected (Treiblmaier and Filzmoser 2010; Reio and Shuck 2015; Matsunaga 2010). As we have an a priori theory of the dimensionality of our indicators, we fixed the extraction to six factors (i.e. innovation climate, user satisfaction, job satisfaction, uncertainty, unreliability, computer self-efficacy), which is generally favoured over unconstrained extraction based on Eigenvalues alone (Costello and Osborne 2005; Reio and Shuck 2015).

Barlett’s test was significant (p < .001) and we reached a KMO (Kaiser-Meyer-Olkin criterion) of .881 (‘meritorious’ according to Kaiser and Rice 1974; values above .90 would be considered ‘marvelous’), which indicates good potential for dimension reduction. The six extracted factors explained 64.20% of the total variance, which is well above the recommended threshold of 40% (Reio and Shuck 2015). After removing items with high cross-loadings or low loadings on their main factor, we ended up with the factor structure presented in Appendix 2. Importantly, though we had to remove several items, no factor includes less than three high loading items (Costello and Osborne 2005; Reio and Shuck 2015). The pattern matrices, including all loadings and cross-loadings for the initial EFA and the final EFA can be found in Appendix 3.

Using our holdout sample, we then tested our measurement model based on a confirmatory factor analysis (CFA) (Matsunaga 2010) using AMOS 25. The model fit for our measurement model was acceptable with a χ² of 447.4, d.f. of 215, χ²/d.f. of 2.08 (<5; Wheaton et al. 1977), CFI of .969 (>.95; Hu and Bentler 1999), RMSEA of .044 (<.60; Hu and Bentler 1999) and SRMR of .041 (<.08; Hu and Bentler 1999). We therefore accepted the factor structure of our measurement model. In (Appendix 4), we summarise the results including the composite reliability for all constructs, which is acceptable (>.7; Nunnally and Bernstein 1994; see also Appendix 2 for the coefficient alpha values as another indicator of reliability, which are also acceptable).

For convergent validity, we assessed the average variance extracted (AVE) for each construct, which should be above .5 (MacKenzie, Podsakoff, and Podsakoff 2011). In the case of job satisfaction, this value was missed slightly, but we retain the construct nonetheless as the specific value for this threshold has been criticised before (e.g. Malhotra and Dash 2011). For discriminant validity, we applied the Fornell-Larcker criterion (i.e. square root of AVE for a construct should be larger than any correlations with other constructs, Fornell and Larcker 1981), which is fulfilled in all cases. In addition, we assessed the heterotrait-monotrait ratio of correlations (HTMT) as a second indicator of discriminant validity (Henseler, Ringle, and Sarstedt 2015). As none of the values reported in Table A7 (Appendix 4) are above the more lenient threshold of .900 or even the more strict threshold of .850 (Henseler, Ringle, and Sarstedt 2015), we can assume discriminant validity for our constructs based on this criterion as well.

As all of our data was measured through the same approach (i.e. self-reports), we tested for the potential influence of common method bias (Podsakoff et al. 2003). In line with current recommendations, we used several methods to assess the influence of CMB on our results (Turel, Serenko, and Giles 2011; Turel 2015). First, we used Harman’s Single Factor test (Harman 1976) in SPSS (i.e. principal component analysis with no rotation), with the first and largest factor explaining 26.59% of the total variance, which is clearly below a majority of the total variance. Second, we inspected the inter-construct correlations matrix (see Table A6 in Appendix 4), looking for significant correlations above .90 (Pavlou, Liang, and Xue 2007). As our correlations range from |.026| to |.678|, this criterion also does not indicate common method bias. Third, we used the common latent factor approach (Podsakoff et al. 2003; Turel 2016; Podsakoff, MacKenzie, and Podsakoff 2012) in AMOS, comparing an unconstrained model (χ² = 447.0, d.f. = 409) to a zero-constrained model (χ² = 447.0, d.f. = 409), which showed no statistically significant difference (Δ χ² = 0, Δ d.f. = 0, p = 1.000). Hence, we can assume that there is no significant influence of common method bias on our results.

As AMOS uses Maximum Likelihood for model estimation, we further had to ensure that multivariate normality could be assumed based on our data (Kline 2011). For this purpose, we first tested for multicollinearity in SPSS for our endogenous variables user satisfaction and job satisfaction. As the variance inflation factors (VIF) are below 10 (Kline 2011) and even below 4 (O’brien 2007) in each case (User satisfaction: unreliability: 1.021, computer self-efficacy: 1.028, innovation climate: 1.042; Job satisfaction: unreliability: 1.021, computer self-efficacy: 1.028, innovation climate: 1.042), we can rule out multicollinearity as an influencing factor. We then also tested for influential outliers in SPSS using Cook’s Distance (Cook 1977). As the largest Cook’s Distance (i.e. .076 between computer self-efficacy and job satisfaction) was clearly below the threshold of 1 (Cook and Weisberg 1982), we can also rule out outliers as an influential factor and can therefore assume multivariate normality. As our data does not show a normal distribution on the univariate level, we used bootstrapping with 2000 samples for all model estimations.

3.3. Model estimation

For our model estimation, we first introduced our control variables (i.e. age, gender, education, computer-self-efficacy) and assessed their influence on each of our five main constructs. Together, the four control variables explained 3.0% of the variance in organisational climate of innovation, 2.8% of the variance in perceived uncertainty, 7.7% of variance in perceived unreliability, 7.1% of the variance in user satisfaction, and 9.2% of the variance in job satisfaction. For the test of our hypotheses, we then only retained the effects between the control variables and our main dependent variables (i.e. user satisfaction and job satisfaction).

The model fit for our research model (see Figure 1), was overall acceptable with a χ² of 572.07, d.f. of 274, χ²/d.f. of 2.09 (<5; Wheaton et al. 1977), CFI of .960 (>.95; Hu and Bentler 1999; not fulfilled), RMSEA of .044 (<.60 / <.80; Hu and Bentler 1999; Hooper, Coughlan, and Mullen 2008) SRMR of .048 (<.05 / <.08; Hu and Bentler 1999; Hooper, Coughlan, and Mullen 2008), GFI of .925 (>.90; Hooper, Coughlan, and Mullen 2008, not fulfilled), AGFI of .904 (>.90; Hooper, Coughlan, and Mullen 2008, not fulfilled), NFI of .927 (>.90 / >.95; Bentler and Bonett 1980; Hu and Bentler 1999, not fulfilled), and TLI of .953 (>.80 / >.95; Hooper, Coughlan, and Mullen 2008; Hu and Bentler 1999). As previously noted, this research model included a number of further relationships for which we did not formulate hypotheses. Hence, we also ran a model in which these relationships were removed (i.e. relationships with dashed lines in Figure 1). The model fit for this simplified model was overall not acceptable with a χ² of 958.63, d.f. of 278, χ²/d.f. of 3.45 (<5; Wheaton et al. 1977), CFI of .909 (>.95; Hu and Bentler 1999; not fulfilled), RMSEA of .067 (<.60 / <.80; Hu and Bentler 1999; Hooper, Coughlan, and Mullen 2008) SRMR of .104 (<.05 / <.08; Hu and Bentler 1999; Hooper, Coughlan, and Mullen 2008), GFI of .878 (>.90; Hooper, Coughlan, and Mullen 2008, not fulfilled), AGFI of .846 (>.90; Hooper, Coughlan, and Mullen 2008, not fulfilled), NFI of .877 (>.90 / >.95; Bentler and Bonett 1980; Hu and Bentler 1999, not fulfilled), and TLI of .894 (>.80 / >.95; Hooper, Coughlan, and Mullen 2008; Hu and Bentler 1999). Hence, the results for these further relationships are also discussed in the next section.

4. Results and discussion

In this section, we present the main results of our model estimation and discuss their implications for research and practice.

4.1. Results of model estimation

In Table 3, we summarise the estimates for the effects included in our research model, their effect sizes and whether the related hypotheses were supported or not.

Table 3. Model estimates and overview of study results.

Effects (hypothesised)	Model estimates	Effect sizes^a	Support
H1a: IC → UC	β = .228, p < .001 (C. R. = 4.193), f² = .055	Small	Yes
H1b: IC → UR	β = −.231, p < .001 (C. R. = −4.607), f² = .085	Small	Yes
H2a: UC → US	β = .009, p = .876* (C. R. = 0.156), f² = -	–	No
H2b: UC → JS	β = .037, p = .499* (C. R. = 0.676), f² = -	–	No
H2c: UR → US	β = −.303, p < .001 (C. R. = −5.720), f² = .064	Small	Yes
H2d: UR → JS	β = −.087, p = .104* (C. R. = −1.625), f² = -	–	No
Effects (not hypothesised)	Model estimates	Effect sizes^a
IC → US	β = .422, p < .001 (C. R. = 8.117), f² = .189	Medium
IC → JS	β = .515, p < .001 (C. R. = 7.831), f² = .336	Medium
UC → UR	β = .537, p < .001 (C. R. = 9.384), f² = .389	Large
US → JS	β = .277, p < .001 (C. R. = 5.192), f² = .007	(Negligible)

* = indicates non-significant effects based on a p < .05 threshold.

IC = organisational climate of innovation; UC = perceived uncertainty; UR = perceived unreliability; US = user satisfaction; JS = job satisfaction.

a = Effect size assessment based on f² thresholds proposed by Cohen (1992).

In the tested model, the support for our hypothesised relationships is mixed and although H1 is fully supported, only one of the four effects hypothesised in H2 is significant. More specifically, we only find that perceived unreliability (e.g. faulty equipment and related time loss) negatively affects user satisfaction (H2c), which is in line with the assumption that reliability is a core characteristic of information systems success (e.g. Petter, DeLone, and McLean 2008). Importantly though, we find support for duality of an organisational climate of innovation related to stress as it increases the perception of uncertainty (H1a), while it decreases the perception of unreliability (H1b).

In addition, we find that the inclusion of direct relationships between an organisational climate of innovation and user satisfaction as well as job satisfaction has merit, as both relationships are significant and part of the model with the overall better model fit. This finding is comparable to the study results reported by Cullen et al. (2014) who found that while change-related uncertainty had no effect on job satisfaction, there was a positive relationship between perceived organisational support and job satisfaction. In their study though, organisational support was included as a mediator of the relationship between stressors and outcomes rather than as a situational characteristic that is an antecedent to stress appraisal.

Regarding the relationships between our stressors and our outcome variables, while we find both relationships to be significant (i.e. perceived uncertainty positively affects perceived unreliability and user satisfaction positively affects job satisfaction), the effect size for the relationship between user satisfaction and job satisfaction is negligible and therefore only of minor importance. The positive effect of perceived uncertainty on perceived unreliability is the most substantial effect in our model (based on Cohen 1992 it can be considered a large effect as the f² value is >.35).

A potential avenue to explain this relationship can be found in research into the emotional effects of uncertainty. For example, in an experimental study which involved the affective evaluation of movies, Bar-Anan, Wilson, and Gilbert (2009) found that introducing uncertainty (through verbal cues) led to more extreme evaluations (positive or negative). This effect can be explained by the human tendency of increasing attention and arousal in uncertain situations (Carretié et al. 2004; Fiorillo, Tobler, and Schultz 2003). It follows that uncertainty that results from technological changes may also lead to increased awareness of potential ICT unreliability (i.e. a positive effect of perceived uncertainty on perceived unreliability).

Hence, while perceived uncertainty itself does not have a negative impact on user satisfaction or job satisfaction, we can assess its indirect effect through perceived unreliability on user satisfaction. We find a standardised effect of β = −.163 (bootstrapped confidence interval: −.224 to −.114 with a standard error of .033). Hence, perceived uncertainty can have a negative impact, but mainly through other stressors (i.e. perceived unreliability in our study). It has to be noted though that this effect is significant, but not substantial due to the small effect size of the relationship between perceived unreliability and user satisfaction (f² = .064).

Overall, the effects in our model explain 5.2% of the variance in perceived uncertainty, 28.5% of the variance in perceived unreliability, 35.5% of the variance in user satisfaction, and 55.3% of the variance in job satisfaction. Based on the relationships included in our research model (Figure 1), it can therefore be stated that while there is also a potential negative side to having an organisational climate of innovation (i.e. increasing perceived uncertainty), overall its positive side (i.e. reduction of perceived unreliability and direct positive effects on user satisfaction and job satisfaction) outweighs its negative effects.

4.2. Implications for research

This study investigated the potential dual nature of an organisational climate of innovation as an antecedent to the appraisal of distress related to technological change in organisations. In contrast to previous research, it was found that such a climate can lead to stress perceptions in the form of heightened uncertainty, while at the same providing organisational members with a way to deal with ICT-related distress caused by the unreliability of technology. This finding creates a more sophisticated picture of the effects that managerial attempts at fostering technological change (by implementing an organisational climate of innovation) can have on individual well-being. Importantly, it highlights that, while technological change can be a source of uncertainty, the way with which it is implemented is crucial. In a climate that allows for mutual support of organisational members, the downsides of technological change (e.g. heightened awareness of ICT unreliability due to uncertainty) can be alleviated substantially. If this facet of an organisational climate of innovation is neglected though, technological change can have detrimental effects. This is, for example, supported by results of a study by Wang, Shu, and Tu (2008) who found that an organisational environment in which innovation is accompanied by competitiveness amongst organisational members will lead to an increase in ICT-related distress appraisal. Edwards (2000) in this context argues that (452): ‘An organization’s ability to deal with […] uncertainty depends […] on the extent to which the innovation can be unpacked and the extent to which individuals have the necessary skills and knowledge to do so.’ As mutual support amongst individuals allows for the transfer of such skills and knowledge, an organisational climate of innovation cannot only be an environment of uncertainty, but also an environment of learning and successful technological change.

The positive side of such an organisational climate was further evidenced by the initially not hypothesised direct effects of an organisational climate of innovation on user satisfaction and job satisfaction. Further, the related effects were more substantial in size than the effects of an organisational climate of innovation on ICT-related distress appraisal (i.e. f² of .189 for user satisfaction and f² of .336 for job satisfaction as compared to an f² of .055 for perceived uncertainty and an f² of .085 for perceived unreliability, see Table 3). Hence, while an organisational climate of innovation plays an important role in the context digital stress, it plays an even more important role for the general job-related well-being of individuals. Nonetheless, additional research on the stress-related effects of such situational characteristics should not be neglected and, in particular, additional research is needed that addresses the unique relationships of such characteristics with ICT-related stressors.

In addition, in their review of previous research that used the Technostress Creators set of digital stressors, Sarabadani, Carter, and Compeau (2018) found that there is a lack of investigations into the antecedents and effects of separate ICT-related stressors. Only one paper in their review (i.e. Ahmad, Amin, and Ismail 2014) investigated separate effects and showed that no consistent pattern emerged (e.g. only overload and uncertainty where significantly related to organisational commitment and, contrary to expectations, both enhanced commitment). In the context of this study too, the widely used conceptualisation of digital stress as a higher-order construct (Ragu-Nathan et al. 2008; Sarabadani, Carter, and Compeau 2018) would not have revealed the dual nature of an organisational climate of innovation. Therefore, conceptualizations such as the one used by Ayyagari, Grover, and Purvis (2011), which investigate situational characteristics, stressors, and outcomes in separation should be favoured. This separation led to more actionable insights, which can be the basis for a number of practical implications.

4.3. Implications for practice

We found that technological change in organisations can have important positive effects (i.e. increased user satisfaction, increased job satisfaction, and a means to deal with the unreliability of ICT). However, we also found negative side effects of an organisational climate of innovation that should not be neglected. It is critical to consider these effects, as change is a constant companion for many organisations, particularly in the current era of digital transformation (Hess et al. 2016). The need for technological change, however, has already been highlighted in the early 2000s. For example, Benamati and Lederer (2001b) argued that ‘[d]ue to today’s highly competitive environment, the failure to capitalize on new IT can lead to lost opportunities and can be especially costly’ (184). In line with this ‘quasi-obligation’ for companies to embrace change, they found that in order to cope with rapid technological change, most companies enforce an organisational climate of innovation that supports learning and procedures to implement technological changes (Benamati and Lederer 2001a).

What our study suggests is the fact that these attempts at handling change are translating environmental uncertainty (e.g. new technologies and resulting competitive challenges) into internal uncertainty (e.g. ambiguity related to employed technologies) which employees have to deal with in their daily business. Further, our study also revealed that this perceived uncertainty may translate into elevated perceptions of unreliability of technology. Because perceived unreliability of technology may negatively affect intention and actual use of computer systems (e.g. Liang and Xue 2009), which, in turn, are necessary preconditions for the positive effects of ICT use (e.g. increased access to information, better decision quality, higher productivity; e.g. Brynjolfsson and Hitt 2000), we argue that organisations should not blindly introduce and establish a climate of innovation. Rather, responsible managers should always consider the potential negative effects of their actions and the potential remedies. As shown in this study, technological change being accompanied by mutual support (instead of fostering competitiveness, Wang, Shu, and Tu 2008) can be beneficial to individuals, though it is not the sole option to deal with uncertainty (e.g. quality of change communication, participation in decision-making, or increased job autonomy are also viable options, Bordia et al. 2003).

5. Conclusion

In this study, we showed that an organisational climate of innovation, which embraces technological change, has benefits, but also downsides. In particular, we found that such an organisational climate has facets that can increase the perception of ICT-related stressors (i.e. perceived uncertainty), but also offers the means to reduce the perception of other ICT-related stressors (i.e. perceived unreliability). From a practical perspective, this finding shows that technological change, if implemented in the right environment (i.e. an environment of mutual support as found in this study) can be coped with effectively by individuals. Our findings also highlight the need for more in-depth investigations into the specific relationships between ICT-related stressors and situational characteristics, as each of the included stressors (i.e. perceived uncertainty and perceived unreliability) was affected differently by an organisational climate of innovation and, in turn, also affected important ICT-related and job-related outcome variables (i.e. user satisfaction and job satisfaction) in different ways.

This study also has some limitations, which can be opportunities for future research. First, as we strived for parsimony in our research model, there are a variety of ways to extend the nomological network that was the basis for this investigation of which we want to highlight a few related to (i) situation, (ii) stressors, and (iii) outcomes.

A first way to extend this line of research on the effects of technological change is to look at different aspects of change, as presented in the study by Rafferty and Griffin (2006) (e.g. frequency of change). In line with their findings, we can, for example, expect that environments with more frequent changes are also accompanied by higher individual levels of perceived uncertainty. Also, further aspects of the organisational environment that help to establish a climate of innovation, such as progressive HRM practices (e.g. Gelade and Ivery 2003), or that enhance its positive effects, such as distributional fairness (e.g. Shih and Susanto 2011), could be added.

In addition, environmental characteristics that are outside of the organisation could have an influence on this relationship such as the country or industry that an organisation operates in and the interest in and support structures for innovation of this country or industry as suggested by Coccia and colleagues (e.g. Coccia 2018, 2019c; Coccia and Watts 2020). In line with the findings of Nybakk, Crespell, and Hansen (2011) it would also be worthwhile to look at more specific aspects aside from the country of operation alone, such as national culture. For example, Shane (1995) found that in cultures characterised by high levels of uncertainty avoidance (Hofstede, Hofstede, and Minkov 2010), individuals were less likely to champion innovation efforts.

Next to environmental characteristics, further individual characteristics aside from the control variables included in this study (i.e. age, gender, education, computer self-efficacy) could help us to understand in which situations technological change can lead to distress and in turn to the successful implementation of technological innovations (Kim and Chung 2017). For example, Cullen et al. (2014) found that individual adaptability is negatively related to perceived uncertainty, while we can expect that individual characteristics such as a dispositional resistance to change (Oreg 2003) may increase the likelihood that individuals feel uncertainty that results from technological change. In this context, it is also important to look at different combinations of personality traits and their influence on stressor perception, as shown recently by Khedhaouria and Cucchi (2019).

Related to stress appraisal, it could be worthwhile to not only focus on distress, but also consider the option of technological change leading to eustress (i.e. situations being perceived as challenge rather than as a hindrance, see, for example, Tarafdar, Cooper, and Stich 2019 or Zhao, Xia, and Huang 2020). In addition, further ICT-related stressors should be included (e.g. complexity or insecurity, Ragu-Nathan et al. 2008) to further investigate the separate effects that an organisational climate of innovation has on stress appraisal. Related to potential outcomes, next to the dependent variables highlighted by Sarabadani, Carter, and Compeau (2018) (e.g. performance, productivity, commitment), it could prove interesting to not only focus on psychological well-being, but also on physiological well-being. For example, Pollard (2001) found that disruptive changes in the workplace (i.e. workplace reorganisation) can lead to substantial reduction of physiological well-being (e.g. elevated levels of blood pressure and heightened levels of total cholesterol), which was particularly evident during phases of highest uncertainty (i.e. closest to the actual implementation of changes).

Second, we used a general definition of ICT based on the classification of technologies presented by Ayyagari, Grover, and Purvis (2011). It would be worthwhile to test whether the relationships we identified can also be found for more specific technologies, and the resulting focus on more specific stressors leads to differences in effect sizes. Existing research streams to which our study could be linked are stress resulting from ICT-related security measures (e.g. D'Arcy, Herath, and Shoss 2014) or stress resulting from the use of social media (e.g. Maier et al. 2015).

Third, analysing the same relationships in the context of more specific populations could lead to different results and we argue that it could particularly influence the effect sizes. Earlier research into the stress of IS professionals specifically hints at that possibility. For example, Sethi, King, and Quick (2004) found that the high demands of continuous training for IS professionals can have detrimental effects such as lower levels of job satisfaction. Based on the recent insights into the work life of software developers by Meyer et al. (2017), who found that software developers, on average, use around 16 different applications per day and more than 40 per month, we can assume that unreliability of provided technologies might have a more substantial impact on overall job satisfaction in such a population. It would therefore be interesting to see additional, context-specific investigations into the effects of techno-unreliability, particularly in job environments that are characterised by a high dependence on technology (such as software engineering or service providers in social media marketing; first evidence of the physiological effects of techno-unreliability in such a sample population is provided in a pilot study by Kalischko, Fischer, and Riedl 2019).

Fourth, we collected our data in a cross-sectional online survey. Though we controlled for common method bias, our data collection approach nonetheless has weaknesses that need to be addressed in future research. For example, from a strict methodological viewpoint, causal inference is only possible based on longitudinal data or experimental designs that imply deliberate manipulation of the independent variable. Such study designs would be needed to investigate the proposed evolutionary process of innovations as outlined by Coccia and colleagues (e.g. Coccia 2017, 2019a, 2019b) or the influence of actual innovative work behaviours on organisational performance (e.g. Shanker et al. 2017). It would also be worthwhile to investigate the potential of other types of data collection in addition to self-reports to shed more light on the effects and coping mechanisms related to perceived uncertainty and unreliability. As shown in a systematic review (Riedl 2013), ICT-related stressors such as unreliability of systems may negatively affect a number of physiological parameters, including elevations of stress hormones (e.g. adrenaline, noradrenaline, or cortisol) or changed patterns of autonomic nervous system activity (e.g. increased heart rate, reduced heart rate variability, or muscle tension).

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research was funded in part by the Upper Austrian Government as part of the project ‘Digitaler Stress in Unternehmen’ (Basisfinanzierungsprojekt) and it was funded in part by the Austrian Science Fund as part of the project ‘Technostress in Organizations’ (grant no. P 30865) at the University of Applied Sciences Upper Austria.

Appendices

Appendix 1

We drew a selection of studies from the review of previous research on digital stress by Fischer and Riedl (2017) that have focused on situational characteristics related to an organisational climate of innovation (e.g. pace of change or technical support). This selection, as listed in Table A1, illustrates that situational characteristics were either included as a beneficial force of the organisational environment (i.e. reducing distress appraisal and/or its negative effects/outcomes) or as a harmful force of the organisational environment (i.e. a source of distress). Only one of these studies considered both options (i.e. Wang, Shu, and Tu 2008), but focused mainly on the harmful effects of a culture of innovation (e.g. by increasing competitiveness amongst members of the organisation, as argued by the authors).

Table A1. Illustrative studies on digital stress that investigated the effect of situational characteristics related to technological change and/or organisational support.

Study	Situational characteristics	Beneficial	Harmful
Ragu-Nathan et al. (2008)	‘Technostress inhibitors’ (involvement facilitation, literacy facilitation, technical support provision)	X
Wang and Shu (2008)	Perceived organisational support	X
Wang, Shu, and Tu (2008)	Culture of innovation, power centralisation		X
Tarafdar et al. (2011)	Innovation support, Involvement facilitation	X
Ayyagari, Grover, and Purvis (2011)	Technology characteristics (anonymity, complexity, pace of change, presenteeism, reliability, usefulness)		X
Fuglseth and Sørebø (2014)	Technostress inhibitors	X
Tarafdar, Pullins, and Ragu-Nathan (2015b)	Technostress inhibitors	X
Califf et al. (2015)	Technology characteristics (pace of change, reliability, usefulness) Technostress inhibitors	X

Appendix 2

In Table A2, we list the instruments that were used to operationalise the constructs in the online survey that was part of this study and the sources that these instruments were drawn from. For each construct, we list the items and their respective loadings on the construct as well as the coefficient alpha (Cronbach and Meehl 1955) for the construct. In addition, in Table A3, we highlight the descriptive statistics for each indicator in both sub-samples, including its mean, standard deviation, skewness and kurtosis.

Table A2. Overview of constructs and items in the online survey.

Constructs and items^a	Loadings
Organisational climate of innovation (Tarafdar, Tu, and Ragu-Nathan 2010) I1_01-We have a very open communications environment. I1_02-Employees and managers are supportive of each other. I1_03-Employees at all levels are rewarded for learning new skills.* I1_04-Management encourages an experimental mind-set and risk taking.* I1_05-In our organisation, new ideas are easy to implement.	α: .796 .776 .759 – – .580
Perceived uncertainty (Ragu-Nathan et al. 2008) I2_01-There are always new developments in the technologies we use in our organisation. I2_02-There are constant changes in computer software in our organisation. I2_03-There are constant changes in computer hardware in our organisation. I2_04-There are frequent upgrades in computer networks in our organisation.	α: .847 – .739 .844 .723
Perceived unreliability (own development) I3_01-I think that the ICT I use at work are reliable. (-)* I3_02-I think that I am too often confronted with unexpected behaviour of the ICT I use at work (e.g. breakdowns or long response times). I3_03-I think that I lose too much time due to technical malfunctions. I3_04-I think that I spend too much time trying to fix technical malfunctions.	α: .847 – .724 .864 .790
User Satisfaction^b (Bhattacherjee 2001) How do you feel about your overall experience of utilising ICT in connection with your work tasks? I4_01-Very dissatisfied/Very satisfied I4_02-Very displeased/Very pleased I4_03-Very frustrated/Very contented I4_04-Absolutely terrible/Absolutely delighted	α: .934 .749 .926 .896 .857
Job satisfaction (Ragu-Nathan et al. 2008) I5_1-I like doing the things I do at work. I5_2-I feel a sense of pride in doing my job. I5_3-My job is enjoyable.	α: .731 .833 .635 .721
Computer self-efficacy (Compeau and Higgins 1995) I could complete my tasks using new ICTs  …  I6_01- … if there was no one around to tell me what to do as I go.* I6_02- … if I had never used an ICT like it before.* I6_03- … if I had only the software manuals for reference.* I6_04- … if I had seen someone else using it before trying it myself. I6_05- … if I could call someone for help if I got stuck. I6_06- … if someone else had helped me get started. I6_07- … if I had a lot of time to complete the job for which the ICT was provided. I6_08- … if I had just the built-in help facility for assistance. I6_09- … if someone showed me how to do it first. I6_10- … if I had used similar ICT before this one to do the same job.	α: .925 –––.751 .861 .855 .814 .562 .859 .816

^a Items were measured on a 7-point Likert scale with 1 – strongly disagree to 7 – strongly agree. The survey was administered in German, with survey items being translated and backtranslated involving a professional translator and native English and German speakers to ensure content validity. The full list of German items can be obtained upon request from the authors.

^b User Satisfaction was measured using adjective pairs on a 7-point scale, with larger values indicating higher values of user satisfaction.

*Indicates items that were removed due to high cross-loadings and/or low loadings on their factor. Low loadings were defined as being below .500 and high cross-loadings were defined as loadings on other factors that were less than .200 different from the highest loading (Costello and Osborne 2005; Reio and Shuck 2015; Matsunaga 2010).

Table A3. Descriptive statistics for indicators in both sub-samples.

Items	Sample for EFA, N = 564				Sample for CFA and SEM, N = 551
	Mean	Std. dev.	Skewness	Kurtosis	Mean	Std. dev.	Skewness	Kurtosis
I01_01	4.883	1.662	−0.587	−0.380	4.882	1.686	−0.595	−0.345
I01_02	4.723	1.779	−0.517	−0.675	4.784	1.726	−0.517	−0.517
I01_05	3.940	1.739	−0.072	−0.881	4.042	1.732	−0.060	−0.829
I02_02	3.832	1.781	−0.008	−0.962	3.898	1.756	−0.061	−0.948
I02_03	3.569	1.787	0.117	−0.994	3.661	1.719	0.106	−0.802
I02_04	3.704	1.793	0.047	−0.978	3.849	1.813	−0.010	−0.985
I03_02	3.926	1.834	−0.029	−1.033	3.869	1.841	−0.017	−1.014
I03_03	3.814	1.774	0.034	−0.924	3.926	1.876	−0.028	−1.069
I03_04	3.007	1.481	0.855	0.545	2.900	1.418	0.769	0.429
I04_01	5.009	1.366	−0.754	0.475	5.049	1.332	−0.655	0.152
I04_02	4.855	1.232	−0.494	0.251	4.951	1.288	−0.488	0.264
I04_03	4.759	1.254	−0.399	0.227	4.760	1.251	−0.261	−0.077
I04_04	4.869	1.235	−0.402	0.092	4.886	1.231	−0.274	−0.080
I05_01	5.335	1.471	−0.854	0.300	5.403	1.416	−0.857	0.313
I05_02	5.410	1.520	−0.961	0.437	5.448	1.526	−1.024	0.570
I05_03	5.184	1.563	−0.824	0.124	5.303	1.441	−0.830	0.374
I06_04	5.043	1.489	−0.688	0.254	4.967	1.597	−0.771	0.244
I06_05	5.562	1.546	−1.126	0.714	5.599	1.657	−1.299	1.059
I06_06	5.447	1.519	−1.039	0.650	5.448	1.634	−1.170	0.853
I06_07	5.158	1.534	−0.673	−0.022	5.218	1.608	−0.890	0.429
I06_08	4.567	1.555	−0.381	−0.262	4.603	1.659	−0.513	−0.325
I06_09	5.676	1.572	−1.190	0.720	5.615	1.666	−1.304	1.011
I06_10	5.576	1.479	−1.205	1.156	5.479	1.599	−1.184	0.896

Appendix 3

In Table A3, we present the results of the initial EFA, which involved all items that were part of latent variables in the survey. The KMO for this initial EFA was .897 and the cumulative explained variance for the six extracted factors was 58.955%. For each factor, we also list its individual contribution to the overall explained variance and its Eigenvalue. For each item, the loadings on the six factors that were extracted are included and the highest loading is highlighted with a grey background. In this initial EFA, there are some major issues with convergent validity (i.e. items only loading weakly on their intended construct) and discriminant validity (i.e. items with substantial cross-loadings).

In Table A4, we present the results of the final EFA, which a reduced number items that were part of latent variables in the survey. The KMO for this final EFA was .881 and the cumulative explained variance for the six extracted factors was 64.201%. For each factor, we also list its individual contribution to the overall explained variance and its Eigenvalue. For each item, the loadings on the six factors that were extracted are included and the highest loading is highlighted with a grey background. In this final EFA, all items loading strongly on their intended factor and there are no substantial cross-loadings.

Table A4. Results of initial EFA.

Items / Factors	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6
% of explained variance	12.784	3.976	2.500	10.599	24.455	4.642
Eigenvalue	4.164	1.606	1.120	3.608	7.724	1.753
I01_01	.717	−.035	.015	−.056	−.014	.027
I01_02	.808	−.064	−.031	−.075	−.011	.011
I01_03	.392	.240	.028	.007	−.336	.254
I01_04	.425	.140	.070	−.010	−.224	.218
I01_05	.612	.074	−.002	−.002	−.188	.153
I02_01	.250	.010	.072	.023	.401	−.102
I02_02	−.108	.767	.107	.007	.093	−.065
I02_03	−.025	.832	−.036	−.040	.004	−.035
I02_04	.055	.760	−.002	.034	.086	−.078
I03_01	−.444	−.109	.168	−.079	−.138	.062
I03_02	−.014	.072	.711	.001	.077	−.056
I03_03	−.010	−.002	.862	.042	.041	−.018
I03_04	.012	−.002	.785	−.059	−.053	.053
I04_01	.070	.036	−.070	.757	.067	−.059
I04_02	−.034	−.040	.023	.930	−.023	.015
I04_03	−.029	.034	.011	.909	−.058	.018
I04_04	.007	−.026	.021	.860	−.051	.038
I05_01	.676	−.089	.069	.055	.207	−.121
I05_02	.611	−.081	.062	.043	.228	−.101
I05_03	.711	−.026	−.056	.037	.145	−.081
I06_01	.066	−.069	.005	−.053	.041	.719
I06_02	−.024	−.090	.025	.039	.013	.835
I06_03	−.064	.018	−.081	.008	.279	.623
I06_04	−.045	.010	.028	.072	.526	.454
I06_05	−.024	−.038	.040	.017	.828	.060
I06_06	.015	−.029	.043	−.030	.825	.069
I06_07	−.017	.032	.019	−.056	.680	.237
I06_08	.022	.045	−.031	.026	.290	.500
I06_09	−.013	.059	−.003	−.009	.930	−.124
I06_10	.068	.090	−.072	−.056	.797	.029

Appendix 4

In Table A5, we present the main results of the reliability and validity assessment for the constructs in our survey, including their composite reliability, average variance extracted, and the test for the Fornell-Larcker criterion as an indicator for the discriminant validity of our constructs on the right-hand side of the table. As an additional means to demonstrate discriminant validity, we also report the HTMT for each construct pair in Tables A6 and A7.

Table A5. Results of final EFA.

Items / Factors	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6
% of explained variance	5.960	3.314	11.367	15.420	1.942	26.198
Eigenvalue	1.698	1.120	2.983	3.872	0.860	6.365
I01_01	.798	−.052	.039	−.010	−.038	.033
I01_02	.779	−.062	−.023	−.025	.051	.025
I01_05	.547	.110	.019	.052	.058	−.061
I02_02	−.074	.739	.107	.003	−.025	.035
I02_03	.015	.844	−.053	−.048	−.015	−.029
I02_04	.024	.723	.004	.034	.045	.032
I03_02	.036	.046	.714	.003	−.039	.029
I03_03	−.053	−.012	.870	.033	.074	.000
I03_04	.058	.008	.782	−.060	−.023	−.034
I04_01	−.002	.020	−.072	.749	.072	.038
I04_02	.009	−.054	.027	.934	−.052	.012
I04_03	−.048	.044	.007	.894	.035	−.042
I04_04	.045	−.020	.013	.859	−.040	.000
I05_01	−.004	−.022	.054	.001	.841	−.016
I05_02	.068	−.020	.037	.011	.642	.059
I05_03	.107	.055	−.088	−.002	.729	−.022
I06_04	.034	.077	.019	.073	−.023	.753
I06_05	−.053	−.087	.032	.020	.014	.859
I06_06	−.047	−.062	.029	−.031	.058	.853
I06_07	−.009	.029	.022	−.044	−.009	.814
I06_08	.147	.124	−.023	.042	−.068	.566
I06_09	−.055	−.047	−.009	−.003	.007	.857
I06_10	.041	.037	−.080	−.042	.002	.818

Table A6. Reliability and validity assessment.

	CR	AVE	IC	UC	UR	US	JS	CS
Organisational climate of innovation (IC)	0.800	0.573	0.757
Perceived uncertainty (UC)	0.787	0.552	0.228**	0.743
Perceived unreliability (UR)	0.848	0.651	−0.107*	0.484**	0.807
User satisfaction (US)	0.935	0.783	0.483**	−0.026	−0.342**	0.885
Job satisfaction (JS)	0.735	0.490	0.678**	0.105*	−0.231**	0.576**	0.700
Computer self-efficacy (CS)	0.926	0.644	0.132**	0.096	0.044	0.227**	0.202**	0.803

Diagonal values in italics indicate the square root of the AVE. * p < .05, ** p < .01

CR = Composite reliability; AVE = Average variance extracted.

Table A7. Discriminant validity assessment based on HTMT.

	IC	UC	UR	US	JS	CS
Organisational climate of innovation (IC)	–
Perceived uncertainty (UC)	0.266	–
Perceived unreliability (UR)	0.089	0.482	–
User satisfaction (US)	0.487	0.019	0.342	–
Job satisfaction (JS)	0.655	0.125	0.199	0.559	–
Computer self-efficacy (CS)	0.146	0.114	0.064	0.234	0.263	–