Reliability of information systems in organization in the context of banking sector: Empirical study from Poland

Abstract

The notion of information systems (IS) reliability in organization is discussed in this article. It contains critical review of the literature, definition of IS reliability and brief overview of the theoretical model of IS reliability in organization developed by the author (including system reliability, information reliability, service reliability and usage reliability). The potential use of the model of IS reliability is demonstrated as a part of empirical research devoted to the analysis and evaluation of IS by the information technology (IT) specialists from inside and outside of the banking sector. The analysis is based on survey conducted among more than 200 IT specialists from and outside of the banking sector in Wroclaw, Lower Silesia, Poland. Banking sector is given as an example due to its uniqueness—extreme sensitivity to IS reliability and accuracy. The analysis of IS reliability in organization is supplemented by the comparison of the way of understanding the notion by the two groups of specialists in order to verify whether the IS reliability is understood in the same way by industry sectors differing in terms of their sensitivity to IS support.

Keywords:

• IS user experience
• IS reliability
• banking sector
• empirical research

PUBLIC INTEREST STATEMENT

This article describes the notion of the reliability of information systems (IS) used in modern organization, considering the banking sector as a business context especially sensitive to information technology (IT) support. The theoretical model of IS reliability developed by the author (including system reliability, information reliability, service reliability and usage reliability) is described. The potential use of the model of IS reliability is demonstrated as a part of empirical research devoted to the analysis and evaluation of IS by the IT specialists from inside and outside of the banking sector. The analysis is based on survey conducted among more than 200 IT specialists from and outside of the banking sector in Wroclaw, Lower Silesia, Poland. The analysis of IS reliability in organization is supplemented by the comparison of the way of understanding the notion by the two groups of specialists in order to verify whether the IS reliability is understood in the same way by industry sectors differing in terms of their sensitivity to IS support.

1. Introduction

Nowadays, when almost every organization operates using some kind of information systems (IS), its reliability seems to be one of the key factors influencing organization’s competitive advantage from using it. Moreover, while facing new global Internet threats (such as WannaCry ransomwares and massive hacker attacks), maintaining the reliability, security and accuracy of the IS in organization is becoming one of the biggest challenges of IS management (Islam, Manivannan, & Zeadally, 2016). The concept of reliability, which is a main topic of this article, is largely related to the IS management and everyday use of IS in the organization. This is mainly because in modern organization, when a critical system is unavailable in any way, it causes significant and immediate business impact (Tsubira & Mulira, 2004). It not only affects the business operation but as such has an outsized effect on the business’ perception of the IS in organization and undermines its potential for supporting business processes. It also affects the ability to be competitive on the market and potentially may affect the customers’ satisfaction (Bieńkowska, Kral, & Zabłocka-Kluczka, 2017; Chan, 2000; Tusubira & Mulira, 2004; Wade & Hulland, 2004). Hence, it makes reliability a critical issue for modern managers and modern businesses. Moreover, it is an issue especially important for sectors, which almost entirely depend on IS to provide the possibility for everyday business operation (Dehning & Richardson, 2002, Tworek, 2018). One of the most obvious examples of those sectors is a banking sector. Electronic banking systems study will be used as an example of IS in organizations especially sensitive to IS reliability issues, to better explain its importance and propose the verified model of factors explaining this notion.

The main aim of this article is to demonstrate the potential use of IS reliability model for analysis and evaluation of IS in organization, and based on that answer, a valid research question is as follows: Is the IS reliability understood in the same way by industry sectors differing in terms of their sensitivity to IS support? The first part of this article presents the theoretical background together with a brief overview of the IS reliability model developed by Tworek (2016, 2018) and verified empirically (Tworek, 2018). It is devoted to addressing the gap, which exists in the current literature concerning the IS in organization: It is no longer enough to analyse the IS in organization just based on its existence and use. The notion of IS reliability is a proposition for a new, complex way of analysing and evaluating IS in organization. The second part of this article presents an empirical research performed among 200 information technology (IT) specialists from inside and outside of the banking sector and is devoted to the verification of a research question indicated above. The results analysis contains a comparison of the understanding of IS reliability notion by the two groups of IT specialists, which performed based on empirically verified model (Tworek, 2018) may be the basis for valid conclusions.

2. IS reliability theoretical concept

The reliability of IS in organization is understood as a measurable property of IS, useful for its control and management, identifying its quality level and pointing out potential problems (Zahedi, 1987), and it is directly linked to the efficiency of IS components, especially those critical to its proper operations (Tworek, 2018). Therefore, it can be said that IS reliability in organization is a notion build by factors connected to three different IS theories. First one is DeLone and McLean (2003) success model, second one is Lyytinen (1987) model indicating four types of IS failure and third one is technology acceptance model (TAM; Davis, 1985).

The IS success model was developed by DeLone and McLean (2003) and to this day is the one of the most recognized and comprehensive one. The first version was published by the authors in 1992. This model indicated that the success of IS in the organization is caused by the positive impact of IS on the entire organization, which has its source in the positive IS impact on individual employees. The impact of IS on users is, according to the authors, dependent on the actual use of it during everyday work, and this, in turn, is conditioned by the quality of the system itself and the information contained in it.

The final model of IS success is constructed by the quality of the system, the quality of information contained in it and the quality of the support service of this system. These factors directly affect the level of actual use of IS by users and their satisfaction with it. It is worth noting here that the intention of use, actual use and satisfaction from it is directly linked to the concept of IS acceptance (TAM; Davis, 1985). Thus, it is reasonable to analyse the above three factors together with the level of IS acceptance in the organization. Figure 1 shows the elements of IS success model and TAM merged together building four elements of proposed IS reliability model.

Figure 1. DeLone and McLean model, TAM and IS failures embedded on IS reliability types.

Source: Own work.

In the context of the IS reliability, it seems also crucial to identify the sources of typical IS failures in the organization in the context of IS success model understood in this way. Lyytinen and Hirschheim’s (1987) indicated that there are four types of IS failures:

1. IS does not meet the pre-defined requirements before starting its implementation;

2. IS is insufficiently efficient;

3. IS users are unsatisfied with the usability;

4. IS is unable to meet the users’ information requirements.

2.1. The model formulation

The analysis of the above four cases in the context of the updated DeLone and McLean (2003) model and TAM (Davis, 1989) clearly shows that each of them is directly and indirectly connected with one or many factors indicated by DeLone and McLean and is building one or more aspects of IS reliability in organization. Therefore, those three concepts embedded on each other were the basis for formulating a theoretical model of IS reliability in organization.

Moreover, in order to fully develop the notion of IS reliability, it was crucial to identify factors that are constructs for each of the four identified variables proposed in the IS reliability model (see Figure 1). To identify all of them, the search of articles published from 2000 to 2018, with keywords “IS in organization” and “measurement,” was conducted with EBSCO and ProQuest databases. From all available publications, those concerning lists of factors describing IS in organization were purposefully selected. Based on those research (Finne, 2005; Irani, 2002; Palmius, 2007), all factors potentially related to IS reliability in the context of above-mentioned three theories were identified and assigned to proposed four variables.

Based on that, model of IS reliability in organization has been developed by the author, and detailed description is published by Tworek (2016, 2018). The model is presented in Figure 1. The reliability of IS in organization consists of four elements: reliability of information included in IS in organization, reliability of support services offered for IS in the organization and reliability of system itself, which also includes the reliability of usage of this system (based on the fact that according to Finne (2005) and Palmius (2007), usability is a notion embedded in the system quality itself). Each element is built by the comprehensive list of items identified based on literature analysis. The final version of a theoretical model is shown in Figure 2 and was presented in Tworek (2018).

Figure 2. Model of reliability of IS in organization.

Source: Tworek (2018).

3. Electronic banking systems field of study

Electronic banking is usually analysed from the perspective of its adoption and acceptance among clients. Hoehle and colleagues (2012) performed a comparative analysis of 247 articles from known, peer-reviewed journals, covering topics related to electronic banking use. They concluded that there are five typical theoretical lenses used to analyse electronic banking: diffusion of innovation (DOI), theory of reasoned action (TRA), TAM, theory of planned behaviour (TPB) and inhibitors.

All those articles were dedicated to analyse the adoption, utilization and resistance of electronic banking. However, methodologies significantly differed from each other. Moreover, there is no statistically verified model listing closed group of factors describing electronic banking systems in organization. Hoehle, Scornavacca and Huff (2012) identified 26 different characteristics analysed in different approaches (see Table 1). Nevertheless, those characteristics are interdependent, influence one another and some of them describe the same feature using a different name. Moreover, in most cases, those characteristics are used to build adoption and acceptance models of electronic banking systems. Moreover, some most recent articles propose a shorter, coherent list of characteristics developed to analyse the adoption of electronic banking (e.g. Lichtenstein & Williamson, 2006).

Table 1. Characteristics of electronic banking systems

Theoretical constructs	Article count	Theoretical constructs	Article count
Relative advantage	51	Consumer and bank loyalty	25
Compatibility with consumer values/culture	26	Confirmation	7
Complexity	39	Costs associated with use	25
Trialability	17	Fun	6
Observability	20	Privacy	15
Attitude towards e-banking	45	Reliability	28
Subjective norm	28	Risk	42
Perceived usefulness	58	Satisfaction	33
Ease of use	97	Security	63
Behavioural control	10	Self-efficacy	33
Awareness	15	Service quality	33
Accessibility and convenience	37/41	Trust	31
Habit	7	Resistance	12

Source: Hoehle et al. (2012).

However, almost none of those articles covers the topic of growing importance in the light of recent event around the world—reliability. It is 1 of the 26 characteristics of electronic banking identified by Hoehle et al. (2012). However, they stated that only 28 out of 247 analysed articles mentioned reliability of electronic banking as one of the characteristics that is gaining value in recent years. It is also mentioned as one of the characteristics in articles not included in Hoehle et al. (2012) survey, for example, Lichtenstein and Williamson (2006). Moreover, in all those articles, reliability is mentioned only as one of many characteristics explaining some other notion concerning electronic banking. There is no clear definition of this notion, and there is no comprehensive research available concerning its nature, value for electronic banking and factors influencing it.

After 30 years of electronic banking, the focus should shift from analysing only its adoption and acceptance (because based on the literature and steadily growing number of users, it can be assumed that those steps already occurred) to analysing the reliability and its importance for both banking institutions and their clients—it seems to be a cognitive gap (Hutchinson & Warren, 2003). Moreover, it is one of the issues important from the point of view of business sustainability—broadly covered in the literature in last couple of years. The authors recognize the growing importance of sustaining reliable elements of every organization in order to survive on the market (Longoni, Golini, & Cagliano, 2014). In case of electronic banking, IS is an important factor for sustainability of organizations from banking industry.

4. Research methodology

The survey was conducted in order to identify crucial factors that can be used to describe four variables that construct IS reliability in organization and those which are perceived as not connected with this notion. The survey was conducted in April 2017, among IT specialists working in various companies located in Wroclaw, Poland, using online survey service: SurveyMonkey. This method of gathering data was used do to its high response rate among specialists used to work with IT every day. As the study was intended to be a pilot study (to initially verify the developed model which was afterwards verified in various business contexts (e.g. Tworek, 2018) as a part of the main study using survey corrected based on the results obtained from this pilot study), no sampling technique ensuring representativeness of the sample was used. The survey was sent to IT specialists working in three different international centres of excellence from banking sector located in Wroclaw, Poland, and to three different branches of global corporations outside of banking sector located in Wroclaw, Poland. A total of 195 valid responses were collected, which was enough to form initial conclusions concerning the given topic. Since the responses were collected using properly prepared form, the online system counted only those fully and correctly filled in. Responses were divided into two groups: first group (91 responses): IT specialists from the banking sector and second group (104 responses): IT specialist from outside of the banking sector.

Respondents were asked to rank the listed factors by their importance for reliability of system, usage, information and service, where 1 was considered as the rank for the most important factor (list of items is presented in Appendix A). Also, the factors might be left without assigned rank, which meant that, according to the respondent, it is considered as not connected with the notion of IS reliability in organization.

There were two main goals of this research. First one was to develop a comprehensive definition of reliable IS in organization, which is crucial for any further research on the subject and verify the initial concept of the model. The definition of IS reliability offered by Zahedi (1987) is outdated, and the developed model (Tworek, 2018) may offer some new insights allowing for better understanding of the notion. Second one was to identify the factors, which in the opinion of IT specialists are the most important factors in building the concept of IS reliability in organization. It allows for the comparison of the differences in understanding the reliability of IS inside and outside of banking industry. Hence, it allows to propose the verified model of factors explaining the notion of reliability of electronic banking systems and underline the factors considered as especially important by IT specialists from the banking sector. It also aims at answering the proposed research question: Is the IS reliability understood in the same way by industry sectors differing in terms of their sensitivity to IS support?

Moreover, there was an important conclusion coming from this pilot study, which was the source of changes in survey used afterwards in the main study. The items building system reliability, usage reliability, information reliability and service reliability were given to respondent without explanation, which caused some misunderstandings and might be a cause for improper responses. It should be noted that in further research, the explanation was included, and feedback obtained from this pilot study was very useful in this way.

5. Empirical research results

The results of the survey are presented below using a web graph and table for each group of variables. They contain information concerning the average of ranks assigned by the two groups of respondents to each variable and the standard deviation for this average (which offers additional information underlining in which part of ranking the variable was usually put). Both groups of respondents considered all elements as important parts in building the notion of IS reliability in organization. Hence, the analysis of differences in perception concerning system reliability, information reliability, service reliability and usage reliability was not possible.

First of all, all factors listed to describe each of IS reliability elements were considered as important and related to the subject. Only 19 out of 195 respondents left several factors unranked, and each of them left different factors. Hence, there was no basis for eliminating any factors from the analysis.

Second of all, there are some differences between responses gathered among specialists from and outside of the banking sector. However, the differences concern the importance of each factor within the selected four elements in building IS reliability, not the notion of IS reliability itself.

5.1. System reliability

The survey results concerning system reliability are presented in Table 2, and web graphs for each group of respondents are presented in Figure 3.

Table 2. Average ranks and standard deviation for system reliability variables

Banking sector			Other sectors
	Average	Standard deviation		Average	Standard deviation
Security	3.29	2.16	Security	2.85	2.15
Hardware stability	4.00	2.12	Availability	4.15	2.55
Availability	4.35	3.31	Compatibility	4.62	1.50
Adaptability	4.85	1.88	Usability	4.77	2.40
Replaceability	5.31	2.83	Hardware stability	5.15	2.04
Hardware performance	5.31	1.86	Adaptability	5.38	2.28
Compatibility	5.77	1.86	Replaceability	5.54	2.89
Usability	5.96	2.49	Hardware performance	6.08	2.07
Saliency	6.17	2.87	Saliency	6.46	3.17

Figure 3. Web graphs of average ranks assigned for system reliability variables by specialists from the banking sector (left graph) and by specialists outside of it (right graph).

In case of software system reliability itself, both groups of specialists specified “security” as the most important factor. Moreover, the standard deviation for this factor is in both cases around 2.15. It means that in most responses, security was ranked in top 3 most important factors explaining IS system reliability. Also, there is also a similar tendency in case of the least important factor—“saliency” was ranked as last by both groups. One of the most visible differences is considering “hardware stability,” which among IT specialists from the banking sector was ranked as second most important factor, while it was not considered as important by others (ranked as fifth on average). In additional comments, several specialists from the banking sector mentioned that it is directly connected with availability of the IS, which was on average ranked as third factor by them and second by specialists from outside of the sector.

Therefore, it seems that in case of reliability of IS itself, security is the most important factor influencing it. The results allow the conclusion that availability of the system, important as it is, is not relevant when system is not secure. Especially in the banking sector, systems are put offline whenever the real threat to their security arises (Hutchinson & Warren, 2003).

5.2. Information reliability

The survey results concerning information reliability are presented in Table 3, and web graphs for each group of respondents are presented in Figure 4.

Table 3. Average ranks and standard deviation for information reliability variables

Banking sector			Other sectors
	Average	Standard deviation		Average	Standard deviation
Accuracy	3.03	2.33	Accessibility	2.92	1.87
Accessibility	3.05	1.55	Portability	3.54	2.15
Relevance	3.92	1.74	Searchability	3.69	1.64
Searchability	4.08	1.83	Relevance	3.92	1.65
Movability	4.34	2.18	Accuracy	4.00	2.12
Portability	4.66	2.10	Achievability	4.92	1.55
Achievability	4.92	1.27	Movability	5.00	2.12

Figure 4. Web graphs of average ranks assigned for information reliability variables by specialists from the banking sector (left graph) and by specialists outside of it (right graph).

Information reliability is the factor in which there are major differences between results from the two analysed groups. IT specialists from the banking sector ranked “accuracy” as the most important factor concerning the reliability of information in the IS and “accessibility” was ranked as second. For IT specialists from outside of the banking sector, “accessibility” was the most important factor, and “accuracy” was usually not even in top 3 factors, and it was ranged as fifth factor on average. Therefore, it shows that there are big differences in perception of information reliability by specialists from and outside of the banking sector. Those coming from banking sector understand and underline the importance of data accuracy. It seems logical that accuracy is considered important in case of financial information. Moreover, considering the case of the least important factors, similarly huge differences occur. Fixed character of IS in banking sector (Hutchinson & Warren, 2003) allows IT specialists to focus less on factors such as portability (which was even left without rank by few respondent from this group), when specialists from outside of the banking sector ranked “portability” as second most important factor. Hence, this part of the survey shows that there are differences in perception of concepts in building IS reliability, depending on sector in which the company is operating and the specificity of its context. Therefore, it might be concluded that the understanding of information reliability differs significantly depending on the organization business context.

5.3. Usage reliability

The survey results concerning usage reliability are presented in Table 4, and web graphs for each group of respondents are presented in Figure 5.

Table 4. Average ranks and standard deviation for usage reliability variables

Banking sector			Other sectors
	Average	Standard deviation		Average	Standard deviation
Responsiveness	2.69	1.69	Responsiveness	2.85	1.62
Error-proneness	3.00	1.37	Acceptance	3.38	1.79
Efficiency	3.69	1.27	Efficiency	3.54	1.79
Memorability	3.77	1.86	Learnability	3.62	1.79
Acceptance	3.85	1.80	Memorability	3.62	1.65
Learnability	4.00	1.81	Error-proneness	4.00	1.42

Figure 5. Web graphs of average ranks assigned for usage reliability variables by specialists from the banking sector (left graph) and by specialists outside of it (right graph).

In case of usage reliability, both groups rank “responsiveness” (system response time) as the most important factor concerning this notion. However, it is interesting that “error-proneness” is considered as second most important factor by specialists from the banking sector, while it usually ranks as the least important one by specialists from outside of the banking sector (and in some cases, it was even left without rank). However, it is logical that in case of financial data, errors may cause problems much more significant and impossible to retract than in any other sector of IS usage. Moreover, IS acceptance, notion well-known in the literature and considered as an important characteristic of any IS (what was confirmed among specialists from outside of the banking sector), is not considered as an important factor in building the IS reliability by specialists from the banking sector. It may be because in this sector, people are generally doomed to work with the given IS, and it is not easy to change it or even improve it. Due to a lot of legal restrictions, it is very hard to make any changes in IS in banking sector, so people, who do not accept the system, are usually leaving due to high dissatisfaction or learning to “like what they have” (Jayawardhena & Foley, 2000).

5.4. Service reliability

The survey results concerning service reliability are presented in Table 5, and web graphs for each group of respondents are presented in Figure 6.

Table 5. Average ranks and standard deviation for service reliability variables

Banking sector			Other sectors
	Average	Standard deviation		Average	Standard deviation
Responsiveness	2.77	1.58	Responsiveness	2.69	1.44
Availability	2.85	1.76	Availability	3.08	1.65
Quality	3.15	1.47	Assurance	3.15	1.41
Fail rate	3.62	1.83	Quality	3.23	1.59
Assurance	4.23	1.26	Empathy	4.08	2.03
Empathy	4.38	1.60	Fail rate	4.77	1.13

Figure 6. Web graphs of average ranks assigned for service reliability variables by specialists from the banking sector (left graph) and by specialists outside of it (right graph).

Service reliability, understood as the reliability of support services available for users, is the factor, in which the “responsiveness” was also ranked as the most important factor by both groups of respondents. However, it is important to remember that responsiveness in case of IS services concerns the support services, and it is a different factor than the “responsiveness” mentioned in the description of previous factor (when it characterized system response time). Moreover, “fail rate” was usually ranked as the least important factor (standard deviation was comparatively small, only 1.13) among specialists from outside of the banking sector, while specialists from this sector usually ranked it in the middle of the list.

6. Conclusions

The notion of IS reliability was analysed during the performed empirical research, and it can be concluded that the model of IS reliability developed by Tworek (2018) can be treated as a proposition for a new, complex way of analysing and evaluating IS in organization. The results of analysis showed the comparison of the understanding of IS reliability notion by the two groups of IT specialists, which performed based on empirically verified model (Tworek, 2018) is the basis for some valid conclusions. First of all, the empirical study showed that in most cases, IS reliability in organization is perceived as a notion connected with IS security, availability and responsiveness. Reliable IS in organization can be characterized as follows:

• IS which has a short response time, high availability connected with high security. However, the security should not be strained by the need for the availability.

• IS in which information is easily accessible, and the accuracy of them is guaranteed.

• IS which is accepted by its users and easy to use.

• IS which has a responsive and available support services.

Therefore, it can be concluded that reliable IS in organization is the IS which is secure, available, accepted and has a short response time and responsive support service.

Moreover, differences between ranks assigned by the two groups of specialists completing the survey showed that there are differences in perception of concepts in building IS reliability, depending on sector in which the company is operating and the specificity of its context. Hence, it can be concluded based on obtained results that the IS reliability is not understood in the same way by industry sectors differing in terms of their sensitivity to IS support. This conclusion is drawn based on differences between specialists’ perception from inside and outside of the banking sector (used as an example of sector sensitive to IS support). The research shows that, on one hand, specialists from the banking sector consider some elements in building IS reliability as more important than specialists from different sectors. This is mainly shown in case of information reliability, where the “accuracy” is labelled as the most important variable—which is understandable considering the business context explained in the first part of this article (Hutchinson & Warren, 2003). On the other hand, issues such as ease of use (considered as a part of TAM and IS acceptance (Davis, 1985)) were included as important factors by specialists outside of the banking sector, even though they have not proven to be significant for specialists from the banking sector. However, as it was mentioned earlier, this sector is not a dependable example in this case, due to fixed nature of systems and a lot of restrictions for its architecture (Hutchinson & Warren, 2003).

Therefore, it is important to remember that this and every other empirical study concerning the notion of IS reliability in organization can be the basis for conclusions limited only to the analysed business context (Tworek, 2018). Especially in this case, when the study was performed as a pilot study and representativeness of the sample was not secured, it is important to underline that obtained conclusions should be verified and have a character of an initial findings.

Hence, the main conclusion is that IS reliability in organization is a notion that has a definition of the more general nature, but the perception of this notion and the detailed theoretical model is highly sensitive to the context. Moreover, the proposed concept of IS reliability is addressing the identified gap, which exists in the field of systems analysis in organizations. The notion of IS reliability was used with success among the tested group as new, complex method of analysing and evaluating an IS in organization. Therefore, after further verification (performed already, for example, in Tworek, 2018), it has a potential to be used as an evaluation tool for IS used in organizations and for the analysis of the notion of IS value for organizations.

Additional information

Funding

The author received no direct funding for this research.

Notes on contributors

Katarzyna Tworek

Katarzyna Tworek, PhD, works as an assistant professor at Wrocław University of Science and Technology, in the Department of Organizational Management and Development. Her PhD concerned the influence of information technology on organizational structure. Currently, she is working on a model of information systems reliability in organization.

Appendix A

The questionnaire measuring the information systems (IS) reliability

1. Please rate the below mentioned features of information systems in your company:

• usability

• security

• hardware stability

• availability

• replicability

• saliency

• compatibility

• hardware performance

• adaptability

(All items were rated, where 1 = the lowest rate)

2. Please rate the below mentioned features of information systems usage in your company:

• learnability

• memorability

• efficiency

• error-proness

• acceptance

• responsiveness

(All items were rated, where 1 = the lowest rate)

3. Please rate the below mentioned features of information included in information systems in your company:

• accessibility

• searchability

• accuracy

• relevance

• achievability

• portability

• movability

(All items were rated, where 1 = the lowest rate)

4. Please rate the below mentioned features of support services for information systems in your company:

• assurance

• empathy

• responsiveness

• fail rate

• quality

• availability

(All items were rated, where 1 = the lowest rate)