Research on IT value: what we have done in Asia and Europe

Introduction

With trillions of U.S. dollars spending on information technology (IT) annually by organizations and individuals worldwide, what value IT can bring to the society is no doubt a big issue in research and practice. Surprisingly, not much has been done in this critical area, especially in Asia and Europe. One possible explanation is that IT value is such a broad term or concept that can encompass different things to different people. Another reason is that the information systems (IS) discipline in Asia and Europe is young compared to that in North America. Although with a tremendous growth in IS research in Asia, particularly in China, in the past decade or so (Chau et al., 2005; Zhang et al., 2006), there is still a long way to catch up.

The theme of the Ninth Pacific Asia Conference on Information Systems (PACIS 2005) is ‘IT and Value Creation’. It focuses on the roles, functions and impacts of information and communication technologies on creating sustainable value, which can be created by a number of innovative business computer applications ranging from efficient management information systems to innovative decision support or e-business technologies. This special section includes three papers that examine the issue from different perspectives: business adoption of B2B exchange, extended use of complex information systems by employees and use of innovative IT by customers in the retail industry. Before introducing these three papers, we would like to present a snapshot of IT value research done in Asia and Europe in the past decade.

In this editorial, we first examine and compare IT value research in Asia and Europe. We chose to use papers published in the proceedings of two major information systems conferences in Asia and Europe, respectively, namely Pacific-Asia Conference on Information Systems (PACIS) and European Conference on Information Systems (ECIS) as the subjects of the examination and comparison because they are the most representative IS conferences and thus provide a good snapshot of IS research undergoing in the regions.

In the next section, we first provide a literature review on IT value research with the objective of ‘setting the stage’ of our subsequent data analysis. With Seddon et al.'s (1999) review article on ‘Dimensions of Information Systems Success’ as a basis, a taxonomy of IT value research is proposed. An examination of the papers published in the conference proceedings of PACIS (from 1993 to 2005) and ECIS (from 2000 to 2005) is then conducted by looking at the dimensions of IT value being studied, research methods, unit of analysis and type of data used, etc. Discussion on what and where we can go in terms of IT value research is then presented. We conclude with a brief description of the papers selected from PACIS 2005 for this special section.

Dimensions of IT value

IT value research has its roots in the IS effectiveness literature (Cronk & Fitzgerald, 1997), in which information is often viewed as the output of an information system. The effectiveness of an information system is then measured in terms of the impact or influence of the output (i.e. information) on the receiver (Shannon & Weaver, 1949; Mason, 1978). Subsequent studies (e.g. DeLone & McLean, 1992; Seddon, 1997; DeLone & McLean, 2003) built on this view of IS effectiveness and proposed four impact or consequence of use measures in their IS success model – individual impact, organizational impact, societal impact and user satisfaction.

Individual, organizational and societal impacts – net benefits

Individual impact refers to the effect of information on the behavior of the recipient, and organizational impact refers to the effect of information on organizational performance (DeLone & McLean, 1992). Other impacts have also been suggested, including inter-organizational and industry impact (Clemons & Row, 1993; Clemons et al., 1993) and societal impact (Seddon, 1997). These impact measures form a continuum and are sometimes referred to as ‘net benefits’ of IT (Seddon, 1997; DeLone & McLean, 2003), which are perhaps the most frequently used measures in IT value research. The positive effects of IT in terms of these benefits (e.g. cost savings, expanded markets, additional sales, etc.) are well-supported and widely documented (DeLone & McLean, 2003).

User satisfaction

Another impact or consequence measure used in IT value research is user satisfaction, referred to as the recipient response to the use of the output of an IS (DeLone & McLean, 1992). However, its role as a separate consequence measure has been controversial. Seddon (1997) argued that user satisfaction is a general subjective evaluation of other consequence measures on a pleasant–unpleasant continuum and should be viewed as proxies for other consequence measures. In other words, user satisfaction should be treated as a general perceptual measure of net benefits of IS use (Seddon, 1997). Given its controversial role, user satisfaction is also included in this study as another dimension of IT value that is separated from net benefits.

Measuring IT value

Besides what constitute the dimensions of IT value, how to measure IT value also poses a number of challenges in IT value research. Among them that were addressed by Seddon et al. (1999) include the stakeholder and the type of system. Drawing from organizational effectiveness and IS effectiveness literature (Cameron & Whetten, 1983; Grover et al., 1996), Seddon et al. (1999) looked into five types of stakeholders; (1) independent observer, (2) individual, (3) group, (4) management or owners and (5) country. For the type of system being evaluated, Seddon et al. (1999) classified different types of system into six categories; (1) an aspect of IT use, (2) a single IT application, (3) a type of IT or IT application, (4) all IT applications used by an organization or sub-organization, (5) an aspect of a system development methodology and (6) an IT function of an organization or sub-organization.

Some other issues in measuring IT value include the level of analysis (individual, group, organization or society) and the type of data (objective or perceptual) (Cameron & Whetten, 1983). For the level of analysis, IT value research in the early 1990s focused mostly on the economic and financial measures of IT value (productivity, economic growth, ROA, ROI, etc.) at the organization level (e.g. Brynjolfsson, 1993; Dos Santos et al., 1993; Hitt & Brynjolfsson, 1996). Subsequent IT value studies took a broader view of IT value and included more measures of IT value at other levels.

For the type of data, IT value can be measured using objective or perceptual data. The advantage of objective measures of IT value is that they tend to be more reliable. Many researchers distrust perceptual measures because people do not necessarily say what they believe or do what they say (Seddon, 1997). However, IT value, just like IS success, involves a value judgment made by an individual from the perspective of some stakeholders and may not always be well-captured. For example, $1 million saving in cost can be significant to one company but insignificant to another. In addition, measures such as user satisfaction are perceptual in nature and cannot be captured objectively.

IT value taxonomy

Based on the review above, especially the works by DeLone & McLean (1992, 2003) and Seddon (1997), this paper reviews the studies on IT value based on the taxonomy as shown in Figure 1. IT value is referred to as the value provided as a consequence of IT use, which includes four major dimensions – user satisfaction, individual impact, organizational impact and societal impact. Furthermore, Cameron & Whetten (1983) raised some important issues in measuring IT value, two of which were addressed by Seddon et al. (1999), that is, the stakeholder's perspective from which IT value is measured, and the type of system being evaluated. This study looks at some other issues in measuring IT value such as unit of analysis, type of data and research method.

Figure 1 A taxonomy of IT value.

Data analysis

We examined all the papers published in the PACIS proceedings and the ECIS proceedings that are available in the AIS e-Library. The collection consists of 876 PACIS papers (1993–2005) and 967 ECIS papers (2000–2005). Among the 1843 conference papers, studies that addressed the value of IT in terms of user satisfaction and net benefits (individual, organizational and societal impacts) were identified. One coauthor and a research assistant reviewed the conference papers independently and then met to resolve disagreements. A total of 41 PACIS papers and 49 ECIS papers were identified. They were further reviewed and categorized in terms of dimension of IT value, research method, unit of analysis and type of data.

The distribution of IT value studies over the years is summarized in Table 1. The interest in IT value studies appeared to subside since the late 1990s and picked up after 2001. For example, the number of IT value studies for both PACIS and ECIS increased from single digit in 2000 (five studies) and 2001 (seven studies) to double digits in 2002 (16 studies), 2003 (17 studies) and 2004 (24 studies). The overall trend appeared to be more prominent for ECIS and less for PACIS (see Figure 2). The overall trend in IT value research may be attributed to the ‘lost of the productivity paradox’ in the mid-1990s (Brynjolfsson & Hitt, 1996), followed by the dot-com boom from 1997 to 2001. The value provided by IT was unequivocal during that period. However, after the dot-com bubble burst in 2001, whether the return of IT justified the investment came into question again, resulting in the reviving interest in IT value research.

Table 1 Number and percentage of IT value studies (1993–2005)

	PACIS			ECIS			Both
Year	ITV	Total	%	ITV	Total	%	ITV	Total	%
1993	2	67	3.0				2	67	3.0
1995	0	86	0.0				0	86	0.0
1997	5	84	6.0				5	84	6.0
2000	0	86	0.0	5	190	2.6	5	276	1.8
2001	4	88	4.5	3	123	2.4	7	211	3.3
2002	7	75	9.3	9	151	6.0	16	226	7.1
2003	4	133	3.0	13	177	7.3	17	310	5.5
2004	12	144	8.3	12	173	6.9	24	317	7.6
2005	7	113	6.2	7	153	4.6	14	266	5.3
2000–2005	34	639	5.3	49	967	5.1	83	1606	5.2
1993–2005	41	876	4.7	49	967	5.1	90	1843	4.9

Figure 2 Number of IT value studies (2000–2005).

The geographic distribution of the IT value studies were also examined and summarized in Table 2. As the most active researchers in the Asia-Pacific region (Chau et al., 2005), Australian researchers contributed in approximately one-fifth of the IT value studies in PACIS (22.0%) and one-fourth of the IT value studies in ECIS (24.5%). U.S. researchers were the highest and third highest contributors in PACIS (24.4%) and ECIS (12.2%), respectively. Taiwan researchers were the third highest contributors in PACIS (19.5%) and U.K. researchers were the second highest contributors in ECIS (16.3%).

Table 2 Geographic distribution of authors

	PACIS		ECIS		PACIS & ECIS
Asia-Pacific
Australia	9	22.0%	12	24.5%	21	23.3%
Taiwan	8	19.5%	0	0.0%	8	8.9%
Hong Kong	5	12.2%	1	2.0%	6	6.7%
Korea	5	12.2%	0	0.0%	5	5.6%
Singapore	2	4.9%	3	6.1%	5	5.6%
New Zealand	3	7.3%	1	2.0%	4	4.4%
Japan	3	7.3%	0	0.0%	3	3.3%
China	2	4.9%	0	0.0%	2	2.2%
Malaysia	0	0.0%	1	2.0%	1	1.1%
Thailand	1	2.4%	0	0.0%	1	1.1%
Europe
United Kingdom	0	0.0%	8	16.3%	8	8.9%
Germany	2	4.9%	4	8.2%	6	6.7%
Italy	0	0.0%	4	8.2%	4	4.4%
The Netherlands	0	0.0%	4	8.2%	4	4.4%
Finland	0	0.0%	2	4.1%	2	2.2%
Greece	0	0.0%	2	4.1%	2	2.2%
Ireland	0	0.0%	2	4.1%	2	2.2%
Spain	0	0.0%	2	4.1%	2	2.2%
Switzerland	1	2.4%	1	2.0%	2	2.2%
Denmark	1	2.4%	0	0.0%	1	1.1%
Norway	0	0.0%	1	2.0%	1	1.1%
Sweden	0	0.0%	1	2.0%	1	1.1%
North America
United States	10	24.4%	6	12.2%	16	17.8%
Canada	1	2.4%	2	4.1%	3	3.3%

The methods used by the studies on IT value for PACIS and ECIS were compared and summarized in Table 3. As the most popular method in IS research (Claver et al., 2000), survey accounted for 40% of IT value studies in PACIS and ECIS. Qualitative methods, including case study, were the second most popular and accounted for 32% of IT value studies. Interestingly, concept description accounted for 20% of the IT value studies in PACIS, while it was used in only 8% of all PACIS studies (Chau et al., 2005) and 8% of IT value studies in ECIS.

Table 3 Distribution of the studies by research method

Research method	PACIS		ECIS		PACIS & ECIS
Survey	16	39%	19	39%	35	40%
Case and qualitative	10	24%	19	39%	29	32%
Concept description	8	20%	4	8%	12	13%
Secondary data	2	5%	5	10%	7	8%
Problem solving and modeling	3	7%	1	2%	4	4%
Experiment	2	5%	1	2%	3	3%
Total	41	100%	49	100%	90	100%

IT value research in the early 1990s focused on the business aspects of IT value using objective financial measures such as firm value and ROI. As a result, the unit of analysis in those studies was typically organization. Table 4 summarizes the IT value studies in PACIS and ECIS in terms of the unit of analysis. As expected, the majority of the IT value studies in PACIS (69%) and ECIS (88%) were conducted and analyzed at the organization level. While there were similar amount of studies in PACIS (7%) and ECIS (4%) at the industry or society level, there was a significantly higher percentage of PACIS studies (24%) than ECIS studies (8%) at the individual level.

Table 4 Distribution of the studies by unit of analysis

Unit of analysis	PACIS		ECIS		PACIS & ECIS
Individual	10	24%	4	8%	14	16%
Organization	28	69%	43	88%	71	78%
Industry/society	3	7%	2	4%	5	6%
Total	41	100%	49	100%	90	100%

Table 5 categorizes the IT value studies in terms of user satisfaction, individual impacts, organizational impacts and industry/societal impacts. Given that most IT value studies focused on the business value of IT at the organization level, it is not surprising that most of the impacts being studied were organizational (84%). Approximately one-fifth of the studies (21%) focused on the individual impacts, 13% of the studies focused on user satisfaction and 7% of the studies focused on industry/societal impacts.

Table 5 Distribution of the studies by dimension of IT value

Dimension of IT value	PACIS		ECIS		Both
User satisfaction	6	15%	6	12%	12	13%
Individual impact	9	22%	10	20%	19	21%
Organizational impact	33	80%	43	88%	76	84%
Industry/societal impact	4	10%	2	4%	6	7%

Table 6 categorizes the IT value studies in terms of whether the measures were objective or perceptual. Examples of objective measures include accounting and financial indicators such as revenues, costs, market value, ROI, etc. Examples of perceptual measures include increased decision quality, better alignment with business strategy, etc. Measures such as benefits and costs are treated as perceptual measures if they are self-reporting measures from the perspectives of the stakeholders. As shown in Table 6, the majority of studies evaluate IT value use perceptual measures for both PACIS (61%) and ECIS (76%). More studies in PACIS tend to use objective measures (27%) than those in ECIS (18%).

Table 6 Distribution of the studies by data type

Data type	PACIS		ECIS		Total
Perceptual	25	61%	37	76%	62	69%
Objective	11	27%	9	18%	20	22%
Both	5	12%	3	6%	8	9%
Total	41	100%	47	100%	90	100%

Conclusions and papers in this special section

IT is often treated as a strategic investment for creating sustainable advantages. Therefore, measuring IT value is an important issue for IS research. There are two major trends for measuring IT value: one is to measure IT adoption, and the other is to measure IT impacts at different levels. In this editorial, we summarize previous research frameworks in IT value and compare papers published in PACIS and ECIS to find the following three points:

• IT value research appeared to go down in late 1990s and picked up after the dot-com burst. This indicates that the value of IT is under scrutiny only when it is in trouble.

• ECIS tends to focus heavily on issues at the organizational level while PACIS has more at the individual level IT value.

• There seems to be a general shift from using objective measures to perceptual measures to study IT value. The difference also exists between publications in Asia Pacific and Europe. PACIS authors tend to use more objective measures.

In this special section, three papers presented at PACIS 2005 have gone through the rigorous review process and been selected from more than 100 papers as representative of recent IT value research from three different countries.

The study by Quaddus & Hofmeyer (2007)explored the adoption behavior of small businesses in Western Australia in relation to B2B Trading Exchanges. It proposed an integrated framework that includes external industrial factors, contextual factors, external control factors and cognitive responses to measure user behavior and intention to adopt the technology. The findings revealed that external influences raise the awareness of an innovation. This awareness leads to the evaluation of the perceived direct and indirect benefits and a positive evaluation leads to a positive attitude towards the innovation. In turn, this leads to the intention to adopt. The findings confirm that external, belief, contextual and control factors drive the attitude towards B2B Trading Exchanges.

The paper by Hsieh & Wang (2007) addressed the issue of continuing use of complex information systems. Once an information system has been adopted, continuous use is essential for realizing its value. In this paper, they proposed a synthesized model of IS Continuance (ISC) model and Technology Acceptance Model (TAM), and conducted a field survey in a large manufacturing company that had successfully implemented a popular ERP solution for more than 2 years. The results indicate that the synthesized model explained higher variances than these two models alone.

The paper by Loebbecke (2007) examined the impact of adopting content integration technologies in a supermarket. The particular case under study was the METRO Group in Germany. The paper is different from the previous two in its case-based research method. The results indicate that the use of state-of-the-art technology, such as RFID (Radio Frequency Identification) in supermarkets has been successful in the case.

These three papers provide three different types of research. This editorial introduction supplements these papers by providing a general framework of IT value. Together, they provide a snapshot of IT value research in PACIS.

Additional information

Notes on contributors

Patrick Y K Chau

About the authors

Patrick YK. Chau is Professor of Information Systems at the School of Business, The University of Hong Kong. He received his Ph.D. in business administration from the Richard Ivey School of Business, The University of Western Ontario, Canada. His research interests include issues related to IS/IT adoption and implementation, information presentation and model visualization, and electronic commerce. He has papers published in major IS journals including MIS Quarterly, Communications of the ACM, Journal of Management Information Systems, Decision Sciences, European Journal of Information Systems and others. He serves on the editorial board of a number of IS journals including Decision Sciences, International Journal of Electronic Commerce, and Information and Management.

Kevin K Y Kuan

Kevin KY. Kuan is an Assistant Professor at the Department of Information Systems, City University of Hong Kong. He received his doctoral degree from the Ross School of Business, University of Michigan. His research interests include human–computer interaction, information presentation and decision making, IS/IT adoption and implementation, and reputation systems.

Ting-Peng Liang

Ting-Peng Liang is Chair Professor of Information Systems and Director of Electronic Commerce Research Center of National Sun Yat-sen University. He received his Ph.D. in Decision Sciences from the Wharton School of the University of Pennsylvania. His research interests include decision support systems, knowledge management and electronic commerce. He has published in journals such as MIS Quarterly, Management Science, Journal of MIS, Decision Sciences, Decision Support Systems, among others. He serves on the editorial board of many information systems journals.