Toward better intelligent learning (iLearning) performance: what makes iLearning work for students in a university setting?

ABSTRACT

We explored the critical factors associated with iLearning that impact students’ learning performance and identified the factors with a notable influence to help managers in higher education institutions increase the effectiveness of iLearning for students. We initially synthesised 4 main dimensions (including 26 criteria): performance expectancy, lecturers’ influence, quality of service, and personal innovativeness. Subsequently, we conducted surveys in two stages. First, by studying a group of students with experience using iLearning at Taiwanese universities, we extracted 5 critical dimensions (including 18 criteria) through a factor analysis. Second, by studying a group of senior educators and practitioners in Taiwan, we prioritised the dimensions and criteria through the analytic hierarchy process (AHP). We found that performance expectancy is the top critical dimension, and the top five critical criteria pertain to enhancing the learning performance, increasing the learning participation, altering learning habits, ensuring access at all times, and enabling prompt use of learning resources. Moreover, we recommend several suggestions for the relevant managers to enhance the students’ iLearning performance.

KEYWORDS:

• Intelligent learning (iLearning)
• higher education
• university
• factor analysis (FA)
• analytic hierarchy process (AHP)

1. Introduction

Technological advancement has significantly changed the way in which students learn and has challenged traditional pedagogy in higher education (Rossing et al. 2012; Thongsri, Shen, and Bao 2019). To learn, students in the past had to attend face-to-face classes or read books (Summers, Waigandt, and Whittaker 2005). However, in recent years, an increasing number of different ways to learn have emerged, for example using Facebook, iTunes, and podcasts, or registering for courses provided by online course venders such as Coursera and Udemy. In other words, students can learn using portable devices such as mobile phones and laptops (i.e. mLearning; e.g. Al-Emran, Arpaci, and Salloum 2020; Irwin et al. 2012; McGarr 2009; McKinney, Dyck, and Luber 2009). Many studies have found that at universities, mLearning is a key way to learn and teach, and this mode contributes to the students’ learning motivation and performance (e.g. Crompton, Burke, and Gregory 2017; Major, Haßler, and Hennessy 2017; Miller and Cuevas 2017). However, mLearning has been criticised in terms of several aspects. For example, mLearning creates a sense of isolation for students and a sense of being out of the loop for both teachers and students (e.g. Rahman 2020); additionally, systems to monitor the learning processes and performance in real time are lacking (e.g. Corbeil and Valdes-Corbeil 2007). Therefore, intelligent learning (iLearning) has emerged as another mainstream way to enable learning and teaching in higher education frameworks because it addresses many of the existing drawbacks of mLearning and provides additional benefits for learning and teaching.

iLearning refers to a smart campus that enables intelligent learning and teaching; related applications, such as interactive learning systems, smart classrooms, synchronous distance teaching and learning, virtual learning communities, and gaming competitions; and collaborative learning across classrooms (e.g. Aini, Rahardja, and Hariguna 2019; Aini, Dhaniarti, and Khoirunisa 2019; Abbasy and Quesada 2017; Araya et al. 2016; AjazMoharkan et al. 2017; Marquez et al. 2016; Muhamad et al. 2017). In the iLearning environment, instructors/lecturers and students are involved in the learning processes in a real-time manner. Although recent studies have indicated that iLearning may contribute to the students’ learning performance (e.g. Nik-Mohammadi and Barekat 2015; Yarmatov, & Ahmedova, 2020), the factors associated with iLearning platforms that influence the students’ learning performance and the relevant factors that must be considered by university managers to enhance the effectiveness of iLearning for students remain unclear. Addressing these gaps in the literature is crucial for universities to effectively improve the overall quality of education and, in turn, their competitive advantages in the long run. The reason is that the resources in any organisation may be limited; hence, it is important for universities to invest their limited resources in addressing the factors that may have a greater impact on the students’ learning performance by adopting iLearning.

Considering these aspects, we aimed to explore the critical factors associated with iLearning platforms that may influence the students’ learning performance and identify the factors with the most notable influence. First, we extracted the factors pertaining to iLearning platforms that have been found to influence the students’ learning performance based on an extensive literature review. Using these factors, we conducted a student survey to identify the key factors for iLearning and prioritise these factors based on an expert survey of other stakeholders (academic staff and practitioners from different industries and the government) to identify the factors that significantly influence the students’ learning performance. We contribute to the literature in the following way: we present novel insights regarding iLearning by specifying the elements derived from iLearning platforms that affect the students’ learning performance, thereby also contributing to the literature related to higher education by demonstrating an efficient technique to increase the effectiveness of iLearning for students. Specifically, although iLearning has gained importance in recent years, few studies have attempted to explore the critical elements (e.g. dimensions and criteria) that may influence students’ iLearning performance. We believe that by focusing on enhancing the key elements, managers of higher education institutions can effectively optimise the students’ iLearning performance in an efficient and effective manner.

2. Key factors derived from ilearning platforms that influence iLearning for students

Learning is a personal act, and learning performance may be affected by the conditions of the learning environment (Milošević et al. 2015). Technological advancements alone may not ensure a student’s learning performance because specific strategies, tools, and resources are necessary to support different types of learning (Milošević et al. 2015). Venkatesh and collaborators (2003) introduced four dimensions that may influence individuals’ use of new technology: performance expectancy, effort expectancy, social influences, and facilitating conditions. These dimensions have been tested in existing studies (e.g. Wang et al. 2020; Wang, Wu, and Wang 2009), and several researchers have demonstrated that the performance expectancy should be particularly emphasised compared to effort expectancy (e.g. Oh, Lehto, and Park 2009). Additionally, the use of new technology may be impacted by not only how effectively and efficiently individuals receive support but also the overall experience of service that the technology delivers (Choi et al. 2020). Moreover, recent studies have highlighted that individual differences such as personal innovativeness may influence the effective use of new technology (e.g. Patil et al. 2020). Considering these aspects, we adopted performance expectancy, lecturers’ influence, quality of service, and personal innovativeness as the measurement dimensions in our research. Notably, due to the nature of our study, rather than focusing on social influence in general, we particularly focused on the lecturers’ influence since lecturers frequently interact with students during the iLearning journey and are therefore more likely to influence students through teacher­ – student interactions.

‘Performance expectancy’ concerns the degree to which individuals believe that adopting an information platform/system will facilitate them in functioning effectively in a given environment or in achieving a given outcome (Venkatesh et al. 2003). When individuals perceive that the use of a specific information platform/system will help them achieve the desired outcome, they may be more likely to use it (Do Nam Hung, Azam, and Khatibi 2019). Empirical studies have supported this observation by demonstrating that enhancing the individuals’ performance expectancy may significantly alter their attitude toward a given information platform/system as well as their behavioural intent (e.g. Lowenthal, 2010; Purwanto and Loisa 2020). Similar findings have been obtained in learning-related studies. In an iLearning context, the performance expectancy may be viewed as the students’ perception of whether the use of a specific learning platform/system (e.g. an iLearning platform) will benefit their learning performance. Wang, Wu, and Wang (2009) and Chaka and Govender (2017) found that as part of learning in a digital and intelligent manner, performance expectancy was a strong determinant that motivated students to adopt this mode of learning. Recent studies have indicated that intelligent and digital learning may enrich students’ learning performance by providing new learning experiences, increasing learning participation (e.g. Chen 2011; Yarmatov & Ahmedova, 2020), and facilitating collaborative learning (e.g. Hirsch and Ng 2011; Abbasy and Quesada 2017) in an integrated environment. Additionally, iLearning may help students strengthen their learning efficiency, enhance their learning performance (e.g. Abu-Al-Aish and Love 2013; Wang, Wu, and Wang 2009), and further extend their learning effect across different domains (e.g. Karimi 2016). Therefore, it is conceivable that these elements may help enhance the students’ iLearning performance expectancy.

‘Lecturers’ influence’ originates from social influence (Abu-Al-Aish and Love 2013). Social influence concerns the degree to which individuals perceive others’ expectations regarding the use of an information platform/system (Sung et al. 2015; Venkatesh et al. 2003). In other words, if individuals strongly perceive that other individuals expect them to use a specific information platform/system, the individuals are more likely to be motivated to use it (Alshurideh et al. 2020; Liu and Li 2010; Lu and Viehland 2008; Nassuora 2012; Venkatesh and Morris 2000). Existing studies have revealed that social influence, similar to performance expectancy, is a key element that can significantly influence the individuals’ behavioural intent to adopt new technology (e.g. Alshurideh et al. 2020; Venkatesh & Davis, 2000; Venkatesh, Morris, and Ackerman 2000). The concept of social influence has been widely extended to the education and learning literature (e.g. Chen 2011; Fagan 2019; Iqbal and Qureshi 2012; Sabah 2016; Vrana 2018).

In a learning context, the ‘lecturers’ influence’ is defined as the degree to which instructors/lecturers encourage, support, and motivate students to adopt a specific learning platform/system for learning (Abu-Al-Aish and Love 2013). Existing studies have reported findings that support this aspect (e.g. Badwelan, Drew, and Bahaddad 2016; Izkair, Lakulu, and Mussa 2020; Milošević et al. 2015). For example, in their study based on a Chinese university, Hao, Dennen, and Mei (2017) discovered that the lecturers’ influence may motivate the students in using a new learning platform for digital learning. Similarly, Nassuora (2012) indicated that lecturers’ positive word of mouth may enhance the students’ willingness to learn through a digital learning platform. Other studies have claimed that to effectively motivate and support students in adopting a digital learning platform (e.g. an iLearning platform), instructors/lecturers may alter their teaching methods from a traditionally physical method to a method that is both intelligent and digital, thereby providing students with a higher learning quality through the digital learning platform, enabling them to experience mobile learning in which they can learn at any location, and increasing real-time discussions between instructors and their students via the platform (e.g. Abu-Al-Aish and Love 2013; Badwelan, Drew, and Bahaddad 2016; Chen 2011; Milošević et al. 2015; Iqbal and Qureshi 2012; Sabah 2016; Vrana 2018). Specifically, iLearning platforms allow a teacher to teach in real time, and the students have opportunities to have face-to-face interactions and discussions with the teacher (e.g. Kose and Deperlioglu 2012). Such frameworks also allow the teacher to teach students to use the platform to learn more promptly and effectively. Therefore, these elements may enhance the lecturers’ influence on the students’ iLearning.

The ‘quality of service’ in the information system and human–computer interaction fields is not the same as that in service industry management (Ameen et al. 2019; Kuan, Bock, and Vathanophas 2008). The former type concerns individuals’ assessment of the reliability and response, content quality, and security of an information platform/system, and the latter type refers to the expectation and satisfaction of customers with regard to the services being provided (Abu-Al-Aish and Love 2013; Celesti et al. 2019; Rai, Lang, and Welker 2002; Venkatesh et al. 2003). Existing studies have found that when a service is provided in a satisfactory manner, individuals are more likely to adopt new technology (Abu-Al-Aish and Love 2013); however, a low quality of service (e.g. inferior facilitating conditions) may impede the adoption of new technology (Lim and Khine 2006). In a learning context, Lee showed that students’ perception of the quality of service of a digital learning platform may significantly affect their willingness to use the platform for learning. Similarly, Abu-Al-Aish and Love (2013) indicated that in a digital learning environment, the quality of service plays a crucial role in predicting the students’ acceptance and adoption of the learning platform in the environment.

To enhance the quality of service of iLearning platforms, recent studies have suggested that instructors should provide ample learning resources (Shukla 2021; Swanson 2020), enhance their teaching quality (Zakariaa, Maatb, and Khalidc 2019), promote a pleasant intelligent campus experience (Lin and Su 2020; Wu, Wu, and Su 2019), and improve traditional learning methods for students (Ehsanpur and Razavi 2020). Moreover, researchers have suggested that learning platform providers should not only ensure the students’ privacy and safety but also assist them in using learning resources efficiently and in logging onto the platform at any time. Additionally, it has been noted that encouraging students to learn via cloud computing technology, increasing the efficiency with which messages are transferred between instructors/lecturers and students, enhancing the operational stability of the learning platform, increasing the number of campus receivers, and making the platform simple and easy to use are critical to advance the quality of service of the learning platform (e.g. Abu-Al-Aish and Love 2013; Badwelan, Drew, and Bahaddad 2016; Hassanzadeh, Kanaani, and Elahi 2012; Kim and Ong 2005; Kim et al. 2012; Lee 2006; Liu, Han, and Li 2010; Milošević et al. 2015; Ng et al. 2010; Mohammadi 2015; Salloum et al. 2019). These findings may be viewed as elements that may enhance the quality of service of iLearning for students.

‘Personal innovativeness’ concerns the degree to which individuals are willing to try and use a new information platform/system (Agarwal and Prasad 1998; Rogers 2003). Compared to their counterparts, individuals with a high degree of personal innovativeness are more likely to adopt and deal with the uncertainty associated with a new information platform/system (Abu-Al-Aish and Love 2013; Lu, Yao, and Yu 2005). Existing studies have reported supportive evidence that indicates that personal innovativeness can motivate an individual to use new technology (e.g. Agarwal and Prasad 1998; Agarwal and Karahanna 2000; Fang, Shao, and Lan 2009; Hung and Chang 2005; Lassar, Manolis, and Lassar 2005; Lewis, Agarwal, and Sambamurthy 2003; Lu, Yao, and Yu 2005; Lian and Lin 2008; Matute-Vallejo and Melero-Polo 2019; Yi et al. 2006). Although personal innovativeness has been widely investigated in different fields, such as online shopping and blogs (e.g. Bhagat and Sambargi 2019; Bigne-Alcaniz et al. 2008; Wang, Chou, and Chang 2010), it is particularly crucial in a digital and intelligent learning context (Abu-Al-Aish and Love 2013; Agarwal and Prasad 1998; Badwelan, Drew, and Bahaddad 2016; Handoko 2019; Joo, Lee, and Ham 2014; Fagan, Kilmon, and Pandey 2012; Liu, Li, and Carlsson 2010; Milošević et al. 2015; Shorfuzzaman et al. 2019; Salloum et al. 2019).

For example, Fagan, Kilmon, and Pandey (2012) discovered that students with a high level of personal innovativeness are more willing to learn by adopting virtual reality simulations. Abu-Al-Aish and Love (2013) and Karimi (2016) indicated that personal innovativeness is crucial in any learning environment involving information technology. To effectively support students’ personal innovativeness, existing studies have highlighted the importance of altering students’ learning habits and strengthening their courage to try and enhance their self-efficacy in learning with a new learning platform (e.g. Abu-Al-Aish and Love 2013; Badwelan, Drew, and Bahaddad 2016; Joo, Lee, and Ham 2014; Karimi 2016; Milošević et al. 2015; Shorfuzzaman et al. 2019; Salloum et al. 2019). Notably, these suggestions may be elements that can enhance the students’ personal innovativeness with regard to iLearning.

In conclusion, through a literature analysis summarising the most referenced and commonly used factors in the relevant studies, we propose four dimensions with 26 criteria (i.e. elements that may help enhance the dimensions) for iLearning. Table 1 summarises the proposed dimensions and sources of the proposed criteria.

Table 1. Dimensions and criteria that influence iLearning of students.

Dimension	References	Criteria	References
Performance Expectancy (A)	Abbasy and Quesada (2017); Abu-Al-Aish and Love (2013); Almahri, Bell, and Merhi (2020); Aini, Rahardja, and Hariguna (2019); Badwelan, Drew, and Bahaddad (2016); Chen (2011); Fagan (2019); Hirsch and Ng (2011); Kumar and Bervell (2019); Karimi (2016); Latip et al. (2020); Milošević et al. (2015); Shukla (2021); Wang et al (2009)	Providing new learning experiences* (A1)	Bai (2019); Darmaji et al. (2019); Kumar and Chand (2019)
		Increasing the learning participation (A2)	Chen (2011); Johnson et al. (2015)
		Extending the learning effect (A3)	Karimi (2016)
		Increasing the learning efficiency (A4)	Abu-Al-Aish and Love (2013); Wang, Wu, and Wang (2009)
		Enhancing the learning performance (A5)	Nik-Mohammadi and Barekat (2015)
		Facilitating collaborative learning* (A6)	Aini, Rahardja, and Hariguna (2019); Aini, Dhaniarti, and Khoirunisa (2019); Abbasy and Quesada (2017)
Lecturers’ Influence (B)	Abu-Al-Aish and Love (2013); Almahri, Bell, and Merhi (2020); Aini, Dhaniarti, and Khoirunisa (2019); Badwelan, Drew, and Bahaddad (2016); Chen (2011); Kumar and Bervell (2019); Iqbal and Qureshi (2012); Latip et al. (2020); Milošević et al. (2015); Sabah (2016); Shukla (2021); Vrana (2018); Wang, Wu, and Wang (2009)	Altering teaching methods (B1)	Sabah (2016); Vrana (2018)
		Teachers’ encouragement (B2)	Kacetl and Klímová (2019)
		Higher learning quality (B3)	Almaiah and Alismaiel (2019)
		Enabling mobile teaching (B4)	Aremu (2019); Wu, Wu, and Su (2019)
		Fostering discussions between teachers and students (B5)	Kose and Deperlioglu (2012)
Quality of Service (C)	Abu-Al-Aish and Love (2013); Almaiah and Alismaiel (2019); Almaiah and Al Mulhem (2019); Alshurideh et al. (2019); Badwelan, Drew, and Bahaddad (2016); Hassanzadeh, Kanaani, and Elahi (2012); Kim and Ong (2005); Kim et al. (2012); Lee (2006); Liu et al (2010); Milošević et al. (2015); Ng et al. (2010); Mohammadi (2015); Salloum et al. (2019)	Additional learning resources (C1)	Shukla (2021); Swanson (2020)
		Privacy and safety (C2)	Palalas and Wark (2020); Thomas and Hajiyev (2020)
		Learning through cloud computing technology* (C3)	Okai-Ugbaje et al. (2020); Sekaran et al. (2019)
		Transferring messages promptly* (C4)	Gonzalez, Gonzalez, and Huidobro (2019); Nyembe and Howard (2019)
		Higher teaching quality (without the limitation of space) (C5)	Zakariaa, Maatb, and Khalidc (2019)
		Numerous campus receivers+ (C6)	Ichaba, Musau, and Mwendia (2020)
		Creating a pleasant campus experience (C7)	Lin and Su (2020); Wu, Wu, and Su (2019)
		Improving traditional learning methods (C8)	Ehsanpur and Razavi (2020)
		Access at all times (C9)	Alshurideh et al. (2020); Cavus (2020)
		Prompt usage of learning resources (C10)	Nielit and Thanuskodi (2020); Okai-Ugbaje et al. (2020)
		Easy to use* (C11)	Alam and Aljohani (2020); Malik et al. (2020)
		Operational stability* (C12)	Seralidou, Douligeris, and Gralista (2019); Wu, Wu, and Su (2019)
Personal Innovativeness (D)	Abu-Al-Aish and Love (2013); Badwelan, Drew, and Bahaddad (2016); Joo, Lee, and Ham (2014); Karimi (2016); Li (2020); Liu et al. (2010); Milošević et al. (2015); Patil et al. (2020); Shorfuzzaman et al. (2019); Salloum et al. (2019); Sidik and Syafar (2020)	Altering learning habits (D1)	Wu, Wu, and Su (2019)
		Promotion of self-efficacy* (D2)	Buabeng-Andoh (2021); Senaratne, Samarasinghe, and Jayewardenepura (2019)
		Having the courage to try (D3)	Joseph et al. (2019); Wu, Wu, and Su (2019)

Notes: 1. ⁺: C6 was deleted after the pretest analysis; 2. *: A1, A6, C3, C4, C11, C12, and D2 were deleted after factor analysis.

3. Research methods

3.1. Research procedure and strategy

The research procedure of this study is presented in Figure 1. We aimed to explore the critical factors that may affect students’ iLearning on campus at a university and identify the factors that have a more notable influence by calculating the relative importance of the critical dimensions and criteria (i.e. factors). To develop an iLearning research framework, we used keywords such as mLearning, eLearning, iLearning, acceptance and use of new technology, and teaching and learning in a high education context to seek the articles necessary for an extensive literature review. The reviewed and considered studies are summarised in Table 1. Based on the extensive literature review, we developed an iLearning research framework that encompasses four critical dimensions with 26 critical criteria. These dimensions and criteria were used to develop a questionnaire. The study was performed in two stages. In the first stage, questionnaire surveys were distributed, through convenience sampling, to respondents (i.e. students who have used iLearning on campus at a university) physically by the researchers. The survey required approximately one month to complete. Subsequently, we performed factor analysis (FA) to extract the critical factors, which were employed to construct the hierarchical evaluation structure of the second-stage questionnaire. In the second stage, the analytic hierarchy process (AHP) was adopted to obtain a consensus based on domain expert opinions. Using the judgment sampling method, the researchers consulted the experts personally across three different sectors (e.g. industry, government, and academia). Next, we analyzed the relative importance of and prioritised each critical dimension and criterion. Based on the analysis results, we formulated managerial and educational recommendations, with this study’s findings serving as a valuable reference for university managers to enhance the effectiveness of iLearning for students. In the following sections, we briefly introduce the relevant data analysis methods (including FA and AHP) used in this study.

Figure 1. Research procedure and strategy.

3.2. FA

FA is a multivariate technique applied to address the problem of interdependence between variables. Factor loadings can be used to extract the main factors based on the explained maximum variance for all variables. The main purpose/advantage of FA is to reduce the number of variables without losing the representativeness of the original data by identifying the hidden dimensions that may/may not be directly apparent from the analyses. However, FA results may be controversial. The interpretations of the results may be debatable since different interpretations may be derived from the same factors. Therefore, naming or renaming factors would require researchers to have specialised knowledge (Navlani 2019). This study used principal component analysis with varimax rotation for factor extraction in the first stage to confirm that no correlations existed among the factors and to satisfy the requirement of independence among the elements of the hierarchical structure in the second-stage AHP analysis. Based on these results, we examined the reliability of the extracted dimensions based on the relevant criteria.

3.3. AHP

The AHP, which was first proposed by Saaty in 1971, is a systematic and hierarchical analysis tool that combines qualitative and quantitative methods (Saaty 1980). This approach can practically and effectively solve complex decision-making problems by dividing the complex decision-making situations into small components. These small components are organised in a tree-like hierarchy, and expert opinions are collected to evaluate the relative importance of each component (i.e. dimension/criterion) on a scale ranging from 1 to 9. Specifically, pairwise comparisons are performed between the evaluation factors; next, comparison matrixes are established to reflect the decision makers’ subjective preferences. The eigenvectors and eigenvalues are calculated, and the eigenvectors determine the relative weights between the criteria, representing the priority of each component (i.e. dimension/criterion) at each level, which can provide decision makers with sufficient decision information, help define the relevant criteria and selected weights, and reduce the risk of decision errors.

In summary, the AHP helps simplify complex problems into hierarchies, thereby facilitating the comparison of factors in a pairwise manner, and it represents a useful tool for addressing the ambiguity and complexity in decision problems (Ramanathan 2001). Existing studies have highlighted several advantages of the AHP. For example, Macharis and collaborators (2004) proposed that the AHP allows a decision problem to be decomposed into constitutive parts that allow the development of hierarchies of the dimensions and criteria, thereby clarifying the level of importance pertaining to each dimension and criterion. Zahir (1999) claimed that the AHP enables group decision-making via group consensus. Millet and Wedley (2002) indicated that the AHP is beneficial in situations that are uncertain or risky because it allows researchers or practitioners to develop scales in cases in which ordinary evaluations cannot be applied. Another advantage of using the AHP is that a large sample size is not required to achieve statistically robust results, and in several cases, the results derived from even a single qualified expert may be sufficiently representative since the AHP is based on judgments of field/specialised experts (e.g. Darko et al. 2019; Tavares, Tavares, and Parry-Jones 2008). The sample size used in many AHP studies normally ranges from 4–9 (e.g. Pan, Dainty, and Gibb 2012; Akadiri, Olomolaiye, and Chinyio 2013; Chou, Pham, and Wang 2013), although a few studies have been known to adopt a sample size larger than 30 (e.g. Ali and Al Nsairat 2009; El-Sayegh 2009). In other words, sample selection is more important than the sample size for an AHP study. This common practice also justifies the appropriateness of the sample size of this study. One of the most criticised aspects of the AHP, however, is that it does not consider the interrelationship between the dimensions and criteria. In this study, the AHP was used as an analytical framework for establishing an evaluation model to determine the order of importance of the critical factors derived from iLearning platforms that may influence students’ iLearning at a university.

Consistency analysis is the main approach that has been used to test the reliability of the experts’ judgments obtained in AHP studies (e.g. Saaty 1980; Saaty and Vargas 2013). This approach has also been used to support the fact that although the data collected are subjective judgments, the AHP can reduce bias and ensure that the judgments are reliable (Abudayyeh et al. 2007; Hsu, Wu, and Li 2008). In this research, based on the results of the AHP, we further adopted consistency analysis with the threshold of consistency (value <1) suggested by Saaty (1980) on the studied dimensions and the overall research framework to test their reliability.

4. Data analysis and results

4.1. Respondents’ demographics

The study adopted questionnaire surveys that were administered in two stages. In the first stage, FA was performed to analyze the data collected by convenience sampling from university students with experience using iLearning. For both the pretest (50 valid responses received) and actual study (355 valid responses received), the participants’ characteristics are summarised in Table 2.

Table 2. Profiles of the respondents in the first-stage surveys.

Characteristic	Category	Pilot Study Frequency (%)	Actual Study Frequency (%)
Gender	Male	22 (44)	140 (39)
	Female	28 (56)	215 (61)
College	Engineering	7 (14)	28 (8)
	Literature	9 (18)	54 (15)
	Medicine	3 (6)	43 (12)
	Agriculture	2 (4)	35 (10)
	Business & Management	23 (46)	148 (42)
	Education	3 (6)	6 (2)
	Law	-	32 (9)
	Science	-	9 (2)
	Other	3 (6)	-
Frequency of using iLearning on campus per week	Less than 1 h	25 (50)	155 (44)
	1–5 h	15 (30)	161 (45)
	5–15 h	4 (8)	29 (8)
	More than 15 h	6 (12)	10 (3)
Frequency of using the Internet to study per week	Less than 1 h	25 (50)	98 (28)
	1–5 h	15 (30)	170 (48)
	5–15 h	4 (8)	65 (18)
	More than 15 h	6 (12)	22 (6)

In the second stage, the AHP was employed to determine the priorities of the dimensions and criteria based on expert consensus. Judgment sampling was used to solicit feedback from experts who had work experience in fields related to iLearning. A total of 17 valid responses were obtained from experts, including 5, 5, and 7 experts from the industry, government, and academia, respectively. Table 3 details the backgrounds of the responding experts. Approximately half of the experts had 10 or more years of relevant work experience.

Table 3. Profiles of the responding experts in the second-stage AHP survey.

Sector	Position	Institution	Tenure (Relevant experience)
Industry (5 experts)	Executive assistant	High-technology company	⩽ 3 y
	Instructor	Computer company	5–10 y
	Engineer	International company	⩽ 3 y
	Sales representative	International company	5–10 y
	Sales representative	International company	⩽ 3 y
Government (5 experts)	President	Department of Education (Secondment from elementary school)	10–15 y
	Division chief	Department of Education	10–15 y
	Officer	Department of Education	3–5 y
	Officer	Department of Education	10–15 y
	Division chief	Department of Education (Secondment from junior high school)	10–15 y
Academia (7 experts)	Professor	Public university	15–20 y
	Assistant professor	Public university	10–15 y
	Professor	Public university	⩾ 20 y
	Assistant professor	Public science and technology university	5–10 y
	Assistant professor	Private college	⩾ 20 y
	Associate professor	Private science and technology university	15–20 y
	Assistant professor	Private university	10–15 y

4.2. Critical factors extracted using FA

During the first stage, the questionnaire (4 dimensions and 26 criteria) was designed based on the related important factors that may affect students’ iLearning with reference to the criteria shown in Table 1. The participants were asked to make their evaluations through a 5-point Likert scale ranging from ‘strongly agree’ (5) to ‘strongly disagree’ (1). After the pretest, one item (numerous campus receivers [C6]) was deleted. In the actual survey, using principal component analysis (PCA) with iterations and varimax rotation, the factors (5 dimensions and 18 criteria) were extracted after eliminating all criteria with factor loadings less than 0.5. The five dimensions were renamed ‘performance expectancy’ (PE) (4 criteria), ‘quality of service’ (QS) (4 criteria), ‘personal innovativeness’ (PI) (3 criteria), ‘convenience of use’ (CU) (3 criteria), and ‘lecturers’ influence’ (LI) (4 criteria), which accounted for 64.65% of the total variance. The data regarding the factor loadings of each criterion, Cronbach’s α values and variance explained by the factors (dimensions), are summarised in Table 4. The extracted critical factors (5 dimensions and 18 criteria) were used to construct a hierarchical structure (Figure 2).

Figure 2. Hierarchical structure of the critical factors for iLearning.

Table 4. Results of the extracted critical dimensions and criteria of iLearning.

Dimensions	Criteria	Factor Loading	Communality	Corrected Item-Total Correlation	Cronbach’s α	Variance (%)	Cumulative (%)
Performance expectancy (PE)	Enhancing the learning performance (PE1)	.782	.748	.719	.823	38.183	38.183
	Increasing the learning participation (PE2)	.713	.652	.626
	Increasing the learning efficiency (PE3)	.707	.645	.634
	Extending the learning effect (PE4)	.694	.656	.611
Quality of service (QS)	Creating a pleasant campus experience (QS1)	.731	.661	.633	.791	7.606	45.789
	Improving traditional learning methods (QS2)	.643	.668	.558
	Higher teaching quality (without the limitation of space) (QS3)	.622	.567	.591
	Higher learning quality (QS4)	.513	.653	.618
Personal innovativeness (PI)	Altering learning habits (PI1)	.729	.693	.559	.700	7.103	52.892
	Having the courage to try (PI2)	.691	.643	.571
	Teachers’ encouragement (PI3)	.642	.630	.432
Convenience of use (CU)	Access at all times (CU1)	.799	.712	.547	.659	6.174	59.066
	Prompt usage of learning resources (CU2)	.654	.684	.510
	Enabling mobile teaching (CU3)	.573	.631	.364
Lecturers’ influence (LI)	Privacy and safety (LI1)	.733	.615	.460	.699	5.583	64.649
	Fostering discussions between teachers and students (LI2)	.641	.543	.485
	Additional learning resources (LI3)	.547	.609	.512
	Altering teaching methods (LI4)	.502	.626	.483

4.3. Relative weights of the critical factors examined in the AHP

In the second stage, as shown in Figure 2, the AHP was employed to prioritise the factors, i.e. the dimensions and criteria. Pairwise comparisons among the factors were performed by experts with work experience and knowledge regarding iLearning. The opinions (assessments) of the 17 experts were averaged to obtain expert consensus. The consistency ratio (C.R.) values of the 5 dimensions, i.e. PE, QS, PI, CU, and LI, were 0.000, 0.076, 0.080, 0.091, and 0.059, respectively, which were less than 0.1 and thus satisfied the threshold of consistency suggested by Saaty (1980). The C.R. for the whole hierarchy (0.062) was also less than 0.1. Therefore, the experts’ opinions were considered consistent. In other words, the AHP analysis results were reliable. Table 5 presents the relative weights (importance) and rankings of the dimensions and criteria. The most critical dimension that affected students’ iLearning was PE (weight=0.349), followed by CU (0.185), PI (0.169), QS (0.168), and LI (0.130). Additionally, the top five critical criteria were enhancing the learning performance (PE1) (0.140), increasing the learning participation (PE2) (0.110), altering learning habits (PI1) (0.092), ensuring access at all times (CU1) (0.075), and enabling prompt usage of learning resources (CU2) (0.067).

Table 5. Results of the prioritised critical factors for iLearning.

Dimensions	Weight	Criteria	Local Weight	Global Weight
Performance expectancy (PE) C.R. = 0.00<0.1	0.349 (1)	Enhancing the learning performance (PE1)	0.402	0.140 (1)
		Increasing the learning participation (PE2)	0.316	0.110 (2)
		Increasing the learning efficiency (PE3)	0.154	0.054 (6)
		Extending the learning effect (PE4)	0.127	0.044 (10)
Quality of service (QS) C.R. = 0.076<0.1	0.168 (4)	Creating a pleasant campus experience (QS1)	0.303	0.051 (7)
		Improving traditional learning methods (QS2)	0.263	0.044 (11)
		Higher teaching quality (without the limitation of space) (QS3)	0.259	0.044 (12)
		Higher learning quality (QS4)	0.175	0.029 (15)
Personal innovativeness (PI) C.R. = 0.080<0.1	0.169 (3)	Altering learning habits (PI1)	0.547	0.092 (3)
		Having the courage to try (PI2)	0.298	0.050 (8)
		Teachers’ encouragement (PI3)	0.155	0.026 (17)
Convenience of use (CU) C.R. = 0.091<0.1	0.185 (2)	Access at all times (CU1)	0.405	0.075 (4)
		Prompt usage of learning resources (CU2)	0.362	0.067 (5)
		Enabling mobile teaching (CU3)	0.233	0.043 (13)
Lecturers’ influence (LI) C.R. = 0.059<0.1	0.130 (5)	Privacy and safety (LI1)	0.349	0.045 (9)
		Fostering discussions between teachers and students (LI2)	0.299	0.039 (14)
		Additional learning resources (LI3)	0.206	0.027 (16)
		Altering teaching methods (LI4)	0.146	0.019 (18)

Notes: 1. C.R.H. = 0.062 < 0.1; 2. (): Ranking of dimensions/criteria.

5. Discussion

This study was aimed at exploring the critical factors associated with iLearning platforms that may impact students’ learning performance and identifying the criteria with a more notable impact to enable university managers to enhance the effectiveness of iLearning for students. First, through an extensive literature review, we synthesised four critical dimensions (e.g. performance expectancy, lecturers’ influence, quality of service, and personal innovativeness) and 26 critical criteria. Subsequently, we extracted five critical dimensions (prompt usage of learning resources is a new dimension extracted based on the FA result) with 18 critical criteria by studying a group of students from universities in Taiwan. We further prioritised these criteria by surveying a group of senior educators and practitioners in Taiwan. Our results indicated that enhancing the learning performance, increasing the learning participation, altering learning habits, ensuring access at all times, and enabling prompt usage of learning resources are the criteria that university managers must emphasise to enhance the effectiveness of iLearning for students.

In terms of the dimensions, our results indicated that PE is the most crucial dimension that may motivate students to adopt iLearning, which aligns with existing related findings (e.g. Chaka and Govender 2017; Lowenthal 2010; Wang, Wu, and Wang 2009). In other words, to successfully implement iLearning in a university, it is crucial for the managers of the university to place emphasis on helping students perceive that they can make progress in their learning performance through iLearning, which in turn may stimulate the students to be more willing to learn by adopting iLearning. In addition, our results indicated that enhancing the learning performance and increasing learning participation are two key factors that strengthen the students’ iLearning PE, which is in line with the findings of many existing studies (e.g. Abu-Al-Aish and Love 2013; Chen 2011; Wang, Wu, and Wang 2009). This finding implies that to effectively help students perceive that they can enhance their learning performance with iLearning, the design of iLearning may need to be goal oriented. For example, a clear yet challenging learning goal may be needed to ensure that the extent to which the students’ learning performance is enhanced can be measured by instructors/lecturers and perceived by the students.

Furthermore, this finding indicates that it may be necessary to focus on designing iLearning that can effectively stimulate students to proactively participate during their learning journey to strengthen the cognitive connection between the use of iLearning and subsequent learning performance. Empirical studies have shown that when students are engaged in learning, they are more likely to perceive an improved learning outcome (e.g. Blasco-Arcas et al. 2013; McLaughlin et al. 2013). However, the learning participation and learning performance may not always be proportional. Therefore, with respect to managerial implications, the relevant units of iLearning management should not only focus on the students’ learning performance but also consider whether the students’ involvement and inputs in learning can be enhanced through iLearning. Existing studies have indicated that if students find that their inputs are polished or advanced, they are more likely to perceive better learning outcomes (e.g. Chemosit 2012). Among the relevant factors, enhancing the learning performance and increasing the learning participation represent the most crucial factors that must be considered the main goals while implementing iLearning in a university settings.

According to our results, CU is the second most important dimension, which is in line with existing findings regarding the importance of the efficiency of using an information platform/system in a digital and intelligent learning context (e.g. Abu-Al-Aish and Love 2013; Badwelan, Drew, and Bahaddad 2016; Hassanzadeh, Kanaani, and Elahi 2012; Kim and Ong 2005; Kim et al. 2012; Lee 2006; Liu et al., 2010; Milošević et al. 2015; Ng et al. 2010; Mohammadi 2015; Salloum et al. 2019). This finding implies that the ease of use of iLearning can enhance the students’ willingness to use and continue to use iLearning. Our results also indicate that ensuring access at all times and enabling the prompt use of learning resources are critical to increase the students’ efficiency in using iLearning platforms. This finding reflects that it may be crucial for university managers to allow students to learn at their own pace. Different students have a different pace of learning, and thus, students may be more likely to learn effectively and successfully when learning at their own pace (Chou and Liu 2005; Vermunt 2005). Other studies based on the iLearning platform infrastructure have suggested investing in smart campuses (e.g. Hirsch and Ng 2011), ensuring seamless connectivity, and supporting integrated learning across various devices (laptops, tablets and smartphones) for ubiquitous learning (e.g. Atif, Mathew, and Lakas 2015).

Additionally, our findings reflect the need for instructors/lecturers to upload learning resources in a timely manner and to ensure the fit and consistency between teaching contents and learning resources to ensure that students can use the learning resources efficiently when needed. Supporting this point, existing studies have highlighted the importance of the timely provision of learning resources and usefulness of learning from/with these resources, especially in a digital and intelligent learning environment (e.g. Ma et al. 2010; Otieno 2010; Ross and Grinder 2002). Certain studies also emphasised the need to ensure that iLearning platforms allow students to use learning resources from any device (e.g. Atif, Mathew, and Lakas 2015; Hirsch and Ng 2011). Notably, ensuring access at all times and enabling prompt usage of learning resources should be considered the main goals to be fulfilled while performing iLearning in a university setting, and these aspects represent two of the top five factors.

PI is ranked as one of the top three dimensions in this study, echoing existing studies that highlight its importance in a digital and intelligent learning environment (Abu-Al-Aish and Love 2013; Agarwal and Prasad 1998; Badwelan, Drew, and Bahaddad 2016; Joo, Lee, and Ham 2014; Fagan, Kilmon, and Pandey 2012; Liu et al., 2010; Milošević et al. 2015; Shorfuzzaman et al. 2019; Salloum et al. 2019). This finding implies that to effectively implement iLearning, university managers may need to develop students’ interest in trying and adopting iLearning. Our results further indicate that altering the learning habits of students is crucial in assisting managers with this task. Recent studies have suggested that iLearning must be designed to provide opportunities for students to frequently interact with learning since an interactive design and application in a digital and intelligent learning environment is beneficial in changing students’ behaviour in specific ways (e.g. Ibrahim, Halim, and Ibrahim 2012).

In general, among the 18 criteria, the accumulated importance weight (0.538) of the top six criteria, i.e. enhancing the learning performance, increasing the learning participation, altering learning habits, ensuring access at all times, enabling prompt usage of learning resources, and increasing the learning efficiency, accounts for more than half of the overall importance weight. These results indicate that except for the criterion of altering learning habits, which ranks third, all five of the remaining criteria belong to the PE and CU dimensions. This result is consistent with the finding of Milošević et al. (2015), who have showed that, especially for higher-level education, iLearning should be an integral mode way of learning that provides prompt access to learning resources to enhance the learning performance. In higher education institutions, investments regarding technologies and specialists in iLearning to enhance the learning performance are a notable concern. The importance of iLearning design in changing students’ learning habits and increasing learning participation should be emphasised. The implication may be that university students are no longer satisfied with traditional ways of learning.

Interestingly, the LI dimension has a relatively low impact on students’ iLearning. This result may be explained by the fact that most students are older than 18 and have their own thoughts and opinions. The students likely adopt iLearning because they perceive that their learning performance is enhanced by iLearning in addition to lectures. Notably, however, although this dimension is the least important compared to the others and corresponds to the lowest ranking criterion, altering teaching methods, in practical applications, the teaching methods of instructors/lecturers still influence the students to a certain extent since students may not proactively adopt this mode of learning without the cooperation of instructors/lecturers. Therefore, instructors/lecturers should be encouraged to adopt new teaching methods related to iLearning-related application systems to facilitate the students’ use of iLearning. Implementing this aspect through the CU dimension, which ranks second and consists of both the fourth and fifth highest ranking criteria (i.e. access at all times and prompt usage of learning resources), instructors/lecturers can successfully alter students’ learning habits, which is the third most important criterion.

6. Research Limitations and future recommendations

Although this study strived to be sufficiently rigorous, it involves certain limitations. The recommendations for future follow-up research are as follows. First, because iLearning is still under development despite its increased global application and technological advancement, more dimensions and criteria may be explored or developed in research and practice in the immediate future. Therefore, we suggest that future research should expand our framework by integrating more relevant dimensions and criteria into the framework for assessment. Through this aspect, the research results can be rendered more practical and applicable. Second, future research may be aimed at having student respondents prioritise the dimensions and criteria and compare the results with those obtained in this study. It is likely that the viewpoints of senior educators and practitioners and those of students are not identical. Since the primary users of iLearning platforms are students, implementing iLearning in universities may be more effective and efficient if the critical factors were identified by incorporating the opinions of students.

Third, while prioritising the dimensions and criteria for iLearning, we did not consider the interrelationships between the dimensions and criteria in our framework. A change in one factor may influence other factors. In this regard, we suggest that future research should adopt the analytic network process (ANP) for prioritisation. The ANP can more effectively address multicriteria decision-making issues, such as those investigated in this study, in which the dimensions and/or criteria may be interdependent (Liao et al. 2018). Fourth, the generalizability of this study may be enhanced. In this study, due to limited resources and time, we focused only on students and experts in Taiwan. Hence, future research may diversify the respondents by studying students and experts from different countries to enhance the generalizability of our results. Fifth, the dimensions and criteria used in this study may be consolidated by using other techniques to determine the significance among the dimensions and criteria, which may be another avenue for future research before our study framework is retested. Finally, subsequent research may be supplemented by case studies, interviews, and other qualitative analyses to more clearly understand and implement iLearning on campus.

7. Conclusion

Technology has significantly changed our way of life, including the way in which individuals gain knowledge. In recent years, students at many universities have ceased relying on conventional ways of learning. Instead, students have increasingly adopted iLearning on campus. While the importance of iLearning has been highlighted in research and practice, little is known regarding the influence of factors derived from iLearning on the students’ learning performance at a university and the factors that must be focused on by relevant management to enhance the effectiveness of iLearning for students. Considering these aspects, we initially proposed a framework encompassing four critical dimensions with 26 critical criteria based on an extensive literature review. Moreover, by conducting questionnaire surveys in two stages, we advanced the framework and prioritised the critical dimensions and criteria for iLearning. The research results may provide university managers with valuable reference to not only promote the use of iLearning on campus but also enhance the effectiveness of iLearning for students.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.