Mobile-assisted or paper-based? The influence of the reading medium on the reading comprehension of English as a foreign language

Abstract

The issues of whether mobile-assisted English as a foreign language (EFL) reading is effective have become increasingly important for EFL teachers and policy-makers. However, empirical studies investigating EFL students’ reading comprehension performance on mobile phones and students’ perceptions of these reading formats are lacking. The present research employed questionnaire surveys and semistructured interviews to investigate whether differences exist between mobile-assisted and paper-based EFL reading in terms of participants’ reading comprehension accuracy, reading speed, and reading strategy used (i.e. general, problem-solving or support strategies) and explore EFL learners’ perceptions of mobile-assisted EFL reading. A sample of 81 first-year college students participated in a quasiexperiment, 6 participants participated in a semistructured interview, and a sample of an additional 10 students participated in a pilot study. The analyses, including qualitative descriptions, mixed-design analysis of variance (ANOVA), paired-sample t-tests, and independent-sample t-tests, indicated that participants achieved superior reading comprehension accuracy when reading from printed paper compared to reading from mobile phones. Superior reading speed from paper was observed only when participants were instructed to read and complete the comprehension test the first time, while no difference in speed between the two reading media was observed the second time. Additionally, participants’ use of reading strategies during mobile-assisted EFL reading was not as effective as that during paper reading. Regarding participants’ preferences, most participants (79%) preferred reading from paper over mobile phones because hard copies provided a better reading experience and promoted active reading engagement. However, participants held positive attitudes toward mobile-assisted EFL learning and wanted to continue to read from mobile phones to assist their foreign language learning due to their convenient built-in functions and portability. The implications for the use of mobile devices to assist language pedagogical practice are discussed.

Keywords:

• Mobile-assisted EFL reading
• reading comprehension accuracy
• reading speed
• reading strategies
• students’ perceptions

1. Introduction

Given the prominence of information technology, mobile-assisted language learning has become an indispensable part of English as a foreign language (EFL) learning and has been recognized as an efficient tool in this setting (Burston, 2015; Chen et al., 2020; Kondo et al., 2012; Şad et al., 2020; Tragant, Pinyana, Mackay, & Andria, 2021). Compared to computer-assisted EFL learning, which has been widely implemented for decades and has been proven successful by multiple studies, the field of mobile-assisted EFL learning is relatively nascent and requires further investigation (Gutiérrez-Colón et al., 2020; Hwang & Fu, 2019; Liu & Huang, 2016; Şad et al., 2020). Although mobile phones are often seen as an extension of computers (Burford & Park, 2014), the nature of mobile-assisted reading differs substantially from reading on desktop computers (Kukulska-Hulme & Shield, 2008; Liu & Huang, 2016). For instance, personal and portable mobile phones support greater spontaneous, connective, and interactive reading across different contexts in comparison to computers (Kukulska-Hulme & Shield, 2008). Moreover, mobile phones may have shaped new reading behavior and styles featured by more browsing and selective reading (Liu & Huang, 2016). Therefore, findings from computer-assisted reading cannot be extended to mobile-assisted reading directly and further studies are required to understand the nature of mobile-assisted language reading to scaffold effective mobile learning. In particular, Lin et al. (2020) noted that further empirical research is necessary on the potential advantages and disadvantages of applying mobile-assisted language learning to EFL reading. During the EFL reading process, readers understand a text through active knowledge construction, which includes selecting useful information, constructing the information into coherent representation, and integrating it into existing knowledge (Mayer, 1996); the mobile phones that readers use might significantly influence these reading practices (Lin et al., 2020). Given that reading medium is recognized as a crucial variable that influences the reading comprehension process (Clinton, 2019; Delgado et al., 2018; Haddock et al., 2019; Kong et al., 2018; Latini et al., 2020; Singer & Alexander, 2017a, 2017b), this study conducted experimental research to investigate the impact of reading media on college EFL readers and further provide insight into language pedagogical practice. The frameworks employed in this study consider reading comprehension an active meaning-construction process in which the reader, the text and the activity play central roles (Alexander & Jetton, 2002). The frameworks identify three levels of reading comprehension, i.e. literal comprehension (understanding the texts and recalling the information), inferential comprehension (inferring the intended meaning of the texts) and evaluative comprehension (making explicit links between the written word and the larger world) (Alonzo et al., 2009; Kintsch, 2005). Specifically, the purposes of understanding the effects of the reading medium on EFL reading were informed by the previous literature, which reported differences in reading comprehension accuracy (e.g. Clinton, 2019; Hsu et al., 2013; Singer & Alexander, 2017a), reading speed (e.g. Kerr & Symons, 2006; Latini et al., 2019), strategy use (e.g. Isaacson, 2017), and perceptions (e.g. Baron et al., 2017) across media. The following literature review section elaborates upon these dimensions.

1.1. EFL learners’ reading comprehension across reading media

The topic of language learners’ reading performance across media has attracted considerable interest. A growing body of empirical research involving both adults (e.g. Haddock et al., 2019; Latini et al., 2019, 2020; Singer & Alexander, 2017a) and children (e.g. Halamish & Elbaz, 2020; Støle et al., 2020) and a series of reviews (e.g. Clinton, 2019; Delgado et al., 2018; Kong et al., 2018; Singer & Alexander, 2017b) have been conducted to address this issue. Generally, the results indicate that reading media influence the reading comprehension process. For the subdomain of mobile-assisted EFL reading, we conducted the following review with a close inspection of the existing literature.

First, the reading medium affects EFL learners’ reading comprehension accuracy. However, prior work yields inconsistent results on this aspect. On the one hand, several studies have shown that language learners can achieve superior reading achievement using mobile devices compared with paper-based reading because mobile-assisted language learning systems provide EFL learners with suitable and personalized reading materials according to their preferences and proficiency levels (Hsu et al., 2013), encourage learners’ self-regulated learning (Kondo et al., 2012), promote their autonomy of interaction (Gheytasi et al., 2015; Hazaea & Alzubi, 2018), and promote their motivation to read (Hazaea & Alzubi, 2018). For instance, Gheytasi and colleagues (2015) explored the effects of smartphones on high school EFL learners’ reading comprehension and found that students who engaged with mobile-assisted EFL learning outperformed their counterparts. On the other hand, a recent study showed that college EFL learners who read from mobile screens achieved lower reading comprehension scores than those who read from paper; this finding is attributable to low reading proficiency, psychological discomfort, and lower metacognitive adaptability (Zou & Ou, 2020). Therefore, the effects of mobile reading on EFL students’ reading efficacy remain unclear, and more empirical evidence is needed in the field of mobile-assisted language learning. In addition, previous research described language learners’ reading performance on different media without considering their different levels of comprehension, representing an important limitation outlined by Singer and Alexander (2017b) based on a comprehensive review of empirical research concerning paper-based and digital reading. This study aimed to extend the existing literature by taking into account the three levels of comprehension (i.e. literal comprehension, inferential comprehension, and evaluative comprehension; Alonzo et al., 2009; Kintsch, 2005) to achieve a better understanding of the nature of mobile-assisted EFL reading.

Second, the reading medium greatly affects EFL learners’ reading speed. Reading speed can reflect readers’ reading proficiency and reading comprehension process (Bell, 2001; De Leeuw & De Leeuw, 1965; Fry, 1963); that is, higher proficiency readers generally achieve faster reading speed and more fluent reading comprehension. Reading comprehension is an active meaning-construction process in which the reader builds a cognitive map of the text; good readers move more fluently through the text. Since the publication of the work by De Leeuw and De Leeuw (1965) and Fry (1963), reading speed has attracted considerable interest, and several factors that influence readers’ reading speed have been identified, such as reading purpose (e.g. faster reading speed occurs with an entertainment purpose than with a study purpose; Narvaez et al., 1999), text type (e.g. faster speed occurs for narrative texts than for expository texts; Narvaez et al., 1999), reading strategies (e.g. global strategies contribute to faster reading speeds; Huang et al., 2009), and reading media (e.g. Latini et al., 2019; Singer et al., 2018, 2019). Regarding the reading medium’s influence on reading comprehension speed, several studies have concluded that the reading speed from a screen is slower than that of reading from paper (e.g. Kerr & Symons, 2006; Latini et al., 2019; Singer et al., 2018, 2019), especially in time-constrained reading (Delgado et al., 2018). According to these studies, the reasons for the slower reading speed on a screen can be summarized as follows: (1) students cannot track electronic text as they can track text on paper; (2) students cannot access the text in its entirety (e.g. Kerr & Symons, 2006; Zou & Ou, 2020); and (3) students show less behavioral engagement and textual integration when reading from a screen (Latini et al., 2019). Some studies, however, have found that learners’ reading speed is faster when reading on screens than when reading from paper (e.g. Singer et al., 2019). It has been argued that due to students’ typically quick interaction with digital devices, they are inclined to process texts on screens more shallowly while engaging in more browsing and scanning (Annisette & Lafreniere, 2017; Latini et al., 2020; Liu & Huang, 2016; Singer et al., 2019), which contributes to a faster reading speed (Singer et al., 2019). While substantial research has been conducted to detect the differences in reading speed between paper and screens and has yielded insightful results, relatively few studies have compared reading speed on paper versus mobile phones in the EFL learning setting (cf., Gutiérrez-Colón et al., 2020). The present study therefore aimed to fill this gap by focusing on this aspect.

Third, as a major factor that affects learners’ reading comprehension, reading strategy has been the focus of a wide variety of studies in the field of EFL learning, such as studies on the taxonomy of EFL reading strategies (e.g. Block, 1986), the differences in strategy use among learners at different language proficiency levels (Huang et al., 2009; Sheorey & Mokhtari, 2001), and the relationship between strategy use and types of text comprehension (e.g. Shang, 2018). According to the existing literature, basic EFL language reading strategies are generally categorized as either top-down or bottom-up strategies (Block, 1986) and can be further classified into global strategies (i.e. strategies used to monitor the reading process, such as previewing the text, predicting the text content, and forming reading purposes), problem-solving strategies (i.e. strategies used to understand particular textual units, such as visualizing the encountered information, adjusting the reading speed, and rereading the text), and support strategies (i.e. strategies used to assist readers’ comprehension, such as taking notes, highlighting, and using dictionaries) (Huang et al., 2009; Mokhtari & Sheorey, 2002).

Readers adopt diversified reading strategies to facilitate their comprehension, and clear instructions regarding reading strategy use enhance language learners’ reading performance (e.g. Anderson, 1991; Block, 1986; Chen et al., 2021; Chang & Lin, 2019; Huang et al., 2009). Given the obvious importance of reading strategies and the various reading media present in readers’ language learning settings, few studies have addressed reading strategy use across different reading media. For instance, Huang et al. (2009) investigated college EFL learners’ online reading strategy use through a computer-based reading program and found that support strategies contributed most to online comprehension achievement and that global strategies were especially important for low proficiency participants. Isaacson (2017) went a step further by comparing EFL learners’ use of support strategies between computer online reading and paper reading and observed that high-intermediate college readers struggled to use support strategies when reading from screens but interacted more easily with the text via support strategies when reading from paper. Nonetheless, previous research has generally studied the effects of the reading medium on reading comprehension performance and reading strategy use separately and has mainly been conducted in the computer-assisted language learning setting. The present study, therefore, aimed to extend previous studies by comparing EFL learners’ reading strategy use when reading from paper to reading from mobile phones, while simultaneously investigating the intersection of reading strategy use across media and reading comprehension performance. By examining the interplay of these variables, this study attempted to offer insight into the nature of reading across media and provide implications for mobile-assisted EFL reading instruction.

1.2. Students’ perceptions of mobile-assisted EFL reading

Students’ perceptions of the reading medium are directly associated with reading proficiency and the effective use of reading strategies, which in turn impact reading comprehension (e.g. Huang, 2013; Huang et al., 2009). Specifically, as noted by Botero and colleagues (Botero et al., 2018), studying language learners’ perceptions of mobile learning is crucial for the successful implementation of mobile-assisted language learning because students’ preferred medium reflects their engagement with different media. Therefore, understanding students’ perceptions of the reading medium is a prerequisite for selecting appropriate reading materials to facilitate learning. Nonetheless, although the importance of language learners’ perceptions of the reading medium has been widely recognized, research focusing on college EFL learners’ perceptions of mobile-assisted EFL reading is comparatively scarce. Additionally, the existing empirical research on this issue has yielded inconsistent results. For instance, Baron and colleagues (2017) were surprised to find that a very high percentage of college students (91%) showed a strong preference for hardcopy books over all digital media (i.e. computer, tablet, eReader, and mobile phone) across all countries they surveyed (i.e. the United States, Japan, Germany, Slovakia, and India). In contrast, recent studies have shown that EFL learners were satisfied with mobile-assisted learning because they could autonomously decide what, how, and where to read, which further motivated them to read and learn (e.g. Hazaea & Alzubi, 2018). Therefore, more empirical studies on college EFL learners’ perceptions of mobile-assisted language reading and its interaction with reading performance and reading strategy use in the EFL learning setting are warranted.

1.3. Research questions

Guided by the issues mentioned above, the present study has two main goals. First, it aims to detect the differences in college EFL learners’ reading performance when they read from mobile phones and printed paper. Second, this study aims to investigate college EFL learners’ perceptions of mobile-assisted EFL reading. Specifically, this study attempts to answer the following research questions:

1. Are there differences in reading comprehension when college EFL learners read from mobile phones versus printed paper?

   1. Are there differences in the reading comprehension accuracy when college EFL learners read from mobile phones versus printed paper?

   2. Are there differences in the reading comprehension speed when college EFL learners read from mobile phones versus printed paper?

   3. Are there differences in reading strategy use when college EFL learners read from mobile phones versus printed paper?

2. What are college EFL learners’ perceptions of mobile-assisted EFL reading?

2. Methods

The study employed both qualitative and quantitative methods in a mixed approach to triangulate the data and simultaneously provide a deeper and more comprehensive understanding of the research topic (e.g. Creswell & Creswell, 2018; Moeller, 2016). The quantitative aspect focused on the collection of information regarding students’ performance across reading media and perceptions of mobile-assisted EFL reading. The independent variable in the study was the reading medium (mobile phone versus printed paper), and the dependent variables were participants’ reading performance (reading comprehension accuracy and reading speed), reading strategy use, and perceptions. Regarding the qualitative aspect, semistructured interviews were conducted to explore the reasons for students’ use of different reading strategies and the reasons for their perceptions of mobile-assisted EFL reading. The results of reading tests, questionnaires, and semistructured interviews were analyzed to determine how the reading medium affected EFL learners’ reading performance, reading strategy use, and perceptions.

2.1. Participants

The participants in this study were all from an urban comprehensive multidisciplinary university in eastern China who were required to take the compulsory College English Band III course to improve their English language skills. They were required to read assigned English language e-books weekly from the beginning of the semester through an online EFL reading system called Iyangcong (http://www.iyangcong.com).¹ According to the policy of the college where the study was conducted, the first-year college students were classified into intermediate or advanced levels based on their English language scores on the college entrance examinations (a national student selection and placement examination administered once a year in China). The participants in the present study were classified as intermediate-level EFL learners. A sample of 10 first-year college students (females = 4, males = 6) with a mean age of 18.17 years participated in the pilot study (but not the quasiexperiment) for payment. Another sample of 84 students (females = 37, males = 47) with a mean age of 18.23 years from two different classes taught by the same teacher (who was not the researcher) participated in the quasiexperiment for a gift. They were from different disciplines (arts = 18, engineering = 25, natural sciences = 22, social sciences = 16). An independent-sample t-test of the scores of a recent college English language proficiency quiz² taken by the participants indicated that the students from these two classes had the same level of English language proficiency, t(83) = −1.975, p = .052. Six students (females = 4, males = 2) who participated in the experiment were also enlisted to participate in semistructured interviews for payment. They were selected based on their willingness to participate and their preferences for the reading medium as indicated in the questionnaire (i.e. 3 students who preferred paper-based EFL reading and 3 students who preferred mobile-assisted EFL reading).

2.2. Instruments

The research instruments used in this study included reading comprehension tests, questionnaire surveys, and a semistructured interview guide. All of the questionnaire surveys and semistructured interviews were administered in the Chinese language. The comprehension tests (see Appendices I & II) used to assess the participants’ reading performance (reading comprehension accuracy and speed) across two reading media (mobile phone and printed paper) were developed by two experts and reviewed by another expert in EFL language teaching and learning with the guidance of the college English language course syllabus and levels of comprehension theory (Alonzo et al., 2009; Kintsch, 2005). They were developed based on one of the assigned reading materials for the students: Chapter 2, The Power of Tragedy (658 words; see Figure 1 for its appearance on a phone screen), and Chapter 10, The Greatest Power (655 words), from Harry Potter Power: Free Your Inner Power (Sykley, 2011). The two texts had similar readability levels (approximately 5.5 grade level) as indicated by the Fry Readability formula (Fry, 1968). For the excerpt, The Power of Tragedy, Questions 1, 3, 5, 6 and 9 were literal comprehension questions; Questions 4, 7 and 10 were inferential comprehension questions; and Questions 2 and 8 were evaluative comprehension questions. For the other excerpt, The Greatest Power, Questions 1, 5, 6, 7 and 8 were literal comprehension questions; Questions 3, 4 and 10 were inferential comprehension questions; and Questions 2 and 9 were evaluative comprehension questions. The texts presented on the mobile phones were presented according to the default setting (i.e. Times New Roman font, size 12, single line spacing within the paragraph, and double spacing between paragraphs). The printed texts were presented in the same way as the texts on the mobile screens and were made into booklets.

Figure 1. Image of the appearance on Iyangcong on a phone screen with an example of an excerpt from The Power of Tragedy.

Strategy questionnaires were used to identify the participants’ use of reading strategies with two different reading media (see Appendices III & IV). In light of the commonly acknowledged taxonomy of EFL reading strategies (i.e. global strategies, problem-solving strategies, and support strategies) (Huang et al., 2009; Mokhtari & Sheorey, 2002), the strategy questionnaires used in this study were adopted from an existing fine-grained strategy survey, the Survey of Reading Strategies (Mokhtari & Sheorey, 2002). This survey has been proven to be reliable and valid and is widely used to assess language learners’ reading strategies. The questionnaire included 27 five-point Likert scale items. Items 1, 3, 4, 6, 8, 12, 16, 19, 20, 22, 23, and 25 concerned global strategies (e.g. I take an overall view of the text to see what it is about before reading it); items 7, 9, 11, 14, 15, 18, 24, and 26 concerned problem-solving strategies (e.g. When I read, I guess the meaning of unknown words or phrases); and items 2, 5, 10, 13, 17, 21, and 27 concerned support strategies (e.g. I refer to a dictionary to help me understand what I read). The validity of the questionnaire was calculated with Pearson product-moment correlations using the Statistical Package for the Social Science (SPSS) version 22.0 by correlating each questionnaire item score with the total score. The statistical results showed that the Pearson’s r values ranged from .24 to .69, indicating that all items were valid (Gliner et al., 2017); therefore, all items were included in the data analyses.

The perception questionnaire was developed by referring to the unified theory of acceptance and use of technology (see Appendix V), which is a widely used and validated framework for designing measurement instruments regarding information technology acceptance (Davis, 1989; Hsu et al., 2013; Lawson-Body et al., 2020). The questionnaire included 12 five-point Likert scale items and two open-ended questions, with items 1, 2, 3, and 4 concerning perceived usefulness (e.g. I expect that reading from mobile phones will improve my English reading proficiency); items 5, 6, and 7 concerning perceived ease of use (e.g. I find it easy to navigate among chapters when reading from mobile phones); items 8, 9, and 10 concerning satisfaction (e.g. I am satisfied with the quality of the texts on the mobile-assisted language reading system); and items 11 and 12 concerning acceptance (e.g. I intend to continue to read from a mobile phone). Two open-ended questions were used to collect the participants’ comments regarding mobile phone reading (Question 13), their preferences for reading media and the reasons for their preferences (Question 14). The validity of this questionnaire was also calculated with Pearson product-moment correlations, and the statistical results showed that the Pearson’s r values ranged from .46 to .83, indicating that all items were valid (Gliner et al., 2017) and were included in the analyses. The Cronbach’s alpha values ranged from .72 to .91, indicating good internal consistency and acceptability of the questionnaires (Gliner et al., 2017). All questionnaires were first translated into the Chinese language by the authors, and three Chinese translation experts amended the translated version in the Chinese language. An additional two education experts, who are native English speakers, testified to the Chinese language version’s accuracy.

Based on the theoretical orientation toward mobile-assisted EFL reading in this study, a follow-up semistructured interview guide (see Appendix VI) was developed by the researchers after the analysis of the participants’ responses to the questionnaires and the results of the pre- and posttests. Using a general, ten-question outline, semistructured interviews with six participants were conducted to gain insight into the participants’ different reading processes across the two media and their perceptions of mobile-assisted EFL reading.

2.3. Data collection

This study was conducted in the middle of the semester when participants had used the Iyangcong English language learning system for EFL reading for more than two months. To counterbalance the order of the delivery of the reading medium, one class with 41 participants first completed the mobile-assisted EFL reading comprehension test, followed by Strategy Questionnaire 1, the paper-based EFL reading comprehension test, Strategy Questionnaire 2, and, finally, the perception questionnaire. Another class with 43 participants completed the paper-based reading comprehension test first, followed by Strategy Questionnaire 2, the mobile-assisted EFL reading comprehension test, Strategy Questionnaire 1, and, finally, the perception questionnaire. All the instruments were administered in paper-and-pen format. There was no time limit for the tests and questionnaires, but before the participants completed them, they were asked to record the amount of time used to read the texts and answer the questions with the embedded stopwatch application on their mobile phones and to write the time on the first page of the booklet. Three days before the experiment, students were asked to ensure that the Iyangcong English language learning system was working well on their phones. They all used smartphones with screen sizes ranging from 4.0 inches to 6.76 inches. The semistructured interviews were conducted with six students recruited from the quasiexperiment participants, including three who preferred mobile-assisted EFL reading and 3 who preferred paper-based EFL reading. The interviews lasted approximately forty minutes; the participants were interviewed individually in an informal way in a quiet room. All interviews were recorded with an audio recorder and then transcribed for further analysis.

Before the quasiexperiment, a pilot study was conducted to ensure the accuracy and readability of the instruments. The pilot study included 10 first-year students who were taking a parallel course to the two classes that would participate in the quasiexperiment. The procedures of the pilot study, which was conducted in a normal classroom, were identical to those of the quasiexperiment. After the participants completed all of the tasks, they were asked to provide comments regarding the tests and questionnaires.

2.4. Data analysis

In this study, quantitative descriptive statistical analysis and qualitative analysis of students’ responses to the reading comprehension tests and questionnaires were performed. Eighty-four copies of printed paper booklets were administered, and all 84 were collected. One copy was excluded from analyses because more than half of the questions were not completed, and two copies were excluded because the same response was provided across almost all items, resulting in 3.5% of the data missing. A two-way mixed-design ANOVA, paired-sample t-tests, and independent-sample t-tests were used to assess the differences in the students’ reading performance across the two reading media. A one-sample t-test was used to assess participants perceptions of mobile-assisted EFL reading. The reading comprehension accuracy data were obtained by scoring the reading comprehension tests, with each correct answer scoring one point. The reading time data were obtained from students’ self-reported times. For the three questionnaires, participants’ responses to the closed-ended items were coded with a 5-point Likert scale, in which ‘1’ indicated ‘Strongly Disagree’ and ‘5’ indicated ‘Strongly Agree’. The responses to the open-ended questions were coded based on Baron et al. (2017) study (participants’ responses were categorized into aesthetic, cognitive, physical and other dimensions; Section 3.2). SPSS version 22.0 was used to perform aggregation and computation of all the statistical data. The interviews were transcribed with Xunfeitingjian (https://www.iflyrec.com) and checked manually by the researchers for further open coding and analyses. Following Glaser’s (1992) grounded-theory procedures, two authors first read the transcripts to obtain a general sense of the interview data; then, they read one of the six transcripts sentence-by-sentence independently to assign labels to the propositions. Next, the two coders discussed the labels to find patterns and classified them into core categories according to their theme (e.g. ‘mobile phones enable reading anytime or anywhere’ was classified into ‘portability’). After reaching an agreement, they coded the remaining five transcripts based on the coding scheme of the first transcript. Disagreements were settled by discussion.

Table 4. Advantages and disadvantages of mobile-assisted EFL reading reported by participants in the perception questionnaire.

Item	Category	Comment	N
Advantages	Aesthetic/Emotional	More pleasant	2
	Cognitive	More motivation to read	10
	Physical	Abundant reading resources	12
		Convenient built-in functions	63
		Portability	55
	Other	Environmentally friendly	3
		Low price	7
Disadvantages	Aesthetic/Emotional	Lack of real reading feeling	15
	Cognitive	Difficulty in finding coherence in the text	9
		Lack of concentration	35
		Poor retention	17
	Physical	Battery-charging problem	5
		Eye strain	38
		Inconvenient operation	16
		Small screen	7
		Inability to take notes	27
	Other	Not being used to it	2

Table 5. Reasons reported by participants for their preferences for the reading medium in the perception questionnaire.

Medium	Category	Comment	N
Mobile Phone	Aesthetic/Emotional	More pleasant	2
	Cognitive	Better focus	3
		More motivation to read	5
	Physical	Convenient built-in functions	12
		Portability	14
	Other	Being used to it	2
		Low cost	2
Printed paper	Aesthetic/Emotional	More real	14
		More pleasant	8
	Cognitive	Better focus	23
		Better retention	5
		More conductive to reflection	6
		More motivation to read	4
	Physical	Better for eyes	8
		More convenient to take notes	25
	Other	Being used to it	4

Note. Data were collected from Question 14 in the Perception Questionnaire (see Appendix V).

3. Results

To answer the first of the three research questions in the present study, namely, that concerning the impact of reading media on EFL learners’ reading accuracy, speed, and strategy used, statistical analyses, including two-way mixed-design ANOVAs, repeated-measures ANOVAs, paired-sample t-tests, and independent-sample t-tests, were employed to analyze the data collected from reading comprehension tests and the questionnaire concerning the reading strategy used. The results of the differences in the reading comprehension accuracy, reading speed, and reading strategy use across the two reading media are presented in Sections 3.1.1, 3.1.2, and 3.1.3, respectively. To answer the fourth research question, namely, that regarding EFL learners’ perceptions of mobile-assisted EFL reading, qualitative and descriptive methods were used to analyze participants’ responses to the perception questionnaire and the interview data, the results of which are presented in Section 3.2.

3.1. Students’ reading comprehension across the two reading media

3.1.1. Students’ reading comprehension accuracy across the two reading media

To determine whether the order in which the text was presented could confound the comprehension accuracy results, we conducted a two-way mixed-design ANOVA with the treatment order (reading from mobile phone or printed paper first) as a between-subjects factor and the reading medium (mobile phones or printed paper) as a within-subjects factor. The analysis yielded main effects of reading medium, F(1, 80) = 4.04, p = .048, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$ = .049, but revealed no main effects of treatment order, F(1, 80) = .83, p = .365, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$ = .011, and no reliable interaction between the reading medium and treatment order, F(1, 80) = 2.36, p = .128, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$= .029, suggesting that the treatment order had no significant effects on reading comprehension accuracy (see Figure 2; for the means and standard deviations of each experimental group, see Table 1). Thus, we conducted a paired-samples t-test on the full sample to determine the effects of the reading medium on participants’ reading comprehension accuracy.

Figure 2. The average scores of reading comprehension tests for each order-medium condition (± standard error bars represent 95% confidence intervals).

Table 1. Means and standard deviations of the students’ reading comprehension scores in each experimental group.

Treatment order	Score of reading from mobile phone(s)		Score of reading from printed paper
	M	SD	M	SD
First mobile phone(s)	8.08	1.46	8.83	1.28
First printed paper	8.63	1.88	8.73	1.09

The analysis indicated that scores were significantly lower for participants who read from mobile phones (M = 8.35, SD = 1.68) than for those who read from paper (M = 8.78, SD = 1.17), t(80) = −2.05, p = .044, d = .298. Furthermore, the results pertaining to the three levels of comprehension showed that the accuracy rates of the literal comprehension questions, inferential questions, and evaluative questions were 84% versus 87%, 79% versus 82%, and 88% versus 91% for mobile-assisted versus paper-based reading comprehension, respectively. To summarize, the participants’ reading comprehension accuracy with mobile screens was significantly inferior to that with paper.

3.1.2. Students’ reading comprehension speed across the two reading media

To determine whether the order in which the text was presented (reading from mobile phones or printed paper first) would confound the reading speed results, we conducted a two-way mixed-design ANOVA. The analysis yielded no main effects of treatment order, F(1, 80) = 1.39, p = .243, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$= .017, but revealed main effects of reading medium, F(1, 80) = 17.47, p < .001, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$ = .181, and a reliable interaction between reading medium and treatment order, F(1, 80) = 25.46, p < .001, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$ = .244 (see Figure 3; for the means and standard deviations of each of the experimental groups, see Table 2). Therefore, to determine the effects of the reading medium on participants’ reading speed, we conducted two independent-sample t-tests for the two reading orders.

Figure 3. The average time spent on reading comprehension tests for each order-medium condition (± standard error bars represent 95% confidence intervals).

Table 2. Means and standard deviations of the students’ reading comprehension speed in each experimental group.

Treatment order	Time of reading from mobile phone		Time of reading from printed paper
	M	SD	M	SD
First mobile phone(s)	987.66	231.52	804.85	229.99
First printed paper	839.43	157.87	856.60	189.41

The analysis showed that reading time was significantly greater for mobile phones (M = 987.66, SD = 231.52) than for printed paper (M = 856.60, SD = 189.41) when participants were instructed to read and complete the comprehension test for the first time, t(79) = 2.79, p = .007, d = .620, but revealed no significant difference between mobile phones (M = 839.43, SD = 157.87) and printed paper (M = 804.85, SD = 229.99) when participants were instructed to read and complete the comprehension test for the second time, t(79) = .79, p = .434, d = .175. In summary, participants’ average reading speed from mobile screens was significantly slower than that from paper when they were instructed to read and complete the comprehension test for the first time, but no difference was observed between the two media when they completed the reading comprehension test for the second time.

3.1.3. Students’ use of reading strategies across the two reading media

To detect the effects of the reading medium on participants’ use of reading strategies, an analysis of the participants’ responses to the two strategy questionnaires was performed. Overall, in both media, problem solving was the most commonly adopted type of strategy, followed by the global strategy and then the support strategy. A 2 (medium type: mobile phone versus paper) × 3 (strategy type: global, problem-solving and support strategies) repeated-measures ANOVA indicated significant main effects of medium type, F(1, 80) = 41.76, p < .001, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$= .343, and main effects of reading strategy type, F(2, 160) = 49.04, p < .001, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$= .380, but revealed no interaction between these two factors, F(2, 160) = .56, p = .570, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$= .007 (see Figure 4). Pairwise comparisons with the Bonferroni correction showed that the reading from printed paper scores (M = 3.76, SD = .45) were significantly higher than the reading from mobile phones scores (M = 3.41, SD = .47) for the global strategy, F(1, 80) = 38.87, p < .001, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$ = .327; the reading from printed paper scores (M = 3.97, SD = .39) were also significantly higher than the reading from mobile phones scores (M = 3.68, SD = .50) for the problem-solving strategy, F(1, 80) = 29.03, p < .001, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$ = .266; the reading from printed paper scores (M = 3.48, SD = .54) were also significantly higher than the reading from mobile phones scores (M = 3.22, SD = .59) for the support strategy, F(1, 80) = 11.91, p = .001, ${\left\{Y_{t,\mathrm{ }c}^l\right\}}_{l=1}^{12}$ = .130. In summary, the participants used all three types of reading strategies more tactically when reading from printed paper than when reading from mobile phones (Table 3)

Figure 4. Mean rating scores of the three reading strategies in the two types of reading media (± standard error bars represent 95% confidence intervals).

Table 3. Descriptive statistics for the items in the perception questionnaire.

Item	M	SD	t
Perceived usefulness
1.I expect that reading from mobile phones will improve my English reading proficiency	3.42	.86	4.37**
2.I am more likely to study for a longer time with mobile-assisted English reading applications than with printed English texts.	2.70	1.02	−2.62*
3.Reading from mobile phones is convenient.	3.58	1.01	5.17**
4.Mobile phones are efficient tools for language learning.	3.62	.93	5.98**
Perceived ease of use
5.I find it easy to navigate among chapters when reading from mobile phones.	3.40	1.17	3.04**
6.The built-in functions of the mobile-assisted language reading application (Iyangcong) are easy to use.	3.21	1.00	1.90
7.I find information faster from texts on mobile phones than in printed texts.	3.17	1.06	1.47
Satisfaction
8.I am satisfied with the quality of the texts on the mobile-assisted language reading system (Iyangcong).	3.49	.92	4.82**
9.I am able to retain what I read when reading from mobile phones.	2.95	.88	-.51
10.I am satisfied with reading English texts from mobile phones.	3.52	.95	4.91**
Acceptance
11.I will recommend the mobile-assisted EFL reading system to my friends.	3.19	1.06	1.57
12.I intend to continue to read from mobile phones.	3.73	.84	7.83**

Note. ^**p < .01, and ^*p < .05 (two-tailed).

The interview data offered detailed information about the strategy questionnaire results. For instance, all six interview participants commented that they were used to scanning texts entirely before reading (Question 2 in the Semistructured Interview; see Appendix VI) because it could give them an overall picture of the reading materials. However, it was inconvenient to go back and forth between pages on mobile phone screens compared to printed paper, and further, mobile phone screens made it difficult to find coherence in the texts (Question 9 in the semistructured interview). In terms of problem-solving strategy, three respondents reported that although there were bold and italicized words in the texts, they seemed less recognizable when presented on mobile phone screens than on paper (Question 3 in the Semistructured Interview). Regarding support strategy, four out of the six interview participants reported that they had to take notes to facilitate their reading and that it was more convenient to take notes on printed paper (Question 8 in the Semistructured Interview section).

3.2. Students’ perceptions of mobile-assisted EFL reading

To explore students’ perceptions of mobile-assisted EFL reading, we conducted a one-sample t-test by comparing the score for each item in the perception questionnaire, with the midpoint being three (for the descriptive statistics for the items in the perception questionnaire, see Table 3). Regarding the usefulness of the mobile-assisted EFL reading system, although participants wanted to spend more time with printed texts than with mobile-assisted reading applications (Item 2, M = 2.70; SD = 1.02), they believed that the mobile-assisted EFL reading system was convenient to use (Item 3, M = 3.58; SD = 1.01) and efficient for language learning (Item 4, M = 3.62; SD = .93). Regarding ease of use, they found it easy to navigate among chapters (Item 5, M = 3.40; SD = 1.17) but did not think that the built-in functions of the mobile-assisted language reading application were easy to use (Item 6, M = 3.21; SD = 1.00). Encouragingly, participants had a high level of satisfaction with mobile-assisted EFL reading (Item 10, M = 3.52; SD = .95) and showed great willingness to continue to read from mobile phones (Item 12, M = 3.73; SD = .84). The advantages and disadvantages of mobile-assisted EFL reading and the reasons for the preference for mobile-assisted EFL reading reported by participants are summarized in Tables 4 and 5, respectively.

Comments from the interview participants could provide more insight into the above results (Question 10 in the Semistructured Interview; see Appendix VI). One participant who preferred mobile-assisted EFL reading remarked in the interview:

I love to read from mobile phones because I can make full use of fragmented spare time to learn English language. I can read when I am waiting in line for lunch or during breaks between classes. Importantly, there is Chinese-English reading mode, which can facilitate my understanding and help me learn English language effectively.

One participant who preferred paper-based EFL reading commented in the interview:

I like reading printed paper books, which gives me a more real, immersed, and enjoyable reading experience. I like writing down what I am thinking when I am reading, and I like underlining the sentences that I think are important or useful. These are helpful for maintaining my focused attention.

4. Discussion

The primary goals of this study were to investigate the impact of reading media on EFL learners’ reading comprehension performance and reading strategy use and explore students’ perceptions of mobile-assisted EFL reading. Overall, the results suggested that students achieved better reading comprehension accuracy and more efficient strategy use when reading from paper than when reading from mobile phones. In terms of reading speed, participants achieved superior reading speed when reading from paper than from mobile phones when they were instructed to read and complete the comprehension test for the first time. Although most respondents (79%) preferred to read from hardcopies for a better reading experience, they were satisfied with the mobile-assisted EFL reading system and displayed a willingness to read from mobile phones to facilitate their foreign language learning.

4.1. Effects of the reading medium on students’ EFL reading comprehension

4.1.1. Effects of the reading medium on students’ EFL reading comprehension accuracy

The present study extended previous research (e.g. Delgado et al., 2018; Kong et al., 2018; Singer & Alexander, 2017a) by investigating language learners’ reading performance across two reading media in the EFL reading setting based on three levels of reading comprehension (i.e. literal, inferential, and evaluative). The results showed that the participants achieved better EFL reading comprehension accuracy at all three comprehension levels during paper reading than during mobile phone reading. First, concerning literal comprehension, participants understood the texts and recalled the information more easily when reading from paper. According to Rupley and Blair (1983), the ability to access the text in its entirety supports readers’ mental construction of the spatial representation of the text by providing fixed spatial cues, which further facilitates readers’ recall and therefore their comprehension. However, the small screen of a mobile phone (the screen sizes ranged from 4.0 inches to 6.76 inches in the present study) restricts readers’ access to the text in its entirety to a great extent (Kerr & Symons, 2006; Zou & Ou, 2020), and therefore makes it difficult for readers to locate useful information.

Moreover, the effects of the presentation format on text legibility may account for variances in reading accuracy. Legibility reflects the adequacy of a text for its intended readership from the surface-level (e.g. typography, colors, and contrast) to deep-level characteristics of the text (e.g. semantic or structural parameters that shape the comprehensibility of a text) (Macedo-Rouet, Rouet, Epstein, & Fayard, 2003). Considering that reading comprehension is an active meaning-construction process (Alexander & Jetton, 2002), legibility may exert a great influence on reading comprehension. On the one hand, although both the printed texts and the electronic texts used in this study had a justified text alignment (i.e. type and size of font; space between lines and paragraphs), the limited size of the phone screen decreased the legibility of the text and therefore restricted the participants’ understanding. This aspect was mentioned by interviewees, who commented that they easily became confused during the mobile reading process because every page seemed identical. In addition, more lines were needed to display a whole sentence on a mobile phone screen, and therefore more cognitive effort might be required for the mental construction of the text meaning. On the other hand, compared with the electronic reading medium, the material substrate of printed paper provides tactile and physical cues of the text (Mangen et al., 2013), which may increase the legibility of the text. Because inferential and evaluative comprehension are largely based on literal comprehension, inferential and evaluation comprehension on mobile phones are consequently restricted. Thus, improving the legibility of the text on mobile phone screens by enhancing typography, colors, and contrast might be the key to promoting a better reading experience and engagement.

Importantly, the inferior use of reading strategies might also lead to lower reading comprehension accuracy during mobile-assisted EFL reading. Previous studies have documented that effective reading strategy use contributes to EFL learners’ superior reading performance (Anderson, 1991; Block, 1986; Chang & Lin, 2019; Huang et al., 2009). According to the results of this study, although the participants had been instructed to read from the mobile-assisted language system for more than two months, they still could not apply the reading strategies (i.e. global, support, and problem-solving strategies) effectively during mobile phone reading. Therefore, it is highly recommended that students should be familiarized with the built-in functions of mobile reading systems and that mobile reading strategy training should be incorporated into language courses to promote students’ reading comprehension performance using these reading media.

4.1.2. Effects of the reading medium on students’ EFL reading comprehension speed

This study contributed important knowledge to the field by revealing a reliable interaction between the reading medium and reading order and showing that when the participants were instructed to read for the first time, they achieved superior reading speed when reading from paper than when reading from mobile phones, while when they read for the second time, no difference occurred in reading speed between the two reading media. These findings are particularly important. Although many previous studies (e.g. Halamish & Elbaz, 2020; Hu & Yu, 2021; Singer & Alexander, 2017a; Singer et al., 2019) have counterbalanced the reading order across media, to the best of the authors’ knowledge, no studies considered the influence of the reading order when reporting and explaining their results. One possible reason for the observed effects of reading order is that participants became familiar with the whole process and the reading materials (the two texts with similar readability levels used in the study were from the same fiction), and they thus achieved a faster speed when they were instructed to read and complete the comprehension test the second time (the reading speed was faster for the second time both from printed paper and mobile phones; although no statistical differences occurred, the reading speed for printed paper was slightly faster than that for mobile phones; see Figure 3). The influence of reading order merits further exploration.

Notably, the results of the perception questionnaire showed that the students believed that they could access information more quickly from mobile phones since they often engage in quick mobile reading in daily life. This finding seems inconsistent with the results of the comprehension tests. However, simply accessing information does not equate to comprehension and retention. For example, evidence from one study indicates that when answering reading comprehension questions, proficient readers quickly scanned a text and deeply processed (e.g. spent more time rereading) only the information relevant to the questions (Lai et al., 2013). When reading from mobile phones, however, students are more likely to process superficially (Annisette & Lafreniere, 2017; Singer et al., 2019), which makes them less aware of the adjustment of their reading speed according to what they are reading. Therefore, instructing students to be more engaged and perform deeper processing in mobile-assisted EFL reading is particularly important to alleviate or eliminate the possible negative effects of mobile reading on students’ reading comprehension performance.

4.1.3. Effects of the reading medium on students’ reading strategy use

It is commonly acknowledged that effective reading strategy use can enhance EFL learners’ reading performance (Anderson, 1991; Block, 1986; Chang & Lin, 2019; Huang et al., 2009). However, this study revealed that participants’ use of all three types of reading strategies with mobile phones was not as effective as that with paper. With respect to global strategies, the limited access to the overview of the organization and structure of the text on mobile phone screens might greatly decrease students’ use of global reading strategies. Four of the six interviewees commented that they could not make an effective global estimation of the information from the mobile phone screen, whereas they had a clear overview of the structure and organization of the printed text. Therefore, on mobile phones, students were inclined to read word by word and sentence by sentence, and retrieving clues and cues was difficult. With regard to problem-solving strategies, as mentioned previously, the participants had difficulty identifying cues and locating useful information in the texts presented on mobile phones due to their unclear mental constructions of the texts; therefore, they could not effectively use the problem-solving strategy on mobile phone screens. Regarding support strategies, although the mobile-assisted EFL reading system was equipped with built-in functions, participants were not familiar with or accustomed to using it. For example, four out of six respondents from the interviews commented that they relied heavily on taking notes when reading to facilitate their reading comprehension, but the electronic note-taking function was not very convenient, and they generally did not use it. Because note-taking is crucial for improving readers’ attention and information encoding and consequently benefits their reading comprehension (Chang & Ku, 2015), it is of great importance for developers to improve note-taking functions and for instructors to familiarize students with built-in functions before engaging them in mobile phone reading.

4.2. Students’ perceptions of mobile-assisted EFL reading

Overall, this study showed that the participants generally had positive perceptions of mobile-assisted EFL reading. They greatly appreciated the portability and convenient built-in functions of the mobile reading system, and they stated that they would continue to read from mobile phones to assist their EFL learning. In the current stage, however, most participants (79%) preferred to read from hardcopies for a better reading experience in terms of the convenience of taking notes, the real feeling of reading, and better reflection, focus, and retention. In particular, the convenience of taking notes and better focus were the two most common responses reported by respondents as reasons for their preference for reading from paper. Because note-taking is an efficient way to improve readers’ attention and facilitate their reading comprehension (Chang & Ku, 2015), it is not surprising that students predominantly like to take notes while reading. Therefore, as mentioned, improving the note-taking function and familiarizing students with the built-in functions are crucial for successful implementation of mobile-assisted EFL reading. With respect to focus, students reported that they easily became distracted due to the communication attributes of mobile phones, which might influence their reading comprehension by reducing their available working memory resources (Liu & Huang, 2016). As Liu and Huang (2016) noted, distraction is not a new problem, but the adventure of mobile learning has pushed this issue to a new level and drawn more attention to it. Furthermore, a less focused state while reading can lead to a comparatively shallow reading style. According to Craik and Lockhart’s levels of processing theory, deeper levels of mental processing contribute to longer retention, while shallow levels of mental processing lead to shorter retention (Craik & Lockhart, 1972). Poor retention of the text, as the present study revealed, was an important variable accounting for students’ dislike of mobile-assisted EFL reading. However, this finding was based on participants’ subjective reports of their text retention. Therefore, future research is required to quantitatively and comprehensively measure the effects of mobile phones on EFL learners’ retention of the text. Overall, it can be concluded that, on the one hand, given the dominant preference for reading from paper in the EFL setting, it is premature to abandon hardcopies. On the other hand, a large proportion of the participants considered the mobile-assisted language learning system useful for their foreign language learning and wanted to continue to read from mobile phones to improve their language proficiency. Therefore, it is strongly recommended that the problems mentioned previously should be addressed, such as developing more user-friendly built-in functions, improving the legibility of the text on mobile phone screens, and instructing students to use reading strategies effectively.

5. Conclusion

The present study contributes to the growing knowledge of college EFL learners’ reading comprehension performance and the intersection of reading comprehension performance with the use of mobile-assisted and paper-based reading media and students’ perceptions of these reading media. Through an understanding of how these factors are intertwined, a better understanding of EFL learners’ reading processes with different media can be achieved. Overall, this study’s results suggest that students had superior reading comprehension performance when reading from paper than when reading from mobile phones. Encouragingly, although most participants preferred to read from hardcopies for a better reading experience, they perceived mobile-assisted EFL reading positively and showed a willingness to continue to read from mobile phones to facilitate their foreign language learning. Because mobile-assisted reading is increasingly common in students’ EFL learning settings, this study sheds light on students’ actual reading experiences with mobile phones and printed paper. Therefore, more insightful decisions can be made regarding how to instruct students to read using different reading media based on the reading tasks and EFL students’ reading experience.

6. Limitations and directions for future research

The main limitation of this study is that only intermediate-level EFL learner samples in the higher education setting were included. Therefore, the findings cannot be directly generalized to other samples, such as low- or advanced-level college EFL learners. As previous studies documented that readers at different language proficiency levels use reading strategies differently in both quality and quantity (e.g. Block, 1986; Huang et al., 2009; Lai et al., 2013; Sheorey & Mokhtari, 2001), future research could include college EFL learners at different language proficiency levels and explore how and why they perform differently during mobile-assisted language reading. Moreover, according to the authors’ observations, there is an increasing incidence of mobile-assisted language learning in the primary and secondary education setting in the post-coronavirus (COVID-19) pandemic world. Therefore, empirical research on this topic in addition to higher education settings is warranted. This research can provide more useful information for the field of mobile-assisted language learning.

Another limitation of this study is that it did not provide a picture of how college EFL performance and perceptions of mobile-assisted language reading change over time. The present study suggests that although the participants preferred the printed-paper format, they showed great willingness to continue to read from mobile phones. It would therefore be interesting for future research to conduct a longitudinal study to investigate changes in perceptions and reading skills by collecting data before and after EFL learners have become familiar with and proficient at mobile reading. This would allow us to obtain deeper insight into the effects of mobile reading on students’ language learning and development.

Notes on Contributors

Miss. Xiaoming Yang is a PhD student, studying at the Department of Linguistics, the School of International Studies, Zhejiang University, Hangzhou 310058, China. Her research interests lie in second language acquisition, reading comprehension studies and computer-assisted English language learning.

Professor Jie Hu, PhD, is a professor working at the Department of Linguistics, the School of International Studies, Zhejiang University, Hangzhou 310058, China. She has been focusing on English language education for more than 10 years after her PhD graduation from the University of Warwick, UK. Her research interests include ICT-based English language education, second language acquisition, educational data mining and learning analysis.

Additional information

Funding

This research is supported by the National Social Science Fund of China, China (Grant number: 21BYY024).

Notes

1 Iyangcong is a paid online reading platform developed by Foreign Language Teaching and Research Press with an aim of fostering EFL learners’ interest in EFL reading and improving their reading ability. It covers a wide range of reading materials, including literature, technology and engineering, history and philosophy, etc. There are Chinese, English and Chinese-English reading modes, and there is a built-in dictionary. In addition, readers can take notes and make comments while reading and then share these notes and comments on Iyangcong forums.

2 The College English Language Quiz is a summative assessment for EFL learners at the university where the study was conducted. It is administered periodically (every 2 months) to evaluate EFL students’ learning outcomes.