Effectiveness of educational video games in English vocabulary acquisition: One case in China classroom context

Abstract

Rote memory (RM) has become the primary method of learning vocabulary for decades in China. However, RM is tedious, leading to reduced motivation and concentration. In contrast, Educational Video Games (EVGs) are attractive and fun, which could be an alternative to RM. Although most studies have investigated EVGs’ effectiveness, empirical research in China’s classrooms is still scarce. Besides, the combination of EVGs and traditional classrooms is constrained by the school bell, English syllabus, hardware, etc. Consequently, their results cannot be directly applied to China’s environment. Therefore, our research compares the learning performance between RM and our Snake Game (SG) in pronunciation, spelling, and recognition. 30 junior high school students tried to remember 20 words through RM (the control group); after days, they managed to learn an additional 20 words presented through the SG (the experimental group). It was found that (1) the SG outperforms the RM in pronunciation; (2) the SG is as effective as the RM in recognition; and (3) although the RM is slightly better than the SG in spelling, the shortfall can be redeemed by the continued enjoyment and motivation of the SG. In summary, students are satisfied with the effectiveness and enjoyment of the SG.

Keywords:

• rote memory
• vocabulary
• educational video games
• effectiveness
• traditional classrooms

Reviewing editor:

• John Lee, Editor-in-Chief, Education University of Hong Kong, Hong Kong

Subjects:

• Computer Science (General)
• Bilingualism / ESL
• Study Skills
• Classroom Practice
• Curriculum Studies

1. Introduction

English plays a crucial role in China’s education system. As a foreign language, English has become a compulsory subject from primary to postgraduate (Di et al., 2019; Hu, 2005). To a large extent, high marks, as the measurement of English proficiency, are one of the criteria for entering universities and companies. Therefore, teachers and students strive to obtain more marks in high-stakes examinations (College Entrance Examination, College English Test, IELTS, and TOEFL, etc.) (Haidar & Fang, 2019; Yu & Liu, 2022).

In China’s education system, English education has been defined as dominantly exam-driven, teacher-centered, and grammar-focused (Yu & Liu, 2022). The major intention of teachers is to cover the national syllabus, and students are trained as reserved, reticent, and passive learners (Wenfeng & Gao, 2008). Since most examinations focus on students’ ability to read and write, vocabulary size and grammar play fundamental parts in obtaining high marks. As the class length is limited, teachers usually teach grammar, and vocabulary is left to students to acquire in self-study sessions. To acquire vocabulary, students employ lots of vocabulary learning strategies, including rote memory (RM), meaning-oriented notetaking, oral and visual repetition, repetition of words’ spelling and sound, bilingual dictionary (Gu, 2018), Chinese equivalents, pronunciation, guessing, word structure, phonological regularity (Zu et al., 2021), to name a few (Fu, 2021). Among these vocabulary learning strategies, RM is the most frequently used in classroom settings (Zu et al., 2021). However, RM is tedious, and it is difficult to maintain sustained concentration and motivation (Di et al., 2019). Consequently, most students fail to acquire the required vocabulary for each lesson. Finally, it is easy to fall into a vicious cycle; poor vocabulary results in ignorance of English lessons further leads to low marks in exams, then causes and erodes students’ confidence, and ultimately a loss of interest in learning English altogether.

The rapid development of technology-related changes has altered teaching and learning methods (Katemba, 2019; Yu, 2018). Since students usually spend much leisure time playing digital games (DGs) of their own volition (Vásquez et al., 2019), researchers strive to employ the motivational power of DGs to keep students motivated to learn vocabulary. Educational video games (EVGs) have been defined as DGs intended to serve a learning purpose (Sánchez-Mena et al., 2019), aiming to simultaneously achieve entertainment and learning performance (Martinez et al., 2022). Compared to RM, EVGs can keep students motivated, foster autonomous learning, and stimulate students’ interest (Katemba, 2022; Zohud, 2019). EVGs create an exciting environment where students learn vocabulary and get mental satisfaction through overcoming challenges (Chen & Hsu, 2020; Lan, 2015; Wu & Huang, 2017). Besides, EVGs not only reduce students’ anxiety but also cultivate their interests and motivation in the acquisition of vocabulary (Che & Lee, 2018; Katemba et al., 2022). Also, EVGs not only maintain students’ intrinsic motivation (Karaaslan et al., 2018) but can also improve their ability to solve problems and develop cooperation skills (Anyaegbu et al., 2012).

Research on the effectiveness of EVGs demonstrates positive impacts (Chen & Hsu, 2020; Lan, 2015; Sandberg et al., 2014; Yu, 2018). Reference (Ebrahimzadeh & Alavi, 2016) leveraged a real-time strategy game called Warcraft where players strategically devise and direct their units and structures, aiming to secure control over various regions of the map while simultaneously defeating their adversaries. It was found that Warcraft could keep students motivated and achieve learning outcomes in the long-term learning process. Reference (Müller et al., 2018) explored the usefulness of a web-based game for learning English idioms. Their results showed that students could acquire low-frequency vocabulary that was repeatedly exposure during gameplay. Reference (Yang & Benazir, 2018) leveraged an online role-playing game to investigate the effects of students’ English proficiency on learning performance. Their results showed that both high and low English proficiency students can improve their learning performance. Reference (Katemba et al., 2022) leveraged a real-time and game-based interactive platform called Kahoot to investigate whether there is a difference between female and male students in learning performance. Kahoot constructs an engaging learning environment for students, enabling them to participate in the classroom through quiz-style questions. It was found that although the male group outperformed the female group, both improved in learning performance and had positive attitudes towards Kahoot.

Some researchers also have compared the effectiveness of the EVG method and other activities. Reference (Sandberg et al., 2014) explored the added value of an adventure game through a comparative experiment. The control group remembered vocabulary via online learning applications, while the experimental group remembered vocabulary using the adventure game. The results showed that learning in a game environment can achieve better learning outcomes. In reference (Mostafa & Haghighatpasand, 2019) research, they allocated 20 university students in each group of 120 hours to learn vocabulary. The results showed that EVGs have better learning achievement than RM. Similarly, reference (Sedigheh & Behbahani, 2013) conducted comparative research at the university. 30 fresh undergraduates learned 35 words in 10 hours, and the results showed that EVGs outperformed the RM in short and long memory. In reference (Katemba, 2022) research, they compared the learning effectiveness of online games, educational videos, and conventional teaching in language learning. The finding showed that online games, and multimedia learning, have a significant enhancement in students’ vocabulary knowledge than the conventional group.

Although some research proved EVGs can increase vocabulary size and have long retention of learned words (Ebrahimzadeh & Alavi, 2016; Sedigheh & Behbahani, 2013), their results cannot be directly applied in China classroom environments. Firstly, EVGs are not appropriate for the hardware at school and for students’ level of computer literacy. Certain commercial-off-the-shelf EVGs, such as Warcraft, Sims, Runaway, etc., were leveraged in some research. This choice stems from the presence of embedded English content within these games. Consequently, players are presented with the exposure to inadvertently cultivate English literacy during gameplay. However, such EVGs tend to be resource hogs (screens with high resolution, large RAM requirements, access to the internet, etc.). On the other hand, the computers that are available for students are traditionally older and less powerful machines and have low processing power (Rice, 2007). Besides, complex operations increase the learning cost, which potentially limits their adoption for classroom use. Secondly, these EVGs do not fit the time constraints of the typical class period (Zohud, 2019). That is, EVGs need to be specifically designed so that one or two learning objectives can be typically achieved within 30-45 minutes (Alsuhaymi & Alzebidi, 2019). Complex EVGs are often challenging, and students can spend several hours playing and learning (Li, 2017), threatening the visual health of students. Thirdly, the effectiveness of these EVGs is not enough to support a busy, fixed schedule and a heavy syllabus. Students have urgent vocabulary demands and a regular syllabus. Students have to remember lots of words to keep pace with the daily English lesson dictated by the syllabus. Fourthly, the assessment type normally focuses on the ability to recognize (multiple-choice questions). The ability to write is also important to get high marks in high-stakes examinations. Lastly, it’s crucial to note that English and Chinese stem from distinct language families, and Chinese students’ exposure to authentic English environments may differ from peers in Western countries. Proficiency levels possibly vary due to factors such as educational resources, motivation, learning methods, and living conditions.

EVGs in China are a newly emerged phenomenon in the educational domain but turn out to be an under-investigated area (Hong et al., 2022; Li, 2017). Also, rare research has studied the application of EVGs in the classroom context. Given the barriers to application in China classrooms, this study developed a game incorporating the learning theory and game characteristics to ensure that the game is suitable for the syllabus, the target audience, the short time slots afforded by school schedules, and the hardware at schools. Moreover, since students usually remember vocabulary through RM, this research will compare the value of EVGs with RM. Apart from the ability to recognize, this study will explore the effectiveness of EVGs in spelling and pronunciation. In this way, this study explores the feasibility of the incorporation of EVGs and classrooms. Based on the above discussion, this study proposes four research questions:

1. In classroom contexts, are EVGs more effective than RM for learning pronunciation?

2. In classroom contexts, are EVGs more effective than RM for recognition and spelling?

3. What types of vocabulary learning strategies do students report?

4. What are the perspectives of students on learning vocabulary through EVGs?

Based on the research question A and question B, this study proposed three hypotheses:

Hypothesis 1:

In classroom contexts, EVGs are more effective than RM for learning pronunciation.

Hypothesis 2

a: In classroom contexts, EVGs are as effective as RM for learning recognition.

Hypothesis 2

b: In classroom contexts, EVGs are as effective as RM for learning spelling.

The structure of the paper is as follows: Section 2 presents the theoretical background. The research methodology is described in detail in Section 3. The research results are presented in Section 4, while a critical discussion is provided in Section 5. Finally, the conclusions are given in Section 6.

2. Theoretical background

This section introduces vocabulary learning theory, fundamental game characteristics, and the method of evaluation, all of which provide a comprehensive grounding for the development of EVGs and the methodology of this study.

2.1. Vocabulary learning theory

There are two methods of learning vocabulary: intentional learning and incidental learning (Hulstijn, 2003). Intentional learning (e.g., RM) refers to always consciously desiring to learn something new, and it is the primary way of gaining vocabulary (Hulstijn, 2003). Incidental learning means vocabulary acquisition is the by-product of doing other activities (e.g., reading books, watching movies, and listening to songs) (Hulstijn, 1992). Incidental learning is usually the ground learning theory of commercial-off-the-shelf EVGs (role-playing games, simulation games, adventure games, etc.). However, enormous scenarios and immersed game environments, dialogues, and complicated operations increase the learning cost. On the other hand, intentional learning requires focused repetition strategies on the pronunciation, form, and spelling of vocabulary, which can be completed in a short time (Amirreza & Bowles, 2019; Batia & Hulstijn, 2001). Therefore, intentional learning (giving the meaning of the target word) is more practical and reasonable for students than incidental learning (inferring the meaning of the target word) (Hulstijn, 1992).

Reference (Nation, 2001) explains three steps of the vocabulary learning theory, including noticing, retrieval, and creative use. Noticing represents that the word can capture the attention of students (e.g., students attempt to complete a task). When it comes to the task, a word-focused task can reinforce vocabulary acquisition and retain more words, compared to incidental learning (Chen et al., 2021). Retrieval means that students can actively retrieve targeted words from memory (e.g., students attempt to recall words’ meaning and form). Besides, multimedia inputs (e.g., pictures, text, sound, etc.) about the targeted words can activate retrieval of relative memory, which facilitates robust memory (Klimesch, 2013). Creative use presents the usage of learned words in different contexts (e.g., students actively use learned words in English lessons).

The effectiveness of a certain task in promoting vocabulary learning is determined by the task-induced involvement that consists of Need, Search, and Evaluation (Laufer & Hulstijn, 2001). Need will be induced when EVGs are interesting and motivating enough for students to continue playing. Search will be induced when students attempt to find the unknown vocabulary during gameplay. Evaluation will be induced when students are encouraged to create original content using acquired words outside EVGs.

2.2. Game characteristics

Although there are generally lots of characteristics of DGs, not all of them facilitate learning performance. Therefore, this subsection will introduce seven typical EVG elements highlighted by previous studies to stimulate learning outcomes (Anyaegbu et al., 2012; Blake, 2011; Mayer, 2003; Sandberg et al., 2014). In this way, EVGs can achieve learning purposes by incorporating the game features that catalyze learning outcomes.

1. ‘Rewards’ in EVGs refers to various incentives, points, prizes, equipment, virtual money, and medals that players receive as a result of their achievements or progress within the game. These rewards are external stimuli to motivate students, enhance engagement, and reinforce learning outcomes.

2. ‘Goals’ refer to specific objectives or targets that students strive to achieve. Goals could be the learning objectives in EVGs. Clear learning goals can motivate them, enhance focus, foster persistence, enable progress tracking, and boost their sense of achievement and confidence, during gameplay.

3. ‘Rules’ are the whole skeleton EVGs. Students are required to learn rules in an explicit manner, which ensures EVGs smoothly run. Furthermore, formal rules should support learning objectives. Also, rules lead students to obtain learning knowledge by achieving goals (Sandberg et al., 2014).

4. ‘Control’ means that students fully control the whole learning process. it encompasses a range of influences on player engagement, learning outcomes, and overall gaming experience. Students feel sad when other people interrupt their learning process (Anyaegbu et al., 2012).

5. ‘Challenge’ is the subjective feeling of students and is the difficulty level of learning objectives. Appropriately calibrated challenges can foster a sense of accomplishment, sustained interest, and intrinsic motivation, contributing to improved learning outcomes and a more immersive educational experience (Sandberg et al., 2014).

6. ‘Sensory stimuli’ typically consist of visual and auditory stimuli. Effective utilization of sensory stimuli (e.g., text, voice, images, etc.) can enhance comprehension, memory recall, retention of memory, and emotional resonance, ultimately contributing to more effective learning outcomes (Mayer, 2003).

7. ‘Feedback’ plays a pivotal role in facilitating learning by providing learners with personalized suggestions for their performance and guiding their progress (Blake, 2011). Timely and constructive feedback can enhance understanding, correct misconceptions, and motivate further exploration. Additionally, well-designed feedback mechanisms can foster self-assessment and metacognitive skills, contributing to more effective learning outcomes and a deeper engagement with educational content.

Integrating the game characteristics that directly facilitate the learning performance can decrease the learning cost and directly catalyze the learning performance.

2.3. Evaluation

To compare the effectiveness of EVGs with RM, a binary method and the more fine-grained Levenshtein distance will be used to measure the students’ learning performance in this study.

2.3.1. Binary method

The binary method is straightforward. If correct, students obtain one point, otherwise, they obtain zero points. The equation of the binary method is presented as Equation 1(1) $$\mathit{\text{score}}=\{\begin{array}{c}0,\mathit{\text{\hspace{1em}Otherwise}}\\ 1,\mathit{\text{\hspace{1em}Correct}}\end{array}\mathrm{ }$$(1) : (1) $$\mathit{\text{score}}=\{\begin{array}{c}0,\mathit{\text{\hspace{1em}Otherwise}}\\ 1,\mathit{\text{\hspace{1em}Correct}}\end{array}\mathrm{ }$$(1)

2.3.2. Levenshtein distance

The Levenshtein distance serves as a metric for gauging the similarity between two strings. Essentially, the greater the likeness between these strings, the shorter the Levenshtein distance. This term quantifies the minimum count of editing operations necessary to transform one character string into another and is also termed the “edit distance.” These operations encompass inserting, deleting, or substituting characters.

The Levenshtein distance finds extensive application in showcasing disparities between words or sentences. Furthermore, it finds utility in representing the divergence between players’ learning sequences (Feng & Yamada, 2021). In this vein, the Levenshtein distance can also be applied to illustrate the nuanced discrepancies between students’ spellings and the correct answers. The Levenshtein distance presents a more refined evaluative approach, affording the capability to quantitatively measure students’ learning performance with greater precision. Therefore, this study will evaluate the competence of spelling by comparing the students’ spelling and correct spelling. The equation of the Levenshtein distance is presented as Equation 2(2) $${\mathit{\text{dis}}}_{x,y}=\{\begin{array}{c}i,\mathit{\text{if j}}=0\\ j,\mathit{\text{if i}}=0\\ \mathit{\text{min}}\{\begin{array}{c}{\mathit{\text{dis}}}_{x,y}\left(i-1,j\right)+1\\ {\mathit{\text{dis}}}_{x,y}\left(i,j-1\right)+1 \\ {\mathit{\text{dis}}}_{x,y}\left(i-1,j-1\right)+1\end{array}\mathrm{ }\mathit{\text{otherwise}}\end{array}\mathrm{ }$$(2) : (2) $${\mathit{\text{dis}}}_{x,y}=\{\begin{array}{c}i,\mathit{\text{if j}}=0\\ j,\mathit{\text{if i}}=0\\ \mathit{\text{min}}\{\begin{array}{c}{\mathit{\text{dis}}}_{x,y}\left(i-1,j\right)+1\\ {\mathit{\text{dis}}}_{x,y}\left(i,j-1\right)+1 \\ {\mathit{\text{dis}}}_{x,y}\left(i-1,j-1\right)+1\end{array}\mathrm{ }\mathit{\text{otherwise}}\end{array}\mathrm{ }$$(2) where the x and y present the two words; i and j present the length of x and y respectively.

3. Research methodology

This section describes the experiment. Our experiment is a quasi-experiment. 40 words were randomly selected from seven grade participants’ mandatory English textbooks. Then, the 40 words were randomly split into two sets of 20 words. The control group learns one set of words with RM, and the experimental group learns the other set of words with EVG. Our comparative study tests the effectiveness of EVGs in a classroom environment. We also conducted an open-ended questionnaire about students’ attitudes toward learning vocabulary in the EVGs fashion.

3.1. Participants

To avoid the influence of other factors (e.g., school, English teacher, educational background, etc.) as much as possible, participants were recruited from one class randomly selected from a junior high school in Shanxi province, China. 30 students (19 males and 11 females, average age 12 years old)) voluntarily participated in the experiment. Participants can be categorized as English beginners because they studied English at school for one year.

3.2. Research instruments

This part introduces the words leveraged in this study, a description of the SG, and assessment approaches used to examine the effectiveness of the SG.

3.2.1. Vocabulary

40 words were randomly selected from seven-grade participants’ mandatory English textbooks, which is in line with their English syllabus. Therefore, the difficulty of the selected words is appropriate for participants. Since the selected words are part of the regular school curriculum, participants consider that the experiment is not an additional load on their schedule. Every student received some stationery (e.g., pens, pencil cases, erasers) after the research as a gift for taking part.

This study randomly divided the 40 chosen words into two groups of 20 words: the control group (learned by RM) and the experimental group (learned by the SG). Owing to their prior exposure to English instruction during primary education, participants inevitably possess a degree of acquaintance with vocabulary selected from junior high school English textbooks. China’s national English syllabus for middle school tailored to students’ age, cognitive development, and subject needs, ensures alignment of content with their learning progression. Hence, prior to the experiment, we employed statistical analysis to prove that there were no significant differences in students’ initial grasp of the two sets of words. This ensured that the learning outcomes could be compared between the two groups after the experiment. The characters of words in the experimental group and control group are presented as Table 1:

Table 1. Characters of words.

	Control group	Experimental group
Total number of letters	113	112
Total number of words	20	20
Mean	5.65	5.60
Variance	1.19	1.20
Median	6	5.5
Maximum length	7	7
Minimum length	3	3

3.2.2. Snake game

The authors developed the SG with the Pygame package in the Python Language. The authors chose SG to conduct the empirical experiment because (1) the study aims to explore the feasibility of applying EVGs in rural classroom environments in China. As a result, we need to take the school infrastructure, English syllabus, and the duration of one lesson into consideration; (2) based on the above situations, most of EVGs are not applicable in rural classroom environments; (3) pre-existing games have limited the flexibility of the research; and (4) given the computer skills of students and the hardware at school, the SG is lightweight, playable offline, and easy to operate. Compared to other commercial-off-the-shelf EVGs, the SG has some differences. Firstly, other EVGs stress the competence of recognizing vocabulary over spelling. The SG emphasizes spelling because it requires students to spell words by eating letters. Secondly, it is easy to understand the rules and operations. The operations include the right, left, up, and down. The snake will get longer as it eats letters, and the game will fail once the snake contacts itself. In the SG, there will be one correct letter and one incorrect letter. Therefore, students will try their best to eat the correct letter, which means they need to remember the correct spelling. Thirdly, the SG provides a word-focused task and leverages the trial-and-error principle. Students need to eat every correct letter according to the Chinese translation. Even though they do not know the spelling, they can try to eat a letter, which will not cause the game to end. Students will finish all tasks one by one in one round. If one word is completed without making mistakes, the word will be removed from the tasks. After one round, their spelling and correct spelling will be displayed on the screen to solidify and rectify their memory. Students play round by round until round until they remember all the words. Therefore, the SG can contribute to research objectives because (1) SG incorporating game elements and learning theories can help explore the effectiveness and enjoyment of learning vocabulary through EVGs; and (2) the eating letters mechanism and embedded official pronunciation of SG can help explore the effectiveness of spelling and pronunciation in EVG method.

Figure 1 shows the screenshot while playing and had been numbered with the red highlights for introducing the SG.

Figure 1. Layout of Snake Game. Information is displayed at the top of the game window and the playing part is below. There are three rows in the information area. The first row has the current score, current task, and phonetics of the current task. In the second row, it has the historically highest score and the track students’ spelling. The third row has the historically highest number of remembered words, the last task that participants finished, and the number of remaining tasks. In the playing part, the background is the grid. There is a letter ‘O’ at the top-center, a letter ‘N’ at the right-top, and a red cross at the right-center. The snake is at the right-bottom corner. It has two yellow circles as the snake head and the rest are letters.

1. The current score.

2. The historically highest score.

3. The historically highest number of remembered words.

4. The current task to be spelled.

5. Helps participants track their spelling.

6. The last task that participants finished.

7. The phonetic of the current task.

8. The number of remaining tasks.

9. The correct letter of the current task.

10. The wrong letter to confuse participants.

11. The tip to help participants.

12. The head of the snake.

13. The letters that the snake has eaten.

Entering the SG, the number 4 presents the current task. Students need to control the snake to consecutively eat the letters in the correct order. If students do not know the spelling of the current word, they can guess according to the phonetic (number 7) and pronunciation. The pronunciation was downloaded from the Oxford dictionary and embedded in the SG. Pressing ‘Q’ will play the pronunciation. In addition, the pronunciation will automatically be played before and after the current task for teaching pronunciation. If students do not want to guess, they can eat the tip (number 11) to get the correct spelling so that they can temporarily get the correct spelling. As shown in Figure 1, the current correct spelling is ‘symbol’, and the student has eaten the ’symb’ which can be tracked on the number 5 or the last 4 letters of the snake tail. The next target will be ‘O’ (number 9), while the ‘N’ (number 10) was used to confuse the student. Either eat ‘O’ or ‘N’, the letter will be added to the snake tail. In every step, there will be one correct letter and one confused letter. If students can correctly spell the task, the spelling of the current task at the snake tail will be removed. Otherwise, the letters will be kept on the snake’s tail. The longer the snake is, the larger the risk of failure will be.

Table 2 describes how the SG fits the vocabulary learning theories and the game characteristics.

Table 2. Principles of SG.

Principles	Descriptions
Intentional learning	The SG directly provides the task and spelling.
Noticing, retrieval, creative use	Participants notice tasks and spelling and then retrieve them from their memory; finally, participants can use the learned words in their English lesson.
Reward	Participants obtain 10 points when eating a correct letter.
Goals	Participants keep playing the SG round by round until the tasks are empty.
Rules	1: Participants control the snake to eat the letters. 2: The game fails once the snake contacts itself. 3: If participants eat the wrong letter, the whole spelling of the current task will be added to the tail of the snake; otherwise, it is deleted when finishing the current task.
Control	Participants totally control the process of learning and the gameplay.
Challenge	1: The length of the snake grows longer if participants constantly make mistakes, which makes the snake hard to control. 2: Unknown words are challenges.
Sensory stimuli	1: The SG provide different sound when participants eat the correct letter, the wrong letter, and the tip. 2: The SG provides background music.
Feedback	1: Correct letter obtains 10 points, incorrect letter decreases 10 points, and using tip decrease 30 points. 2: the SG provides the words and their spelling after every round, which facilitates a review of their mistake and enhances memory.

3.2.3. Assessment approaches

To evaluate the effectiveness of SG, this study will test pronunciation, recognition, and spelling. In the experimental group, participants can hear the pronunciation from SG, and in the control group, participants can ask the English teacher and classmates. Every participant is required to pronounce every word based on the vocabulary and phonetics (See Figure 2) face-to-face with the English teacher. The English teacher scores their pronunciation through the binary method. The English teacher has been teaching at the junior high school for over 15 years and is experienced enough to score their pronunciation.

Figure 2. Material for testing pronunciation. There are two columns in the picture. The first column is the list of English words, and the second column is the counterpart phonetics.

To test the competence of recognition, this study leveraged a matching test (See Figure 3) that is more difficult than multiple-choice questions. The left column is the Chinese counterpart characters of the 20 English words, and the right column is 20 English words with five alternative words to confuse students. The two columns were separately randomly shuffled when testing. Participants are required to connect the Chinese characters with the correct vocabulary with a line. The score of recognition is accumulating the number of correctly connected words by the binary method.

Figure 3. Material for testing recognition. There are two columns in the figure. The first column is the 20 Chinese characters, and the second column is the 20 English words for matching together.[AQ]

This study used a productive approach to test the competence of spelling. Participants were required to write down the English spelling according to Chinese translations of the selected words (See Figure 4). It is too blunt and coarse-grained to allocate one point for correct spelling, as some words are longer than others. In addition, a score of zero does not mean participants do not remember anything when they cannot completely spell a word. Therefore, this study calculates the distance between participants’ spelling and the correct spelling per vocabulary by Levenshtein distance and then sums all the distances as the spelling score, and thus gives the student partial credit for partially spelling a word correctly. The smaller the score, the better the ability to spell.

Figure 4. Material for testing spelling. There are two columns in the picture. The first column is Chinese characters, and the second column is the blank for writing spelling.

3.3. Experimental procedure

This section describes the details of the pretest, posttest, questionnaire, and experimental treatments.

3.3.1. Pretest

To measure the improvement of participants for comparative analysis, every participant was required to do a pretest to assess the initial scores concerning pronunciation, recognition, and spelling. In the pretest stage, the English teacher scored every participant on their pronunciation (see Figure 2), spelling (see Figure 4), and recognition (see Figure 3) of the 40 words (the authors just cite a few words as an example).

3.3.2. Posttest

Participants received a one-day delayed posttest. The difference between the pretest and posttest is the sequence of words randomly changed when assessing the competence of pronunciation, recognition, and spelling.

3.2.3. Questionnaire

The questionnaire was the last step of the experimental procedure. The questionnaire consists of 11 open-ended questions for two primary purposes. The first purpose was to ask about the general situations of participants in learning vocabulary (e.g., How do you usually learn vocabulary?). The second purpose was to explore the attitudes of participants toward the SG (e.g., Does the SG waste time?).

3.2.4. Experimental treatments

This experiment was conducted on the computers (Windows 7 64-bit flagship version) in the computer laboratory and the participants’ classroom at the school. Before experimenting, the English teacher met participants twice (two class sessions) to teach SG. In the first session, the English teacher presented all the rules and operations of the SG. In the rest of the class, participants familiarized themselves with SG and made sure SG could run smoothly on their computers. In the second session, participants got enough practice and the English teacher ensured that every participant understood the mechanism of SG.

Initially, participants learned the 20 words for 30 minutes through RM, which is normally a workload for one self-study session. Next, they did a one-day delayed posttest. After a few days, the same participants learned another 20 words through the SG for 30 minutes. Then, they did a one-day delayed posttest. Finally, a questionnaire was handed out asking about participants’ attitudes toward the SG.

4. Research results

This section answers the four questions raised in this study. To statistically analyze the data, the Shapiro-Wilk test, t-test, and ANOVA were employed to compare the learning performance between SG and RM (Qiao et al., 2022).

4.1. In classroom contexts, are EVGs more effective than RM for learning pronunciation?

An experienced English teacher scores students’ pronunciation after the experiment. The data were collected to test Hypothesis 1: In classroom contexts, EVG is more effective than RM for learning pronunciation. Firstly, Shapiro-Wilk was used to test the normality of data as the sample is small. Next, T-test or ANOVA test was used based on the results of the normality of the data. According to the Shapiro-Wilk test, the posttest scores (p = 0.000) did not follow a normal distribution. Therefore, the ANOVA test was leveraged to compare the mean of the two groups statistically. The results showed a significant difference in posttest scores between the SG and RM at the p < 0.05 level (F = 0.353, p = 0.048). Therefore, the Hypothesis 1 is not rejected. As shown in Table 3, importantly, the mean score of SG is higher than RM. The results indicated that the embedded pronunciation in SG is more effective than RM in pronunciation.

Table 3. Pronunciation data in pretest and posttest.

	Pretest		Posttest
	Game	Rote	Game	Rote
Mean	17.6	17.17	19.5	18.13
Variance	6.11	9.87	1.707	12.05
Shapiro-Wilk	0.000	0.000	0.000	0.000

4.2. In classroom contexts, are EVGs more effective than RM for recognition and spelling?

The pretest and posttest of recognition and spelling were evaluated in binary method and Levenshtein distance respectively to test Hypothesis 2a: In classroom contexts, EVG is as effective as RM for learning recognition, and Hypothesis 2b: In classroom contexts, EVG is as effective as RM for learning spelling. Firstly, Shapiro-Wilk was used to test the normality of data as the sample is small. Next, T-test or ANOVA test was used based on the results of the normality of the data. According to the Shapiro-Wilk test (see Table 4), the pretest and posttest scores did follow a normally distributed population. The results of paired sample t-test showed that the average score of both methods improved at the p < 0.05 level (p = 0.000), and the mean scores increased by 2.97 and 4.1 respectively. Besides, the independent samples t-test was leveraged to compare the posttest scores between the two groups. The results showed that there is not a significant difference in posttest scores between the SG and RM at the p < 0.05 level (F = 0.353, p = 0.048), which indicated that SG is as effective as RM in recognition. Therefore, the Hypothesis 2a is not rejected.

Table 4. Recognition data in pretest and posttest.

	Pretest		Posttest
	Game	Rote	Game	Rote
Mean	8	7.87	10.97	11.97
Variance	3.034	2.395	5.689	2.378
Shapiro-Wilk	0.72	0.82	0.54	0.177

Regarding spelling, according to the Shapiro-Wilk test (see Table 5), the pretest and posttest scores did follow a normally distributed population. The results of paired sample t-test showed that the average distance of the SG and RM statistically decreased at the p < 0.05 level (p = 0.000), and the mean scores decreased by 21.16 and 34.1 separately. Additionally, the independent samples t-test was leveraged to compare the average distance in posttest scores between the two groups. The results showed that there was a significant difference in posttest scores between the SG and RM at the p < 0.05 level (t=-3.489, p = 0.001), and the average score of RM was lower than the SG. The statistical data indicated that although the SG can help students improve their competence in spelling, its’ effect on spelling is not as effective as RM. Therefore, Hypothesis 2b is rejected. The main reason for the results is that even though students were trained to play SG before the experiment, students heavily rely on the fixed mindset of RM. Consequently, students did not take full advantage of the learning theories integrated in SG.

Table 5. Spelling data in pretest and posttest.

	Pretest		Posttest
	Game	Rote	Game	Rote
Mean	61.53	61.07	40.37	26.97
Variance	90.809	99.444	207.551	235.068
Shapiro-Wilk	0.2	0.2	0.2	0.2

4.3. What types of vocabulary learning strategies do students report?

A total of 30 participants submitted questionnaires, and there were 5 open-ended questions to explore the general situations of participants in learning vocabulary (see Table 6).

Table 6. General situations.

Questions	Answers
1: How do you usually learn vocabulary?	29 (RM)	1 (Oline Learning)
2: Do you often use online resources to learn vocabulary?	17 (Yes)	13 (No)
3: Do you often use phonetics to assist learn vocabulary?	26 (Yes)	4 (No)
4: Do you often use pronunciation to assist learn vocabulary?	27 (Yes)	3 (No)
5: Are you confident with your pronunciation?	10 (Yes)	20 (No)

According to the findings from question 1, 29 participants learn vocabulary through RM, and 1 participant uses online learning. According to their answers, there are two self-study sessions per week for participants to learn vocabulary, which is the main period to acquire the required words for the weekly lessons. Under this circumstance, RM is the primary and expert way of acquiring words. Therefore, participants highly rely on RM to help them keep pace with the English syllabus. Question 2 is to investigate the role of online resources in learning vocabulary. 17 participants often use mobile applications to learn vocabulary (mainly searching for unknown words and official pronunciation), while 13 participants do not. According to their answers, they rarely learn vocabulary after school. The usage of online resources is during doing English homework, which is more convenient than searching in textbooks. Consequently, although the Internet and mobile devices have been developed a lot, participants rarely benefit much from online resources in learning vocabulary.

Based on findings from questions 3 and 4, it can be concluded that most participants consider phonetics (26 participants) and pronunciation (27 participants) to be important to help them learn vocabulary in the RM way. According to their answers, most of the vowels and consonants have a relation with one letter or a combination of letters, which facilitates the acquisition of vocabulary. Additionally, participants said that they often feel embarrassed to ask for help from classmates or teachers. Consequently, participants heavily rely on phonetics to guess or infer pronunciation in the classroom context. Also, since English exams do not test the ability to speak, pronunciation is not the priority during self-study sessions. According to question 5, 10 participants are confident with the accuracy of pronunciation, while 20 participants are not. The finding further showed that participants do not require themselves to have correct pronunciation during learning vocabulary during self-study sessions.

Based on the above discussion, RM is still the popular way to learn vocabulary, and participants benefit little from online resources in learning vocabulary. Besides, participants highly rely on phonetics to learn vocabulary and infer pronunciation. However, most participants are not confident with the accuracy of pronunciation.

4.4. What are the perspectives of students on learning vocabulary through EVGs?

Six open-ended questions in the questionnaire investigated the perspectives of participants in learning vocabulary through EVGs (see Table 7).

Table 7. Attitudes toward snake game.

Questions	Answers
6: What do you think is the effectiveness of the Snake Game?	26 (Yes)	4 (No)
7: Does Snake Game waste time?	5 (Yes)	25 (No)
8: Do you like pronunciation incorporated in Snake Game?	20 (Yes)	10 (No)
9: Do the think the snake game is difficult?	2 (Yes)	28 (No)
10: Can Snake Game motivate you to learn vocabulary?	27 (Yes)	3 (No)
11: Any comments about the Snake Game?	–	–

Question 6 and question 7 were to consult the subjective perspective on the effectiveness of the SG. 26 participants were satisfied with the learning speed; 25 participants did not consider that learning vocabulary through SG wastes time. Therefore, most students have a positive attitude toward the effectiveness of the SG. Some comments to support the conclusion: ‘I think the SG does not waste time because it is an appropriate method for me to learn vocabulary’. ‘The SG makes learning interesting with less study pressure’. ‘The SG can capture the attention of learning. Other students who are not satisfied with the effectiveness claimed that they prefer RM to SG.

According to question 8, there are 20 out of 30 did actively listen to the real pronunciation. The SG provides the real-time pronunciation of the current task as long as participants press the ‘Q’ on the keyboard. Also, the pronunciation will automatically be played at the beginning and end of the current task. Most participants said that the official pronunciation helped them rectify their pronunciation. Some participants who do not like using the pronunciation said they are confident with their pronunciation and could pronounce the current task according to phonetics. Also, they sometimes forget to press the ‘Q’. Therefore, the function of automatically playing the pronunciation in SG is quite important to unconsciously improve participants’ pronunciation.

Regarding question 9, 28 participants agreed that the SG is simple. Some comments said: ‘I can get familiar with the rules and operations in a short time’. 2 participants thought the game was difficult because they had not played any DGs ever before.

Regarding question 10, 27 participants consented that the SG could motivate them to learn vocabulary. There are some comments to support this conclusion: ‘This game is interesting because I want to get more scores than my classmates’. ‘Eating letter as the way of spelling is novel’. ‘I like the pronunciation because it is more convenient than RM’. ‘I can learn vocabulary while having fun’.

Participants provided lots of valuable suggestions to improve the SG in terms of question 11. For example, ‘Apart from the score, the game can add badges for rewarding improvement’. ‘It would be better if the game can look through all vocabulary before playing.’ ‘The game can add the function of customized background music’. ‘The game can add more play modes’.

Based on the above discussion, students have a positive perspective on the enjoyment and effectiveness of the SG.

5. Discussion

English is a crucial subject for Chinese students. However, students do not receive enough assistance with vocabulary from teachers in the exam-driven, teacher-centered, and grammar-focused educational system. RM, as the main method of acquiring vocabulary, is tedious and monotonous. Moreover, since students have an enormous workload in all compulsory subjects, time for learning vocabulary is scarce. As a result, students expect a novel and effective method that incorporates the English syllabus. Therefore, our research explored the feasibility of the combination of EVGs and traditional classroom environments.

The authors developed the SG in accordance with the practicalities of stakeholders. Firstly, both students and teachers can become familiar with SG in a short time. Therefore, the mechanics of SG itself are not a hurdle for them. Secondly, the SG considers the length of a class and the English syllabus. Therefore, students can achieve the learning objectives in one classroom period. Finally, the SG does not require high-quality hardware. Consequently, the SG is friendly and practical for a low budget, because the SG can be played on old school computers.

According to the findings from question A, the learning outcome in pronunciation of SG is better than RM. Obviously, participants improve in pronunciation in SG fashion because the SG not only automatically play the pronunciation of the current task, but also allows participants to listen to it whenever they press ‘Q’. On the other hand, students feel awkward when asking teachers about pronunciation in a traditional classroom environment (Wang & Roopchund, 2015). Reference (Al-Ahdal, 2020) also found that conventional teaching methodology cannot sufficiently train students, while podcasts that help students frequently be exposed to pronunciation improve their English pronunciation. Therefore, compared to RM, students’ pronunciation can improve in SG fashion at the same learning time. This finding implies that game developers should incorporate audio files featuring native speakers as pronunciation models for students. This integration enhances students’ exposure to and practice of correct pronunciation in EVGs.

According to the findings from question B, the learning outcome in recognition of SG is as effective as RM. However, when it comes to spelling, the statistical analysis concluded that RM outperforms SG. The method of testing recognition in this study is more similar to the exam paper than multiple-choice questions because there are lots of words on the paper instead of a few choices. Consequently, participants must clearly remember every word’s format to finish the recognition test. Otherwise, participants get confused when encountering all the learned words because every word on the paper seems the correct answer. During playing the SG, participants need to clearly know the pair of Chinese characters (game tasks) and words to finish tasks, which is good for improving their competence in recognition. Also, the game provides feedback after every round, including participants’ spelling, correct spelling, and Chinese characters, which is good for enhancing their memory. Therefore, the effectiveness of the SG in recognition can be as effective as RM. In the reference (Chen et al., 2021) research, a game with word-focus exercises can help students improve their recognition, which is in line with the findings of our research. On the other hand, SG does not perform better in spelling than RM, because participants can repeat all words more times in RM fashion than in SG in a fixed time. During playing SG, participants need time to eat letters in the correct order, which decreases the exposure of words. In contrast, participants can frequently learn words by RM. Reference (Chen & Hsu, 2020) found that repetitive exposure to a target word can enhance the retention of words over fewer occurrences, and students can gain vocabulary knowledge in game environments. Nevertheless, based on the difference in average Levenshtein Distance in each method, the RM only exceeds around 2 words than SG. Therefore, the learning performance in spelling is acceptable. To enhance the learning outcomes of EVGs, we put forward three suggestions: (1) The initial approach entails increasing word exposure frequency without extending gameplay duration; (2) Adapting the game structure to incorporate additional vocabulary learning theories; and (3) Enhancing the gaming environment by introducing supplementary game elements and diversifying the gameplay components.

According to the findings from question C, participants still learn vocabulary by RM at schools. There are two reasons. Firstly, students are chronically affected by traditional Confucian-heritage cultures of learning, and lack of a native-like English learning environment (Zu et al., 2021). Secondly, exam-oriented education and poor teaching techniques cause RM to be the prevailing low-cost method in schools (Zu et al., 2021). Participants normally use online resources, phonetics, and pronunciation to learn vocabulary, which is in line with the findings of (Fu, 2021; Zu et al., 2021) that students use bilingual dictionaries, pronunciation, and Phonological Regularity to learn and consolidate vocabulary. Consequently, students do not take full advantage of novel technologies to serve their studies (Wang & Roopchund, 2015). As participants require phonetics and pronunciation to assist in learning vocabulary (Zu et al., 2021), it is essential to provide such information in EVGs. Therefore, schools should improve the infrastructure so that teachers and students can reform and diversify their teaching and learning methods. Teachers need to, to the most extent, create an English-friendly and learning-supportive environment where students increase vocabulary volume by actively participating and integrating into the class activities. EVGs are supposed to provide word-related information (e.g., text, voice, images, phrases, phonetics, etc.) to teach and demonstrate the multilevel meanings of words.

According to the findings from question D, participants are satisfied with the effectiveness and enjoyment of the SG, whose results are in line with the reference (Anyaegbu et al., 2012). Participants do not consider the SG to be a waste of time because SG incorporates the learning theory and game characteristics that facilitate learning outcomes, which has the same finding from reference (Sandberg et al., 2014). Participants sometimes forget to press ‘Q’ for listening pronunciation. As a result, EVGs should not only automatically play the pronunciation during gameplay, but also have the function to serve the demands of individuals for actively listening to pronunciation audios. In SG way, students can avoid awkwardness and shyness in traditional classrooms, which is good for self-study sessions. During gameplay, participants get excited when they collect scores and control the snake. Reference (Sandberg et al., 2014) found that rewards in EVGs can motivate students, enhance engagement, and reinforce learning outcomes. Also, participants feel satisfied when they control the whole learning process in SG, reference (Anyaegbu et al., 2012) found that Students feel sad when other people interrupt their learning process. As a result, student-centered EVGs could increase students’ intrinsic motivation. Therefore, they agree that SG is interesting and can maintain sustained attention and motivation.

Overall, the results of this study are in line with some previous works in which EVGs not only motivate students but also facilitate the competence of recognition (Ebrahimzadeh & Alavi, 2016; Lan, 2015), and spelling (Wang & Roopchund, 2015). Besides, this study also found that EVGs can improve students’ ability to pronounce if there are pronunciation audios in EVGs.

6. Conclusion

To explore the feasibility of the combination of EVGs and the traditional classroom environment in China, this study compares the effectiveness between SG and RM. Given the educational setting, syllabus, computer literacy of students, and hardware at school, the authors developed SG. This study reported that EVGs facilitate competence in pronunciation, recognition, and spelling, and most participants are satisfied with the effectiveness and enjoyment of EVGs. Therefore, students would like to spend more time learning vocabulary in EVGs fashion. Therefore, this little shortfall in spelling can be redeemed by the enjoyment and motivation of the SG, which can facilitate students being lifelong English learners. In summary, EVGs can replace or supplement RM for students to acquire vocabulary in classroom environments.

There are five limitations of this study. Firstly, the research only applied to Chinese students. SG is not friendly to foreign students who do not know Chinese because most information in SG is Chinese. Secondly, the target group of the research is students in developing or rural areas where a great proportion of Chinese students get an education. Since students in developed areas can access lots of resources (high-quality hardware, excellent teachers, good learning environments, etc.), they have lots of effective ways to learn vocabulary. Thirdly, the participants in the research are junior high school students. Consequently, the findings from the research possibly do not apply to fundamental and senior high school students. Fourthly, the selection of words in this study relies on China’s education system, which has a national English syllabus from primary school to graduate studies. Therefore, we compare the learning outcomes of the two methods by ensuring that students’ prior knowledge of each set of words is consistent before the experiment. One potential solution in the future is the implementation of a scalar measurement to precisely evaluate the levels of word difficulty between the two groups. Finally, the research only explores the effectiveness of SG. Nevertheless, the effectiveness and enjoyment of SG are practical and acceptable for students and schools. Based on the above limitation, the authors will explore the effectiveness of SG in fundamental and senior high schools in developing areas in China. Also, the authors will continue exploring how to improve the effectiveness and enjoyment of SG. Besides, more EVGs research considering the limited education resources in rural areas is urgent, where students rarely benefit from the advanced techniques as students in developed areas. Consequently, students’ English proficiency in rural areas is not competitive with students in developed areas. Given that English proficiency differs among students, EVGs with artificial intelligence are necessary to dynamically adjust the learning pace according to the individual student’s English proficiency. A general framework for developing an EVG that can facilitate the listening, speaking, recognition, and spelling of vocabulary in limited education resources is pressing. When EVGs that fulfill real situations of stakeholders can achieve more learning outcomes and enjoyment than RM, students in rural areas can use EVGs in the classroom. Then, they further improve their English proficiency at a low learning cost and become lifelong English learners.

Acknowledgement

The authors would like to thank the head of the junior high school who agreed with the conduction of the experiment at school. The authors would like to thank the 30 participants and the English teacher, without whom the experiment could not be conducted. Thanks, John R Woodward and Atm S Alam, who supported methodology, and statistical analysis. Massive thanks to Lei Liu, Sha Wang, John R Woodward, Atm S Alam, who provided valuable advice on the development of the SG.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the China Scholarship Council with the Queen Mary University of London under Grant 202006930007.

Notes on contributors

Jianshu Qiao

Jianshu Qiao is a candidate Ph.D. in the Game AI group at the Queen Mary University of London. My research focuses on developing a framework for the design of EVG that sustain students’ durable motivation and concentration in learning English, since a self-designed EVG gives more flexibility in research. This paper confirmed that lightweight EVG not only can integrate into classrooms but also has acceptable effectiveness and enjoyment in learning vocabulary. Based on this foundation, we plan to incorporate artificial intelligence to personalize learning experience based on individual students’ English literacy. Then, we will improve the entertainment of EVG and integrate more learning theories to maximize their learning potential. By sharing findings, we hope to provide a blueprint for other researchers to develop effective, engaging, and intelligent EVGs that align with the specific needs of their school infrastructures, schedules, English syllabus, and the computer literacy of students and teachers, etc.

John R. Woodward

Dr. John R. Woodward is Reader in Computer Science and is the Head of Department (Computer Science) at Loughborough University. He received the B.Sc. degree in theoretical physics, M.S. degree in cognitive science, and Ph.D. degree in artificial intelligence from the University of Birmingham, Birmingham. He has previously led the Operational Research group at Queen Mary, University of London (http://or.qmul.ac.uk/people.html) and prior to that lectured at the Universities of Stirling, Nottingham and Birmingham. He was with the European Organization for Nuclear Research (CERN), Switzerland, where he conducted research into particle physics, the Royal Air Force as an Environmental Noise Scientist, and Electronic Data Systems as a Systems Engineer. His research interests include Machine Learning, Operational Research, Optimization, Artificial Intelligence, Computational Intelligence And Automatically Design Algorithms.

Atm S. Alam

Dr. Atm S. Alam is currently an Assistant Professor at the Queen Mary University of London. He received the BSc degree in information and communication engineering from the University of Rajshahi, Bangladesh, the MSc degree in telecommunications and computer networks engineering from London South Bank University, and the Ph.D. degree in wireless communications from The Open University, UK. His research interests include the areas of intelligent wireless communications and networks, enhanced teaching and learning, gamification for creative teaching and learning, intelligent transport systems. He is also an Associate Fellow of Higher Education Academy. Previously, he worked on several European and U.K., funded projects as a Post-Doctoral Research Fellow.