Investigating preuniversity EFL teachers’ perceived wash-back effects of university entrance exams and teaching materials on students’ learning objectives and teachers’ class performance

Abstract

This study investigated the preuniversity EFL teachers’ perceptions toward the wash-back effects of the University Entrance Exams and the preuniversity students’ textbook (English 1 & 2) on the students’ communicative learning objectives and their teachers’ class performance in Shiraz, Iran. Likewise, to investigate if the two sources are aligned toward the objectives of the educational system in our EFL setting was pursued. To this end, The Teacher Textbook Evaluation Scale questionnaire was administered to 40 preuniversity EFL teachers. Using the SPSS version 17 and a two-way ANOVA, the results of participants’ perceptions showed that although the English textbook was designed to follow the communicative teaching and learning objectives, the teachers were influenced by these types of exams being against the purposes of the syllabus. Also, these tests and the supplementary materials showed negative wash-effects on the students’ learning objectives and teachers’ class practices. Teachers’ different years of experience and their views on the effect of university entrance exams on the education goals and the way teachers use the materials were statistically significant, but their gender distinction was nonsignificant. These results coped with those of the previous studies at different educational levels.

Keywords:

• Wash-back effect
• high-stakes tests
• textbook evaluation
• teaching materials
• communicative goal

PUBLIC INTEREST STATEMENT

University Entrance Exams, as high-stakes tests, affect many candidates to be selected from a population registered. The selected candidates continue their studies in a university seeking a BA/BS degree. The case under study in Shiraz, Iran is to find the preuniversity English teachers’ views toward the negative impact of these exams on their students’ communicative goal of learning English. To this end, a questionnaire was given to 40 English teachers seeking their perceptions. Results showed that these teachers were influenced by these types of tests helping their students to be accepted in a university. Thereby, they change their teaching methods and the materials used against the communicative goal of the English textbook to engage the EFL learners in academic activities as expected. This type of study and the like would provide a platform for education authorities, test designers, teachers, students, parents to cooperate and prepare programs and syllabi toward the English learning goal in non-English speaking environments.

1. Introduction

Wash-back refers to the impact that a test has on teaching and learning. Bachman and Palmer (1996) make a distinction between the extent or intensity of wash-back and its direction (beneficial or damaging). Bailey (1996) states that the design of the test and the tasks it includes can guide wash-back in a helpful or harmful direction. Regarding intensity, Cheng (2005) believes that it is the degree to which the testees adjust their behavior to the demands of a test. If wash-back encourages the forms of teaching or learning intended by the test developers, one can say it has a beneficial direction; however, if it discourages the forms of teaching or learning, it has a damaging trend. Hamp-Lyons (1998) states that the appropriateness of wash-back depends on both the judgments which are made and the educational goals that are pursued. There are also some wash-back studies on high-stakes examinations in different countries such as Sri Lanka (Wall & Alderson, 1996), Hong Kong (Cheng, 1997a, 1997b, 1999), and Israel (Shohamy, Donitsa-Schmidt, & Ferman, 1996) where English is used as a foreign or second language. They investigated the effects of high-stakes tests on teaching and learning (Andrews, Fullilove, & Wong, 2002; Burrows, 1998, 2005; Cheng, 1997a; Lam, 1993; Watanabe, 1997). High-stakes tests influence different stakeholders such as students, classroom teachers, administrators, materials developers, and publishers. Likewise, the results of Nur Cholis and Rizqi (2018) study in Indonesia showed a positive wash-back effect on Senior High School English teachers’ attitudes and their teaching methods. In the same vein, Gokturk Saglam (2018) investigated the effect of wash-back of a university English language proficiency test in the Turkish context showing both negative and positive wash-back effects exerted on teaching. Another study carried out by Akpinar & Cakildere (2013, cited in Gokturk Saglam) in Turkey was related to the wash-back impacts of high-stakes tests on both written and reading skills of the personnel. Their findings demonstrated strong negative wash-back effects on their respondents’ performance. According to Bailey (1999), learners are the most important participants because the wash-back processes have a direct relationship with language learning. Moreover, there is a lack of specific study on the analysis of textbooks and their effects on learners, teachers, learning processes, and teaching methodology in our EFL setting. So, this study is going to fill this gap by focusing on the analysis of the preuniversity English textbook using Bailey’s framework. The researchers have mainly concentrated on the possible impact that University Entrance Examination (UEE), as high-stakes tests, may have on the way English teachers at the preuniversity level teach and use the assigned textbook as the first objective of the study. The second aim is to examine if the direction of the two sides of the coin, UEEs, and the English textbook, is mutually harmonized to support each other toward the communicative objectives of teaching and learning English in our EFL setting. The third goal is to find out if English teachers’ years of teaching English and their gender distinction may be influential factors to change their views against the wash-effect of UEEs and the teaching materials employed against the goals specified by the education authorities in Iran.

Following the above objectives, the research questions are as follows:

1. How are the preuniversity English teachers’ perceptions toward the UEEs’ wash-back effect on the preuniversity EFL students’ communicative learning objectives and the way they teach?

2. How are the preuniversity English teachers’ perceptions about the mutual alignment of the English textbook and the UEEs toward the students’ communicative learning objectives?

3. How do teachers’ gender and years of experience affect their views about the wash-back effect of UEEs and the English textbook on EFL students’ learning goals and their class activities?

1.1. Theoretical models of wash-back

Various models of wash-back have been developed; yet, wash-back studies are so diverse in whether and how teachers, learners, and others change their behavior to address test demands. Wash-back effects depend highly on context. Therefore, researchers have proposed different theoretical frameworks to account for the variation they have observed. One of the researchers who proposed a basic model of wash-back was Hughes (1993). He stated that there were some distinctions between the effects on participants such as those who were affected by tests (namely, teachers, learners, and materials writers), processes (namely, teaching and learning activities). Finally, products which were the outcome of these processes (e.g. scores on tests, courses, teaching materials and so on).

Later, Bailey (1996) developed a model and represented it in the form of a diagram and stated that there were relationships among test design considerations, participant values, motivations and resources, and the perceived importance and difficulty of the test. Because the second and third issues are related to social and individual differences, participants may be affected by a test differently in the same general context. Figure 1.1 illustrates the model proposed by Bailey (1996):

Figure 1. A basic model of wash-back (Bailey, 1996, p. 264)

Bailey’s (1996) framework was used in this study because she specified some different participants, including researchers, and the types of products that might be affected by an examination. Furthermore, she mentioned that these products might change other products as well (e.g. research results can feed into materials, curriculum design, and teaching). Then, she made a distinction between “wash-back to the learners”, and “wash-back to the program.” By the former, she meant the result of supplying “test-derived information” to the test-takers, and by the latter, she meant the result of providing information to all the other participants.

1.2. Background literature on wash-back effect of high-stake tests and English textbooks

Lam (1993) defined textbook wash-back as the impact of high-stake tests on teaching materials which led material writers to publish exam-oriented materials designed based on the needs of students and their teachers who wanted to prepare them for such trials. The available studies discussed wash-back on materials regarding their content, classroom use, and their effect on test performance. Initial research has supported the fact that the content of exam-oriented materials had influenced textbook wash-back on teaching materials. For instance, in Hong Kong, Lam (1993) analyzed the content of two books designed for a public proficiency exam (NUE) based on their range of skills, the authenticity of materials and task types contained. Lam concluded that most of the textbooks included exercises which had the exam format. Similarly, Watanabe (1996) in Japan worked on teaching materials before university entrance examinations. The materials were based on the past exam papers. The analysis of the content of these materials was based on the calculation of the percentage of task types reflected in the exam. The results showed that wash-back existed on materials.

However, the following studies suggested that exam-preparation materials could have an uneven relationship to the exam they are preparing for. Hamp-Lyons (1996, 1998)) focused on the content of five TOEFL test-preparation textbooks. The results of the study indicated that “the skills promoted by the books consist of (a) test-taking strategies and (b) mastery of language structure’s, lexis and discourse semantics that have been observed on previous TOEFLs” (Hamp-Lyons, 1996, p. 6). The researcher concluded that the materials could have negative wash-back on teaching and learning as teachers and learners would find themselves teaching and learning discrete chunks of language rules and vocabulary items without context or even much co-text and might lead to “curricular alignment.”

Furthermore, IELTS exams’ teaching materials used to prepare students had impressive results. Wang (1997) used the Instrument of Analysis of Textbook Materials (IATM), designed by Bonkowski (1996). He analyzed sample units of two IELTS exam-preparation textbooks and concluded that the IELTS test did have an influence on the content and format of the preparation books concerning content except on book methodology.

Lam (1994) in his study on the classroom use of exam-oriented materials in Hong Kong concluded that teachers were like “textbook slaves” and “exam slaves.” The former refers to focusing too much on the exam manuals and the latter concentrates on past papers in exam classes rather than using materials devoting to students’ learning. He believed that this dependency on exam preparation materials was a kind of negative wash-back because instead of authentic materials, they used commercial ones. In another study in Hong Kong by Cheng (1997b), he concluded that teachers paid more attention to the syllabus of the exam rather than the manuals. This indicates the effect of wash-back on the content of teaching.

Based on extensive interviews with teachers in Europe, Wall and Horak (2006) found that the TOEFL preparation textbooks were the center of the courses investigated. The researchers believed that the teachers used the exam books as the content and sequence of teaching. Moreover, according to them, this was a sign of negative wash-back. Wall and Horak stated that teachers relied on the exam textbooks to satisfy what the students expect, to ensure adequate coverage of exam materials, and to compensate for lack of other resources. On the other hand, Saif’s (2006) observation reported that the teacher modified and adapted the test materials based on the students’ needs and the test objectives.

Besides exam textbooks, supplementary materials were used by teachers to prepare students for examinations. For instance, Nikolov (1999) stated that those materials were local publications that focused on grammar and represented exam wash-back effect. Wall (1999) concluded the same results which were the use of supplementary material books to compensate for lack of grammar in the course-book. In Hawkey’s (2006) study, based on class observation and also a teacher questionnaire about whether the materials other than the primary textbook were used, the researcher revealed that teachers, other than test-related materials, also used additional materials targeting specific language skills and components from a variety of sources outside the exam, e.g. the press, TV, radio, and Internet.

Based on the study done in Japan and his classroom observation, Watanabe (1996, 1997)) found out that translation and grammatical explanation were used in two types of university exam preparation lessons. He believed that educational background, beliefs about teaching and attitudes toward the exam, and the time of the exam play a significant role in how teachers teach and what method they use.

Hargreaves (1997) analyzed the relationship between examinations and teaching practices. Based on classroom observations conducted in Egyptian secondary schools, she suggested that while the secondary leaving examination dominated teaching methods, it did not dictate them. Reform on one was not dependent on the other. Other factors such as poor quality teacher education and a general lack of resources contributed to low-quality pedagogy and assessment. Finally, Glover (2006) based on teacher talk tried to analyze wash-back on how teachers teach in Hungary. He concluded that there were many differences between faculties and stressed how teachers teach in examination lessons may be different because they have a different pedagogic purpose that can be teaching the test and not the language.

Rostampour (2015), through using a questionnaire, sought for the teachers’ and learners’ perceptions toward the wash-back effects on the reading comprehension courses in undergraduate EFL classes at Shiraz, Iran. The findings indicated strong positive and negative wash-back effects on teaching and learning of these courses in our EFL setting.

Despite several pieces of research carried out on wash-back and its good/bad effects on teaching and learning; there are still more openings to research in different contexts. For example, in our EFL setting studies can be done about textbooks and high-stakes exams and their wash-back effects on each other. The results can provide teachers and materials developers and other beneficiaries to take advantage of. According to some scholars such as Shepard (1990, 1991, 1993, 1993)), there needs to be a close connection between the examinations (high-stakes or low-stakes) and the content and format of the curriculum or textbook called “curriculum alignment.” The claim is that the primary goal of this kind of adjustment is to improve teaching and learning. It is called “systemic validity” Frederiksen and Collins (1989); “consequential validity” Messick (1994), Messick (1996)); and test impact Bachman and Palmer (1996) and Baker (1991).

Following this kind of reasoning, the researchers of the present study have investigated the possible existence of a mutual alignment between the teaching materials used in the last year of high schools and the UEEs administered every academic school year by the authorities in our country.

2. Method

2.1. Participants

The sample of the study included 40 female and male teachers from different preuniversity schools in District Education No. One in Shiraz. Teachers’ years of teaching experience varied between 2 and 11 years. All the participants were Persian native speakers. Information on the subject group including their gender and teaching experience is demonstrated in Table 1 below. The sampling procedure was a non-probability sampling of the accidental type due to the problems related to the availability of the subjects.

Table 1. Information on the teachers’ gender and years of teaching experience

Gender	Variables	N
	Female Male Total	25 15 40
Teaching experience	2–4 years	15
	5–7 years	5
	8–9 years	9
	10/11 years	11

2.2. Instruments and materials

The Teacher Textbook Evaluation Scale (TTES) questionnaire, which elicits the criteria of a textbook for the English course, was taken from Maniruzzaman (2012). The original version was developed at Sung Kyun Kwan University in 2000. The items were designed, analyzed, and examined one by one with the consultation of the supervisors to obtain information about the relevance and clarity of the statements, the format, and the amount of time required to fill in the questionnaire. The questionnaire was revised accordingly. The revised version had four dimensions and consisted of 18 items. The first aspect was based on the “Activities” of the textbook with 6 items (items 1–6). The second dimension was the “Language Type,” consisting of 4 items (items 7–10). “Subject and Content” was the third dimension containing 5 items (items 11–15). Finally, the fourth dimension, the “Whole Aspect,” had 3 items (items 16–18). Using a five-point scale, the statements in the inventory were assigned values from 1 to 5. Namely, “strongly agree (1),” “agree (2),” “no opinion (3),” “disagree (4),” and “strongly disagree (5).” The researchers applied the Cronbach’s alpha coefficients to test the reliability of the “Teacher Textbook Evaluation Scale,” as a whole and each of the four dimensions separately. On the whole, the “Teacher Textbook Evaluation Scale” had a high degree of reliability (α = 0.9478). Additionally, Table 2 presents the Cronbach’s alpha coefficients of this instrument regarding its four dimensions. As for the validity, the questionnaire was evaluated content-wise by several experts in the TEFL field including the ones supervising this dissertation, who agreed on the content validity of the questionnaire.

Table 2. Item distribution and Cronbach’s alpha coefficients in the textbook evaluation questionnaire

Dimensions	Items	Alpha
Activities	1, 2, 3, 4, 5, 6	0.866
Language type	7, 8, 9, 10	0.750
Subject and content	11, 12, 13, 14, 15	0.670
Whole aspect	16, 17, 18	0.798
Total	18	0.947

In Table 2, the categorization and distribution of the 18 items of the questionnaire such as activities, language type, subject and content, and finally whole aspect as well as the reliability indices of the teacher textbook evaluation scale are displayed.

The target teaching materials under evaluation were “English 1 & 2” textbook used for the last year of high schools. It contained different sections starting with “Before You Read” section with some questions followed by “the Reading” section with some comprehension questions, “Vocabulary,” “Grammar,” and “Writing.”

2.3. Data collection procedure and analyses

For the sake of ethical consideration and with the permission and cooperation of the principals of the preuniversity schools located in District Education No. One in Shiraz and before the administration of the questionnaire, an orientation session was held with the selected 40 EFL teachers in one of the schools. In this session, the researcher introduced herself to them and then started to explain the main goal of her research project as a PHD candidate at Shiraz Islamic Azad University. Some questions raised by some of the participants were answered. They finally all agreed to take part in the main session to answer the items in the questionnaire and their consent was obtained. The researcher assured them that their responses would remain confidential.

To pursue the aims of this study, the testing administration took place during the spring of 2015. Data collection for this research involved the use of the questionnaire consisting of 18 statements in four categorizations, as was pointed out above, to provide detailed information about what and how teachers teach in their assigned preuniversity English classes. The questionnaire was given to the participants, taking 15–20 min to respond. The exam session was preceded by a brief introduction and orientation as mentioned above. To analyze the data, their responses were collected and analyzed descriptively by the researchers using a quantitative paradigm by applying the SPSS program module concerning the four categories. The statistical measurements included means and standard deviations for each item of the questionnaire. Likewise, a two-way ANOVA was conducted, aiming at responding to the research questions of the present experiment to explore which group of the teachers had different views regarding the effect of the UEEs and the preuniversity English textbook on the communicative objectives of the course and the EFL teachers’ practices in their classes.

3. Results

3.1. Results of the textbook evaluation scale questionnaire

The results of the study focusing on the effectiveness of the EFL textbook on teaching and learning English from the perspectives of teachers are discussed. The following table illustrates the results of the textbook evaluation questionnaire.

3.1.1. Evaluation scale of activities

The questionnaire data from 40 female and male teachers from different high schools, located at District Education No. One in Shiraz were collected and analyzed. The items were examined and refined through consultation with the supervisors to be sure they elicited the information intended through questionnaire surveys. The results were analyzed quantitatively as reported in the following sections in detail.

Based on Table 3, the mean score and standard deviation of Item No. 1 were 1.50 and 0.71, respectively. Based on Table 4, 62.5% of the respondents thought that the textbook would provide a balance of activities. There was an even distribution of free vs. controlled exercises and tasks focusing on both smooth and accurate production. Similarly, Table 4 indicates that 62.5% of the teachers agreed that the activities encourage sufficient communicative and meaningful practice. The mean score and standard deviation of Item No. 2 (Table 3) revealed the same results (2.50 and 1.51, respectively).

Table 3. The mean scores and standard deviations of the items of the textbook evaluation questionnaire

No.	Item	Mean	Standard deviation
	1. Activities
1	The textbook provides a balance of activities (E.g. There is an even distribution of free vs. controlled exercises and tasks that focus on both smooth and accurate production).	1.50	0.71
2	The activities encourage sufficient communicative and meaningful practice.	2.50	1.51
3	The activities incorporate individual pair and group work.	2.00	1.51
4	The grammar points and vocabulary items are introduced in motivating and realistic contexts.	3.37	1.42
5	The activities promote creative original and independent responses.	2.97	1.34
6	The textbook’s activities can be modified or supplemented quickly.		1.22
	2. Language type
7	The terms used in the textbook are authentic, that is, like real-life English.	3.02	1.68
8	The language used is at the right level for students’ current English ability.	3.67	1.14
9	The progression of grammar points and vocabulary items is appropriate.	1.50	0.81
10	The grammar points are presented with brief and easy examples and explanations.	3.77	1.42
	3. Subject and content
11	The subject and content of the textbook are relevant to students’ needs as an English language learner.	3.00	1.53
12	The subject and content of the textbook are realistic.	2.25	1.21
13	The subject and content of the book are exciting, challenging and motivating.	3.37	1.23
14	There is sufficient variety in the subject and content of the textbook.	2.45	1.33
15	The materials are not culturally biased and they do not portray any negative stereotypes.	3.65	1.38
	4. Whole aspect
16	The textbook is appropriate for the language-learning aims of my institution.	2.95	1.69
17	The textbook raises my (students’) interest in further English language study.	3.12	1.28
18	I would choose to study/teach this textbook again.	3.25	1.49

Table 4. Frequency and percentage of the activities section of the textbook evaluation scale questionnaire

Item number		Strongly agree	Agree	No opinion	Disagree	Strongly disagree
1	Frequency	25	10	5	0	0
	Percent	62.5%	25%	12.5%	0%	0%
2	Frequency	15	10	0	10	5
	Percent	37.5%	25%	0%	25%	12.5%
3	Frequency	25	5	0	5	5
	Percent	62.5	12.5%	0%	12.5%	12.5%
4	Frequency	5	10	0	15	10
	Percent	12.5%	25%	0%	37%	25%
5	Frequency	7	10	5	13	5
	Percent	17.5%	25%	12.5%	32.5%	12.5%
6	Frequency	7	10	10	10	3
	Percent	17.5%	25%	25%	25%	7.5%

The findings displayed in Table 4 (i.e. 75% of the respondents chose “agree” plus strongly agree) indicate that the activities contained in the textbook incorporate pair and group work. It is not possible to say the same favorable things for grammar point and vocabulary items. This is because more than half of the respondents think that grammar points and vocabulary items are not introduced to motivate in a realistic context. These findings can be confirmed by the mean scores of Item No. 4 are 3.37 and 1.42 (Table 3), respectively and also the fact that 62.5% of the respondents disagreed that the grammar points and vocabulary items are introduced in motivating and realistic contexts.

The mean score and standard deviation of item No. 5 (Table 3) are 2.97 and 1.34, and Table 4 reveals that 12.5% of teachers do not have any opinion and 45% of them (strongly disagree plus disagree) stated that the activities do not promote creative original and independent responses. Item No. 6 (Table 3) has the mean score of 2.80 and the standard deviation of 1.22, and Table 4 exhibits that 42.5% of the teachers believe that the textbook’s activities can be modified or supplemented quickly whereas 25% of them did not have an opinion.

3.1.2. Evaluation scale of language type

Table 5 indicates that 50% of the teachers (strongly disagree plus disagree) did not believe that the language used in the textbook is authentic, that is, like real-life English. The mean score and standard deviation of Item No. 7 (Table 3) confirm the findings (i.e. 3.02 and 1.68, respectively).

Table 5. Frequency and percentage of the language type section of the textbook evaluation scale questionnaire

Item number		Strongly agree	Agree	No opinion	Disagree	Strongly disagree
7	Frequency	12	7	1	8	12
	Percent	30%	17.5%	2.5%	20%	30%
8	Frequency	10	0	4	15	11
	Percent	25%	0%	10%	37%	27%
9	Frequency	26	10	2	2	0
	Percent	65%	25%	5%	5%	0%
10	Frequency	25	13	0	2	0
	Percent	62.5%	32.5%	0%	5%	0%

Similarly, Table 5 shows that 65% teachers (disagree plus strongly disagree) stated that the language is not at the right level for students’ current English ability (Item No. 8) with the mean of 3.67 and standard deviation of 1.14 (Table 3). The average score and standard deviation of Item No. 9 are 1.50 and 0.81, respectively (Table 3). Table 5 indicates that 90% of the teachers (agree plus strongly agree) confirmed that the progression of grammar points and vocabulary items is appropriate. Item No. 10 has the mean score of 3.77 and the standard deviation of 1.42 (Tables 3 and 5) exhibits that 95% of the teachers believe that the grammar points are presented with brief and easy examples and explanations.

3.1.3. Evaluation scale of subject and content

The mean score and standard deviation of Item No. 11 are 3.00 and 1.53 (Table 3). Table 6 reveals that 50% of teachers (strongly disagree plus disagree) stated that the subject and content of the textbook is relevant to students’ needs as English language learners. Item No. 12 has the mean score of 2.25 and the standard deviation of 1.21 (Table 3). Table 6 exhibits that 62.5% of the teachers believed that the subject and content of the textbook is realistic. As the mean score of Item No. 13 is 3.37 while its standard deviation is 1.23 (Table 3), it is clear that the subject and content of the textbook is not attractive, challenging, and motivating. In other words, 62.5% of the teachers believed that the textbook content should be more motivating (Table 6).

Table 6. Frequency and percentage of the subject and content section of the textbook evaluation scale questionnaire

Item number		Strongly agree	Agree	No opinion	Disagree	Strongly disagree
11	Frequency	10	8	2	12	8
	Percent	25%	20%	5%	30%	20%
12	Frequency	15	10	5	10	0
	Percent	37.5%	25%	12.5%	25%	0%
13	Frequency	5	5	5	20	5
	Percent	12.5%	12.5%	12.5%	50%	12.5%
14	Frequency	10	17	3	5	5
	Percent	25%	42.5%	7.5%	12.5%	12.5%
15	Frequency	5	5	2	15	13
	Percent	12.5%	12.5%	5%	37.5%`	32.5%

Based on Table 6 (Item No. 14), 67.5% of the teachers stated that there is enough variation in the subject and content of the textbook. The mean score and standard deviation of Item No. 6 are indicative of such a tendency (i.e. 2.45 and 1.33, respectively, Table 3).

As the average score and standard deviation of Item No. 15 are 3.65 and 1.38 and also based on Table 6 which is an indicator of the fact that 70% of teachers disagreed that the materials are not culturally biased, they do not portray any negative stereotypes.

3.1.4. Evaluation scale of the whole aspect

The findings displayed in Table 7 (Item No. 16) reveals that 62.5% of the respondents chose “agree plus strongly agree” supporting their view that the textbook is appropriate for the language-learning aims of the education. The mean score can confirm these findings and a standard deviation of Item No. 16 (Table 3) being 2.95 and 1.69, respectively. In other words, teachers focused on those kinds of materials that help students do well in examinations. The overall results of the teachers’ perceptions toward the wash-back effect of the UEEs on the way they teach and what they teach and on the EFL learners’ communicative English language learning revealed that the UEEs exerted negative wash-back impact. The results also indicated that the two sources of input, the textbook and the UEEs, were not aligned to support each other, conveying a negative answer to the second research question

Table 7. Frequency and percentage of the whole aspect section of the textbook evaluation scale questionnaire

Item number		Strongly agree	Agree	No opinion	Disagree	Strongly disagree
16	Frequency	15	10	5	5	5
	Percent	37.5%	25%	12.5%	12.5%	12.5%
17	Frequency	5	10	5	15	5
	Percent	12.5%	25%	12.5%	37.5%	12.5%
18	Frequency	7	8	4	10	11
	Percent	17.5%	20%	10%	25%	27.5%

3.2. Results of the teachers’ teaching experience and their gender distinction regarding the effect of UEEs on the preuniversity English textbook

Based on the information gathered from the EFL teachers displayed in Table 1, a two-way ANOVA was conducted to examine if EFL teachers with more years of teaching experience and their gender distinction had different opinions toward the influence of UEEs on the preuniversity textbook and their class activities. The results are represented in the following Table

Table 8. Two-way ANOVA on the views of teachers regarding the effect of university entrance examination on the preuniversity English textbook evaluation scale questionnaire

Expteachers	1.00	2–5	15
	2.00	5–7	5
	3.00	7–9	9
	4.00	9–11	11
Gender	1.00	Female	24
	2.00	Male	16
Descriptive statistics
Expteachers	Gender	Mean	Std. Deviation	N
2–5	Female	52.1250	8.35699	8
	Male	54.0000	8.32666	7
	Total	53.0000	8.09762	15
5–7	Female	33.4000	3.71484	5
	Total	33.4000	3.71484	5
7–9	Female	23.2000	4.02492	5
	Male	20.2500	2.50000	4
	Total	21.8889	3.58624	9
9–11	Female	78.3333	6.74290	6
	Male	79.0000	6.40312	5
	Total	78.6364	6.26534	11
Total	Female	48.7500	21.48862	24
	Male	53.3750	23.48581	16
	Total	50.6000	22.12992	40
Tests of between-subjects effects
Multiple comparisons
Source	Type III sum of squares	df	Mean square	F	Sig.	Partial Eta squared
Corrected Model	17,664.642^a	6	2944.107	67.706	.000	.925
Intercept	80,707.263	1	80,707.263	1856.040	.000	.983
Expteachers	17,359.066	3	5786.355	133.070	.000	.924
Gender	.154	1	.154	.004	.953	.000
Expteachers* gender	33.086	2	16.543	.380	.687	.023
Error	1434.958	33	43.484
Total	121,514.000	40
Corrected total	19,099.600	39
						95% Confidence interval
	(I) expteachers	(J) expteachers	Mean difference (I–J)	Std. Error	Sig.	Lower bound	Upper bound
Scheffer	2–5	5–7	19.6000*	3.40523	.000	9.5706	29.6294
		7–9	31.1111*	2.78036	.000	22.9222	39.3001
		9–11	−25.6364*	2.61762	.000	−33.3460	−17.9267
	5–7	2–5	−19.6000*	3.40523	.000	−29.6294	−9.5706
		7–9	11.5111*	3.67807	.034	.6781	22.3441
		9–11	−45.2364*	3.55665	.000	−55.7117	−34.7610
	7–9	2–5	−31.1111*	2.78036	.000	−39.3001	−22.9222
		5–7	−11.5111*	3.67807	.034	−22.3441	−.6781
		9–11	−56.7475*	2.96388	.000	−65.4769	−48.0180
	9–11	2–5	25.6364*	2.61762	.000	17.9267	33.3460
		5–7	45.2364*	3.55665	.000	34.7610	55.7117
		7–9	56.7475*	2.96388	.000	48.0180	65.4769

^aR² = .925 (adjusted R² = .911)

Based on the observed means.

The error term is mean square (error) = 43.484.

*The mean difference is significant at the .05 level.

Expteachers = experience of the teachers.

Table 8 shows that diverse levels of the teachers’ experience were a determining factor in forming the views of teachers about the effect of UEEs on the preuniversity English textbook, (F _{(3, 33)} = 133.07, p = .000). However, their gender was not a significant factor in changing their views about the effect of UEEs on the preuniversity English textbook (p = .95).

4. Discussion

The studies done on the wash-back effect of the high-stakes tests on the educational curriculum (Akpinar & Cakildere, 2013, cited in Gogturk Saglam; Li,1990; Alderson & Wall, 1993; Nikolov, 1999; Shohamy et al., 1996; Hargreaves, 1997) have contended that high-stakes tests exert more effect on the content of teaching. All of them believe that teachers in different educational settings adhere to the skills needed to be tested in these high-stakes exams and they spend most of their class time to prepare their students for these types of exams administered in different EFL contexts. Hargreaves (1997) claims that this, in turn, would negatively affect tertiary level learning and teaching. The results of our study corroborate the findings of these studies. Even though our EFL teachers believed that the textbook would provide a balance of activities and an even distribution of free vs. controlled exercises and tasks that focus on both smooth and accurate production, they used supplementary materials to prepare their students for the UEEs in our country, being adverse with the set goal.

Our findings are also in line with Cheng’s (1997b) who concluded that teachers did not pay attention to the goal of the course and taught to the university entrance examination. Likewise, data gathered from the questionnaire on our teachers’ perceptions implied a strong negative wash-back of the UEEs on the way they teach and their students’ communicative learning goal complying with Gokturk Saglam’s (2018) findings who investigated the wash-back effect of a university English language proficiency test in Turkey on how and what teachers teach.

Most of our participants believed that the grammar points and vocabulary items introduced in the textbook could be modified or supplemented as 75% of the teachers mentioned that the grammar points were not presented with brief and easy examples and they were demotivating. They believe that the leading English book employed to teach our EFL preuniversity students does not follow the university entrance exams and they do not support each other. However, supplementary materials used were geared toward these types of examinations, in sharp contrast with the primary English book design and objective. Cheng (1997a), Hawkey (2006), Wall & Horak (2006), and Lam (1993) reached the same conclusion that additional materials were used other than their main textbook because the leading textbook could not prepare students for the test. They believe that this condition would induce leading material developers and writers to publish exam-oriented materials designed for the needs of students and their teachers who were interested in preparing them for such high-stakes tests

Aligned with the findings of the above studies, the results of our research project strongly highlight the negative wash-back impact of the UEEs on the how and what they teach, and on the students’ learning aim. These tests are administered every year in our country to screen the registered candidates to get accepted in a university or an institute of higher education to continue their studies in their selected field. The last finding of this study is related to two independent variables, gender and years of the teachers’ teaching experience. As it is evident in Table 8, the mean difference between the groups who had more years of experience is higher indicating that seniors believe that the UEEs have considerable adverse effects on the preuniversity English textbook. In other words, they think that using the main textbook does not follow the goal of the UEEs, which is to prepare the stakeholders to pass the test, but not to enable them to communicate effectively. However, gender distinction among the teachers was not found to be significant (.95).

5. Conclusion and implications

The findings of the EFL teachers’ perspectives of our study highlighted the negative wash-back impact on their class practices and their students’ communicative learning objectives, corroborating the studies carried out by other researchers conveying similar lines of argument (cf. Caine, 2005; Cheng, 1997b; Wall & Horak, 2006; Hawkey, 2006; Gokturk Saglam, 2018). Our EFL teachers’ perceptions showed that they did not pay attention to the aim of the primary English textbook to help their students to communicate efficiently. Instead, they spent more time on grammar explanation and practices. They only taught grammar because it was going to be tested in the UEEs. In other words, the achievement of communicative competence was neglected, and the teachers taught whatever they thought would be tested in these types of exams. This supports the fact they mostly adhere to and follow the format and content of the UEEs administered nationwide in our country, showing conformity with the results of the previous studies investigating the wash-back effect of high-stakes test on the teachers’ teaching methods and their views in their EFL classrooms.

Another ancillary issue investigated in our experiment is the factors of teachers’ years of teaching experience and their gender distinction. The results have shown that there is a significant difference between different groups of teachers with differing degrees of teaching experience revealing their differentiation of views toward the negative effect of UEEs on the preuniversity English textbook communicative goal. This issue would be used as a sign of senior teachers’ being more attentive in their critical analysis of a problem or responsibility assigned to them, which deserves more credit.

Considering the importance of wash-back effect concept in mind, the results of this research will have implications for materials developers, universities and institutes authorities, teachers, and learners. There is public concern regarding the effects of testing and test scores in teaching and materials development and what and how students and teachers treat during classroom instructions. Living in the testing world leaves no doubt that in education, testing effects methods, and methodology.

Firstly, teachers should be aware of both the objectives of EFL teaching and learning as stated in the textbook and the syllabus, and also they should be oriented to teach following the syllabus and curriculum. So the Ministry of Education should arrange some awareness raising programs for teachers to orient them to teach according to the learning objectives stated in the textbooks to develop the students’ communicative competence in all the core skills.

Secondly, school authorities should monitor the performance of teachers in classrooms. In this way, they can observe their class activities to find out whether or not the teachers teach to the test and university entrance examinations by giving test-related exercises and skipping some lessons that are not likely to be tested in the exam. Alternatively, instead, they teach based on the objectives mentioned in the curriculum and develop the skills needed for communication in different situations.

Thirdly, the material developers should design their teaching and learning texts to cope with the learning objectives and the needs and interests of the learners. Finally, the university entrance examination authorities should prepare and design their high-stakes tests to be consistent with the principles of communicative testing measuring the communicative competence of the students. In other words, the test should determine how well the students can perform in real life situations. All four skills including listening, reading, speaking, and writing should be integrated and receive equal attention in testing.

Additional information

Funding

The authors received no direct funding for this research.

Notes on contributors

Shiva Sadighi

Shiva Sadighi I should say that I am working on my PHD dissertation in TEFL at the Islamic Azad University Shiraz center, as the first author of the accepted paper. Members of the committee are Dr Mortaza Yamini and Dr Mohammad Sadegh Bagheri as supervisors and Dr Lotfollah Yarmohammadi as an advisor. They have expertise in teaching and researching in different domains of applied linguistics. The rationale for the selection of the topic by the researcher with the collaboration of the members includes several reasons. The researcher’s interest and experience as a student and a teacher and the way EFL teachers taught English prompted me to conduct this study. Another reasoning is that teachers ignored the stakeholders’ communicative competence while teaching English. The negative wash-back effects of the high-stakes tests on teachers’ class activities and students’ learning conditions as another impetus to investigate this issue. Therefore, the results and their implications might help educational authorities to provide opportunities and facilities to solve such problems facing the stakeholders.