Exploring language learners’ perception of the effectiveness of an English Language Teaching (ELT) program in Iran

Abstract

This study aimed to explore the effectiveness of an English Language Teaching (ELT) program at a language institute in Iran from learners’ perspective. Qualitative and quantitative data were collected using a questionnaire and semi-structured interviews. The analysis of the quantitative data showed that the learners found the reading, writing, listening, vocabulary, and grammar materials they received at the institute sufficient, but felt the need for more speaking materials. They also believed that the program overemphasized non-communicative teaching techniques and paid little attention to communicative authentic language learning tasks. The learners were dissatisfied with the difficulty and fairness of the assessment method at the institute and complained that the staff members were uncooperative and little attention was paid to the learners’ voice. The analysis of the qualitative data indicated that the learners viewed “teachers” and “accessibility” as the strengths and “staff’s sense of cooperation,” “class arrangements and rules,” “facilities,” and certain aspects of the “teaching and assessment” methods as the weaknesses of the institute. From the findings, it can be concluded that communicatively oriented teaching materials need to be injected into the ELT program, the teaching method needs to be overhauled, and the assessment system needs to be steered toward standard unbiased methods. Moreover, given the impact of facilities and administrative issues on learners’ evaluation of and satisfaction with a program, the study suggests that supportive policies be made on logistics to secure the learners’ satisfaction and the sustenance of the institute.

Keywords:

• evaluation
• ELT
• language teaching program
• learners
• language institutes

PUBLIC INTEREST STATEMENT

Despite the large number of English Language Teaching (ELT) institutes in Iran and the desire to learn English among Iranians, few studies have investigated the effectiveness of ELT programs at the institutes in the country. This study researched the effectiveness of an ELT program at an institute from the learners’ perspective. It was revealed that except for speaking materials the learners received sufficient materials for other language skills. They also believed that the teaching method should focus on communicative authentic language teaching tasks, complained about the fairness and difficulty of the tests, and demanded more heed to the learners’ voice. The learners viewed teachers and accessibility as the strengths and staff’s sense of cooperation, class arrangements, and facilities as the weaknesses of the institute. The study concludes that the teaching method at the institute needs to be overhauled, communicative materials should be used, and standard testing methods should be employed.

1. Introduction

Many Iranian youth and students attend English Language Teaching (ELT) programs at private language institutes across the country to learn English as an international language. Following the trend, having personal interests, having a practical need for proficiency in English, and working at the spoken English are reported as the main purposes for which Iranians go to language institutes (Sadeghi & Richards, 2015). Research has also shown that English language institutes play an “increasingly important role” in Iran (Haghighi & Norton, 2016), which suggests “the increasing value of the English language” in the country (Sheibani, 2012). Likewise, Sadeghi and Richards (2015) noted, “The demand for learning English has greatly increased in Iran” in recent years, which has led to “the growth of the private language institute industry.” Given that there exists great interest in learning English in the country, ELT institutes are expected to provide effective language teaching programs to aid learners in achieving their goals, which will, in turn, bring learners’ satisfaction with the institutes. With regard to the effects of learners’ satisfaction on educational processes, it has been shown that students’ satisfaction “indirectly influences organizational performance” of educational institutions (Hanssen & Solvoll, 2015), and is “associated with the perceived quality” of the institutions (Athiyaman, 1997). However, research into language learners’ views about the efficacy of institutes in offering appropriate language teaching programs has shown that learners are not satisfied with the ELT programs and do not achieve their goals at the institutes (Mirhosseini & Khodakarami, 2015). Sadeghi and Richards (2015) underscored the inability of language institutes in Iran to teach spoken English to the learners, describing the curriculum and the teaching materials used at the institutes as the factors contributing to the failure. Explaining the reasons for the failure of language institutes to provide communicatively acceptable ELT programs in Iran, Mirhosseini and Khodakarami (2015) also argued that directors and supervisors of institutes did not have a clear understanding of what communication in English means. Similarly, Balagafshe, Vahdany, and Arjmandi (2014) noted that language institutes in the Iranian context employ different methodologies to teach English, and “there is no agreement upon the same method even between two institutes.” Therefore, it seems that despite the burgeoning interest in learning English and the growing number of ELT institutes across the country, language learners do not feel satisfied with the results they obtain from the ELT programs at the institutes. To discover the reasons for the institutes’ failure to meet the learners’ needs for learning English, the present study set out with the aim of exploring the effectiveness of the ELT program at a language teaching institute in the Iranian context from the learners’ perspective.

1.1. The ELT program at the institute

The target ELT institute, which has its center in the capital city of Tehran, has branches in most cities across the country and has a high number of annual registration. In addition to English, the institute offers other language courses, including French, in some branches. The ELT program at the institute consists of basic, elementary, pre-intermediate, intermediate, higher intermediate, and advanced levels of English language proficiency. Each proficiency level is further divided into three sublevels of 1, 2, and 3, which are offered in consecutive semesters. Each semester lasts almost 3 months, and the learners attend English classes twice a week. The classes are not coed and are held in the institute both in the morning and in the afternoon. The prescribed language teaching method at the institute is an eclectic approach to teaching English as a foreign language. The teachers are to follow directions and procedures prescribed by the authorities at the institute; however, they can use innovative techniques to improve the teaching task. All language teaching activities performed in the classroom pivot on the student’s books. The ELT materials at the institute, called the English Series, consist of Student’s Books, Workbooks, and CDs. The title of each series is determined by the proficiency level for which the series is developed. For example, the Student’s Books for the Pre-Intermediate level, which is divided into the three sublevels of 1, 2, and 3, are entitled English Series: Pre-Intermediate 1 student’s book, English Series: Pre-Intermediate 2 student’s book, and English Series: Pre-Intermediate 3 student’s book. The books have the same format and are planned, compiled, and revised by the research and planning department of the institute. The number of pages of the books varies according to the level for which they are developed. For instance, the Pre-Intermediate 1 Student’s Book is a 120-page book, while the number of pages of the Elementary 3 Student’s book is 148. The number of units of the books also varies according to the level. For example, the Student’s books in the Pre-Intermediate levels consist of 8 units, whereas the Student’s books in the elementary levels contain 10 units. The books are developed based on audio-lingual material design procedures and have the same format. Each unit of the books consists of a vocabulary list, a dialog, a reading passage, a structure or grammar section, and a listening section. In the vocabulary list section, which appears at the beginning of each unit, all new words in the unit, together with their meanings and/or illustrations, are given. In the dialog section, which follows the vocabulary list section, a dialog aiming to present a grammar point in a real-life satiation is given. To increase the learners’ general knowledge of English, boost their comprehension, and expand their vocabulary, a passage has been incorporated in each unit. Grammatical and structural points are taught in the structure or grammar section, which is followed by drills planned to expose the learners to the structure of English. The listening section comes in the last part of each unit and aims to improve the learners’ listening skill through exposing them to the materials thematically related to the dialog and reading passage. A productive activity appears at the end of the listening part section, giving the learners the opportunity to practice what they have learned in the unit. The student’s books are also accompanied by a workbook to provide the learners with the opportunity to practice, review, and enhance what they have been taught in the classroom. The books are also accompanied by a CD, which contains audio files needed for the listening sections. A pocket dictionary covering the new words in the student’s book is also available for the learners.

The institute prescribes criterion-referenced assessment. The pass mark at the institute is set at 75 out of 100. It should be noted that 40% of the learners’ final score is calculated based on their score on the final exam and the remaining 60% is allotted to their class activities during the term. The teachers normally check the learners’ reading comprehension or ask the learners to have a role-play based on the dialog in the student’s book every session. After completing the reading comprehension task or performing the role-play, the teachers score the learners’ performance, correct their mistakes or errors, and provide feedback on their performance. An extracurricular book containing sample tests of the different levels of language proficiency at the institute is also available.

1.2. The significance of discovering the effectiveness of educational programs

One of the techniques which educational authorities can use to check the efficacy of an educational program and thereby the learners’ satisfaction with the program is to evaluate the program (Patton, 2008). Evaluation has always been of great significance in the field of language teaching (Cummins & Davison, 2007), is the “basis for rational decisions about future educational practices” (Quintero, 2003), and shows “where to improve future teaching and learning practices”. According to Norris (2016), language program evaluation “enables a variety of evidence-based decisions and actions, from designing programs and implementing practices to judging effectiveness and improving outcomes.” Norris (2016) described that the impetus for language program evaluation can be “accountability evaluation, student learning outcomes assessment, program development, monitoring and improvement, effectiveness evaluation, and language teacher evaluation.” “Effectiveness evaluation” and “program development, monitoring, and improvement” were the impetus behind the present research which aimed to evaluate the ELT program at a language institute in Iran from learners’ perspective. The evaluation task in the present study was performed by discovering the learners’ views on the sufficiency of the teaching materials, the teaching method, the assessment system, and the heed paid to the learners’ voice at the institute. Norris (2016) explained that an evaluation with program development, monitoring, and improvement impetus underscores “formative, local, and largely internal language program evaluation efforts.” He further pointed out that such an evaluation is conducted to glean “locally useful information” to find a way out of the “immediate challenges.” He introduced needs analysis as a means to such an end. Elaborating on the “effectiveness evaluation,” he stressed that this type of evaluation focuses on the “effectiveness” of “teaching approaches, materials, or curricula.” Likewise, Irambona and Kumaidi (2015) underscored that in educational contexts evaluation “depicts the effectiveness, strengths, weaknesses, or the failures of a given program.” They further noted that educational evaluation indicates the extent to which “students have reached the desired standards,” adding, “An effective evaluation will be based on that set of objectives which students are intended to meet.” As the learners themselves know better than anyone else the set of language learning objectives they aim to achieve, this study set out to discover the learners’ evaluation of the ELT program at the institute.

Program evaluation can also help policymakers run a check on educational programs and the outcomes of such programs, which can directly or indirectly influence the stakeholders’ satisfaction. Therefore, the evaluation of educational programs, including language teaching programs, can contribute to the learners’ satisfaction, which can, according to Hanssen and Solvoll (2015), “increase the chances of the long-term success” of educational institutions. The burgeoning interest in learning English among Iranians has motivated many individuals to invest in teaching English, and as Haghighi and Norton (2016) noted, “The surge in the number of Iranian English Language institutes is well documented.” Research in the Iranian context has also proved that official English language learning programs have failed to achieve the specified goals (Bagheri, 1994; Rahimi, 1996; Rashidi, 1995). Despite the substantial amount of time, money, and energy allocated for the teaching and learning of English, the learners appear not to be satisfied with the outcomes of the ELT programs. Evaluating the existing ELT programs at the institutes from the perspective of the learners can contribute to the identification of the shortcomings and weaknesses of the programs. The existing literature also shows that a few studies have been conducted in the area of ELT program evaluation in the Iranian context (e.g., Atai & Mazlum, 2013; Birjandi & Nosratinia, 2009; Foroozandeh, Riazi, & Sadighi, 2008; Razmjoo & Riazi, 2006) and there has been a growing interest in the studies evaluating the use of technology in teaching English (e.g., Alizadeh, 2018, 2012; Ebadi, 2016; Marandi & Seyyedrezaie, 2017). There is, however, a paucity of research evaluating the effectiveness of ELT programs implemented in ELT institutes from the learners’ perspective in the Iranian context. In this regard, Balagafshe et al. (2014) stressed, “There is, hardly ever, any scientific evaluation on the efficiency of the methods” of ELT in the Iranian context. As a consequence, issues like the adequacy of the language teaching materials, the appropriateness and fairness of the assessment methods, the effectiveness of the language teaching methods, the heed paid to the learners’ opinions and rights, and the learners’ overall satisfaction with the ELT programs at the institutes have remained unexplored. Moreover, discovering the learners’ perceptions of the effectiveness of the ELT programs at language institutes will contribute to the enrichment of the literature on ELT program evaluation. It will also contribute to the identification of the ELT problems at language institutes. Therefore, this study set out with the aim of discovering the effectiveness of the ELT program at the target language teaching institute in the Iranian context from the language learners’ perspective. Discovering the learners’ evaluation of the program at the institute can provide teachers, educational authorities, and administrators with insights into the ways to have an ELT program which suits the learners’ needs, meets the learners’ satisfaction, and secures the sustenance of the institute. Having these goals in mind, the researcher tried to answer the following questions:

1. What is the language learners’ view about the sufficiency of the teaching materials and the effectiveness of the teaching method at the language institute?

2. What is the language learners’ view about the assessment method and the heed paid to the learners’ voice at the language institute?

3. What are the strengths of the ELT program at the institute from the Iranian learners’ perspective?

4. What are the weaknesses of the ELT program at the institute from the Iranian learners’ perspective?

2. Literature review

A review of the literature on the evaluation of ELT programs showed that the studies in this area have approached the task of evaluation differently. Some of the studies have targeted ELT curriculum planning at the national level (e.g., Atai & Mazlum, 2013; Birjandi & Nosratinia, 2009; Foroozandeh et al., 2008), some of the researchers have researched the effectiveness of ELT at English language institutes and schools (e.g., Irambona & Kumaidi, 2015; Khattak, Usman, Khan, Abbasi, & Ahmad, 2011; Ortiz & Garzón, 2007; Tabatabaei & Loni, 2015), and many other researchers have investigated certain aspects of an ELT program (e.g., Baleghizadeh & Karamzade, 2017; Razmjoo & Riazi, 2006). The literature on the ELT program evaluation in the Iranian context and abroad also shows that few studies have specifically investigated the effectiveness of a language teaching program at a language institute from the learners’ perspective. Khattak et al. (2011), for instance, evaluated the effectiveness of ELT at English language institutes in the Pakistani city of Mardan by seeking the learners’ perception regarding the efficacy of the course content, class activities, and teaching methods in the language institutes. The results of the study showed that the learners found “the teachers’ ineptness, focusing on the content than on the learners’ needs, and the unavailability of the proper resources” as the main weaknesses of the program at the institutes; it was, however, found that the learners were quite satisfied with the improvement in their accent and pronunciation. In a similar study, Ortiz and Garzón (2007) investigated the English teaching practice at a language institute at a private university in Colombia to describe the pedagogical practices at the institute. Observation of the classes, questionnaires, and interviews with the teachers and students were the three techniques used for data collection. The results of the study showed that the institute lacked a clear methodology model. Most of the teachers benefitted from the Presentation, Practice, and Production model to language teaching, some of them utilized cooperative learning approaches, and others used a communicative approach. It was also discovered that the students used direct and indirect strategies to learn English and complained about the unavailability of educational resources.

Some of the studies in this area have focused on the evaluation of the ELT programs offered at public schools. Irambona and Kumaidi (2015), for example, evaluated the effectiveness of the ELT program in the eleventh grade of senior high school, using Stufflebeam's (2000) Context, Input, Process and Product (CIPP) evaluation model. The results from the context component of the model showed that the program objectives, classroom condition, students’ needs, and barriers were in the effective category of the model. The results from the input component of the model showed that the teachers, students’ textbooks, and course designs were also in the effective category, but “the teaching training” was in the “not effective” category. The analysis of the data in the process component revealed that the teaching materials, teaching methods, teaching activities, and assessments were in the effective category. The findings from the product component showed that the English marks, students’ needs, and barriers were in the effective category, but the teaching materials were in the “not effective” category. Likewise, Tabatabaei and Loni (2015) investigated the problems of teaching and learning English in high schools in Lorestan Province, Iran. They researched the relationship between the availability of educational facilities and the quality of learning and attempted to discover the problems by searching through learners and teachers’ opinions. The results showed that there was no significant relationship between the availability of educational facilities and the quality of learning from the language learners’ perspective. The main problems of the teaching and learning English, which were identified in the study, were “lack of target language use in the class, restricting the evaluation merely to midterm and final exams, classes being overcrowded, the absence of language laboratories and also the mere use of textbook ignoring any inclusion of supplementary materials such as CDs, movies and other instructional aids to create a more effective methodology.”

There are also studies that have approached the task of evaluating ELT programs at larger scales than institutes. Atai and Mazlum (2013), for instance, carried out a comprehensive study in the Iranian context, investigating ELT curriculum planning in Iran’s Ministry of Education and its implementation by teachers. The main objectives of the study consisted of “Program evaluation along with needs analysis; the ELT-specific documents; communication channels between planning and practice levels; teacher evaluation and student assessment; interpretation and re-examination of national policies within the Ministry; and the criteria set for ELT material development.” The findings attested to the lack of a program evaluation as well as an ELT evaluation model in the country. It was asserted that there existed a gap between planning and practice in the Iranian ELT system, originating from “a highly centralized policymaking process in which local policymakers (i.e. teachers) are not involved.”

A number of studies in the area of program evaluation have targeted some specific dimensions of an ELT program. Razmjoo and Riazi (2006), as a case in point, investigated language teaching methodologies used in language learning institutes in the Iranian city of Shiraz. They also aimed to discover learners’ views of the efficacy of the language teaching methodologies. The results of their study revealed that the methodologies used in the language institutes had traces of communicative approaches. As for the learners’ view, it was discovered that most of the learners believed that except for the writing skill their teachers paid enough attention to other language skills and components; they also expressed that the textbooks were suitable and were satisfied with the general issues at the institutes. They, however, expressed dissatisfaction with the unavailability of supplementary materials, asking for more supplementary materials and equipment. Baleghizadeh and Karamzade (2017) also researched Iranian English language learners’ views about their preferred language learning activities in an ideal ELT class. The study focused on the three main lesson styles of communication-oriented, control-oriented, and form-oriented. The effect of the learner variables on the students’ preferred activities was also investigated in the study. The results revealed that the female learners showed a stronger preference for form-oriented lessons than the male learners. There was, however, no significant difference in the learners’ attitudes toward communication-oriented and control-oriented lessons. The findings also indicated that the learners’ age and their language proficiency did not significantly affect their attitudes toward and preferences for any of the three main lesson types.

3. Method

3.1. Participants

The participants in the present study were 250 English language learners attending language classes at the target language teaching institute. The learners were all male, and their age ranged from 14 to 45. The participants were at elementary, pre-intermediate, and intermediate levels of language proficiency and had at least a three-term experience of learning English as a foreign language at the institute. To have a comprehensive evaluation of the effectiveness of the ELT program at the institute, the learners were selected from the three different levels of language proficiency. The participants volunteered to take part in the study and oral consent was obtained from them. They were informed in a meeting that participation in the study would have no impact on their scores or their language learning at the institute.

3.2. Instruments

To collect data in the present study, a 23-item researcher-developed questionnaire was used to elicit the learners’ evaluation of the adequacy of the teaching materials, the teaching method/techniques, the assessment method, and the heed paid to the learners’ voice at the institute. The questionnaire was developed after conducting an in-depth study of the relevant literature, conducting interviews with the language learners and having consultations with senior and experienced teachers at the institute. The reliability of the questionnaire, which was calculated using Cronbach’s alpha coefficient, was estimated at 0.78. Close consultations were held with three experts in the field to ensure the validity of the questionnaire. Some items on the questionnaire were adapted following the consultations.

A semi-structured interview was also conducted with the language learners to explore their views about the strengths and weaknesses of the institute in providing the language learners with an effective ELT program. The researcher reached the data saturation point after interviewing 42 learners. Each of the interviews lasted between 7 to 15 min, depending on the learners’ answers.

3.3. Data collection and analysis

The data collection phase started after obtaining permission from the director of the institute to conduct the study at the language institute. After receiving the approval from the director, the researcher explained the purposes of the study to the learners and ensured them that their answers would remain confidential and would not affect their scores or language learning at the institute. Then, the questionnaire was distributed among the learners and they were instructed how to complete it. Finally, they were asked to return the questionnaire in 2 weeks. After collecting the questionnaires, the semi-structured interviews were conducted. The interviews were conducted with the leaners who were at elementary, pre-intermediate, and intermediate levels of language proficiency. During the interviews, the learners were asked about educational and non-educational issues at the institute. They were also advised to resort to the Persian language if they found expressing their ideas in English difficult. The interviews normally began with some warm-up questions and continued with questions like, “Are you satisfied with the language program at the institute?”, “In your opinion, what are the strengths of the ELT program at the institute?”, and “In your opinion, what are the weaknesses of the ELT program at the institute?” The interviews were recorded and were subsequently transcribed. The quantitative and qualitative data were analyzed separately. To analyze the quantitative data, the Statistical Package for the Social Sciences (SPSS version 19 [IBM Corp., Released 2010. IBM SPSS Statistics for Windows, Version 19.0. Armonk, NY: IBM Corp., USA]), which is a software package for statistical analyses, was used. As the data in the present study were categorical, frequencies and percentages were used as descriptive statistics to analyze the data. To analyze the qualitative data, the Framework Analysis Method, which is a qualitative content analysis method, was used. To do so, first, the learners’ answers were transcribed. Next, two raters read the learners’ responses carefully and underlined important concepts in the responses. Then, the raters independently rated three responses, describing the key concepts in the responses with a label or code. Next, the raters met to discuss the codes, remove the differences, and develop an initial analytical framework. The raters rated three more responses, writing down any new concepts not fitting the existing set of codes. They met again and revised the initial framework. The process of applying the revisions and refining the framework continued until no new codes were generated. The framework was then applied to all responses. In the final stage of the analysis, the related codes were clustered together to form a category.

4. Results

After collecting the questionnaires and finishing the interviews, the data were categorized into two groups of quantitative and qualitative data. To analyze the quantitative data, SPSS software version 19 was used. To analyze the qualitative data, the framework analysis method was used.

4.1. The learners’ evaluation of the sufficiency of the teaching materials for language skills

To discover the learners’ evaluation of the sufficiency of the language teaching materials used at the institute, they were asked to show how sufficient the (1) reading materials, (2) listening materials, (3) writing materials, (4) grammar materials, (5) speaking materials, and (6) vocabulary materials were. The results are given in Table 1.

Table 1. The learners’ evaluation of the sufficiency of the teaching materials

The materials	Percentage of responses
	Quite sufficient	Sufficient	Not sufficient	Quite insufficient
Reading materials	39.8	43.5	14.8	1.9
Listening materials	13.9	49.1	28.7	8.3
Writing materials	29.6	36.1	29.6	4.6
Grammar materials	33.3	45.4	17.6	3.7
Speaking materials	6.5	36.1	40.7	16.7
Vocabulary materials	25.9	45.0	20.8	8.3

As can be seen in the table, the majority of the learners believed that the teaching materials they received for grammar (45.4%), vocabulary (45.0%), reading (43.5%), writing (36.1%), and listening (49.1%) skills were sufficient; they, however, believed that the teaching materials they received for the speaking skill (40.7%) were not sufficient.

4.2. The learners’ evaluation of the language teaching method

The study also aimed to see how frequently certain language teaching techniques and methods were used at the language institute. The results from this part are shown in Table 2.

Table 2. The learners’ evaluation of the teaching method/techniques used at the institute

The teaching method/techniques	Percentages of responses
	Always	Often	Sometimes	Rarely	Never
The English language teachers at the institute model correct use of English language and employ pattern practice and various drills to enhance learning	38.9	38.9	17.6	4.6	0.0
The English language learners at the institute are encouraged to do pair work and perform role-plays to learn the language better	39.8	29.6	14.8	8.3	7.4
The English language teachers at the institute employ real-life tasks to teach English effectively	9.3	12.0	14.8	24.1	39.8
The English teachers at the institute correct the language learners’ errors and mistakes and provide them with necessary feedback on their errors and mistakes	47.2	36.1	12.0	2.8	1.9
English is the only language that is used as the medium of instruction by the teachers in the classrooms at the institute	39.8	42.6	14.8	0.9	1.9
The language teaching method at the institute stresses developing learners’ communicative ability by encouraging interaction between the learners or between the teachers and the learners	16.1	18.4	43.1	15.9	6.5

As the table shows, the majority of the learners believed that the teaching method at the institute emphasized modeling correct use of English and pattern drills (38.9%), doing pair work and performing role-plays (39.8%), doing error correction (47.2%), and using English as the only medium of instruction in the class (42.6%). They, however, expressed that authentic tasks were “never” (39.8%) used in the process of teaching and the learners’ communicative ability was “sometimes” (43.1%) tapped.

4.3. The learners’ evaluation of the assessment method at the institute

To discover the learners’ evaluation of the ELT program at the institute more comprehensively, they were asked to evaluate the assessment method used at the institute. The learners’ evaluation of the assessment method at the language institute is given in Table 3.

Table 3. The learners’ evaluation of the assessment method at the institute

The assessment method	Percentages of responses
	Strongly agree	Agree	Disagree	Strongly disagree
The oral quizzes administered every session at the institute are a good strategy to check the improvement of the learners’ language skills	30.6	56.5	8.3	4.6
The quizzes given at the institute provide the learners with constructive feedback and aid them in discovering their strengths and weaknesses in learning English	28.7	47.2	7.4	16.7
The tests administered during the term and the finals at the institute are neither easy nor difficult	24.1	27.8	39.8	8.3
The teachers' final class activity report at the institute is a fair and acceptably accurate assessment of the learners’ performance during the term	22.4	30.4	38.9	8.3
The learners are given as many mocks and written tests as they need during the term and before the final exam at the institute	18.5	21.3	38	22.2
The content of the quizzes and finals given at the institute closely corresponds to the content of instructions and textbooks	30.0	49.8	14.6	5.6

The table shows that the majority of the learners agreed that the oral quizzes given at the institute were a good strategy to check their improvement (56.5%), the quizzes provided the learners with constructive feedback (47.2%), and the content of the quizzes and finals corresponded to the content of instructions and textbooks (49.8%). The learners, however, believed that they were not given enough mocks and written tests (38%), and the teachers’ final reports were not fair and accurate (38.9%).

4.4. The language learners’ evaluation of the learners’ voice at the institute

The study also intended to discover how learners’ voice was treated when decisions about educational and logistic issues were made at the institute. Table 4 shows the learners’ evaluation of the heed paid to their voice and needs at the institute.

Table 4. The learners’ evaluation of learners’ voice at the institute

Questions	Percentages of the responses
	Strongly agree	Agree	Disagree	Strongly disagree
I can easily have contact with my teachers at the institute when I need to do so	20.4	40.7	19.4	19.4
I can ask my questions, if any, from my teachers in the classes at the institute	37.0	46.3	15.7	.9
I can freely express my ideas about the issues at hand in the classes at the institute	27.9	38.9	19.3	13.9
The learners' ideas about educational, logistical, and administrative issues at the institute are sough and taken into consideration	8.3	21.3	33.3	37.0
The staff members, administrators, and director at the language institute have a good sense of cooperation and assist learners in overcoming their problems at the institute	8.3	21.3	20.4	50.0

As the table indicates, the learners mostly believed that they could communicate with their teachers easily (40.7%), ask their questions form the teachers in the classes (46.3%), and freely express their ideas in the class (38.9%). However, the majority of the learners expressed that the staff members, administrators, and director at the institute were not cooperative enough (50.0%), and their ideas about educational, logistical, and administrative issues were not duly considered (37.0%).

4.5. The learners’ evaluation of the strengths and weaknesses of the institute

Semi-structured interviews with key questions on the weaknesses and strengths of the ELT program at the institute were conducted with the learners. The codes and categories emerging from the qualitative content analysis of the learners’ evaluation of the strengths and weakness of the institute along with excerpts of the learners’ responses are given in Table 5.

Table 5. Results from the qualitative content analysis of the learners’ responses

Category	Codes	Excerpts of the learners’ responses
Educational advantages	Teachers	1. The teachers at the institute are well-educated and resourceful. 2. The teachers know how to teach the materials and help the learners learn better. 3. The teachers use effective techniques to get across the concepts. 4. The teachers are highly credentialed, skillful and helpful.
	Accessibility	1. I am a university student in the capital and study English at the language institute. As the language institute has branches in my hometown, when I go back to my hometown, I do not miss my classes and can continue my language learning there. 2. When we travel to other cities during summer vacations, we can have our language learning classes almost everywhere we go. 3. As the institute has many branches across the country, if we travel to other cities, there will be no interruption in our language learning at the institute. 4. The institute has many branches across the country and even in some cases in the same city, which accept students from all other branches of the institute.
Educational disadvantages	Teaching method	1. Little attention is paid to communicative language skills at the institute. Add more free discussion time to the class activities. 2. Despite taking placement tests and final exams, the learners in the same class are usually at different levels of language proficiency, which disrupts the flow of instruction and learning in the class. 3. Using only English as the language of instruction of words in the Vocabulary Section seems to be a major problem. Providing students with Persian equivalents of the new words at all proficiency levels at the institute seems to be helpful.
	Assessment	1. The reading part of the final exams is difficult. 2. The set pass mark at the institute is too high. 3. The exams administered at the institute are generally difficult. 4. The final exams at the institute lack items checking the learners’ speaking skill. 5. There is no writing part in the exams given at the institute.
Logistical issues	Class arrangements and rules	1. The number of sessions that the learners can be absent from the language classes at the institute is limited (three sessions). 2. The time of each class (105) is too long and the number of classes each term (21 sessions) is too many. 3. The learners are not given the right and freedom to choose the days they wish to go to the institute. They also cannot choose their favorite teachers. 4. The number of language learners in the classes at the institute is too high.
	Staff’s cooperation	1. The staff members and administrators are not cooperative enough. 2. The staff members are weak at informing the learners about the educational issues at the institute. 3. The time for paying the tuition fee is to be after the final exam. 4. Most of the time, the administrative staff do not answer the phone, or if they do, they are not cooperative enough. 5. On the exam days or registration days, the personnel are not cooperative enough.
	Facilities	1. The website and communication systems of the institute are weak and outdated. 2. There is no café at the institute. 3. The cooling and heating systems are weak. 4. No parking lot has been devised for the learners’ vehicles. 5. The speed of the Internet at the institute is slow and poor, and the learners cannot connect to the net.

The qualitative content analysis of the learners’ responses showed that the majority of the learners commented that one of the distinguishing features of the language institute was its “resourceful” teachers. The learners also stressed that having branches in all cities across the country was one of the strengths of the institute. The weak points the learners raised in their evaluations of the institute could be categorized as educational and logistic. The educational weaknesses were related to the teaching and assessment methods and the materials used. The logistical weaknesses indicated by the learners were the staff members’ sense of cooperation and facilities. The learners believed that the way issues such as the time of the classes, the size of the classes, and the learners’ ideas about the arrangement and management of the classes were handled was not acceptable.

5. Discussion

This study aimed to discover the language learners’ perception of the effectiveness of the ELT program at an institute in the Iranian context. The learners evaluated the sufficiency of the teaching materials, the teaching method, the assessment method, and the status of the learners’ voice at the institute. They also expressed their views about the strengths and weaknesses of the prescribed ELT program at the institute. The significant findings of the study are discussed below.

5.1. The sufficiency of the teaching materials

With regard to the sufficiency of the teaching materials used at the institute, most learners indicated that the reading, writing, listening, grammar, and vocabulary materials they received at the institute were sufficient. The learners, however, showed that they needed more teaching materials for the speaking skill. The finding corresponds to the results of other studies investigating the significance attached to different language skills in the Iranian context. Describing the materials used based on the current ELT syllabus in the Iranian official education system, Sadeghi and Richards (2015) noted, “The course materials at middle school primarily address alphabet recognition, pronunciation, and limited vocabulary development, while those used at high school continue to focus on reading comprehension, grammar and vocabulary development, with little emphasis on writing beyond de-contextualized sentence practice. Listening is almost absent in the syllabus, and speaking is limited to a few drills (mainly intended to practice grammar) and short dialogues to introduce language functions.” Moreover, English language is a compulsory subject from grade 7 to grade 12 in the Iranian school curriculum, which, according to Hayati and Mashhadi (2010), underscores “grammar and reading.” Therefore, one likely reason for the way the learners evaluated the materials in the present study could be the fact that they received enough materials for the reading, writing, grammar, and vocabulary skills at public schools. A second reason for the way the learners evaluated the materials in the present study could be the fact that a specific section for the reading, writing, listening, grammar, and vocabulary skills has been incorporated in every unit of the student’s book at the language institute. The follow-up subsection of the listening part, which requires learners to discuss their ideas about the listening part, seems to be the only part in the student’s book focusing on the speaking skill. According to Sadeghi and Richards (2015), “Iranian learners often evaluate their success in language learning, as well as the effectiveness of their English course, on the basis of how well they feel they have improved in their spoken-language proficiency.” As a result, it can be argued that as the Iranian learners have little exposure to English language out of the institute, they go to the institute to compensate for the lack of speaking practice, and hence their demand for more materials for the skill. The insufficiency of the speaking materials seems to exacerbate the condition for the language learners as they go to language institutes to compensate for the lack of communicative activities at public schools. Dahmardeh (2009), in this regard, noted, “Teaching communicative skills remains a neglected component in many foreign language classrooms.” Moreover, public school teachers focus their pedagogy on the explicit teaching of grammar rather than English communication skills (Baleghizadeh & Farshchi, 2009). Hayati and Mashhadi (2010) also argued, “Iranian educational policy for English mostly centers on grammar and reading elements incommensurate with the ever-increasing demands of society.” Similarly, Haghighi and Norton (2016) held that Iranian learners aspiring to improve their communicative English skills “look for support beyond public schooling, and therefore choose to study at private English language institute.” Therefore, the insufficiency of communicative materials, especially speaking materials, at the institute becomes one of the major drawbacks of the ELT program at the institute from the learners’ perspective.

5.2. The teaching method and techniques

The language learners’ evaluation of the teaching method and techniques used at the institute showed that except for lack of using “real life tasks” and working on “learners’ communicative abilities”, other language teaching techniques were used appropriately. The drawbacks seem to have roots in the audiolingualistic language teaching method used at the institute. Sedighi and Nazari (2007) described the prescribed method of language teaching at the institute as audio-lingual. The language teaching method emphasizes repetition drills, substitution drills, and pronunciation drills. It also prescribes that the language of instruction should be only English. Another line of argument in relation to the weaknesses of the teaching methods and techniques at the institute could be the lack of authentic tasks and the scarcity of opportunities for developing the learners’ communicative skills at the majority of language teaching institutes and the official ELT program in Iran. With regard to Iranian institutes’ failure to meet the learners’ language learning needs, Mirhosseini and Khodakarami (2015) described that Iranian learners felt dissatisfied with the efficacy of language institutes as they mostly studied grammar and vocabulary and the class activities did not develop their speaking skill. Likewise, Razmjoo and Riazi’s (2006) study on the language teaching methods used at Iranian institutes indicated that only less than 14% of the average English class time was spent on speaking and the rest of the time was devoted to other skills and sub-skills. Similarly, Sadeghi and Richards (2015) reported the lack of communicative activities in the classes at private institutes, noting, “Students often report that in such classes, they receive similar kinds of lessons to those encountered in high school, and many move from one institute to another to find courses that will improve their spoken English.” Moreover, underscoring the paucity of communicative activities in the official ELT program in Iran, Borjian (2013) held, “Only phonological, morphological, and syntactical aspects of English” are stressed in school curricula. Sadeghi and Richards (2015) also stressed, “Unless students have taken additional courses in a private institute, they normally have minimal communication skills in English.” Mirhosseini and Khodakarami (2015) also blamed the directors of private language institutes for the lack of emphasis on the communicative language skills, arguing that they believed that they were preparing the learners for English communication without having a clear understanding of communication in English.

5.3. The assessment method

With regard to the assessment method used at the institute, the learners expressed satisfaction with the oral quizzes administered every class, the feedback they received from the quizzes, and the correspondence between the content of the tests and the content of the instructions they received. They were, however, dissatisfied with the number of mocks they were given, the fairness and accuracy of the teachers’ final reports, and the difficulty level of the quizzes and finals. In a similar study, Tabatabaei and Loni (2015) investigated the problems of teaching and learning English in the Iranian high schools, reporting that the learners had complaints about restricting the evaluation merely to midterm and final exams. As for the correspondence between the content of the instructions and the content of the tests, it should be said that the tests at the institute are normally prepared by a panel of experienced teachers and based on the content of the student’s book and the workbook. This relates to the content validity of the tests defined by Brown (1996) as “any validity strategies that focus on the content of the test. To demonstrate content validity, testers investigate the degree to which a test is a representative sample of the content of whatever objectives or specifications the test was originally designed to measure.” Therefore, it can be safely claimed that the reason for the learners’ positive evaluation of the tests at the institute could be the precision and expertise with which the tests are developed.

The use of mocks as preparatory tests was one of the demands of the language learners at the institute. The effect of taking practice tests before official tests has been a source of debate in the literature. A number of studies have supported the effect of such tests, holding that they can cause a positive washback effect (Khodabakhshzadeh, Zardkanloo, & Alipoor, 2017; Yang & Badger, 2015), while others have cast doubt on the positive effects of such tests, arguing that they are context-bound (Green, 2007). As for the case in the present study, it can be argued that three self-assessment and remedial tests are incorporated in the workbook of the learners. In addition, an extracurricular book on sample tests of the different language proficiency levels at the language institute is available for the learners.

With regard to the fairness of the final reports, which also relates to the difficulty of the tests and quizzes, it should be said that 40% of the learners’ final score at the institute is calculated based on their score on the final exam and the remaining 60% is allotted to their class activities. The finals are scored objectively using a machine or a key sheet. The reason why many learners did not agree on the fairness and accuracy of the assessment system at the institute could be the subjective nature of 60% of the total score. According to the Educational Testing Service (ETS), subjective scoring is susceptible to evaluator bias, which can affect fairness and validity (ETS, 2002, p. 17). The ETS also holds that fairness is “secured by cutting down construct-irrelevant personal characteristics [that] have no appreciable effect on test results or their interpretation” (Educational Testing Service, 2002, p. 17). Moreover, Willingham (1999) viewed test fairness as an important aspect of validity. To allay concerns in this regard, Davies (1999) argued that for the interest of fairness two or more raters should mark subjectively scored tests.

The results also showed that setting a high pass mark was a weakness of the language institute from the learners’ perspective. With regard to the pass mark, it can be said that the prescribed scoring system at the institute is criterion-referenced. The pass mark at the language institute has been set at 75 out of 100. It could be said that the set score is to guarantee that the learners have become competent enough to move to the next level of language learning. In this regard, Alderson, Clapham, and Wall (1995, p. 155) discussed, “In many contexts, the setting of a pass mark is not thought to be a problem, and a fixed percentage is considered to be appropriate. This may be 50%, or 75% or some other magic number. This is simply a matter of historical tradition, yet there is no reason why any particular figure must be the pass/fail boundary.” They, furthermore, noted that for many tests the final score could be “a combination of subjectively and objectively marked items.” As for the difficulty level of the tests, it can be said that all test items are prepared based on the information the learners receive at the institute. The tests at the institute enjoy content validity; therefore, if the learners manage to cover the teaching materials at the institute well and if they are not affected by extraneous factors, they will have no problem with the difficulty level of the tests.

5.4. The learners’ voice and administrative issues

With regard to the learners’ voice at the institute, most of the learners complained that the administrators and staff members were not cooperative enough and the learners’ views about the arrangement of the classes were not considered. The learners, however, expressed that they could easily communicate with their teachers and ask questions from them. In a similar study, Tabatabaei and Pourakbari (2012) investigated the problems of teaching and learning English in the high schools of Isfahan, Iran. It was found that despite the existence of numerous problems with the ELT program, the learners and their English teachers had a friendly relationship. In addition, the concept of the learners’ voice can be discussed in terms of the learners’ needs, expectations, and rights. Therefore, the concerns raised in this regard could be discussed within the framework of the needs analysis. According to West (1994), needs analysis covers the “full educational process.” Brindley (1984) also defined needs in the educational context as wants, desires, demands, expectations, lacks, constraints, and requirements. Along the same line of discussion and stressing the significance of the learners’ expectations from a language teaching program, Nunan (1989) noted, “The effectiveness of a language program will be dictated as much by the attitudes and expectations of the learners as by the specifications of the official curriculum.” Therefore, it can be argued that the learners’ concerns about the heed paid to their voice at the institute relate to their language needs and expectations from the institute. The learners in the present study expected the administrators and staff members to be cooperative and demanded consideration for their views about the arrangement of the classes. As for the sense of cooperation among the staff members at the institute, it is noteworthy that in each branch of the institute there is an office for the learners’ affairs, whose responsibilities range from registration to issuing report cards and certificates. However, there could be some staff members who are not cooperative enough, and hence the learners’ dissatisfaction with the administrative issues at the language institute. As for considering the learners’ views about the arrangement of classes, it should be noted that the learners normally attend language classes at the institute twice a week on a Saturday–Wednesday, Sunday–Tuesday, or Monday–Thursday schedule. A reason for not considering the learners’ preferences in the arrangement of the classes could be the high number of learners enrolling for language learning courses at the institute. As the institute has limited space and personnel, it cannot live up to the expectations of all learners.

5.5. The teachers

The learners regarded the teachers at the language institute as one of its strengths. One of the learners, as a case in point, said in the interviews that the teachers at the institute were “well-educated and resourceful.” Another learner said, “The teachers know how to teach the materials and help the learners learn better.” In keeping with the findings in the present study, Freeman and Johnson (1998) emphasized the role of teachers in language teaching, saying, “language teacher education has begun to recognize that teachers, apart from the method or materials they may use, are central to understanding and improving English language teaching.” Galluzzo (2005) also stressed that among “the most oft-expressed statements about teaching is that nothing is more central to student learning than the quality of teacher.” Likewise, expanding the significant part teachers can play in education, Shohel and Banks (2010) maintained that teachers “are at the core of attempts to expand, improve, and reform the education system of any country.” Therefore, it can be argued that the language institute has been largely successful as long as educational issues are concerned. Unlike the finding of the present study, the results of the study by Tabatabaei and Pourakbari (2012) showed that the learners were dissatisfied with the performance of their teachers, complaining that they did not use teaching aids and did not teach in English. One of the reasons why the language learners in the present study gave credit to the language institute for its teachers could be the teacher recruitment policy there. To be employed at the language institute, the regulations of the institute require that the applicants should hold a degree in ELT or a certificate of high English language proficiency, pass the institute’s recruitment exam, and pass the oral interview of the institute, which involves a demo. As can be understood, the applicants should be highly qualified to meet the recruitment criteria of the institute. The second reason could be the class observation policy that is strictly implemented at the institute. Almost all teachers recruited at the institute start teaching from the basic levels. After being observed by a senior head teacher and meeting the required criteria, they are allowed to teach a higher level class. The language institute also organizes in-service teacher training courses with a focus on empowering the teachers in the areas of classroom management, teaching skills as well as language skills. Another reason could be the institute’s promotion policy. All teachers who manage to meet the required criteria to teach at a higher level class receive a raise. As meeting the criteria requires a high level of expertise in the field, the raise can act as an incentive for the teachers to improve and upgrade their level of knowledge in the field. More knowledgeable and skilled teachers can naturally provide learners with better teaching, which can guarantee the learners’ confidence in the teachers and their satisfaction with the program. In keeping with this line of argument, Noriss (2016) stressed that though teachers do not form the whole of a language program, they are one of the key components, if not the most important one, in many language teaching programs.

5.6. The facilities and accessibility

One of the significant findings of the study was issues concerning educational facilities and logistics at the institute. The results showed that the majority of the learners had complaints about the facilities at the institute, which corresponds to the findings of many other studies in this area of investigation in the Iranian context. Describing a typical English class at language institutes in the Iranian context, Sadeghi and Richards (2015) noted, “Most classes in institutes are held in traditional classrooms where seats are arranged in a linear format with students sitting back-to-face, with the teacher’s desk in a front corner … Almost all classes are equipped with a whiteboard; most have TV and video/CD-player facilities; a very limited number of institutes have access to computers, the Internet, intelligent whiteboards and language laboratories … In small cities, language schools use chalkboards and a portable CD player.” The results of the present study showed that the learners named the website, the Internet, the cooling and heating systems, and the parking lot as the facilities in which the institute was lacking. In a similar study, Tabatabaei and Pourakbari (2012) reported that in relation to facilities school libraries in high schools in the Iranian province of Isfahan were not well equipped and classrooms were poor in terms of facilities and physical conditions. In addition, the learners in the study by Razmjoo and Riazi (2006), which investigated the language teaching methodologies used in language learning institutes in the Iranian city of Shiraz, were satisfied with the general issues at the institutes. They, however, expressed dissatisfaction with the unavailability of supplementary materials, asking for more items of equipment. From the findings, it can be argued that the lack of facilities is mainly a financial issue. Upgrading the website of the language institute, equipping classes with better heating and cooling systems, and providing a parking lot for language learners need a large budget. The only source of income of the language institute is the tuition it receives from the learners; therefore, there is a mismatch between the income and the expenditures. In this regard, Hanssen and Solvoll (2015) described both investing in facilities and the cost of investments as significant, adding that it is of paramount significance for the institutes to invest in facilities contributing to the students’ satisfaction. While high-quality facilities have been proved to have a great influence on learning (Lewis, 2000; Tanner, 2009), inappropriate facilities have been reported to adversely affect students’ motivation (Hassanbeigi & Askari, 2010). The findings of a study by Hanssen and Solvoll (2015) showed that the quality of social areas, auditoriums, and libraries influenced students’ satisfaction with the educational institution, whereas computer access on campus did not influence the students’ satisfaction. They argued that the quality of the facilities at educational institutions had a great impact on the students’ satisfaction and the long-term success of the institutions. Likewise, Mai (2005) reported that the quality and accessibility of IT facilities were a predictor of students’ satisfaction. Lee et al. (2012) in a study described “thermal comfort, indoor air quality and audio and visual comforts” as facilities affecting learning and students’ satisfaction. The learners’ satisfaction with the logistics at institutes is noteworthy as educational institutions are a business-like sector and meeting students’ needs is their primary focus (Gruber, Fub, Voss, & Glaser-Zikuda, 2010); there is also a growing competition among education institutions, making them adopt market-oriented policies to distinguish themselves (Butt & Rehman, 2010).

The learners also described the institute’s branches across the country as one of its strengths. The language learners’ responses in the interviews clearly explain why they viewed the accessibility of the institute as one of its strengths. One of the learners commented that he is “from another city,” is “a university student in the capital,” and after finishing his studies in the capital, he may decide to go back to his hometown. As the language institute has “many branches across the country,” he can resume his language learning at other branches of the institute without the need to take placement tests or switch to another language teaching program. A line of argument for this could be that there is no institution or body in the country coordinating policies on teaching English as a foreign language at language institutes and every institute has its own rules and regulations. Balagafshe et al. (2014) noted that language institutes in the Iranian context employ different methodologies to teach English, and “there is no agreement upon the same method even between two institutes.” What worsens the situation is that the language institutes do not accept certificates of language proficiency from other institutes. Therefore, if language institutes have branches in other cities, it will help learners continue their language learning based on the same language teaching program in different cities and avoid the potential problems of switching to a different language teaching program.

6. Conclusion

This study investigated Iranian language learners’ perception of the effectiveness of the ELT program at a language teaching institute in terms of the sufficiency of the teaching materials, the language teaching method, the assessment method, and the attention paid to the learners’ voice. It also researched the strengths and weaknesses of the institute from the learners’ perspective. It was discovered that the learners received sufficient materials for the reading, writing, listening, grammar and vocabulary skills, but felt the need for more materials for the speaking skill. The findings also indicated that the institute prescribed structural methods and techniques of language teaching and limped along in teaching communicative language skills and using authentic tasks. From the findings, it can be concluded that the language teaching method at the institute should be replaced with a communicative method of language teaching. Moreover, with a metamorphosis to a communicative language teaching method, the teaching materials at the institute should naturally undergo a transformation to fit the requirements of the teaching method. The results also showed that the learners complained about the fairness of the teachers’ final reports, the difficulty of the final exams, and the number of mocks they were given before the final exams. They were, however, satisfied with other aspects of the assessment method. It seems that using objective rather than subjective assessment, employing standardized tests, and giving more preparatory tests can alleviate the problems with the assessment method at the institute to a great extent. The study also discovered that despite being satisfied with the way the teachers treated the learners’ voice in the classroom atmosphere, the learners had complaints about the heed other officials at the institute paid to the learners’ voice. To address this issue at the institute, consultation sessions with the learners can be organized, learner committees or councils can be established, and suggestion boxes can be used.

The findings also revealed that the learners described the teachers and accessibility as the strengths of the institute and had complaints about the staff members’ sense of cooperation and the logistic factors. The findings in relation to the staff members’ sense of cooperation indicate that the institute has failed to establish a strong relationship with the learners. To allay the learners’ concerns in this regard, the institute can set rules and incentives requiring all staff members and administrators to be cooperative, friendly, and sociable. In addition, in-service training courses and workshops on interaction, communication, and cooperation can be organized for the staff at the institute. The learners’ responses also showed that the logistic issues had a substantial influence on their evaluation of the effectiveness of the ELT program, and thereby on their satisfaction with the institute. Therefore, it is suggested that the authorities at the institute make supportive policies on logistics to meet the learners’ needs and secure the sustenance of the institute in the competitive education market.

Additional information

Funding

This author received no direct funding for this research.

Notes on contributors

Iman Alizadeh

Dr. Iman Alizadeh holds a Ph.D. in Teaching English as a Foreign Language (TEFL) from Allameh Tabataba’i University, Tehran, Iran. He is a faculty member of Guilan University of Medical Sciences, Rasht, Iran. He is an assistant professor of TEFL and teaches General English courses, ESP courses, Research in SLA, Second Language Acquisition/Learning, and Applied Linguistics. He has published research papers in different areas of TEFL. His main research interests, however, center around language testing/assessment, vocabulary learning/teaching, Psycholinguistics, and e-learning.