The effect of evaluative language on high- and low-graded post-graduate students’ academic writing ability across gender

Abstract

Inspired by appraisal theory, the current study looked into the effect of using evaluative language on high- and low-graded post-graduate students’ academic writing in an EFL context across male and female groups. Participants were 60 MA students of TEFL who were randomly assigned to experimental (n = 30) and control (n = 30) groups. Attitudinal and Graduation resources of evaluative language were taught to raise the students’ critical stance in academic writing. Conventional writing instruction was provided for the control group. The writing pre- and post-test was the main instrument in order to investigate the difference between the students’ performance in the study groups. Findings revealed that evaluative language could significantly result in increasing the post-graduate students’ academic writing ability, while no significant difference was resulted between the male and female groups. High-graded students were also found to act significantly better than the low-graded ones in writing academically. The results of the study revealed that evaluative language can contribute to the post-graduate students’ possessions of critical stance in writing academically, which helps the writer to make a dialogic interaction with the reader.

Keywords:

• Academic writing
• appraisal theory
• attitude
• evaluative language
• graduation
• high- and low-graded students

PUBLIC INTEREST STATEMENT

Drawing on Martin and White’s (2005) Appraisal Theory, the present study focused deeply on the effect of using evaluative language on high- and low-graded post-graduate students’ academic writing in an EFL context across male and female groups. In this study, attitudinal and graduation resources of evaluative language were taught to raise the students’ critical stance in academic writing. The writing pre- and post-test was the main instrument in order to investigate the difference between the students’ performance in the study groups. Findings revealed that evaluative language could significantly result in increasing the post-graduate students’ academic writing ability, while no significant difference was resulted between the male and female groups. High-graded students were also found to act significantly better than the low-graded ones in writing academically. The results of the study revealed that evaluative language can contribute to the post-graduate students’ possessions of critical stance in writing academically, which helps the writer to make a dialogic interaction with the reader.

1. Introduction

Academic writing for English for Academic Purposes (EAP) students has been considered as one the crucial aspects of academic courses. The ability to write academically can be regarded as the predictor of the EAP students’ academic success (Hyland, 2002) although it has always been a challenging task for students regardless of being high- or low-graded ones (Hyland, 2006). In fact, students find it problematic to write academically, which might result from their reluctance to be proficient writers (Turner, 2015). From another point of view, it has been claimed that lack of critical stance in academic writing can lead to their failure (Arsyad, 2000; Lee, 2006). Pedagogically, academic writing instruction has apparently been taken for granted in English Language Teaching (ELT) research (Connor, 1996; Hyland, 2003), which demands teacher education reforms in raising the EAP teachers’ awareness of adopting a more interactive approach for meeting the needs of students.

It is widely agreed that writing provides an interactional platform for the writer and the reader. Through writing, the writer is able to develop unity and interaction with the target audience (Thompson, 2001). Thus, in the case of academic writing, a writer is required to be equipped with an ability to use evaluative and interactive language. In this regard, various terminologies for teaching academic writing through interpersonal resources have been introduced the examples of which can be attitude (Halliday, 1994), evaluation (Hunston & Thompson, 2000), and appraisal (Martin, 2000). In order help EAP students use evaluative language to show their critical stance in academic writing, teachers can employ appraisal theory (Martin, 1992, 1997; Martin & White, 2005) to meet EAP students’ communicative needs in writing. Evaluative language in appraisal involves the use of interpersonal resources at the level of discourse and grammar through which the writer can prove his/her position as the writer and show his/her mastery over the provided arguments and claims, acknowledging the quality of writing piece.

Appraisal theory is considered as a recently-developed analytical framework derived from Systemic Functional Linguistics (SFL) (Halliday & Matthiessen, 2004). It aims at describing the ways in which linguistic realization of interpersonal meaning takes place when using language. The three categories of appraisal include Attitude, Engagement, and Graduation (Martin & White, 2005). Figure 1 shows the overview of appraisal system and its main categories and subcategories. Attitude refers to the students’ use of evaluative language in showing their emotive responses. Engagement is the students’ interpersonal language use when they are interacting with the text and are dialogically engaged with their readers by adjusting and negotiating their claims. Graduation refers to the students’ use of language for evaluation in terms of intensifying or downgrading the discourse (Martin & Rose, 2003). Each of these categories justifies the use of evaluative language by EAP students to benefit from appraisal in their academic writing and acknowledge their critical stance in developing their argument or persuasion in writing (Lee, 2015). Therefore, graduate and post-graduate students, regardless of their writing proficiency, can be taught regarding appropriate and conscious employment of appraisal resources in developing their academic writing. Besides, as learners of different gender groups might have different performance in using evaluative language, investigating their academic writing affected by teaching appraisal resources can be worth attention. More importantly, teaching Attitude and Graduation resources to EFL learners in an academic context seems to have not been sufficiently addressed in an EAP context, comparing high- and low-graded learners’ writing. The most significantly, appraisal system can be effective on the students’ creating of interpersonal meanings in their writing and production of discourse. Lee (2015), however, argues that Attitude and Graduation can be more applicable for the students’ academic writing, and the instructor can direct the students’ attention towards using these resources in order to promote more creativity and enable students to show their emotive responses and then do the valuation of a previously generated claim.

Figure 1. Appraisal system categories and subcategories (adapted from Martin & White, 2005)

2. Literature review

The related studies on evaluative language can be divided into experimental and non-experimental researches conducted in EFL and ESL settings. Each part is explained in the following.

Regarding the use of evaluative language framed in appraisal theory (Martin, 1992, 1997; Martin & White, 2005), there is a paucity of research conducted to explore the effect of benefiting from language of evaluation as an instructional approach to improve the students’ academic writing or other language skills as well. In this regard, Lee (2010) has been one the supporters of appraisal theory, suggesting its application in the students’ development of interpersonal relationship in their persuasive essays by tertiary students. Lee took into account the essay writings of six East Asian and six Australian students from the Faculty of Arts at a university in Australia. The main appraisal was Engagement aiming to investigate the students’ use of Attribution in their writings. The research particularly aimed at concentrating on the high- and low-graded students’ use of explicit/manifest intertextual resources. It has to be added that no specific instruction of appraisal theory was conducted in the study. However, there were sessions of scaffolding for the students in order to resolve any possible ambiguities during the course in which the participants were requested to write four persuasive essays.

Lee (2010) employed the model presented by White (1998) and Martin and Rose (2007) in order to look into “propositions and proposals and both Monogloss and Heterogloss utterances” (p. 187) as Attribution systems. Through coding the corpus of the study, the identification of significant grade-based differences in applying Attribution in the students’ writing was observed. The most highlighting point was the finding that successful writers were in favor of using Extragloss in their persuasive writing. The researcher pointed out EAP students can be taught regarding the use of Attribution and intertextual resources in their academic writing, particularly when they are providing evidence for their claims. When instruction is successfully done for the students, they can convey their message as clearly as possible and make a balance in their interpersonal and socio-cultural relationship with the audience.

Investigating the Iranian EFL students’ critical reading comprehension ability, Akbarzade Haromi (2014) benefited from appraisal theory to assist university students in doing the reading tasks through critical stance. The researcher argued that due the time-consuming nature of adopting the entire appraisal framework of Attitude, Engagement, and Graduation, attempt was made not to bore the students and benefit from engagement resources in order to develop the students’ critical reading comprehension ability. The researcher provided the students with the instruction of Engagement resource in which the linguistic devices having the function of commenting on a claim in the selected articles were specified by the instructors and the students were given time to find similar strategies in the studied texts. The conventional instruction of reading comprehension was conducted by familiarizing the students with the strategies of brainstorming and introducing some key vocabularies. Akbarzade Haromi found that the BA students’ critical stance in reading comprehension was developed through teaching Engagement resources and the students could reflect their critical stance in doing the reading comprehension tasks. The researcher recommended ELT teachers and researcher to carry out more experimental research, although it might be challenging to implement appraisal resources, in order to promote the students’ evaluative language use, which can enhance their mental ability in using a variety of linguistic resources in communicating reflected in their writing or speaking, particularly in an academic setting.

The literature on using appraisal theory includes a number of studies that direct EFL and ESL scholars to take into account various dimensions of evaluative language in investigating the students’ academic writing ability in general and persuasive and argumentative writing in particular. For example, Wu (2008) investigated the role of evaluative language in analyzing the statement types in high- and low-rated essays through the Hunston’s (1989) concept of evaluation status. According to Hunston, statements can follow two main purposes of informing and focusing, which can be used by the write to convey the message. First-year undergraduate students of the National University of Singapore were required to write essay on a particular topic. Textual analysis of the essays revealed that the statement types of world-creating statements, interpretation statement type, and factual statements were among the most common types of structures used by the undergraduate students.

Low-rated students used minimum number of factual statements in their essays although Wu (2008) believed that it cannot be resulted from their less knowledge of academic writing. Findings contributed to the practicality of teaching statement types as an evaluative tool in developing the EAP students’ academic writing ability in generating argument or persuasion in a writing task more appropriately. As high-rated students were found to use interpretation statement types, it can be recommended that academic courses can fulfill the students’ needs by raising the students’ awareness of using interpretive language while writing in academic genre. In fact, no matter when a student has the least or most knowledge of academic writing, helping undergraduate, graduate, and post-graduate students to enhance their understanding of statements in a written language can be efficient in their academic success, particularly in the field of academic writing.

In an analytical research on the EAP students’ use of Attitudinal resources in persuasive writing, Lee (2008) collected the data from 12 EAP students of the Faculty of Arts an Australian university. Lee employed appraisal theory as an analytical approach to look into the students’ essay writing during the course in which each student had to write four assignments by constructing a persuasive or argumentative piece of writing. The subcategories of Affect, judgment, and Appreciation were investigated in the students’ writing to see the extent to which the participants were able to use such expressions of evaluation in developing their argument or persuasion. Results highlighted significant differences in the use of Attitudinal resources between high- and low-graded students.

As high-graded writers were shown to use more evaluative language in their argumentative writing, Lee (2008) asserted that theses students’ writing is preferred to be argumentative rather than descriptive. In order to make a judgment to develop their argument, high-graded writers employed various instance of interpersonal resources to clarify their point of view. As to the implication side, the researcher added that “incorporating the appraisal system into teaching can help undergraduate and international students to enhance skills associated with expression” (p. 55). Although the study suffers from generalizing the results due to small sample size, Lee further suggests that teaching evaluative language should be strongly attended EAP writing teachers to teaching students the language of evaluation for enhancing their critical ability in writing.

Inspired by the complete framework of appraisal theory, Liu (2013) took into account the use of Attitude, Engagement, and Graduation resources in Chinese EFL students’ writing. The students were divided into high- and low-graded writers according to the holistic rating in which the linguistic elements were not examined by the two raters of the study. The analysis of high- and low-rated essays resulted in the existence of similarities and difference in using evaluative resources in their argumentative writing. Comparing the two groups of students in terms of using Attitudinal resources indicated that both groups were in favor of using Appreciation resources in comparison with Affect and judgment, showing that the students’ writing was found to be more likely appreciative rather than personal, which might be due to the topic of Internet. Textual analysis confirmed the successful employment of Affect resources by high-rated writers. Liu also found that “in the high-rated essay, there was a lower frequency of Engagement occurrences but a higher frequency of Monoglossic resources” (p. 51).

The employment of Graduation resources was also observed in high-rated essays in the sense that the writers could make a meaningful interplay concerning the use of Attitude and Graduation interpersonal instances. The researcher insisted on the pedagogical implications of the study by claiming that “instead of primarily focusing on the correctness of grammar use, EFL/ESL writing instruction and learning needs to pay more attention to evaluative meanings conveyed through linguistic construction” (p. 51). The author also recommended father research by investigating the effect of evaluative language in experimental studies and explore its effectiveness on Chinese and English writings of Chinese EFL students.

In a similar study on the utilization of appraisal theory as a framework for investigating the use of evaluative resources in the argumentative writings of 15 Kurdish-speaking MA students of TEFL, Jalilifar and Hemmati (2013) conducted quantitative and qualitative analyses of high- and low-graded essays. It was generally found that quantitative and qualitative data confirmed that high-graded writers were able to successfully apply Attitudinal resources in developing their argument in academic writing. The Engagement resources were weakly used by the low-graded writers compared to high-graded ones. There was marginal difference concerning the use of Graduation resources by both groups of students. The researchers recognized the role of appraisal theory as an analytical framework for taking into account the students’ academic writing ability. By mentioning the limitation of the study regarding excluding the students’ linguistic background, the researchers suggested a further study to consider this limitation and investigate whether the writers’ first and second language can have impact on using evaluative resources in academic writing.

In a more recent study, Lee (2015) conducted a meta-analysis of using evaluative stances of Attitude by EAP students from an Australian university. The research took into account the students’ use of Appreciation, as a subcategory of Attitude, in their argumentative writing. Lee found that successful writers could benefit from Appreciation instances in order to do Valuation in their argumentative writing. The researcher aimed to broaden the scholars’ insights regarding the pedagogical implication of evaluative language framed in appraisal theory. In fact, it was argued that teaching evaluative language can be a valuable practice for EFL and ESL teachers of academic writing to benefit from this instructional approach in developing the students’ ability in writing argumentatively rather than descriptively.

Bouwer et al. (2018) encouraged language teachers to teach learners how to analyze texts through using evaluative language, which can pave the way for their quality writing, particularly academic one. Researchers found that learners’ awareness of appraisal resources leads to their high potentials in developing self-regulated writing ability and boosting their critical voice in generating their claims more effectively.

In the same vein, Morton and Llinares (2018) investigated whether L2 learners could use appraisal resources in their written and oral production. Similarities and differences observed between high- and low-rated learners regarding the use of evaluative language in their writing and speaking about history in a bilingual education program. Researchers acknowledged the instruction of evaluative language as an effective strategy to help learners take a critical stance in their arguments, while suggesting further research in broader and different contexts to take into account its effectiveness on the learners’ use of evaluative voice in their production.

The literature on appraisal theory proves its recognition by EFL and ESL scholars. In fact, the majority of the research has benefited from the theory as a tool to analyze students’ use of evaluative language in their academic writing. In addition, in the context of EFL students, there exists a paucity of research on appraisal theory. Last but not least, teaching evaluative language through conducting an experimental study and taking into account the students’ gender and their comparison of high- and low-graded writing ability appear to have not been investigated. Hence, the following research questions were addressed:

RQ1. Is there any significant difference between the effect of teaching writing through evaluative language regarding the male and female students’ academic writing ability?

RQ2. Is there any significant difference between the effect of teaching writing through evaluative language regarding the high- and low-graded students’ academic writing ability?

3. Methodology

The current study investigated university students’ academic writing ability affected by evaluative language. The methodology applied in the study was quantitative. Information on the design and the context of the study as well as the characteristics of the participants is provided below. The main instruments, data collection procedure, and explanation of data analysis are subsequently presented.

3.1. Design and context of the study

The present study was conducted through an experimental design in which appraisal resources were employed by the researchers in order to investigate its effectiveness on the post-graduate students’ academic writing ability. The setting for conducting the study was Islamic Azad University, Ahvaz Branch situated in Khuzestan Province, Iran. The target context was the Faculty of Foreign Languages where the participants of the study were chosen for the first semester of 2018–2019 academic year.

3.2. Participants

There were 60 post-graduate MA students of TEFL who constituted the main participants of the study. They were the first-year students of Islamic Azad University, Ahvaz Branch at the time of the study. Thirty students formed the experimental group (16 low-graded and 14 high-graded) and the remaining ones were assigned as the control group randomly. Purposeful sampling was adopted by the researcher to select post-graduate students who had received at least one of the basic courses of Essay Writing, and Paragraph Development. In addition, based on the researchers’ observations and conversations with the university teachers, no Advanced Writing course was taken by the students which ensured participants’ homogeneity in terms of their educational background as they were all doing their MA courses. It had been widely acknowledged that by running merely a proficiency test, the researcher cannot thoroughly ensure the homogeneity of the participants (Artiles et al., 2005; MaCswan & Rolstad, 2006). The participants’ age range was from 23 to 31. Table 1 shows demographic information of the participants.

Table 1. Demographic background of the participants

No. of Students	60 (30 experimental; 30 control; MA students of TEFL)
Gender	36 Females & 24 Males
Native Language	Persian
Major	TEFL
University	IAU, Ahvaz Branch
Academic Year	2018–2019

As to Table 1, the participants’ gender distribution included 36 female and 24 male students, who were studying TEFL and doing their MA course at the time of conducting the study.

3.3. Instruments

In order to collect the data, two main instruments were used; writing pre- and post-test to check the students’ academic writing ability and the rating scale used to rate the students’ writing. More detail for each instrument is provided below.

3.3.1. Writing pre- and post-test

The writing pre-test was used in order to examine the students’ initial ability in writing academically. Then, to check the effect of teaching evaluative language on the students’ academic writing, students took the post-test. They were requested to write an essay of at least 250 words within 50 minutes. The topic was “Learning about the past has no value for those of us living in the present. Do you agree or disagree? Use specific reasons and examples to support your answer”, which was extracted from IELTS exam (IELTS 5). Students were expected to provide sufficient reasons for their claims and support their position. The writing pre- and post-tests were rated by the raters to identify their writing quality.

3.3.2. Rating scale

The scale provided for the raters was developed by Jalilifar and Hemmati (2013) (see Appendix). It aimed at assessing the students’ writing by considering content, organization, vocabulary, language use, and mechanics. The grading was estimated to divide students into four levels: excellent to very good (83–100 points), good to average (63–82 points), fair to poor (52–62 points), and very poor (34–51 points). Jalilifar and Hemmati (2013) claimed that the cut-off score was 63, and the scores below (between 34 to 62) and higher than 63 were considered as low-graded and high-graded writers, respectively. Three teachers of academic writing at BA and MA levels of TEFL who had been teaching IELTS for more than six years were the raters. They had consultation with the researchers to be thoroughly aware of how to rate the students’ writing.

3.4. Data collection procedure

In order to collect the data, initially, the researchers were concerned with the selection of participants. They included 60 post-graduate MA students of TEFL, who were studying at Islamic Azad University, Ahvaz Branch. The administrative procedures for conducting the study were done in that one of the researchers met the head of the University and the dean of the Faculty of Foreign Language for official permission. Two Faculty professors who were PhD holders in TEFL were cordially invited to teach academic writing to the participants of the study. Both students and instructors were thoroughly informed regarding the objectives of the study and informed consent was obtained to meet the ethical considerations of the research. Additional sessions were also held between the instructors and the researchers concerning the instructional procedures and using appraisal resources as the medium of instruction. The materials covered during the instruction included the essays adopted from Bailey (2003) and Bill (2011) in which the instructors could direct the students’ attention toward using evaluative language. The participants in the experimental and control groups underwent ten 2-hour sessions of academic writing instruction that is explained in what follows.

The experimental group was exposed to teaching evaluative language. Appraisal resources were employed by the instructor (Martin & White, 2005) in order to raise the students’ awareness of Attitude and Graduation resources in writing academically. Students were provided with general and specific explanations of the appraisal resources of Attitude and Graduation, and any ambiguities were scaffolded. The selected essays were taken into consideration by the students to get familiar with the instances of appraisal resources used in the writing and investigate their functions. The instructor directed the students’ attention to the quality of the statements in the essays with and without language of evaluation to help students figure out how the author had developed his/her ideas in maneuvering on the argument or claim. Interactive classroom was targeted in which students were encouraged to analyze the writing tasks in terms of appraisal resources of Attitude and Graduation in groups and then discuss them with their peers as a whole with the instructor monitoring them. Scrutinizing the essays was conducted sentence by sentence and paragraph by paragraph for discovering the most frequent phrases for appraisal resources, leading the students to self-recognize the importance of evaluative language in developing discourse in academic writing. The students themselves were also required to write on their own and use Attitudinal and Graduation resources in their essays and discuss their correctness with their peers and receive the instructor’s feedback as well.

By contrast the students in the control group underwent conventional instruction of academic writing in which they were initially familiarized with the general definition and structure of academic writing. Students were taught the linguistic rules. Modeling of the essay writing was conducted by the instructor in order to prepare a sample paragraph and show how different elements can be developed in a paragraph. The general intention was to help the students understand the general format of the paragraphs and the point that learners had to provide evidence for their thesis statement in academic writing. The use of cohesive devices was practiced by the instructor. Then, the students were required to select their personal topic and elaborate on it to produce an essay.

Finally, it can be stated that the pre- and post-test of writing were given to the students in order to investigate the difference among the study groups regarding their academic writing.

3.5. Data analysis procedure

In order to meet the objectives of the study quantitative methodology including descriptive and inferential statistics were used. More specifically, in order to address the research questions of the study with respect to investigating male and female students’ academic writing and their differences in experimental and control groups, descriptive measures containing the students’ development of mean scores were applied. One-way analysis of variance (one-way ANOVA) was also run to inferentially investigate gender differences. Regarding the high- and low-graded students’ academic writing, one-way analysis of covariance (one-way ANCOVA) was conducted.

As to the reliability of the pre- and post-tests, Cronbach’s alpha was used to run inter-rater reliability. The pre- and post-test reliability coefficients were .88 and .91, respectively, enjoying the acceptable estimate of reliability (Farhady et al., 1994).

4. Results

The present study investigated the effect of evaluative language on the high- and low-graded university students’ academic writing ability regarding their gender. Therefore, the results of the study were taken into consideration concerning the participants’ gender and grade as follows.

4.1. Male and female students’ academic writing

As to the academic writing of the female and male participants, initially descriptive statistics for the pre- and post-test of writing for two gender groups were provided as shown in Table 2.

Table 2. Descriptive statistics for the writing pre- and post-test of the female and male groups

		N	Mean	Std. Deviation	Std. Error	95% Confidence Interval for Mean		Minimum	Maximum
						Lower Bound	Upper Bound
FEMALE	PRE EXPERIMENTAL	18	63.0000	5.48795	1.29352	60.2709	65.7291	53.00	71.00
	POST EXPERIMENTAL	18	67.8333	3.66622	.86414	66.0102	69.6565	59.00	74.00
	PRE CONTROL	18	61.3333	5.28037	1.24460	58.7075	63.9592	51.00	71.00
	POST CONTROL	18	62.0000	4.83857	1.14046	59.5938	64.4062	53.00	72.00
	Total	72	63.5417	5.41239	.63786	62.2698	64.8135	51.00	74.00
MALE	PRE EXPERIMENTAL	12	63.92	3.059	.883	61.97	65.86	59	69
	POST EXPERIMENTAL	12	69.17	1.899	.548	67.96	70.37	66	72
	PRE CONTROL	12	64.33	4.376	1.263	61.55	67.11	59	72
	POST CONTROL	12	64.42	4.719	1.362	61.42	67.41	59	73
	Total	48	65.46	4.177	.603	64.25	66.67	59	73

Table 2 shows that there was an increase from the pre-test of female (M = 63.00; SD = 5.48) and male (M = 63.92; SD = 3.059) students to the writing post-test of female (M = 67.83; SD = 3.66) and male (M = 69.17; SD = 1.89) participants. Descriptive data reveals that female and male participants in the experimental group appeared to perform better than the control group that had no considerable increase in their mean scores. In order to investigate the level of significance between the gender groups, one-way ANOVA was run. To run this test, initially equality of variances has to be checked as in Table 3.

Table 3. Test of homogeneity of variances

	Levene Statistic	df1	df2	Sig.
FEMALE	1.177	3	68	.325
MALE	2.581	3	44	.065

As to Table 3, the p value for the female (sig. = .325) and male (sig. = .065) groups are greater than .05 and, therefore, acknowledging that the variances are equal. Table 4 shows one-way ANOVA results for the comparison of female and male students. for the experimental and control groups.

Table 4. One-way ANOVA for the writing pre- and post-test of the female and male groups

		Sum of Squares	df	Mean Square	F	Sig.
FEMALE	Between Groups	467.375	3	155.792	6.570	.001
	Within Groups	1612.500	68	23.713
	Total	2079.875	71
MALE	Between Groups	221.750	3	73.917	5.437	.003
	Within Groups	598.167	44	13.595
	Total	819.917	47

Inferential analysis in Table 4 shows that teaching writing through evaluative language significantly improved the female (F_3,68 = 6.750; p = .001) and male (F_3,44 = 5.437; p = .003) students’ academic writing ability because the significance level is less than .05. In order to compare female and male participants’ writing ability in the post-test, Table of multiple comparisons is presented below.

Table 5 shows that no significant difference was found between the male and female students in the experimental group on the post-test (mean difference = 1.34; p = .909) because the significance level is more than .05. In other words, there was no significant difference between the effect of teaching writing through evaluative language regarding the male and female students’ academic writing ability.

Table 5. multiple comparisons for the writing-post-test of the female and male groups

(I) Group MF	(J) Group MF	Mean Difference (I-J)	Std. Error	Sig.	95% Confidence Interval
					Lower Bound	Upper Bound
Ex Female	Ex Male	−1.34000	1.30061	.909	−4.3334	2.6001
	Con Female	5.83333*	1.62321	.001	2.5943	9.0724
	Con Male	3.41000*	1.20721	.003	1.2694	7.7050
Ex Male	Ex Female	1.34000	1.30061	.909	−2.6001	4.3334
	Con Female	7.17000*	1.36409	.001	1.8140	9.0860
	Con Male	4.750*	1.505	.003	1.72	7.78
Con Female	Ex Female	−5.83333*	1.62321	.001	2.5943	9.0724
	Ex Male	−7.17000*	1.36409	.001	−9.0860	−1.8140
	Con Male	−2.42000	1.27535	.801	−3.4956	3.3033
Con Male	Ex Female	−3.41000*	1.20721	.003	−7.7050	−1.2694
	Ex Male	4.750*	1.505	.003	1.72	7.78
	Con Female	.2.42000	1.27535	.801	−3.3033	3.4956
*. The mean difference is significant at the 0.05 level.

4.2. High- and low-graded students’ academic Writing

The current study also aimed at investigating the effect of appraisal resources on high and low-graded post-graduate students’ academic writing. Based on the rating scale, there were 16 low-graded and 14 high-graded students in terms of their writing ability. It is worth mentioning that the students in the experimental group were focused. Table 6 provides descriptive statistics for the high- and low-graded students’ writing.

Table 6. Descriptive statistics for the writing pre- and post-test of the high- and low-graded groups

		N	Mean	Std. Deviation	Std. Error	95% Confidence Interval for Mean		Minimum	Maximum
						Lower Bound	Upper Bound
E.PRE	LOW GRADE	16	59.81	2.810	.702	58.32	61.31	53	63
	HIGH GRADE	14	67.43	2.311	.618	66.09	68.76	64	71
	Total	30	63.37	4.627	.845	61.64	65.09	53	71
E.POST	LOW GRADE	16	66.56	2.920	.730	65.01	68.12	59	70
	HIGH GRADE	14	70.43	1.785	.477	69.40	71.46	67	74
	Total	30	68.37	3.113	.568	67.20	69.53	59	74

Descriptive results in Table 6 show that high-graded students performed better than the low-graded ones in the pre- and post-test of writing. In other words, the mean scores for the post-test of high- (M = 70.43; SD = 1.78) and low-graded (M = 66.54; SD = 2.92) students indicate the outperformance of high-graded students’ academic writing descriptively. One-way ANCOVA was run in Table 7 to present inferential results.

Table 7. One-way ANCOVA for the writing post-test of the high- and low-graded groups

Source	Type III Sum of Squares	df	Mean Square	F	Sig.	Partial Eta Squared	Noncent. Parameter	Observed Power^b
Corrected Model	213.141^a	2	106.571	42.424	.000	.759	84.847	1.000
Intercept	21.868	1	21.868	8.705	.006	.244	8.705	.812
E.PRE	101.541	1	101.541	40.421	.000	.600	40.421	1.000
E.PRE.CODE	6.785	1	6.785	2.701	.002	.091	2.701	.354
Error	67.825	27	2.512
Total	140,501.000	30
Corrected Total	280.967	29
a. R Squared = .759 (Adjusted R Squared = .741)
b. Computed using alpha = .05

ANCOVA results demonstrate that the mean difference between the post-test scores of the two groups is significant (F_1,27 = 2.70, p = .002), which shows that there was a significant difference between the effect of teaching writing through evaluative language regarding the high- and low-graded students’ academic writing ability in the sense that high-graded students outperformed the low-graded ones.

5. Discussion

The current research examined the effect of using evaluative language on the high- and low-graded post-graduate students’ academic writing ability across gender. Quantitative analysis of the pre- and post-test of writing indicated the male and female students’ significant improvement of academic writing. In addition, high-graded students could significantly perform better that low-graded ones. Finally, no significant difference was observed between the male and female participants’ academic writing.

Comparing and contrasting the results of this study with the previous literature on appraisal resources, it can be inferred that the results of the study can be in alignment with previous research (Lee, 2015; Liu, 2013; Martin & White, 2005) that confirmed the effectiveness of using appraisal resources in paving the way for students to develop their academic writing. It also be stated that the present study is the replication of the studies done by Jalilifar and Hemmati (2013), Lee (2008, 2010), and Liu (2013) who highlighted the practical use of evaluative language in persuasive and argumentative writing for EAP students. Furthermore, the researchers mentioned above found that high-graded students can benefit from evaluative language more than the low-graded ones in creating an interaction with the reader in a dialogic manner. Therefore, findings of the study can be supported by these non-experimental studies that acknowledged the high-graded students’ outperformance in using Attitudinal, Engagement, and Graduation resources. The current study found that teaching evaluative language can be pedagogically beneficial for post-graduate students’ interpersonal communication with the interlocutor at the level of discourse, which can be in agreement with Akbarzade Haromi (2013) and Lee’s (2010) experimental studies on the contribution of appraisal resources as an educational treatment.

Comparing and contrasting the findings of the study with previous research, researchers in this study found that when learners were taught to use evaluative language in their writing, regardless of their writing grades, they could develop their critical voice in academic writing. Accordingly, Morton and Llinares (2018) encouraged teachers to make learners familiarized with evaluative sources and enhance their evaluative knowledge in doing the writing tasks more efficiently and critically. Bouwer et al. (2018) also argues that the learners’ ability to analyze the text in a critical manner relies on their ability to use evaku8atoive language consciously, which lies at the learners’ exposure to an appropriate medium of instruction to learner appraisal resources.

It has to be motioned that findings of the study can, be however, in contrast with Akbarzade Haromi’s (2013) study as the concentration was on the students’ critical reading comprehension. In addition, Lee’s (2010) study was conducted in an ESL context, acknowledging the findings of the study for students of EFL classroom in an academic setting. Although the above-mentioned differences exist, the supporters of appraisal theory (Jalilifar & Hemmati, 2013; Lee, 2015; Martin & White, 2005) concur that using evaluative language can help student self-develop their critical voice in generating their claims in writing and be more purposeful in managing dialogic interaction with the target reader they are communicating with. Such quality can be considered as an important element of academic writing as a write needs to intensify or downgrade his/her arguments in writing or showing his/her emotions and give comments on particular phenomena all of which can be fulfilled by conscious use of evaluative language.

6. Conclusion

The present study was an attempt to have a quantitative investigation of using evaluative language on high- and low-graded students’ academic writing across gender groups. Findings showed that evaluative language could be the significant predictor of male and female students’ academic writing development, and high-graded students significantly outperformed low-graded students in writing academically. As to the implication side, the results of the study can pedagogically be of benefit for EAP teachers to adopt appraisal theory as an instructional approach in paving the way for students, particularly the graduate and post-graduate ones, to raise their critical awareness of academic writing and develop their literacy of using interpersonal resources. However, implementing appraisal-based instruction in an EAP classroom requires the teachers’ sufficient expertise regarding evaluative resources and how they should be applied by students to write critically. Teacher education programs can be helpful for teachers to raise their awareness of appraisal theory, and how its sub-categories can be used at the service of EAP teachers in writing instruction. When the teachers’ knowledge is satisfied, EAP students’ concerns of academic writing can be fulfilled by evaluative language as appraisal resources direct the students’ attention to be a mature critique rather than a simple writer.

The current study suffers from generalizing of the results due to small sample. The context of the study was also limited to a university in Ahvaz. The students’ major can be another source of limitation. Therefore, further studies can be conducted experimentally or comparatively with a participation of a large number of participants from different academic contexts, such as Azad and Public universities to shed light on the efficiency of evaluative language on the students’ academic writing. Another recommendation is called for studying other language components, such as reading comprehension, listening comprehension, or even speaking affected by appraisal resources. Last but not least, appraisal theory can be considered as a psychological treatment on the students’ critical thinking ability, which appears to have not been taken into account previously. Therefore, appraisal resources can be acknowledged as an effective tool in language pedagogy, which has to be further investigated in order to broaden our insights regarding its efficiency.

Additional information

Funding

The authors received no direct funding for this research.

Notes on contributors

Azizeh Chalak

Mona Bahmani ([email protected]) is a Ph.D. candidate in TEFL at Islamic Azad University, Isfahan (Khorasgan) Branch, Iran. Her principal research interests are critical discourse analysis, cognitive semantics, and TEFL studies. Her latest publication “Interpersonal Rhetoric of Attitude in News: CNN vs. AJE” co-authored with Dr. Ahlam Alharbi published in Pragmatics and Society, Vol. 10:2 (2019).

Azizeh Chalak ([email protected]) is an associate professor of TEFL at Islamic Azad University, Isfahan (Khorasgan) Branch, Iran. Her research works mainly include discourse analysis, sociolinguistics, E-mail communications and CMC.

Hossein Heidari Tabrizi ([email protected]) is an associate professor of TEFL at Islamic Azad University, Isfahan (Khorasgan) Branch, Iran. His research works mainly include foreign/second language assessment and translation studies.

Appendix

Rating scale for assessing the learners’ essays (adapted from Jalilifar & Hemmati, 2013).

Table

Score	Range	Content
	27–30	EXCELLENT TO VERY GOOD: knowledgeable • substantive • thorough development of thesis• relevant to assigned topic • audience/reader awareness
	22–26	GOOD TO AVERAGE: some knowledge of subject •adequate range • limited development of thesis• mostly relevant to topic, but lacks detail • somewhat the awareness of audience/reader
	17–21	FAIR TO POOR: limited knowledge of subject • little substance • inadequate development of topic • inadequate audience/reader awareness
	13–16	VERY POOR: does not show knowledge of subject • non-substantive • not pertinent • OR not enough to evaluate • no awareness of audience/reader
Organization plan for an argument
	18–20	EXCELLENT TO VERY GOOD: presents the complete introduction (attention grabber, writer’s thesis) • presents the reason(s) supporting the claim • summarizes the views differing from writer’s • discusses for and against the views • brings essay to closure
	14–17	GOOD TO AVERAGE: somewhat introduces the thesis • presents and supports the reason(s) to some extent • loosely presents the main body of essay • limited discussion for and against the views • loosely sums up argument
	10–13	FAIR TO POOR: incomplete thesis • disconnected reasons • incomplete discussion of views • ambiguous and incomplete summary of argument
	7–9	VERY POOR: no organization of thesis, reason(s) and conclusion
Vocabulary
	18–20	EXCELLENT TO VERY GOOD: sophisticated range • effective word/idiom choice and usage • word form mastery • appropriate register
	14–17	GOOD TO AVERAGE: adequate range • occasional errors of word/idiom form, choice, usage but meaning not obscured
	10–13	FAIR TO POOR: limited range • frequent errors of word/idiom form, choice, usage • meaning confused or obscured
	7–9	VERY POOR: essentially translation • little knowledge of English vocabulary, idioms, word form • OR not enough to evaluate
Language use
	22–25	EXCELLENT TO VERY GOOD: effective complex constructions • few errors of agreement, tense, number, word order/function, articles, pronouns, prepositions
	18–21	GOOD TO AVERAGE: effective but simple constructions • minor problems in complex constructions • several errors of agreement, tense, number, word order/function, articles, pronouns, prepositions but meaning seldom obscured
	11–17	FAIR TO POOR: major problems in simple/complex constructions • frequent errors of negation, agreement, tense, number, word order/function, articles, pronouns, prepositions and/or fragments, run-ons, deletions • meaning confused or obscured
	5–10	VERY POOR: virtually no mastery of sentence construction rules • dominated by errors • does not communicate • OR not enough to evaluate
Mechanics
	5	EXCELLENT TO VERY GOOD: demonstrates mastery of conventions • few errors of spelling, punctuation, capitalization, paragraphing
	4	GOOD TO AVERAGE: occasional errors of spelling, punctuation, capitalization, paragraphing but meaning not obscured
	3	FAIR TO POOR: frequent errors of spelling, punctuation, capitalization, paragraphing • poor handwriting • meaning confused or obscured
	2	VERY POOR: no mastery of conventions • dominated by errors of spelling, punctuation, capitalization, paragraphing • handwriting illegible • OR not enough to evaluate