Online peer feedback patterns of success and failure in argumentative essay writing

Figures & data

Table 1. Coding scheme details to analyze the quality of students’ argumentative essay.

Variables	Points	Labels	Descriptions
Introduction on the topic	0	Not mentioned at all	Introduction on the topic is not presented at all.
	1	Just mentioned	Introduction on the topic is just presented, but not elaborated and justified.
	2	Mentioned and elaborated	Introduction on the topic is presented and elaborated, but not justified.
	3	Mentioned, elaborated, and justified	Introduction on the topic is presented, elaborated, and justified.
Taking a position on the topic	0	Not mentioned at all	Position on the topic is not presented at all.
	1	Just mentioned	Position on the topic is just presented, but not elaborated and justified.
	2	Mentioned and elaborated	Position on the topic is presented and elaborated, but not justified.
	3	Mentioned, elaborated, and justified	Position on the topic is presented, elaborated, and justified.
Arguments for the position	0	Not mentioned at all	No argument in favor of the position is presented.
	1	Mentioned to a small extent	Only one argument in favor of the position is presented.
	2	Mentioned to a moderate extent	Only two arguments in favor of the position are presented.
	3	Mentioned to a great extent	More than two arguments in favor of the position are presented.
Justifications for arguments for the position	0	Not justified at all	Justification for arguments for the position is not presented at all.
	1	Justified to a small extent	Only one argument for the position is justified.
	2	Justified to a moderate extent	Some but not all arguments for the position are justified.
	3	Justified to a great extent	All arguments for the position are justified.
Arguments against the position (counter-arguments)	0	Not mentioned at all	No argument against the position is presented.
	1	Mentioned to a small extent	Only one argument against the position is presented.
	2	Mentioned to a moderate extent	Only two arguments against the position are presented.
	3	Mentioned to a great extent	More than two arguments against the position are presented.
Justifications for arguments against the position	0	Not justified at all	Justification for arguments against the position is not presented at all.
	1	Justified to a small extent	Only one argument against the position is justified.
	2	Justified to a moderate extent	Some but not all arguments against the position are justified.
	3	Justified to a great extent	All arguments against the position are justified.
Response to counter-arguments	0	Not mentioned at all	Response to counter-arguments is not presented at all.
	1	Just mentioned	Response to counter-arguments is just presented, but not elaborated and justified.
	2	Mentioned and elaborated	Response to counter-arguments is presented and elaborated, but not justified.
	3	Mentioned, elaborated, and justified	Response to counter-arguments is presented, elaborated, and justified.
Conclusion and implications	0	Not mentioned at all	Conclusion and/or implications are not presented at all.
	1	Just mentioned	Conclusion and/or implications are just presented, but not elaborated and justified.
	2	Mentioned and elaborated	Conclusion and/or implications are presented and elaborated, but not justified.
	3	Mentioned, elaborated, and justified	Conclusion and/or implications are presented, elaborated, and justified.

Table 2. Coding scheme to analyze features of received peer feedback.

Nature of feedback	Feedback features	Points	Label	Description
Affective		0	Poor – discouraging	The comment included discouraging and negative emotions such as anger or disappointments
		1	Average – neutral/not mentioned	The comment did not include either negative or positive emotions
		2	Good – encouraging	The comment included encouraging and positive emotions such as praise or compliments
Cognitive	Description	0	Poor – not mentioned	The comment did not include a summary statement such as description of content or taken action
		1	Average – mentioned to small extent	The comment included a summary statement such as description of content or taken action but to a small extent
		2	Good – mentioned to a large extent	The comment included a summary statement such as description of content or taken action to a large extent
	Identification	0	Poor – not mentioned	The comment did not include explicit identification of problem
		1	Average – mentioned but not localized	The comment included identification of problem without localization of identified problem
		2	Good – mentioned and localized	The comment included explicit and localized identification of problem
	Justification	0	Poor – not mentioned	The comment did not include elaborations and justifications of identified problem
		1	Average – mentioned, elaborated, but not justified	The comment included elaborations but not justifications of identified problem
		2	Good – mentioned, elaborated, and justified	The comment included elaborations and justifications of identified problem
Constructive		0	Poor – not mentioned	The comment did not include any recommendations or action plans for further improvements.
		1	Average – only recommendation is mentioned	The comment included recommendations but not action plans for further improvements.
		2	Good – both recommendation and action plan are mentioned	The comment included recommendations and action plans for further improvements.

Table 3. Differences among successful, less successful, and unsuccessful students in terms of mean scores for received peer feedback quality.

Variables		Group	Peer feedback received quality		pairwise comparisons	Difference between among unsuccessful, less successful, and successful statistics
			Mean	SD
Affective		Unsuccessful	1.64	0.17	Successful < Unsuccessful **	F (2, 269) = 5.1, p < 0.01**, η2 = 0.03
		Less successful	1.63	0.17	Successful < Less successful *
		Successful	1.57	0.17
		Total	1.61	0.17
Cognitive	Description	Unsuccessful	1.32	0.33	Successful < Unsuccessful **	F (2, 269) = 4.80, p < 0.01**, η2 = 0.03
		Less successful	1.28	0.35
		Successful	1.18	0.34
		Total	1.26	0.34
	Identification	Unsuccessful	0.53	0.29	Successful > Unsuccessful *	F (2, 269) = 2.92, p < 0.05*, η2 = 0.02
		Less successful	0.58	0.32
		Successful	0.65	0.41
		Total	0.59	0.35
	Justification	Unsuccessful	0.04	0.08		F (2, 269) = 2.01, p = 0.13
		Less successful	0.06	0.09
		Successful	0.08	0.13
		Total	0.06	0.10
Constructive		Unsuccessful	0.79	0.34		F (2, 269) = 1.61, p = 0.20
		Less successful	0.87	0.32
		Successful	0.87	0.36
		Total	0.84	0.34

(P < 0.01)**, (P < 0.05)*

Table 4. Multinomial logistic regression the essay writing improvement and received peer feedback quality.

Improvement categories	Peer feedback features		B	SE	Exp (B)	95% CI for Exp (B)
						Lower bound	Upper bound
Less successful	Affective		0.30	1.19	1.34	0.14	13.97
	Cognitive	Description	−0.97	0.60	0.37	0.12	1.24
		Identification	0.11	0.64	1.12	0.36	3.60
		Justification	0.80	2.41	2.23	0.03	260.19
	Constructive		−1.18	0.98	0.30	0.05	2.12
	Course	Course A	−0.08	0.70	0.92	0.23	3.66
		Course B	−0.25	0.67	0.77	0.20	2.88
		Course C	−0.04	0.54	0.96	0.33	2.77
		Course D	0.03	0.50	1.03	0.38	2.78
	Educational level		−0.07	0.35	0.92	0.46	1.84
Successful	Affective		−1.40	1.08	0.25	0.03	2.09
	Cognitive	Description	−1.58	0.54	0.20**	0.07	0.61
		Identification	−0.07	0.57	0.93	0.34	2.71
		Justification	2.75	2.09	15.78	0.42	1085.633
	Constructive		−1.76	0.87	0.17*	0.03	0.99
	Course	Course A	−0.31	0.62	0.73	0.21	2.50
		Course B	−0.51	0.58	0.60	0.18	1.88
		Course C	−0.27	0.48	0.76	0.29	1.97
		Course D	−0.09	0.45	0.90	0.37	2.19
	Educational level		−0.03	0.31	0.96	0.52	1.78

Note: Model χ ² = 33.26; P < 0.05, −2 log likelihood = 557.28, Pearson χ ²=  548.97, p >0.05; Deviance χ ²=  557.28, p > 0.05; Pseudo R² (Cox and Snell  = 0.11, Nagelkerke = 0.12, McFadden = 0.05).

Improvement categories: dependent variable; Peer feedback features: independent variable

The reference category is: unsuccessful