Students’ perceptions of the peer-feedback experience in MOOCs

Figures & data

Figure 1. Study design showing activities and measurements

Table 1. Mixed ANOVA results for students’ peer-feedback perceptions

Predictor	dfNum	dfDen	ε	F	P	dfNum	dfDen	ε	F	p
	Willingness						Usefulness
Condition	1	43	.01	0.43	.52	1	43	.00	0.00	1.00
Measurement	1	43	.02	2.71	.11	1	43	.00	0.13	.72
Condition x Measurement	1	43	.02	2.81	.10	1	43	.00	0.81	.38
	Preparedness					General attitude
Condition	1	43	.02	1.10	.30	1	43	.00	0.05	.82
Measurement	1	43	.02	4.46	.04	1	43	.01	1.02	.32
Condition x Measurement	1	43	.00	0.14	.71	1	43	.01	0.79	.38

Table 2. Ordinal regression analyses with the peer-feedback perception as dependent and peer-feedback experience as independent variables

	Willingness			Usefulness			Preparedness			General attitude
	B	SE	t	B	SE	t	B	SE	t	B	SE	t
EX2: Did you find receiving and/or providing peer-feedback valuable for your own learning?
Intercept	1.88 ***	0.08	23.32	1.88***	0.07	28.67	2.94***	0.08	38.34	1.81***	0.07	26.68
Sometimes	-.61 **	0.22	-2.73	-.39*	0.18	-2.14	0.05	0.21	0.24	-.45*	0.19	-2.36
Often	.47 *	0.2	2.33	.37*	0.17	2.26	.85***	0.19	4.42	.42*	0.17	2.49
Always	.37 *	0.15	2.51	.58***	0.12	4.78	.74***	0.14	5.21	.57***	0.13	4.53
N/A	0.07	0.13	0.51	0.01	0.11	0.09	.25*	0.12	2.01	-0.04	0.11	-0.34
R^2	0.13			0.19			0.19			0.2
EX3: Did you ever review someone’s work or provide feedback to someone in a regular course (NOT a MOOC)?
Intercept	1.74 ***	0.07	25.04	1.76***	0.06	29.87	2.71***	0.06	43.5	1.68***	0.06	27.46
Yes, once	-.32*	0.13	-2.44	-.24*	0.11	-2.09	-.87 ***	0.12	-7.39	-0.21	0.12	-1.78
Yes, 2–5 times	.34*	0.14	2.36	0.31**	0.12	2.6	.39**	0.12	3.2	0.29	0.12	2.43
Yes, more than 5 times	-0.12	0.15	-0.83	-0.19	0.12	-1.55	-0.1	0.13	-0.77	-.21**	0.13	-1.6
R^2	0.05			0.05			0.22			0.04
EX6: Did you ever participate in peer-feedback training in a MOOC?
Intercept	1.52***	0.14	10.58	1.59***	0.12	13.06	2.12 ***	0.14	15.66	1.56***	0.13	12.42
Yes, once	-0.48	0.31	-1.54	-0.48	0.27	-1.82	-.99***	0.29	-3.35	-0.36	0.27	-1.33
Yes, 2–5 times	-0.19	0.29	-0.66	-0.35	0.24	-1.43	0.04	0.27	0.14	-0.37	0.25	-1.46
Yes, more than 5 times	-0.29	0.26	-1.13	-0.41	0.22	-1.87	-0.33	0.24	-1.35	-.59*	0.23	-2.59
R^2	0.02			0.02			0.11			0.03

Table 3. Ordinal regression analyses with the peer-feedback perception and peer-feedback experience as dependent and independent variables respectively

	Willingness			Usefulness			Preparedness			General attitude
	B	SE	t	B	SE	t	B	SE	t	B	SE	t
EX7: Did the peer-feedback training/exercises help you to provide feedback?
Intercept	1.78 ***	0.07	24.73	1.78***	0.06	29.39	2.71***	0.07	39.13	1.71***	0.06	27.27
Sometimes	-.42 *	0.17	-2.44	-0.24	0.14	-1.68	-0.29	0.17	-1.78	-0.29*	0.15	-1.99
Often	0.28	0.16	1.72	0.22	0.14	1.59	.59***	0.16	3.72	.29*	0.14	2.07
Always	.35 *	0.16	2.2	0.48***	0.13	3.66	.42**	0.15	2.79	.41**	0.14	2.96
N/A	0.23	0.15	1.54	0.09	0.13	0.75	0.28	0.14	1.94	0.13	0.13	0.97
R^2	0.06			0.08			0.1			0.07
EX8: Have you ever used a rubric to review someone or provide feedback?
Intercept	1.76***	0.09	19.35	1.78***	0.08	22.92	2.58***	0.08	31.53	1.73***	0.08	21.57
Sometimes	0.11	0.2	0.52	-0.01	0.17	-0.05	0.08	0.18	0.42	0.09	0.18	0.54
Often	0.2	0.2	1	0.06	0.17	0.36	1.15***	0.18	6.4	0.12	0.18	0.66
Always	0.04	0.21	0.19	0.01	0.18	0.07	.44*	0.19	2.34	0.14	0.18	0.77
N/A	0.05	0.2	0.23	0.05	0.17	0.29	.43*	0.18	2.36	-0.02	0.18	-0.06
R^2	0			0			0.18			0
EX9: The DPSIR framework characterizes the problem of marine litter as consisting of Drivers, Pressures, State, Impacts and Responses. Are you familiar with the DPSIR framework?
Intercept	1.90***	0.09	20.66	1.86***	0.08	23.58	2.96***	0.09	33.06	1.82***	0.08	22.26
No	.31*	0.13	2.36	0.15	0.11	1.35	.51***	0.13	4.01	0.21	0.12	1.82
R^2	0.02			0.01			0.06			0.01

Table 4. Relationship between peer-feedback experience, value, and students’ perceptions: Extract of significant results from Table 2

Perception	EX2: Did you find receiving and/or providing peer-feedback valuable for your own learning?	EX3: Did you ever review someone’s work or provide feedback to someone in a regular course (NOT a MOOC)?
Willingness	never > sometimes often > never always > never	never > once 2–5 times > never
Usefulness	never > sometimes often > never always > never	never > once 2–5 times > never
Preparedness	often > never always > never	never > once 2–5 times > never
General attitude	never > sometimes often > never always > never	2–5 times > never

Note. Table 4 shows significant results of two pre-test questionnaire items (EX2 and EX3) regarding students’ experience with peer-feedback. For each perception variable, the significant differences are shown and depicted with angle brackets. In the case of item EX2 and perception variable willingness, students who never experienced peer-feedback as valuable scored significantly higher on willingness than those who experienced this sometimes.

Table 5. Relationship between peer-feedback training, rubric, and students’ perceptions: Extract of significant results from Table 3

Perception	EX7: Did the peer-feedback training/exercises help you to provide feedback?	EX8: Have you ever used a rubric to review someone or provide feedback?
Willingness	never > sometimes always > never	no significant differences
Usefulness	always > never	no significant differences
Preparedness	often > never always > never	often > never always > never not applicable > never
General attitude	never > sometimes often > never always > never	no significant differences

Note. Table 5 shows significant results of two pre-test questionnaire items (EX7 and EX8) regarding students’ experience with peer-feedback. For each perception variable, the significant differences are shown and depicted with angle brackets. In the case of item EX7 and perception variable willingness, students who always found peer-feedback training valuable scored significantly higher on willingness than those who never found it useful.

Table 6. Relationship between domain knowledge and students’ perceptions: Extract of significant results from Table 3

Perception	EX9: The DPSIR framework characterizes the problem of marine litter as consisting of drivers, pressures, state, impacts and responses. Are you familiar with the DPSIR framework?
Willingness	no > yes
Usefulness	no significant differences
Preparedness	no > yes
General attitude	no significant differences

Note. Table 6 is an extract of Table 3 and shows the significant differences for the pre-test questionnaire item EX9. Significant differences are depicted with angle brackets. Students who were not familiar with the DPSIR framework scored significantly higher on willingness than those who were familiar with it.

Table

	Items pre-questionnaire	Items post-questionnaire peer-feedback activity
Demographics	DG1: What are the first two letters of your first and last name First name: Last name:
	DG2: What is the email address you used when enrolling to this MOOC?:
	DG3: What is your gender?: Male Female Other	-
	DG4: What is your birth year?: (dropdown option)	-
	DG5: What is your nationality? (dropdown option)	-
	DG6: What is your native language? (dropdown option)	-
	DG7: What is the highest degree or level of school you have completed? (dropdown option)	-
	DG8: What is the highest level of education you attended? (dropdown option)	-
Experience with peer-feedback	EX1: Did you ever participate in a peer-feedback activity at all? No, never Yes, once Yes, 2 - 5 times Yes, more than 5 times * Peer feedback is about giving or receiving feedback from/to fellow students, colleagues and friends. It is not about feedback from a teacher.	-
	EX2: Did you find receiving and/or providing peer-feedback valuable for your own learning? Never Sometimes Often Always Not applicable	-
	EX3: Did you ever review someone’s work or provide feedback to someone in a regular course (NOT a MOOC)? No, never Yes, once Yes, 2 - 5 times Yes, more than 5 times	-
	EX4: Before enrolling in this MOOC, did you ever review someone’s work or provide feedback to someone in a MOOC? No, never Yes, once Yes, 2 - 5 times Yes, more than 5 times *MOOC (Massive Open Online Course like this course about Marine Litter)	-
	EX5: Peer-feedback training provides you with information and examples/exercises about how and why feedback is given/needed. Did you ever participate in peer-feedback training in a regular course (NOT a MOOC)? No, never Yes, once Yes, 2–5 times Yes, more than 5 times	-
	EX6: Did you ever participate in peer-feedback training in a MOOC? No, never Yes, once Yes, 2 - 5 times Yes, more than 5 times	-
	EX7: Did the peer-feedback training/exercises help you to provide feedback? Never Sometimes Often Always Not applicable	-
	EX8: Have you ever used a rubric to review someone or provide feedback? Never Sometimes Often Always Not applicable/Don’t know * A rubric is often used when giving written feedback. A rubric consists of a set of criteria used during reviewing. This is particularly useful if you want to ensure that everyone gives feedback on the same criteria.	-
Experience with DPSIR	EX9: The DPSIR framework characterizes the problem of marine litter as consisting of Drivers, Pressures, State, Impacts and Responses. Are you familiar with the DPSIR framework? Yes No	-
	EX10: Have you used the DPSIR framework before? No, never Yes, once Yes, 2 - 5 times Yes, more than 5 times Not applicable	-
Willingness (7-point Likert scale ranging from agree to disagree)	W1: I am willing to review someone’s assignment and provide feedback.
	W2: I am willing to upload my assignment in the MOOC and get it reviewed by other MOOC learners.
	W3: I am willing to learn how to review someone’s work and use a rubric.
	W4: I am willing to use peer-feedback in order to improve my assignment.
	W5: I am willing to help fellow students by reviewing their work.
Usefulness (7-point Likert scale ranging from agree to disagree)	U1: I find it useful to provide peer-feedback on my assignment.
	U2: I find it useful to receive peer-feedback on my peer’s assignment.
	U3: Peer-feedback is useful because it helps me to improve my own work.
	U4: Reviewing others’ work is useful because it helps me to reflect on my own work.
	U5: I find it useful to get examples on how to review and provide peer-feedback.
	U6: I find it useful to watch a short video in which the peer-feedback process is explained.
	U7: I find it useful to practice with a short quiz before having to review someone else’s work.
	U8: I find it useful to know why I have to provide peer-feedback.
Preparedness (7-point Likert scale ranging from agree to disagree)	P1: I know how to review someone’s work and provide feedback.
	P2: I feel confident about my peer-feedback/review skills.
	P3: I know how to use a rubric when providing peer-feedback.
	P4: Examples and instruction on how to review and provide feedback make me feel prepared to give peer-feedback.
	P5: Instructions on why to review and provide feedback make me feel prepared to give and receive peer-feedback.
	P6: I have enough knowledge about the DPSIR to provide feedback on someone else’s DPSIR.
	P7: I need more examples on how to review the DPSIR of a fellow student.
General attitude	GA1. Receiving peer-feedback is valuable for my learning process.
	GA2. Providing peer-feedback is valuable for my learning process.
	GA3. I am interested in the perspectives of others on my work.
	GA4. Peer-feedback should always be explained before asking students to review each other’s work.
Peer-feedback quality (7-point Likert scale ranging from agree to disagree)	-	How valuable did you find the received peer-feedback? Please indicate below! PFQ1: The received feedback helped me to see strong points in my descriptions
	-	PFQ2: The received feedback helped me to see weak points in my descriptions
	-	PFQ3: The received feedback helped me to see how I could improve my DPSIR scheme
	-	PFQ4: Do you have any suggestions regarding the examples and instruction regarding the peer-feedback activity?

Note. The items of the pre-test and post-test questionnaires are shown in this table. Items are numbered and indicated with an abbreviation, for example, W1 = first items of the willingness variable. Items that were included in both pre- and post-test span over two columns.