Reliability of the Reflective Learning Framework for Assessing Higher-Order Thinking in Geography and Sustainability Courses

Abstract

Experiential education partnered with guided reflection is thought to support students with higher-order thinking skills. In this study, 44 reflections from two university-level sustainability courses were compared. In both courses students were asked to write a reflection, but only one course used the Reflective Learning Framework (RLF). Tests of interrater reliability support the consistency of the RLF when used by trained raters. Furthermore, comparison of means using t-tests shows significant differences between mean ratings for the two courses. This provides evidence of the effectiveness of the RLF for students to apply and demonstrate the use of higher-order thinking skills.

Keywords:

• Reflection
• experiential learning
• interrater reliability
• comparison of means
• reflective learning framework

Acknowledgements

We first and foremost thank the students who have participated in the course reflection assignment and supported this research through their participation by providing access to their reflections. We thank the course instructor, Dr. Michael Mikulak, and teaching assistants, Robert Etherington and Mohammad Abdul Aziz, who guided the students through their experiential learning and reflection. We are grateful for Robert and Mohammad for continuing to support this research as returning evaluators. We send our sincerest appreciation to our colleagues Hannah King, who provided creative vision and expertise to produce the RLF video workshop, and Abbie Little, who provided creative vision and design expertise for the creation of the reflection handout as well as provided valuable feedback on the various instructional resources that have been developed to support teaching students about and engaging them in reflection using the RLF. Faculty members of McMaster University’s School of Earth, Environment & Society have been a source of guidance and valuable feedback since inception of this study, particularly Dr. Maureen Padden and Dr. Carolyn Eyles. Any errors or omissions are the responsibility of the authors.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 Up-to-date versions of all RLF resources can be found online at https://asp.mcmaster.ca/experiential-learning-project-opportunities-2/experiential-learning-project-resources/