Validation of a reflection rubric for higher education

ABSTRACT

Reflection in learning is a popular and widely studied concept that has historical roots in higher education. Previous authors reported little agreement on the assessment methods and procedures for measuring reflection. Moreover, many studies have not provided sufficient detail on the development or implementation of their reflection rubrics, and only some reported psychometric support for reliability or validity. These challenges make it difficult to identify a standardized assessment process for reflection that is reliable and valid for use across different disciplines in higher education. The current study provides preliminary support for a reflection rubric for college students that is theoretically- and empirically-based and psychometrically supported, using student data that represents several academic disciplines and a variety of student groups. The authors describe the development of a reflection rubric in detail; present results that support its reliability, validity, and use among college students from different disciplines to promote and actively practice a rigorous and standardized process for written reflection; present the rubric’s limitations; and describe plans for future research, including additional tests of validity.

KEYWORDS:

• Reflection
• higher education
• psychometrically valid
• learning

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. Cohen’s kappa for nominal scales is a more robust measure of reliability, compared to percent agreement, because it accounts for level of agreement due to chance. Weighted Cohen’s kappa ensures that disagreements of varying degrees between raters are weighted accordingly, as opposed to Cohen’s kappa that weights all disagreements equally.

2. The authors did not run an exploratory factor analysis (EFA) because of the amount of existing theoretical (King & Kitchener, 1994, 2004) and empirical support (Wood, 1997) that informed the development of this study’s reflection rubric and its factor structure.

3. One-hundred fifty students did not have a cumulative grade point average because it was their first semester.

Additional information

Notes on contributors

Jordan Rogers

Jordan Rogers is a Doctoral Candidate in the Counseling Program at the University of North Texas. She holds a Master of Science degree in Counseling & Development from Texas Woman’s University and is licensed as a Professional Counselor in the state of Texas. She is currently a faculty development specialist in the Quality Enhancement Plan, Career Connect at UNT, where she collaborates with faculty to promote experiential learning into courses. She also supervises and teaches at the university where she places an emphasis on student-centered initiatives to foster reflection and deeper learning in curricular and co-curricular spaces.

Scott Peecksen

Dr. Scott Peecksen holds his Ph.D. from the University of Arizona and is an Outcomes Assessment Specialist for the Career Connect department at the University of North Texas.  He manages the assessment of the university’s QEP and the evaluation research conducted as a result of QEP data collection.  He has more than 15 years of research and evaluation experience encompassing study design, data collection, analysis, and reporting.  He is well versed in descriptive and inferential analysis methods and group-comparison designs, and is a What Works Clearinghouse reviewer for the U.S.  Department of Education.

Megan Douglas

Dr. Megan Douglas holds her Ph.D. in Clinical Health Psychology from the University of North Texas.  She completed her pre-doctoral clinical psychology internship in Health Services Psychology at the Department of Veterans Affairs Connecticut Healthcare System in West Haven, CT.  She is currently a Post-Doctoral Fellow in Rehabilitation Research at Baylor Scott and White Health in Dallas, TX. Prior to this, she was a graduate research assistant for Career Connect as part of the University’s QEP. Her QEP work primarily focused on analyzing student reflections through a university’s ePortfolio system to promote deeper learning.

Mike Simmons

Dr. Mike Simmons holds his Ph.D. from the University of Texas at Arlington and is the Assistant Vice President of Academic Affairs at the University of North Texas, leading the Career Connect department and other academic and technology innovation projects.  He was previously the Assistant Vice President for Lifelong Learning at Texas Woman’s University where he was responsible for online education, faculty support, and continuing education.