Exploring internal structure of a performance-based critical thinking assessment for new students in higher education

Abstract

Critical thinking is a combination of complex cognitive skills. It is important for the successful acquisition of disciplinary skills in higher education and thus, it is a valuable competency for a new student. The complex nature of critical thinking leads to challenges for its assessment even in performance assessments such as CLA + International (Collegiate Learning Assessment). The aim with this study is to examine internal associations of a critical thinking assessment for new students in higher education. The sample consisted of 1469 first-year students in 18 higher education institutions in Finland. An open-ended performance task and multiple-choice tasks were used to assess six measures of critical thinking, namely analysis and problem solving, writing effectiveness, writing mechanics, scientific and quantitative reasoning, critical reading and critiquing an argument. Exploratory and confirmatory factor analyses were used to analyse the data. A latent component indicated by the measures derived from the performance task was identified. The measures derived from the multiple-choice tasks did not form a factorial structure. Multiple-choice questions are problematic in critical thinking assessment as they focus on individual skills instead of holistic use of skills. Implications for assessment development and higher education are discussed.

Keywords:

• Critical thinking
• performance assessment
• internal structure
• higher education

Acknowledgements

The authors would like to thank the Council for Aid to Education (CAE) for providing the CLA + International, the participating institutions for administering the assessments and recruiting participants and the Kappas project team for their efforts. This work was supported by the Finnish Ministry of Education and Culture (grant numbers: OKM/44/240/2018 and OKM/280/523/2017).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Auli Toom

Katri Kleemola, MA, is a PhD Candidate at the Centre for University Teaching and Learning in University of Helsinki. Katri’s research focuses on critical thinking, argumentative writing and other generic skills in transition to higher education.

Heidi Hyytinen, PhD, is Senior Lecturer in Higher Education at the Centre for University Teaching and Learning in University of Helsinki. Heidi’s research focuses on generic skills, performance-based assessment, self-regulation, and pedagogy in higher education.

Auli Toom, PhD, is Professor of Higher Education and the director of the Centre for University Teaching and Learning in University of Helsinki. Auli’s research focuses on teacher education, teacher knowing, professional agency, generic skills, and pedagogy in higher education.