Automatic Assessment of Mathematical Creativity using Natural Language Processing

ABSTRACT

Creativity is now accepted as a core 21st-century competency and is increasingly an explicit part of school curricula around the world. Therefore, the ability to assess creativity for both formative and summative purposes is vital. However, the fitness-for-purpose of creativity tests has recently come under scrutiny. Current creativity assessments have up to five key weaknesses that create a barrier to their widespread use in educational settings. These are: (a) A lack of domain/subject specificity; (b) Inconsistency, leading to a lack of trust; (c) A lack of authenticity in classroom settings; (d) Slowness (in providing useful results); (e) High cost to administer. The aim of the present study is to explore the feasibility of the automated assessment of mathematical creativity, drawing on tools and techniques from the field of natural language processing, as a means to address these weaknesses. This paper describes the performance of a machine learning algorithm, relative to human judges, demonstrating the practicality of automated creativity assessment for large-scale, school-based assessments.

Plain Language Summary

The importance of creativity is recognized in education systems globally. The ability to assess creativity, for both formative and summative purposes, is therefore vital. However, the quality of creativity tests (their validity and reliability) tends to come at a cost. In simple terms, the better the creativity test, the greater the effort, and therefore cost, required to deliver and score that test. The aim of the present study is to explore the feasibility of the automated assessment of mathematical creativity, drawing on tools and techniques from the field of natural language processing. This paper describes how a machine learning algorithm assesses mathematical creativity and compares this to human judges.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Due to the nature of this research, participants of this study did not agree for their data to be shared publicly, so supporting data is not available. The code used to analyze the data can be found here: https://github.com/ericwanga/math-creativity-assessment

Notes

1. https://pisa2022-math.oecd.org/#Twenty-First-Century-Skills

2. https://github.com/jasonwei20/eda_nlp

Additional information

Funding

This work was not supported by any funding bodies