What can ChatGPT not do in education? Evaluating its effectiveness in assessing educational learning outcomes

ABSTRACT

This paper examines ChatGPT’s capability in evaluating educational learning outcomes, investigating its effectiveness in assessing SMART criteria alignment and identifying the presence of fundamental components. The hypothesis posits that ChatGPT can proficiently accomplish these tasks, offering potential benefits to educational design and assessment processes. Through exemplar learning outcomes, the study showcases ChatGPT’s ability to discern SMART criteria alignment. Furthermore, it demonstrates ChatGPT’s competence in identifying the fundamental components, substantiated by cogent explanations. The analysis underscores the congruence between ChatGPT’s evaluations and human assessors’ judgements, underscoring its potential utility in educational quality assurance. Implications for educational practice emphasise ChatGPT’s potential to assist educators in formulating effective learning objectives, meeting SMART criteria and encapsulating crucial components. While ChatGPT’s capabilities are promising, human expertise remains vital for nuanced evaluation. In conclusion, this paper illuminates ChatGPT’s role in shaping educational outcomes and encourages further exploration into AI’s potential impact on educational processes.

KEYWORDS:

• Learning outcome
• education
• ChatGPT
• language model
• SMART criteria, curriculum design

Introduction

In outcome-based education, the formulation of effective learning outcomes or objectives stands as a foundation in the design of educational modules, courses, and lectures (Schoepp, 2017). Learning outcomes represent clear statements that articulate the anticipated knowledge, skills, or abilities that learners should possess at the conclusion of an educational experience (Kent et al., 2016; Okutsu et al., 2013; Prøitz, 2010). These outcomes provide a roadmap for outcome-focused educators, guiding their instructional strategies and enabling learners to assess their progress and achievements.

It can be argued that a well-constructed learning outcome comprises three fundamental components: verbs, content, and context (How to Write Learning Outcomes n.d.; Learning Outcomes, n.d.; Writing Learning Outcomes, 2018), recognising the existence of other related evaluative concepts such as the ABCD formula – Audience, Behaviour, Condition and Degree (Kurt, 2020b). Verbs encapsulate the behaviours that learners are expected to exhibit, manifesting the operations they should perform. The content encapsulates the subject matter or the central theme of the learning, indicating what knowledge or skills are the focus. Context, on the other hand, situates the learning within a specific environment, revealing the conditions under which the knowledge or skills are to be applied.

To enhance the effectiveness of learning outcomes, outcome-focused educators often rely on the SMART criteria – an acronym for Specific, Measurable, Achievable, Relevant, and Time-bound (Adams, n.d.; Chatterjee & Corral, 2017; Creating Learning Outcomes, n.d.; Edly, 2022; Smart Tips to Write Smart Learning Objectives, 2016), acknowledging the existence of other evaluating concepts such as SMARTTT – Speak to the learner, Measurable, Applicable, Realistic, Time-Bound, Transparent, Transferable (Clark, 2015). The SMART criteria, adopted in this study due to its wide usage, serve as a guiding framework to ensure that learning outcomes are well-defined, measurable, attainable, aligned with desired results, and bound by a defined timeframe. However, the process of manually assessing the presence of verbs, content, context, and evaluating adherence to the SMART criteria can be resource-intensive. In light of advancements in natural language processing, particularly exemplified by the capabilities of the ChatGPT language model (Feng et al., 2023; Kohnke et al., 2023), there is an opportunity to explore its existing capacity to automatically identify the essential components of learning outcomes as well as assess the compliance of learning outcomes with the SMART criteria.

Contribution

Through a theoretically grounded analysis that involves the exploration of ChatGPT’s current capabilities, this research aims to shed light on the potential of leveraging AI-driven approaches in educational contexts. By examining how ChatGPT can aid in the analysis of learning outcomes and in evaluating their alignment with established criteria, outcome-focused educators can gain valuable insights to refine instructional strategies and improve curriculum design.

Debates surrounding the use of learning outcomes in education

While the focus of this study is on evaluating the effectiveness of ChatGPT in recognising SMART attributes and the fundamental components within learning outcomes, it is crucial to remember that human expertise remains essential for nuanced assessment and contextual considerations. This is particularly important considering the arguments against the use of learning outcomes in education in the literature.

For instance, Brady (1996) criticised the narrow and instrumental nature of outcome-based education, claiming it treats all teaching and learning as a means to achieve predefined ends, causing students to miss out on other important educational goals or values which are inherent in certain educational activities and encounters. That position was supported by Erikson and Erikson (2019) who stated that, when educational goals are expressed as learning outcomes, there is a risk of not been able to assess some educational goals that cannot be expressed in that manner, in line with the constructive alignment. Critics of outcome-based education have also pointed out the danger in fixing learning outcomes which leaves no room for the achievement of emerging or unintended learning outcomes (Martin, 2016). Brady (1996) argued that, due to the unpredictable nature of outcomes from lessons which are built upon unguided discovery, the use of learning outcomes imposes a limit on the students’ creativity and enquiry. Yu (2016) was also critical of outcome-based education’s lack of accommodation for unpredictable outcomes, and the consequent production of graduates who are homogeneous group of followers instead of pioneers of spontaneous and creative ideas. Erikson and Erikson (2019) buttressed that point when they noted that outcome-based education prevents teachers from rewarding learners who explore outcomes other than those predefined, thus creating a ceiling that prevents the students from going the extra mile, negatively impacting their ability to develop creative solutions.

Thirumoorthy (2021) highlighted another problem with using learning outcomes which is the difficulty in accounting for educational goals which are related to feelings, values, attitudes, and beliefs. This, according to Thirumoorthy (2021), is particularly important when the teachers’ plan is to develop learning outcomes which are measurable and assessable. According to Brady (1996), there is also the fear that affective based skills (e.g. social, appreciation, etc.), which are not only hard to formulate as targeted learning outcomes but also difficult to assess, can be ignored. Erikson and Erikson (2019) had their say on educational goals which are rooted in affective skills when they noted their concern about neglecting goals that cannot be measured when teaching and learning is focused mainly on a set of measurable learning outcomes.

Another objections raised by Eldeeb and Shatakumari (2013) is the allocation of the same work to different groups of students (for instance, disadvantaged and advantaged students) or assessment of those groups using the same test in outcome-based education. According to Brady (1996), due to its focus on the achievement of predefined goals, outcome-based education favours remediation over enrichment, leaving capable learners deprived. This, as stated by Brady (1996), is different to what is applicable in mastery learning where regrouping of students takes place at some point according to the students’ level of understanding to allow remediation (for students requiring additional support) and enrichment of capable learners. Brady (1996) went further to lay emphasis on the creation of additional challenge and enrichment for capable learners. Mcglynn and Kelly (2017) buttressed the need for extra challenge for capable learners, which is evident in their saying, ‘No one should be bored, and no one should be lost’.

Notwithstanding the arguments against the use of learning outcomes, there are notable benefits which those in the learning outcome movement continue to capitalise on. For instance, in outcome-based education, educators use learning outcomes to set the aims of an educational experience, clarifying to the learners the skills or attributes they are expected to possess at the end of such experience (Aziz et al., 2012). Presenting learners with learning outcomes or criteria for successful learning can enhance their learning experience (Hattie, 2015; Marsh, 2007). Additionally, outcome-based education can enhance students’ employability through the formulation (as learning outcomes) and attainment of skills that employers seek in applicants (Jorre de St Jorre & Oliver, 2018; Soares et al., 2017). Learning outcomes have become integral parts of important educational frameworks. For instance, learning outcomes are a stage in the BOPPPS framework (Foxe et al., 2016) where the different constituents of the framework work together to increase the students’ interest (in learning) and engagement (Fan et al., 2020). Setting learning outcomes is also a stage in constructive alignment (Biggs, 1996) where both the assessment and learning activities are tailored to help the students achieve predefined learning outcomes, consequently preventing unnecessary detour from important learning activities (and skills) and ensuring judicious use of allotted time (Kurt, 2020a).When setting learning outcomes, in line with Bloom’s taxonomy (Adams, 2015), the use of verbs like apply, theorise, design, etc. is recommended to not only increase the learners’ critical thinking and engagement level but also to ensure that the learning process is deep (Dolmans et al., 2016). In deep learning, learners are equipped with knowledge for use beyond the university walls. This is different from surface learning (Dolmans et al., 2016) where learners acquire knowledge solely for examinations purpose.

Considering the opposing views on the impacts of learning outcomes in education, it is important that educators initially decide, using relevant frameworks, the suitability of learning outcomes in their practice as a fore-running step to the application of ChatGPT in reviewing learning outcomes. This again emphasises the continued role that humans need to play in applying AI models in education. Furthermore, while arguments against the use of learning outcomes in education exist, this study investigates the use of ChatGPT in reviewing learning outcomes due to the wide acceptance of learning outcomes (and the benefits they offer as canvassed) by those in the learning outcome movement.

The subsequent sections detail the methodology employed in this study and present the findings that contribute to advancing the field of educational technology and pedagogy.

Methods

Data collection

In order to evaluate ChatGPT’s familiarity with the research subject and its capacity to analyse learning outcomes according to the SMART criteria, a multi-faceted approach was devised. To assess ChatGPT’s capability, we used a collection of learning outcomes (lecture or session based), educational goals and pre-requisites¹ used in a level 2 computer science based module (S.D. Fabiyi, personal communication, October 2022) at the University of Leeds, Leeds, England. These formed parts of a series of instructions provided to ChatGPT to assess the learning outcomes for SMART compliance and the presence of fundamental components. This information set ensured a comprehensive evaluation of ChatGPT’s capabilities in reviewing and refining learning outcomes in a higher educational domain.

Qualitative analysis

The approach adopted for data analysis are structural analysis (Schroeder, 2022) and conversational analysis (Meredith, 2019; Tabone & De Winter, 2023). When applying structural analysis, ChatGPT was tasked with evaluating selected learning outcomes for the following criteria, based on their structure and content:

• SMART criteria compliance: ChatGPT was asked to analyse each learning outcome and determine whether it met the SMART criteria. For this purpose, ChatGPT’s analysis was expected to provide reasons and explanations for its assessments, highlighting specific aspects of each outcome that aligned or deviated from the SMART criteria. Specifically, straightforward instructions were provided for the criteria of Specific, Measurable, and Time-bound, where ChatGPT was expected to review the learning outcomes for clarity, precision, assessability (quantitatively or qualitatively), and the presence of a time limit. For the criteria of Achievable and Relevant, distinct strategies were employed. For Achievable, to gauge ChatGPT’s ability to evaluate the achievability of learning outcomes, the model was instructed to align the outcomes with specified prerequisites. These prerequisites served as benchmarks against which ChatGPT evaluated whether the learning outcomes could realistically be accomplished within the given conditions. For Relevant, in the context of assessing the relevance of learning outcomes, ChatGPT was prompted to align the outcomes with the module’s learning goals. These contextual factors acted as reference points for ChatGPT to ascertain whether the learning outcomes were relevant to the specific educational context.

• Identification of fundamental components: Another dimension of ChatGPT’s evaluation involved identifying the presence of fundamental components in the learning outcomes. ChatGPT’s capacity to recognise and isolate these elements was explored, and the results were used to gauge its ability to comprehend and deconstruct learning outcomes.

ChatGPT is becoming a useful tool in analysing qualitative data as evident by its successful deployment in recent research (Morgan, 2023; Şen et al., 2023), hence its use for related assignments in this study.

Additionally, conversational analysis was used to examine the results of our interactions with ChatGPT to reach a conclusion on whether ChatGPT can be effective in reviewing and refining learning outcomes. The analyses focused on the coherence, accuracy, and reasoning of ChatGPT’s assessments, shedding light on its proficiency in aligning outcomes with prerequisites, context, and the SMART criteria, as well as its ability to identify fundamental components. This methodological approach sought to unveil ChatGPT’s current capabilities in evaluating existing learning outcomes for SMART criteria adherence and component presence. The subsequent section presents the results of these analyses and explores the implications for the potential integration of AI-driven tools in the realm of educational design and assessment.

Result

Evaluation of learning outcome’s SMARTness

In the exploration of ChatGPT’s capacity to review learning outcomes, a set of learning outcomes was presented to ChatGPT for examination. These outcomes were evaluated by ChatGPT for their alignment with the SMART criteria. The results of this evaluation are presented below.

Author to ChatGPT: Given the provided prerequisites and educational goals, determine if the following learning outcome is SMART with reasons:

By the end of this class, students should be able to apply control statements to control the flow of a program in C++.

ChatGPT’s response: The learning outcome is SMART, considering the provided prerequisites and educational goals. Here’s the breakdown:

1. Specific: The outcome specifies that students should be able to apply control statements in C++. It clearly defines the skill that students are expected to acquire.

2. Measurable:The outcome is measurable as it involves the application of control statements. The assessment can involve coding exercises, where students demonstrate their ability to use control statements effectively.

3. Achievable: Given the listed prerequisites of procedural programming and object-oriented programming, students should have the foundational knowledge required to understand and implement control statements in C++. Therefore, the outcome is achievable within the scope of the class.

4. Relevant: Understanding control statements is a fundamental aspect of programming. It aligns with the broader educational goals of the class, which include understanding basic types, manipulating strings, and simple I/O operations.

5. Time-Bound: The outcome doesn’t explicitly mention a specific time frame, but since it is framed within the context of ‘by the end of this class’, it implies a time-bound achievement, making it suitable for a course duration.

In summary, the provided learning outcome is SMART because it is specific, measurable, achievable, relevant, and time-bound in the context of the class’s prerequisites and goals.

Author to ChatGPT: Given the provided prerequisites and educational goals, determine if the following learning outcome is SMART with reasons:

By the end of this class, students should be able to apply control statements

ChatGPT’s response: The learning outcome is not entirely SMART. Let’s break it down:

1. Specific: The outcome is not very specific. ‘Apply control statements’ does not clearly define what aspects of control statements the students are expected to understand or utilise. It lacks details on the types of control statements or the complexity level.

2. Measurable: While it mentions applying control statements, it doesn’t provide clear criteria for measuring success. Without specific details, it might be challenging to assess whether students have achieved the desired level of proficiency.

3. Achievable: Assuming the prerequisites are adequately covered, it could be achievable for students to apply control statements. However, the lack of specificity makes it difficult to determine the exact scope of what is expected.

4. Relevant: Understanding control statements is relevant to programming, but without specifying which control statements or what level of proficiency is expected, it might lack relevance to the broader goals of the class.

5. Time-Bound: The outcome specifies ‘by the end of this class’, providing a time-bound context, which is good. However, it would be more effective if coupled with more specific details.

In summary, while the outcome is time-bound, it lacks specificity and measurability. A more detailed and specific version of the outcome would be more aligned with the SMART criteria.

Author to ChatGPT: Given the provided prerequisites and educational goals, determine if the following learning outcome is SMART with reasons:

Students should be able to understand computer processor design.

ChatGPT’s response: The learning outcome ‘students should be able to understand computer processor design’ is not SMART based on the provided prerequisites and educational goals.

Here’s the breakdown:

1. Specific: The learning outcome is not specific to C++ programming. It introduces a different topic related to computer architecture, which is not aligned with the specified goals of understanding basic types, manipulating strings, I/O, control statements, operators, overloading, pointers, memory management, and the const modifier in C++.

2. Measurable: It is challenging to measure or assess understanding of computer processor design within the context of a C++ programming course. The provided goals focus on specific C++ concepts, and the learning outcome should align with those.

3. Achievable: Given that the class is focused on C++ programming and the provided prerequisites, diving into computer processor design may not be achievable or relevant within the scope of the course.

4. Relevant: Understanding computer processor design is relevant in a broader computer science context, but it may not align with the specific objectives of a C++ programming course with the mentioned prerequisites and goals.

5. Time-bound: The learning outcome doesn’t specify a timeframe for achieving the understanding of computer processor design. Additionally, introducing a new and complex topic may exceed the time constraints of the course.

In summary, a SMART learning outcome within the context of the provided prerequisites and educational goals for a C++ programming class would be more focused on C++ concepts and their applications.

Evaluation of learning outcome’s structure

In our exploration of ChatGPT’s proficiency in evaluating learning outcomes for the presence of essential components of verbs, content, and context, we present a detailed analysis of its responses to authors’ queries. Below, we provide a comprehensive breakdown of ChatGPT’s responses.

Author to ChatGPT: Determine if the following learning outcome contains the three basic components of a learning outcome (verb, content, and context), with reasons:

By the end of this class, students should be able to apply control statements to control the flow of a program in C++

ChatGPT’s response: Certainly! Let’s analyse the provided learning outcome to determine if it contains the three basic components:

1. Verb: The verb in this learning outcome is ‘apply’. It indicates the action that the students are expected to perform, which is to use or employ control statements.

2. Content: The content is ‘control statements’ and the application of these statements to ‘control the flow of a program in C++’. This specifies what the students will be learning and applying.

3. Context: The context is implied by the phrase ‘in C++’. This indicates the environment or platform in which the students are expected to apply their knowledge of control statements.

In summary, the learning outcome does contain all three basic components: Verb: Apply, Content: Control statements and Context: Control the flow of a program in C++. This is well-structured and provides a clear expectation of what students should be able to do by the end of the class.

Author to ChatGPT: Determine if the following learning outcome contains the three basic components of a learning outcome (verb, content, and context), with reasons:

By the end of this class, students should be able to apply control statements

ChatGPT’s response: The learning outcome provided is somewhat clear but lacks specificity. Let’s break it down into the three basic components:

1. Verb: The verb in the learning outcome is ‘apply’, indicating the desired action the students should be able to perform. This is a good choice as it implies a higher level of cognitive engagement than, for example, ‘understand’ or ‘know’.

2. Content: The content in this learning outcome is ‘control statements’. While this term is clear to someone familiar with programming or a related field, it might benefit from a bit more specificity. Control statements in programming can include if statements, loops, and switch statements. Providing more detail about which specific control statements the students are expected to apply would enhance clarity.

3. Context: The context is missing in the given learning outcome. The context typically specifies the conditions under which the learning is expected to be applied. For example, it might involve a particular programming language or a specific problem-solving scenario. Without context, it’s challenging to understand the real-world application or setting in which the control statements are expected to be used.

Author to ChatGPT: Determine if the following learning outcome contains the three basic components of a learning outcome (verb, content, and context), with reasons:

Students should be able to understand.

ChatGPT’s response: The learning outcome provided is incomplete as it lacks specific details, particularly a clear verb and content. Let’s break it down:

1. Verb: The learning outcome starts with ‘students should be able to understand’, which is not a specific verb. ‘Understand’ is a vague term that does not clearly indicate what students are expected to do. It would be better to use a more specific action verb that describes the desired cognitive process, such as analyse, evaluate, apply, or create.

2. Content: The learning outcome does not specify the content or subject matter that students are supposed to understand. A complete learning outcome should provide information about what knowledge, skills, or concepts students are expected to grasp.

3. Context: The learning outcome does not include information about the context in which the understanding should take place. The context helps to define the conditions under which the learning is expected to occur. It might include the type of problem, situation, or setting in which the knowledge or skill is applied.

Discussion

From our conversation with ChatGPT when evaluating learning outcomes for their alignment with the SMART criteria, we observed that the learning outcomes that were deemed specific by ChatGPT clearly and concisely articulate the goals or objectives which the students are expected to accomplish. For instance, ‘By the end of this class, students should be able to apply control statements to control the flow of a program in C++’ contains no ambiguity in understanding the skills which the students are expected to gain from the class. For the learning outcomes that were classified as not specific by ChatGPT, we observed that they do not provide a clear and concise articulation of the intended accomplishment. For instance, ‘By the end of this class, students should be able to apply control statements’ is vague since it does not include details of the particular programming language or areas of learning e.g. C++. We noted that the learning outcomes which were deemed measurable by ChatGPT include the expected skills that can be quantified and assessed, allowing educators to measure the students’ success level. For those learning outcomes, it is possible to determine whether the students have achieved them or not. For the learning outcomes that were classified as not measurable by ChatGPT, we noted that the learning outcomes lack the criteria for evaluation (they are abstracts in nature). An example of these is ‘By the end of this class, students should be able to apply control statements’ which does not focus on any particular area or programming language. We also noted that ChatGPT classified ‘Students should be able to understand computer processor design’ as not measurable within the provided educational context. We agree with ChatGPT on this since there is a misalignment between the provided goals and intended outcomes. Looking at the learning outcomes considered achievable by ChatGPT, we can see that a connection exists between the targeted skills and the prerequisites (foundational knowledge), ensuring that the targeted skills are within the students’ reach. For instance, students with a background knowledge in procedural programming and object-oriented programming (provided prerequisites – see Appendices) can understand and implement control statements in C++, making ‘By the end of this class, students should be able to apply control statements to control the flow of a program in C++’ an achievable learning outcome.

For the learning outcomes that were classified as not achievable by ChatGPT, we noted that such learning outcomes include objectives which do not align with the provided prerequisites. For instance, ‘Students should be able to understand computer processor design’ can be considered unrealistic and unattainable within the given educational context. In other words, the targeted skill is not within the students’ reach since it has no link with the pre-requisites. As can be seen in our conversation with ChatGPT, the learning outcomes considered relevant by ChatGPT align with the provided educational goals (see Appendices). For instance, ‘By the end of this class, students should be able to apply control statements to control the flow of a program in C++’ is relevant to the broader objectives of understanding basic types, manipulating strings, simple I/O operations and control statements. For the learning outcomes that were classified as not relevant by ChatGPT, it can be seen that such learning outcomes have no alignment with the educational goals. For instance, learning outcomes like ‘Students should be able to understand computer processor design’ are completely detached from the specified educational goals. We observed that the learning outcomes considered time-bound by ChatGPT provide a timeframe within which the objectives are to be accomplished while those classified by ChatGPT as not time-bound do not. For instance, ‘By the end of this class, students should be able to apply control statements to control the flow of a program in C++’ provides a time boundary for achieving the outcome by incorporating ‘by the end of the class’. In another instance, ‘Students should be able to understand computer processor design’ provides no time boundary for achieving the intended outcome.

From our conversation with ChatGPT when evaluating learning outcomes for the presence of fundamental components, we observed that the learning outcomes which ChatGPT classified as having a verb incorporate a verb which indicates the action that students are expected to perform. For instance, the verb (apply) in ‘By the end of this class, students should be able to apply control statements’ indicates the targeted action the students should be able to perform. However, the classification of ‘Students should be able to understand’ as not having a verb by ChatGPT can be attributed to the absence of a verb which indicates the action that students are expected to perform. We noted that the learning outcomes which ChatGPT classified as having a content incorporate the topic, concept or subject matter to be learned by the students. For instance, ‘By the end of this class, students should be able to apply control statements to control the flow of a program in C++’ specifies what the students are supposed to learn and apply. However, the classification of ‘Students should be able to understand’ as not having a content by ChatGPT can be attributed to the absence of what the students are supposed to understand. We observed that the learning outcomes which ChatGPT classified as having a context specify the circumstance in which students are expected to apply acquired skills. For instance, the phrase ‘the C++’ in ‘By the end of this class, students should be able to apply control statements to control the flow of a program in C++’ indicates the platform for the application of the knowledge of control statements by the students. However, the classification of ‘By the end of this class, students should be able to apply control statements’ as not having a context by ChatGPT can be attributed to the absence of such a platform.

ChatGPT’s responses demonstrated a commendable capability to evaluate learning outcomes against the SMART criteria. Specifically, it demonstrated proficiency in assessing specificity, measurability, achievability, relevance, and time-bound attributes. Additionally, the model showcased its ability to identify the three essential components in learning outcomes. Generally, its analyses were coherent and well-reasoned, showcasing an understanding of the principles that underpin effective learning outcomes. The model’s analysis of learning outcome structure and SMART criteria compliance holds promise for educational contexts, potentially providing educators with automated tools to evaluate and refine their learning outcomes. The ability of ChatGPT to recognise SMART attributes and the fundamental components within learning outcomes can streamline the process of instructional planning and quality assurance. This research therefore marks a significant step in exploring AI’s role in educational design and assessment, bridging the gap between technological innovation and pedagogical practices.

Conclusion

In this study, we embarked on a comprehensive exploration of ChatGPT’s capacity to evaluate learning outcomes in alignment with the SMART criteria. Through a comprehensive analysis of presented learning outcomes and their adherence to the SMART principles, we gained valuable insights into ChatGPT’s potential in aiding educational design and assessment. The results revealed that ChatGPT exhibited proficiency in assessing specificity, measurability, achievability, relevance, and time-bound attributes, which are foundational for effective educational planning. The results further underscores the capacity of ChatGPT to contribute to educational practices by evaluating learning outcomes.

Building on the findings of this study, we propose a direction for further exploration; given the increasing significance of fostering creativity in education, we recommend that future research delve into ChatGPT’s ability to analyse the creativity level of learning outcomes within the framework of Bloom’s Taxonomy (Adams, 2015). This involves assessing how well ChatGPT can recognise and evaluate the creativity level of learning outcomes. By incorporating creativity as a dimension of learning outcome assessment, educators can gain insights into their pedagogical strategies and their potential to inspire imaginative thinking among students.

Furthermore, it is crucial to acknowledge that while AI-driven tools like ChatGPT can offer valuable insights and automation, they do not replace the nuanced expertise of educators. Human judgement, experience, and contextual understanding remain integral to the educational process. This is particularly important considering the debates surrounding the use of learning outcomes in education. Additionally, while learning outcomes are evaluated in this study for their alignment with the SMART criteria and the presence of fundamental components, it is noteworthy that this is a complex field with contests on the best strategy to learning outcome and curriculum design. Individual educational institutions also have specific requirements and policies guiding the use of AI-based tools in the design and development of educational resources. It is important that educators are aware of such requirements and policies and be guided by them when deploying AI-based tools in their practice. Finally, while the current study is focused on evaluating learning outcomes which are lecture- or session-based, the use of ChatGPT in reviewing learning outcomes could be extended to modules or programs, providing a mechanism for ensuring that such outcomes align with national frameworks such as Framework for Higher Education Qualifications, European Qualifications Framework, and United States Qualifications Framework.

Acknowledgments

Sections of this paper include materials generated by ChatGPT (version 3.5). The author(s) reviewed and edited the generated materials as needed and take(s) full responsibility for the content of the publication.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Descriptions of the data used for this study are provided within the article.

Additional information

Funding

The author received no grant from any funding agency for this study. For the purpose of open access, the authors have applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.

Notes on contributors

Samson Damilola Fabiyi

Samson Damilola Fabiyi has been a Teaching Fellow with the University of Leeds, Leeds. He received the M.Sc. and Ph.D. degrees from the University of Strathclyde, Glasgow, U.K. in 2018 and 2022 respectively. He is a Fellow of the Higher Education Academy.

Notes

1. The educational goals and prerequisites used in this study are included as appendices.

Appendices

Pre-requisites

1. Procedural Programming

2. Object Oriented Programming

Educational goals for the class:

Understanding what is C++ and why are we learning it.

1. Understanding the basic types and how to use them.

2. Understanding how to manipulate strings.

3. Understanding simple I/O

4. Understanding control statements

5. Understanding operators and overloading.

6. Understanding pointers in C++ and memory management

7. Understanding the const modifier