Metacognitive self-knowledge and cognitive skills in project-based learning of high school electronics students

ABSTRACT

Self-knowledge is an important element of metacognition. It encompasses understanding one's own strengths and weaknesses and includes belief and self-assessment of one’s capacity to perform a task. Cognitive skills comprise a range of mental processes, such as attention, memory, problem solving, systems thinking, abstract thinking, and critical thinking, that are essential for acquiring and applying knowledge. The research described in this paper examined the ability for self-knowledge and cognitive skills of electronics students engaged in project-based learning. The study, which used quantitative and qualitative tools, involved 140 twelve-grade electronics students and explored the self-knowledge and cognitive skills they exhibited while executing their final projects. The correlation and paired-samples t-test between the students’ self-report questionnaire (indirect assessment of cognitive skills) and achievement test (direct assessment of cognitive skills) scores were examined to assess the students’ self-knowledge and cognitive skills. The findings revealed a significant difference between self-report questionnaire scores and achievement test scores, indicating that the students’ self-knowledge did not reflect their actual low level of cognitive skills. Measuring self-knowledge and cognitive skills in project-based learning is essential for assessing the effectiveness of the learning process, resulting in improved project outcomes and a firmer foundation for future academic and professional success.

KEYWORDS:

• Metacognitive knowledge
• self-knowledge
• cognitive skills
• project-based learning (PBL)
• high-school electronics students

1. Introduction

Many factors drive curriculum development in higher engineering education to offer alternatives to disciplined theoretical courses (Sheppard et al. 2009). To attract prospective students, reduce attrition, improve professional practice preparation, and contribute to sustainable development, innovation, and job creation, education quality and processes must be improved. Project-based learning (PBL) uses authentic tasks that facilitate the acquisition of key engineering principles and require the use of critical thinking skills. Implementing PBL in engineering courses facilitates the collective conceptualisation of engineering foundations among students, enabling them to produce comprehensive and viable solutions to engineering challenges (Stewart 2007).

Both systems thinking and abstract thinking require cognitive abilities. Systems thinking provides a framework for investigating the interrelationships between system components (Senge 1990). Systems thinking is seen as crucial for the analysis and design of engineering systems (Jaradat 2015) while abstract thinking focuses on the details relevant to the current viewpoint, while temporarily ignoring the less significant details at the specific stage (Timothy 2008). Given the importance of cognitive skills such as systems thinking and abstract thinking, numerous studies have focused on the characterisation and development of each of these abilities in several contexts: high schools (Gero, Shekh-Abed, and Hazzan 2021; Shekh-Abed and Barakat 2023; Shekh-Abed, Hazzan, and Gero 2021), academia (Hazzan and Kramer 2016), and industry (Frank and Koral-Kordova 2009, 2013).

To design and maintain complex engineering systems, engineers must possess high levels of metacognition and cognitive skills. Flavell (1976; 1979) defined metacognition as thinking about one’s own mental activities. In more precise terms, metacognition refers to planning, monitoring, and assessing one’s understanding and performance (Chick 2013). The concept of metacognition is based on the ability to recognise one's thinking and learning as well as one's role as a thinker and learner. Metacognitive thinking is a key feature of expert thinking that can be taught to novices so that they may improve their learning skills (Stevenson, Azuma, and Hakuta 1986). It is widely considered to be an important aspect of learning and problem solving in a variety of fields (Georghiades 2000; Paris and Paris 2001; Pintrich 2002).

According to Krathwohl (2002), the revised Bloom's taxonomy consists of four broad areas of cognitive knowledge: factual, conceptual, procedural, and metacognitive. The metacognitive knowledge category was an addition to the original taxonomy, which already included the other three categories. Metacognitive knowledge consists of both general knowledge about cognition and awareness and understanding of one's own cognition (Pintrich 2002). Researchers concur that as students progress, they become more aware of their own thinking and more knowledgeable about cognition in general. Furthermore, such students tend to learn better when they act on this understanding (Bransford, Brown, and Cocking 1999). Georghiades (2000) found that the impact of metacognitive education on student performance could be seen in the experimental groups’ classroom interactions. Students who received metacognitive education appeared to retain more taught material from earlier lessons (e.g. terminology, definitions, examples, and applications). In his classic article on metacognition, Flavell (1979) suggested three types of metacognitive knowledge: strategic knowledge, knowledge about cognitive tasks, and self-knowledge.

The study described here investigated the self-knowledge ability (one of three types of metacognitive knowledge abilities) and cognitive skills among senior high school students majoring in electronics by examining their performance on final projects that combined hardware and software.

The paper begins with a review of metacognition. Then, the research purpose and questions are formulated, and the research methodology is described. The main findings are then presented, and the paper ends with a discussion and conclusions.

2. Theoretical background

Problem-based learning has appeared in engineering curricula for over 20 years. Many universities have included PBL in their engineering curricula, and numerous publications on PBL have been published in engineering education journals (Codur, Karatas, and Doğru 2012; Gibson 2001; Lou et al. 2010; Schachterle and Vinther 1996; Zhou, Kolmos, and Nielsen 2012). For effective PBL, Goodrich (1995) recommends projects with genuine interest, natural context, clear goals and steps, flexibility, and self-direction. Researchers also emphasise mentoring, modelling, continual assessment, training students in critical thinking, and focusing on learning goals in projects. The motivational aspect of PBL is a crucial factor in learning. Blumenfeld et al. (1991) found that PBL is most effective when it motivates students. There are numerous advantages to PBL. It fosters higher-order thinking skills in the cognitive domain, such as systems thinking (Nagarajan and Overton 2019), abstract thinking (Shekh-Abed, Hazzan, and Gero 2021), critical thinking (Pan & Allison, 2010), metacognition, and problem-solving ability (Miranda et al. 2020).

The term ‘cognitive skills’ refers to a broad range of mental abilities that are required for effective learning and problem solving. Memory, attention, problem solving, critical thinking, language processing, perceptual abilities, and executive processes are among the cognitive tasks covered by these mental abilities (Diamond 2013; Halpern 1998; Treisman and Gelade 1980). Individuals with strong cognitive skills can adapt to new conditions, learn from experience, and make sound decisions. They are important in the attainment of education attainment, professional performance, and daily living (Sternberg 2003), as they contribute to an individual's ability to process and apply knowledge, solve issues, and negotiate the intricacies of cognitive activities in a variety of contexts. A thorough grasp of cognitive skills is required for educators, psychologists, and researchers seeking to improve learning outcomes and address cognitive difficulties in a variety of situations (Atkinson and Shiffrin 1968; Gazzaniga, Ivry, and Mangun 2018).

Systems thinking helps integrate people, purpose, process, and performance by focusing on the whole and its relationships (Senge 1994; Senge 2008). It enables to investigate a topic/territory by connecting systems to their environment and to each other, and understand complex problems. Considering the whole system and its subsystems maximises results and minimises unintended consequences. Given the increasing complexity of engineering systems, the use of engineering systems thinking is becoming increasingly important (Monat and Gannon 2018).

Abstract thinking, on the other hand, enables individuals to concentrate on the pertinent details pertaining to the present perspective, while momentarily disregarding the less crucial information for the given stage (Timothy 2008). Consequently, it can be applied by engineers across many domains, who engage in both hardware and software development (Mishali, Dubinsky, and Maman 2008).

Systems thinking and abstract thinking both involve cognitive skills, but they differ in terms of the required knowledge, background, and interpersonal skills (Gero, Shekh-Abed, and Hazzan 2021; Shekh-Abed, Hazzan, and Gero 2021). Within the cognitive domain, the comparison between the two forms of thinking involves three specific abilities: requirements analysis, seeing the overall picture, and understanding the interactions and interconnections between system elements (Gero, Shekh-Abed, and Hazzan 2021).

The concept of metacognition can be defined as cognition about cognition. The term is used to refer to people's awareness and control, not only of their cognitive processes, but also of their emotions and motivations (Papleontiou-louca 2003). The development of independent learners and the ability to learn for life are some of the educational benefits of applying metacognitive strategies such as self-awareness and self-monitoring. Learners should be aware when they know something, when they do not know something, and when they do not know what to do. When referring to the monitoring and correcting of learning processes, students can learn about their strengths and weaknesses through metacognitive practices as learners, writers, readers, test takers, and group members. It is essential to recognise the limits of one’s knowledge and abilities and then find ways of expanding them (Bransford, Brown, and Cocking 1999).

Nelson (1990) proposed that the cognitive system is divided into two layers. The object level represents lower-order cognitive activities required for task execution, while the meta level is made up of higher-order evaluation and planning processes that govern the object level. Monitoring processes send information about the state of the object level to the meta level, whereas control processes send commands from the meta level to the object level. When errors occur at the object level, metacognitive monitoring processes notify the meta level and control processes are activated to resolve the problem. Because anomalies in cognitive activity elicit metacognitive control, this is essentially a bottom-up model.

Metacognition has two dimensions: metacognitive knowledge and metacognitive regulation. A learner’s metacognitive knowledge includes his or her own cognitive abilities (e.g. I have difficulty remembering dates), the nature of particular tasks (e.g. these ideas are complex), and the learner’s knowledge of different strategies and when to use them (e.g. breaking telephone numbers into chunks will help me remember them) (Brown 1987; Flavell 1979). The concept of metacognitive regulation refers to the monitoring and controlling of the learners’ own cognitive processes, for example, trying another approach after realising that the approach they are using is not working (Nelson 1990).

Self-knowledge, as noted in Section 1, is one of three categories of metacognitive knowledge ability (Flavell 1979). Flavell claimed that self-knowledge is an important component of metacognition, along with knowledge of different strategies and knowledge of cognitive tasks. Self-knowledge encompasses knowledge of one’s strengths and weaknesses. For example, a student who knows that she generally understands Ohm's Law in electricity has some metacognitive self-knowledge about her ability to understand Ohm's Law. One of the characteristics of experts is that they recognise when they do not know something and must rely on general strategies to obtain the necessary information. This self-awareness of the breadth and depth of one's own knowledge base is an important part of self-knowledge (Pintrich 2002) and included assessments of one’s capacity to complete a task (self-efficacy), achieve the goals of completing a task (learning or just getting a good grade), and the interest and worth that the task holds for them. It is highly vital to have accurate views and assessments of one's knowledge base and skills as opposed to exaggerated and incorrect self-knowledge (Pintrich and Schunk 1996). Students are unlikely to make an attempt to acquire or develop new information if they are unaware that they lack some part of factual, conceptual, or procedural knowledge. As a result, we emphasise the importance of teachers assisting students in making correct appraisals of their self-knowledge.

Students who know their strengths as well as their weaknesses can adapt their cognition and thinking to varied tasks, thus facilitating their learning (Pintrich 2002). A student may pay more attention to a topic when reading, and will apply alternative ways to grasp it if he knows it is a topic about which he does not know much. If a student recognises her weakness in a certain subject (e.g. electronics), she can study appropriately for forthcoming electronics tests. Without knowing their strengths and weaknesses, students are less likely to adapt to changing contexts and regulate their learning. If a student reads a text and thinks he already is knowledgeable on the topic, but in fact is not, he will be less likely to reread or review the material. A student who thinks she understands the content will not study as much for a test as a student who knows she do not. A student who says he understands the content but in fact does not, will score poorly on the test since he likely will not study as hard as the student who recognised his lack of understanding. Lack of self-knowledge may well hinder learning.

Teaching metacognition requires intentional instruction, which teachers sometimes do. Often, however, they teach implicitly. Many times, they presume their students can acquire metacognitive knowledge independently; some indeed can, while others cannot. Metacognitive knowledge can be acquired through experience and age, although many students struggle to do so. Studies with students (Hofer, Yu, and Pintrich 1998; Pintrich, McKeachie, and Lin 1987) reveal that many of them lack metacognitive knowledge, including strategies, cognitive tasks, and self-knowledge. The fact that college students tend to perform better academically alludes to the need to teach metacognitive information in K-12 settings.

Metacognitive practices assist students in planning, monitoring, and evaluating their own progress as they read, write, and solve problems in the classroom. According to research, metacognition is a powerful predictor of learning. Metacognitive practices contribute to learning in a unique way that goes beyond the influence of cognitive ability (Veenman, Wilhelm, and Beishuizen 2004). They have been shown to improve academic performance across a variety of ages, cognitive abilities, and learning domains, including reading and text comprehension, writing, mathematics, reasoning and problem solving, and memory (Dignath and Büttner 2008; Hattie 2009). Metacognitive skills can help students transfer what they have learned from one context to another, or from one task to the next, and teachers can assist by explaining how to do so.

Teachers can employ various approaches to support the metacognitive development of their students and encourage the monitoring and regulation of their cognitive processes. Schraw (1998) outlined four instructional strategies for fostering the construction and acquisition of metacognitive awareness. First, educators should highlight the significance of metacognition, emphasising its unique role in self-regulated learning. They should also model metacognitive processes, dedicating time to group discussions and reflection. Second, enhancing students’ knowledge of cognition involves using tools like the Strategy Evaluation Matrix to improve declarative, procedural, and conditional knowledge. Third, the regulation of cognition can be facilitated through the use of a regulatory checklist, guiding learners in planning, monitoring, and evaluating their performance. Fourth, creating environments that promote metacognitive awareness involves fostering a mastery-oriented approach, emphasising the importance of effort, persistence, and strategizing over performance goals. By implementing these four strategies, educators can effectively nurture students’ metacognitive skills and enhance their ability to regulate their own learning.

According to Silver et al. (2023) and Shekh-Abed and Stav (2023), constructive feedback and self-reflection boost students’ metacognitive and cognitive skills, improving project outcomes. These methods have several benefits: First, they raise students’ awareness of mental processes, allowing instructors and classmates to provide feedback on their problem-solving tactics (Shekh-Abed and Stav 2023). Constructive feedback helps students identify their strengths and shortcomings, allowing them to capitalise on their strengths and actively develop their flaws, fostering a well-rounded skill set. Self-reflection helps students examine project decisions and assumptions, improving future approaches. Students evaluate their work, set goals for improvement, and plan for future engineering project success. Analysing their own problem-solving methods broadens their perspectives and gives them diverse approaches to challenging engineering problems (Meyer et al. 2010). When given constructive comments, students gain confidence, and self-reflection motivates them to take on more challenging undertakings. Reflecting on teamwork improves collaboration and project outcomes. Finally, proactive learning from past failures improves productivity and project success (Siegesmund 2016).

3. Research purpose and questions

The research purpose was to measure the self-knowledge and cognitive skills of 12th grade high school students (seniors) when performing hands-on activities that are scaffolded to form a comprehensive engineering assignment. The objective was to gain a better understanding of the causes of the many challenges the student encounter, the responses to those challenges, and the students’ perceptions during these activities. The research questions were as follows:

• To what extent do high school seniors engaged in electronics project-based learning have self-knowledge of their cognitive skills?

• How are cognitive skills expressed among high school seniors engaged in electronics project-based learning?

4. Methodology

4.1. Participants

The study involved 140 high school electronics seniors whose educational background and level were similar to those of average Israeli 12th grade students (17–18 years old) majoring in electronics. The number of participants from each school ranged between 7 and 39. The participants reflected the ethnic distribution in their regions. Students executed their final projects throughout the school year, working in teams of two, under their teacher’s guidance.

4.2. Procedure

The projects focused on the design and implementation of a system that combined hardware and software. Products were based on a programmable device, such as a PC or an Arduino, and featured hardware components such as sensors, motors, drivers, and screens. The projects were carried out in the school’s electronics laboratories, which were outfitted with measuring instruments (such as multi-metres and oscilloscopes). Each team submitted a project proposal at the beginning of the school year and a final report at the end of the school year. Table 1 presents examples of final projects.

Table 1. Examples of final projects.

Subject Description
Smartphone-controlled robotic arm	The robot is based on an Arduino Mega 2560 microcontroller. The robot’s 6-axis arm movement is controlled by an app that is connected to the robot via Bluetooth.
Tank robot	The robot is based on an Arduino Uno microcontroller. The robot uses continuous tracks (treads) instead of wheels, as commonly used on rough, uneven, and loose terrain. The robot is designed to enter dangerous places and rescue people.

Students spent about six hours per week in the school's electronics laboratories, where they collaborated, experimented, and troubleshooted, fostering a hands-on learning environment. The coursework gave students a basic background in electronics that was supported by a thorough curriculum beginning in the tenth grade, enabling a smooth transition to more challenging projects in their senior year. Additional courses, strategically integrated, covered programming languages, circuit design, and hardware application.

The study used quantitative and qualitative methods to analyze quantitative and qualitative data collected by various instruments. At the end of the school year, the students completed a self-report questionnaire (indirect assessment of cognitive skills ability) and took an achievement test (direct assessment of cognitive skills ability). The self-report questionnaire had indirect assessment statements that corresponded directly to the direct assessment of cognitive skills. For example, to assess the students’ self-knowledge about Ohm’s Law (a type of metacognition knowledge), we examine the correlation and difference between the self-report statement score and the achievement question score (see Table 2). (This example was not included in the current study.)

Table 2. Example of indirect vs. direct assessment of cognitive skills ability.

Self-report statement	Achievement question
I have an understanding of Ohm’s Law: 1 – Strongly disagree to 5 – Strongly agree	A 12 V battery is connected to a circuit with a resistance of 4 ohms. What is the current flowing through the circuit?

To assess the students’ self-knowledge (type of metacognition knowledge), the researcher examined the correlation between the average score for each statement on the self-report questionnaire and the students’ average score for the question on the achievement test that corresponds to the same cognitive skill. The researcher also calculated the difference between the two scores, for each item-pair on the students’ self-report questionnaire and achievement test.

The quantitative data from the two instruments were analyzed using:

• Spearman's correlation coefficient between the score for each statement on the self-report questionnaire and score for question on the achievement test that correlates to the same cognitive skill.

• Paired-samples t-test between self-report questionnaire and achievement test scores.

Throughout the school year, an in-depth qualitative investigation was conducted, capturing participant observations and pertinent documents related to high school electronics projects, including project proposals, preliminary designs, and final reports. A directed content analysis approach (Hsieh and Shannon 2005) was used that was guided by the cognitive skills of systems thinking and abstract thinking, and involved five specific skills: requirements analysis, seeing the overall picture, understanding the interaction and interconnection between the system elements, identifying problems in implementing the solution, and solving problems in the implementation stage (Gero, Shekh-Abed, and Hazzan 2021; Shekh-Abed, Hazzan, and Gero 2021). The data underwent a meticulous coding process. An initial codebook was created, predefining codes aligned with the theoretical framework, and multiple iterative rounds of analysis ensued. Drawing methodological inspiration from Hsieh and Shannon's Directed Content Analysis, the analysis aimed to identify patterns and themes that reflect/describe how high school students applied cognitive skills at different project stages. The number of iterations was determined by reaching themes saturation, ensuring comprehensive exploration of the dataset. The resulting themes provide a nuanced understanding of systems and abstract thinking in the context of high school students engaging in electronics projects.

4.3. Instruments

The self-report questionnaire and the achievement test were based on the cognitive skills required of engineers, namely systems thinking and abstract thinking (Frank and Koral-Kordova 2009, 2013; Hazzan and Kramer 2016; Koral-Kordova and Frank 2012; Ye and Salvendy 1996), and were adapted for high school students majoring in electronics (Gero, Shekh-Abed, and Hazzan 2021; Shekh-Abed, Hazzan, and Gero 2021). The characteristics of these cognitive skills were validated by two experts in engineering education and seven electronics teachers with rich experience guiding projects that combine hardware and software.

The self-report questionnaire comprised 18 statements that the students were asked to rate on a 5-point Likert scale (ranging from 1 – Strongly Disagree to 5 – Strongly Agree). Thus, for example, agreement with the statement ‘When I am involved in an engineering project, I do not understand the data flow in the system of the proposed product’ expresses a relatively low cognitive skill level, whereas agreement with the statement ‘When I am involved, as a student, in an engineering project that combines software and hardware, I understand the syntax of the software I am developing’ indicates a relatively high cognitive skill level. These two examples demonstrate the definition of low and high cognitive skill levels using the described Likert scale. The internal consistency of the questionnaire statements was good (Cronbach’s α = 0.802). The left column of Table 3 presents examples of questionnaire items.

Table 3. Examples of statements from the questionnaire vs. questions from the achievement test.

Self-report questionnaire		Achievement test
As a student in an engineering project team, I understand the project requirements of the proposed product.		What are the project requirements for the proposed product? The system will be resistant to malfunctions that can be expected to occur due to reasonable use, such as communication problems. The system will move a gate that weighs up to 200 kg. The system will enable the user to open and close the gate from a maximum distance of 50 metres. The application will work on Android operating systems.
When I am involved, as a student, in an engineering project that combines software and hardware, I can build the structure chart of the software (graphical representation of the software according to its main operations and functions, including their connections).		What does the structure chart of the software in the proposed system describe (the diagram is shown in the Introduction)? The Car_Gate, Order_Gate, and Con_Gate functions pass values to the Main function and the Open_CloseGate function receives values from the Main function. The Car_Gate, Order_Gate, and Con_Gate functions pass values to the Open_CloseGate function. All functions return values to the Main function. All functions receive values from the Main function.
When I am involved, as a student, in an engineering project that combines software and hardware, I understand the syntax of the software I am developing?		What is the value of ‘a’ after running the following code? a = 11 a = 8 a = 10 a = 9

The achievement test focused on the analysis of a system that opens and closes a parking lot gate and is controlled via smartphone. This system, which combines hardware and software components, was not part of any of the students’ final projects. The test comprised 18 multiple-choice questions (one correct answer and three distractors) that dealt with cognitive skills. The right column of Table 3 presents several typical test questions. The internal consistency of the achievement test questions was acceptable (Cronbach’s α = 0.662).

Indirect assessment statements on the self-report questionnaire were directly correlated to the direct assessment of cognitive abilities. For example, as indicated in the second row of Table 3, we must investigate the correlation and difference between the students’ self-report statement and corresponding achievement test score to measure the students’ self-knowledge about project requirements (a type of metacognition knowledge). The correlation was determined for the 18 self-report questionnaire statements vs. achievement test questions scores. Ethical approval was received from the Office of the Chief Scientist, Ministry of Education, State of Israel (Approval No. 9572).

5. Findings

An analysis of Spearman's correlation coefficient between the scores of each statement in the self-report questionnaire and the scores of corresponding questions on the achievement test revealed a lack of statistical significance for the nine cognitive skills items (see Table 4). In contrast, for the remaining eight items, a weak but statistically significant correlation was observed.

Table 4. Correlations between questionnaire statements and scores of achievement test questions.

	Cognitive skill	Spearman's correlation coefficient
1	Understanding the project requirements of the proposed product	Weak, positive, and not significant (r = 0.275, p > 0.05)
2	Understanding the block diagram of the proposed product	Weak and positive, and significant (r = 0.164, p < 0.05)
3	Understanding which is the optimal alternative for the proposed product	Weak, positive, and not significant (r = 0.077, p > 0.05)
4	Understanding which sensor cannot be used as an alternative for the proposed product	Weak and positive, and significant (r = 0.165, p < 0.05)
5	Understanding what change should be made in the system when replacing another component	Weak, positive, and not significant (r = 0.010, p > 0.05)
6	Understanding which circuit more correctly describes the connection of digital inputs in the circuit	Weak and positive, and significant (r = 0.163, p < 0.05)
7	Understanding the software requirements of the proposed product (understanding the input and output specifications of the system)	Weak, positive, and not significant (r = 0.046, p > 0.05)
8	Understanding the performance requirements of the software	Weak, positive, and not significant (r = 0.110, p > 0.05)
9	Identifying the type of software required (e.g. Windows programming, computer-embedded system, real-time, app programming)	Weak, negative, and not significant (r = −0.037, p > 0.05)
10	Understanding the range of voltage values entering an Arduino microcontroller from an analog input	Weak, positive, and not significant (r = 0.123, p > 0.05)
11	Understanding the range of digital values entering an Arduino microcontroller from an analog input	Weak and positive, and significant (r = 0.202, p < 0.01)
12	Understanding what the software’s structure chart describes in the proposed system	Weak and positive, and significant (r = 0.199, p < 0.01)
13	Understanding the concept of the software’s implementation for the proposed system's functions	Weak, positive, and not significant (r = 0.042, p > 0.05)
14	Understanding how the flowchart describes the concept of the software’s function	Weak and positive, and significant (r = 0.170, p < 0.05)
15	Understanding which result correctly represents the values of the variable after running the programme	Weak, positive, and not significant (r = 0.036, p > 0.05)
16	Identifying the software syntax error in the programme	Weak and positive, and significant (r = 0.233, p < 0.01)
17	Understanding the software from different perspectives (e.g. different user perspectives)	Weak, positive, and not significant (r = 0.126, p > 0.05)
18	Understanding the software in detail	Weak and negative, and significant (r = −0.162, p < 0.05)

Table 5 presents the students’ mean score, M, and standard deviation, SD, for the self-report questionnaire (self-report questionnaire scores ranged between 20 and 100) and the achievement test (achievement test scores ranged between 0 and 100).

Table 5. Self-report questionnaire and achievement test scores.

Test
	M	SD
Self-report questionnaire	81.32	9.14
Achievement test	46.43	18.32

A paired-samples t-test conducted to determine the difference between the self-report questionnaire and achievement test scores revealed a significant difference between self-report questionnaire scores (M = 81.32; SD = 9.14) and achievement test scores (M = 46.43; SD = 18.32); [t(139) = 22.826, p < 0.001]. On average, report questionnaire scores were 34.89 points higher than achievement test scores (95% CI [31.87, 37.91]).

Analysis of the qualitative data (final reports) shows that at the end of the school year, the students did not understand the cognitive skills involved in their projects. Thus, for example, students wrote in their final report about the project requirements: ‘We are developing a portable, smart baby formula maker that is easy to use’, but in fact, they built a heavy product that is difficult to move from place to place. They used heavy AC motor (∼2 kg) instead of small step motor (∼200 g), and incorporated other heavy materials in their product. In addition, they did not address issues that may arise due to reasonable use, such as the need to press two buttons simultaneously, and instead of presenting an overall picture in their abstract, they used it to describe technical details:

Each of the four buttons on the system's front is connected to an Arduino digital input via a pull-down resistor. When one of the buttons is pressed, 5 V enters the Arduino input. Otherwise, 0 V is input.

The system includes Arduino, motor, DS1880 sensor, I2C LCD 16*4, pump, valve and heater.

Then, they described the components of the block diagram rather than the interaction and interdependence between components:

The buttons allow the user to select the desired amount of milk for the baby. A flow sensor measures the amount of water required. The LCD function is to display the desired amount of milk. The function of the equation is to lower the water level in the bottle. The function of the motor is to lower the required amount of milk formula into the bottle.

The students also described the functions of their code but failed to describe the relationships between the functions:

The Temp function measures the temperature of the water. The Flow function is used to calculate the amount of water. The Heat function activates the heating element.

Some students had difficulty identifying problems with the project's implementation, as expressed in their final reports:

I asked my instructor to help me figure out a few problems.

I faced many problems regarding the hardware.

Students also had difficulty resolving problems that arose during the project's implementation:

There were some bugs in the codes, so we worked hard and got help from our instructor to finish the code correctly.

I came across a component connection … that I couldn't activate, but after many attempts and with the assistance of my instructor, I succeeded.

Table 6 summarises the manifestation of students’ cognitive skills based on final reports and observations of students while they worked on their projects in the lab.

6. Discussion

The findings of this study highlight a noteworthy gap between students’ self-reported cognitive skills and their actual performance on achievement tests. The Spearman's correlation coefficient, which aimed to establish a relationship between the self-report questionnaire scores and the achievement test scores, demonstrated insignificance for nine cognitive skills and weak significance for the remaining eight. This lack of a robust correlation suggests a notable discrepancy between how students perceive their cognitive abilities and their demonstrated proficiency on objective assessments. The significant difference identified through a paired-sample t-test between self-report questionnaire scores and achievement test scores further underscores this disjunction. The implication is clear: students’ self-knowledge does not correspond to their actual cognitive skills, indicating a potential overestimation or misperception of their abilities.

Table 6. Students’ expression of their cognitive skills.

Skill	Manifestation of students’ cognitive skills
	Final reports	Observations
Requirements analysis	‘The system should be portable.’	‘The majority of projects completed by the students do not meet the criteria stated in the final report.’
Seeing the overall picture	‘The button is connected to an Arduino digital input via a pull-up resistor. When the button is pressed, 0 V is applied to the Arduino input. Otherwise, 5 V is used as input.’	‘Students struggle to describe the big picture of the project and instead focus on the project's components.’
Understanding the interaction and interdependence between components	‘The user selects the number of pharmaceutical pills/capsules using a 3*4 keypad. The system contains an I2C LCD that displays data and messages … .’	‘Students tend to describe the components themselves rather than the relationships between them in the system.’
Describing the relationships between software functions	‘The function Distance measures the robot's distance from nearby objects. The Forward function moves the robot forward. The Right function moves the robot to the right. The Left function moves the robot to the left.’	‘Students struggle to describe relationships between software functions and are content to describe the functions themselves.’
Identifying problems in implementing the solution	‘I asked my instructor to assist me in figuring out a few problems … ’	‘Students have difficulty identifying problems when implementing a system that combines several hardware and software components.’
Solving problems in the implementation stage	‘There were some issues with component connections, but we got assistance from our instructor and the system worked properly.’	‘Students struggle to solve problems when implementing a system that combines multiple hardware and software components.’

Requirements analysis is a critical step in the development of successful projects developed by high school electronics students through engineering PBL. It involves understanding, defining, and documenting the needs, constraints, and objectives of the project to ensure that the final outcome meets the desired goals. Despite the fact that the students formulated requirements for their products in their final reports, they did not implement them in their actual projects. According to the research observations, the students did not understand the importance of requirements analysis and how it is expressed in their projects. The findings also revealed that the students did not understand how to express the other five cognitive skills in their projects:

• Seeing the big picture: This skill involves understanding the overall scope and objectives of the project. It enables students to grasp the broader context in which their product will function and how it aligns with the broader goals.

• Understanding the interaction and interdependence of components: Software systems are often composed of various interconnected parts. Students need to comprehend how these components interact with each other and how changes in one part can impact other parts.

• Describing the relationships between software functions: This skill pertains to the ability to articulate how different software functions or features relate to each other and contribute to the overall functionality of the product.

• Identifying problems while implementing the solution: During the development process, students should be able to recognise any issues or challenges that arise while trying to implement their proposed solutions.

• Solving problems during the implementation stage: Problems will inevitably arise during the implementation phase, and students should be equipped with problem-solving skills to address these issues effectively.

The students did not understand how to express these skills, which is why they were not reflected in the performance of their projects or in the final reports. To address this issue, it is crucial for educators to focus on enhancing students’ cognitive skills during their engineering studies. Interactive discussions and real-world case studies can help students grasp the practical aspects of these skills and improve their ability to apply them effectively in their projects.

In the context of a high school smart home automation project, for example, students tended to describe individual components, such as motion sensors, smart thermostats, and lighting controls, without effectively conveying the relationships between these elements in the overall system. While their presentations excelled in detailing the functionalities of each component, the lack of emphasis on how these components interacted hindered the creation of a cohesive smart home ecosystem. To address this challenge, a recommended pedagogical strategy involves incorporating system-level design projects. By exposing students to tasks that require them to consider the holistic functionality of the entire system, they can transition from isolated component descriptions to understanding the intricate connections and interdependencies between components. This approach fosters a systems thinking mindset, enhancing students’ ability to comprehend and articulate the relationships within complex systems, as demonstrated by the transformation observed in the smart home automation project.

Additionally, providing constructive feedback (Nicol and Macfarlane-Dick 2006) and self-reflection (Shekh-Abed and Stav 2023; Silver et al. 2023) can also help students improve their metacognition (including self-knowledge) and cognitive skills, leading to better project outcomes. Here are some examples in which constructive feedback and self-reflection can positively impact students’ project outcomes (Meyer et al. 2010; Siegesmund 2016):

• Increased awareness of thought processes: After a presentation on a sustainable energy project, a student receives feedback highlighting the effectiveness of his algorithm and suggesting an alternative approach for optimising energy consumption. This feedback prompts the student to reflect on his thought process, realising the potential for improvement. Subsequently, he actively adjusts his algorithm, showcasing an increased awareness of his thinking patterns and a proactive effort to enhance problem-solving strategies.

• Identification of strengths and weaknesses: A student receives constructive feedback praising her detailed research and pointing out a lack of clarity in explaining project implications. Recognising this as valuable input, the student acknowledges her weakness in communication and actively seeks opportunities to improve, capitalising on her research strengths. This process fosters a well-rounded skill set, as she addresses her weaknesses while leveraging her existing strengths.

• Encouraging critical thinking: During a software development project, a student engages in self-reflection on his coding decisions. He critically evaluates the efficiency of his chosen algorithms, questions assumptions about user interactions, and explores alternative methodologies. This habit of critical thinking refines his approach, leading to more informed choices in subsequent projects as he consistently questions and improves his decision-making processes.

• Promoting continuous improvement: A student, reflecting on a series of engineering projects, identifies a pattern of improvement in her coding skills. She sets specific goals to further enhance her programming proficiency and consistently monitors her progress. This commitment to continuous improvement not only benefits the current project but also prepares her for future engineering endeavours, creating a trajectory of ongoing development.

• Enhanced problem-solving strategies: A student reflects on a design challenge in which a traditional approach fell short. Analyzing his problem-solving process, he identifies alternative materials and manufacturing techniques not initially considered. This experience broadens his perspective, equipping him with a diverse set of problem-solving strategies for addressing complex engineering challenges in future projects.

• Boosting confidence and motivation: A student, initially uncertain about her ability to lead a team in a robotics competition, receives supportive feedback praising her leadership skills. This constructive reinforcement boosts the student’s confidence. Through self-reflection, she recognises personal growth, fostering a sense of accomplishment. Motivated by this positive experience, the student becomes more willing to tackle and excel in more ambitious projects.

• Effective team collaboration: Reflecting on a group project, a student identifies his role in resolving conflicts within the team and facilitating communication. Learning from this experience, he actively seeks opportunities to enhance collaboration in future projects, recognising the importance of effective teamwork in improving overall team dynamics and achieving successful project outcomes.

• Preventing future mistakes: A student reflects on a coding error that caused delays in a software project. Learning from this mistake, she implements rigorous testing protocols in subsequent projects, preventing similar errors. This proactive approach ensures increased efficiency and more successful outcomes in future projects by applying lessons learned from past experiences.

Instructors and mentors play a crucial role in fostering an environment that encourages feedback and self-reflection (Ritter and Barnett 2016). They can guide students to ask themselves questions such as:

• What did I learn from this project experience?

• What worked well in my approach, and what could have been improved?

• Did I communicate my ideas and solutions effectively to others?

• How did I handle challenges and setbacks, and what could I do differently next time?

• How did I contribute to the team's success, and how can I be a more valuable team member in the future?

The practice of providing constructive feedback and encouraging self-reflection can have a profoundly positive impact on the metacognition of high school electronics students and on their project outcomes (Brandt 2020). By engaging in these processes, students are encouraged to think critically about their work, identify strengths and areas for improvement, and develop a deeper understanding of their learning process. Constructive feedback from teachers and peers enables students to gain valuable insight into their projects, helping them identify blind spots, correct errors, and refine their ideas (Marzano, Pickering, and McTighe 1993). This feedback not only improves the overall quality of the projects but also fosters a growth mindset, encouraging students to embrace challenges and view mistakes as learning opportunities. Moreover, self-reflection complements the feedback process by prompting students to assess their own performance and thought processes independently (Schunk and Zimmerman 2012). Such an introspective approach enables them to recognise their learning strategies, identify areas for improvement, and set goals for future projects. Metacognition, or the ability to think about one's thinking, is enhanced through self-reflection, leading also to improved problem-solving skills.

Further research will investigate whether incorporating constructive feedback and self-reflection into the learning process of high school electronics students can positively transform their educational journey. By nurturing metacognition, students can be empowered to become more self-directed learners, leading to enhanced project outcomes and a stronger foundation for future academic and professional success.

7. Conclusions

The findings of this study highlight a noteworthy disparity between students’ self-reported cognitive skills and their actual performance on achievement tests. Several factors may contribute to this gap. It could be rooted in the inherent difficulty of using indirect assessment (such as self-report questionnaires), as individuals may struggle to accurately gauge their cognitive strengths and weaknesses. Additionally, social desirability bias may play a role, as students may naturally be inclined to present themselves in a more favourable light on self-report measures.

These findings hold critical implications for both educators and educational psychologists. The misalignment between perceived and actual cognitive skills underscores the need for interventions that enhance students’ self-knowledge and improve the accuracy of their self-assessments. This could involve the implementation of metacognitive strategies within the curriculum, encouraging students to reflect on their learning processes and actively monitor their cognitive growth.

Moreover, the results call for a reconsideration of the assessment methods used to evaluate students’ cognitive skills. While self-report questionnaires provide valuable insights into students’ perceptions, they may not fully capture the nuances of cognitive development. A more comprehensive approach, perhaps integrating multiple assessment tools and methodologies, could offer a more accurate representation of students’ cognitive abilities. In addition, it is important to consider the potential influence of instructional quality and students’ reflexivity on their ability to express their cognitive skills, as well as the disparity between students’ self-knowledge of cognitive skills and actual performance.

In conclusion, the discrepancy between self-reported cognitive skills and objective measures observed in this study highlights the complexity of understanding and evaluating student cognition. Educators and researchers must continue to explore effective strategies for fostering accurate self-knowledge and refining assessment practices to ensure a more holistic understanding of students’ cognitive capabilities. Addressing this discrepancy can ultimately contribute to more targeted and impactful educational interventions tailored to the genuine needs and potentials of each student.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Aziz Shekh-Abed

Aziz Shekh-Abed is a lecturer in the Department of Electrical and Computer Engineering at Ruppin Academic Center, Israel. He holds a PhD in engineering education from the Technion – Israel Institute of Technology. His research focuses on systems thinking, abstract thinking, soft skills, and metacognition. Dr Shekh-Abed holds an MA in science education and a BSc in technology education, both from Tel Aviv University, Tel Aviv, Israel. He is listed in the Israeli Engineers and Architects Register, under the section of electronic engineering.