Exploring undergraduate students’ scientific reasoning in the force and motion concept

Abstract

This research investigates the abilities of undergraduate students to apply scientific reasoning in Indonesia, with a particular focus on the concept of force and motion. Forty-three first-year undergraduate students from an Indonesian private institution, comprising 20 males and 23 females, performed the Scientific Reasoning Test of Motion (SRTM) in five different patterns of scientific reasoning. The current research combines quantitative and qualitative analysis. The findings reveal that the SRTM is a valid and reliable instrument. While no significant gender-based differences were observed in Correlational Thinking (CT), differences emerged in Control of Variables (CV), Proportional Thinking (PPT), Probabilistic Thinking (PBT), and Hypothetical-Deductive Reasoning (HDR). These results indicate that PBT, HDR, and CT are applicable across gender groups, although students struggle with scientific reasoning. Specifically, the students had misconceptions regarding projectile motion and applying Newton’s second law in practical situations. These findings indicate the necessity for further developing students’ abilities to apply scientific reasoning.

Keywords:

• Scientific reasoning
• gender
• motion
• force
• physics

REVIEWING EDITOR:

• Sammy King Fai HUI, the Education University of Hong Kong, HONG KONG

SUBJECTS:

• Physics
• Thinking, Reasoning & Problem Solving
• Assessment

1. Introduction

Students need to learn science as it is a means of interpreting daily routines. Scientific reasoning, which encompasses the cognitive abilities needed for investigation, inference, and argumentation, has become a primary goal and crucial learning outcome in science education standards across many countries (Bolger et al., 2021; Omarchevska et al., 2022). Some scholars have proposed that developing students’ abilities for scientific reasoning is essential to academic achievement and daily life (Sutiani et al., 2021).

Examining the various studies conducted in this field is necessary to gain a deeper understanding of scientific reasoning. The term ‘scientific reasoning’ describes a set of abilities used in scientific practices, including ‘science inquiry,’ ‘experimentation,’ ‘evidence evaluation,’ and ‘inference.’ Zimmerman et al. (2019) mentioned that scientific reasoning encompasses the cognitive abilities needed for investigation, inference, and argumentation. The reasoning is reaching conclusions based on premises and supporting evidence (Schlatter et al., 2021; Van Vo & Csapó, 2021). The conceptual framework of scientific reasoning by Engelmann et al. (2016) comprises three interrelated dimensions: (a) scientific reasoning as a scientific discovery process, (b) scientific reasoning as a focus of scientific argumentation, and (c) scientific reasoning in terms of understanding the nature of science.

Many studies have explored the nature of scientific reasoning to understand and develop students’ scientific reasoning (Lawson et al., 2000), to comprehend the development of scientific reasoning tests (Lawson et al., 2000; Van Vo & Csapó, 2021), and to develop scientific reasoning (Choowong & Worapun, 2021; Erlina et al., 2018). Additionally, studies have been conducted on scientific reasoning development, with a focus on cognitive development and academic level (Boğar, 2019), gender (Piraksa et al., 2014; Yang, 2004), and student’s perceptions of the learning environment (Bezci & Sungur, 2021).

Further, the results of gender-based studies showed contradictory findings. Piraksa et al. (2014) concluded that there was no discernible gender difference in scientific reasoning. In contrast, Sagala et al. (2019) found that males and females demonstrated relatively distinct patterns of cognitive characteristics. Kersey et al. (2018) examined the influence of aptitudes and personal circumstances on gender differences in scientific reasoning, and they found that men were likelier than women to complete tasks requiring scientific thinking. It aligns with the evidence by Nieminen et al. (2013), which indicated that men demonstrated superior performance in creating and applying theories. Valanides (1996) argued that men performed significantly better than women when employing probabilistic reasoning. The inconsistency of findings across these studies is the primary reason for the knowledge gap regarding gender differences in scientific reasoning abilities. Therefore, by addressing these gaps, we can better understand the nature and scope of gender inequalities in scientific reasoning, specifically in the context of force and motion.

For centuries, the concepts of force and motion have been fundamental to studying physics. These concepts are crucial to understanding mechanics and other challenging ideas in physics, such as electricity, heat, and waves (Tomara et al., 2017). Without an adequate grasp of mechanics, students cannot develop a correct understanding of force and motion law (Wancham et al., 2023). A lack of engagement in physics lessons, which focused on complex mathematical equations over the development of conceptual understanding, could potentially lead to misconceptions (Erlangga et al., 2021; Levin et al., 1990). A common misconception of projectile motion is that the horizontal and vertical motions are independent. Students might think that simultaneously launching an object horizontally and vertically will result in the horizontal motion remaining unaffected by the vertical movement. Indeed, the vertical and horizontal motions are interrelated (Hidayatulloh et al., 2021).

Learning about physics should be done through observation and experiments related to scientific reasoning. Force and motion, as a crucial concept of mechanics, can help undergraduate students in their first year effectively learn physics. Therefore, it is necessary to carry out an initial investigation of their scientific reasoning about the concept of force and motion. This research emphasised several relevant issues, including (1) assessing the instrument’s quality using Rasch analysis, (2) a description of the Rasch analysis-based gender inequalities in scientific reasoning, and (3) evaluation of students’ scientific reasoning on the concepts of force and motion. It is expected that the findings can be a resource in developing appropriate media, techniques, strategies, models, and methods so that they can develop their scientific reasoning.

2. Literature review

The Scientific Discovery as Dual Search (SDDS) paradigm represents scholars’ prevailing framework for scientific reasoning (Dunbar & Klahr, 2013; Zimmerman, 2000). The SDDS conceptualises scientific reasoning as a problem-solving process involving developing and revising hypotheses within two distinct spaces: the hypothesis space and the experiment space. Subsequent studies concerned more specific aspects of scientific reasoning, including integrating diverse causal variables (Kuhn & Pearsall, 2000). Moreover, recent developments in scientific reasoning have highlighted the importance of epistemic cognition, or beliefs about knowledge and its development, and the cognition that emerges from these beliefs (Reith & Nehring, 2020). Argumentation is critical in this field (Fischer et al., 2018).

This research concerns scientific reasoning, the most widely employed topic in science education research. Lawson (2004), who defined scientific reasoning as creating causal inferences for a phenomenon that cannot be directly observable by using mental rules, plans, or tactics, was the initiator. This concept focuses on the mental processes that enable one to extrapolate unobservable mechanisms from observable facts (Lawson, 2004, 2010). Five subsets of skills, including hypothetical-deductive reasoning, control of variables, proportional thinking, correlational thinking, and probabilistic thinking, are identified as essential elements of Lawson’s conceptualisation of scientific reasoning (Lawson, 2004; Zimmerman, 2000). These competencies represent various processes frequently essential for successful scientific inquiry (Lazonder & Egberink, 2014). Established tests in psychometric and educational research typically measure and assess scientific reasoning. These include Lawson’s Classroom Test of Scientific Reasoning [LCTSR], The Test of Logical Thinking [TOLT], and Group Assessment of Logical Thinking [GALT]). However, few studies have used self-developed instruments, despite the potential benefits of such approaches (Dolan & Grady, 2010; Luo et al., 2020).

In order to gain a deeper comprehension of scientific reasoning in undergraduate students, it is imperative that we take into account two important theoretical stances: information processing theory and cognitive constructivism theory. As per Jean Piaget’s theory of cognitive development, children go through discrete stages of mental development that are characterized by changes in the thinking processes. This theory offers a framework for comprehending the ways in which people learn and create their worldview. According to Piaget’s thesis, an undergraduate student’s capacity for scientific reasoning is influenced by their cognitive development stage, which in turn affects their capacity for problem-solving and scientific inquiry. Lawson (1985) found that undergraduates in the formal operational stage are able to understand complex scientific concepts and exhibit more abstract thinking than those in the concrete operational stage, which is consistent with Piaget’s theory. In their scientific reasoning, concrete operational stage students typically rely on empirical evidence and observable phenomena. Furthermore, research conducted by Lourenço (2016) revealed that undergraduate students are generally at the formal operational stage, which is defined by hypothetical reasoning and abstract thought. This stance explains how students use reasoning, utilize abstract concepts, and model argumentation to approach scientific thinking at this level.

Furthermore, highlighting the cognitive processes involved in learning, Robert Gagné’s Information Processing Theory concentrates on how people encode, store, and retrieve information. Undergraduate students’ capacity for scientific thinking is impacted by their information processing and manipulation skills, according to Gagné’s notion. According to research by Winne (2013), comprehension and application of scientific concepts are more proficient among undergraduate students who possess strong cognitive processing abilities, such as rehearsal, elaboration, and attentiveness. Gagné’s theory also highlights the importance of practice and feedback in the learning process, implying that students’ capacity for scientific reasoning can be improved via repeated exposure to scientific challenges and constructive criticism (Hasson et al., 2015). Avargil et al. (2018) have conducted studies which emphasize the significance of metacognitive techniques, including planning, monitoring, and assessing, in scientific thinking for undergraduate students. Gagné’s Information Processing Theory thus have the potential to inform science teachers’ creation of instructional tactics that maximize students’ cognitive processes and foster more potent scientific reasoning abilities.

Several factors must be considered when evaluating scientific reasoning. Firstly, it is essential to acknowledge the gender difference as a variable. Secondly, more studies are required to clarify the contradictory findings of previous studies on gender differences in scientific reasoning, thinking, or intelligence quotient (Luo et al., 2021; Pezzuti et al., 2020). For instance, studies by Luo et al. (2021), Nieminen et al. (2013) and Yang (2004) found distinguished scientific reasoning skills between men and women. In contrast, studies by Piraksa et al. (2014) and Waschl and Burns (2020) found no significant difference. A study by Piraksa et al. (2014) stated that the gender variable had no discernible impact on students’ scientific reasoning across all LCTSR sub-dimensions. On the contrary, men performed better than women in formulating and making judgments (Yang, 2004). Consequently, further investigation from multiple perspectives is essential to ascertain the underlying causes of these disparate outcomes.

3. Method

3.1 Research design and procedure

The mixed-methods approach integrates quantitative and qualitative approaches, prioritising quantitative over qualitative analysis. The combined results assist students in comprehending scientific reasoning in greater depth. The first stage of this research was to administer the Scientific Reasoning Test of Motion (SRTM) to a group of students. After collecting the test data, Rasch analysis was carried out to assess several factors: dimensionality and fit, item and individual reliability, independence, and interpretation of the Wright map of scientific reasoning. The researchers then examined the written test, which expressed the students’ ideas and thoughts, to comprehend the students’ reasoning in detail. After analysing the responses, a discrepancy was found between the student’s answers and their scientific reasoning. In the end, the researchers combined and discussed the quantitative and qualitative findings to formulate conclusions and offered suggestions based on the information gathered from administering the SRTM and using Rasch analysis to analyse the results. All phases in this research are shown in Figure 1.

Figure 1. Process diagram for the mixed method research.

3.2 Participants

The participants were 43 first-year students (aged 18–19) enrolled in introductory physics courses at a private institution in Sidoarjo, Indonesia. Table 1 gives a detailed overview of the demographic characteristics of the sample.

Table 1. The participants’ demographic characteristics.

No	Demographic characteristics	Students
		Total	%
1.	Gender
	Male	20	46.51
	Female	23	53.49
2.	Domicile
	City	18	41.86
	Village	25	58.14
3.	Major
	Natural Science Education	14	32.56
	Informatics Engineering	29	67.44
4.	GPA
	2.75–3.25	15	34.88
	3.25–4.00	28	65.12

3.3 Instrument

The Scientific Reasoning Test of Motion (SRTM) is the instrument in this research. The SRTM consists of 20 items. There are ten questions on two levels. It has a follow-up question to measure the student’s scientific understanding of the process. The concepts measured by the instrument were Proportional Thinking (PPT) (items 1 and 2), Control of Variables (CV) (items 3 and 4), Probabilistic Thinking (PBT) (items 5 and 6), Correlational Thinking (CT) (items 7 and 8), and Hypothetical-Deductive Reasoning (HDR) (items 9 and 10). Three experts examined this SRTM instrument. All the experts have taught physics courses in higher education for the last five years. The evaluation criteria include the relevance and appropriateness of the items to prompt the student’s abilities of scientific reasoning on the topic of force and motion and clarity of items to the population of test-takers, such as these first-year students of STEM majors who were taking introductory physics courses. The results show that SRTM is feasible and has a percentage of agreement (R) of 83%.

Each item in the SRTM has four major components. The first element is a stimulus, which can be data, images, phenomena, or an explanation of a concept. The second is a question to prompt scientific reasoning. The third is a choice of four or five statements from which to choose as replies to prompt scientific reasoning. The last is a blank space where students can express their reasons for choosing those answers. See Figure 2 for an example of a scientific reasoning test used in this research.

Figure 2. An example of scientific reasoning test of motion.

3.4 The scientific reasoning abilities’ domains

Patterns of scientific reasoning are PPT, CV, PBT, CT and HDR. The following examples show how to conceptualise the PPT. The process entails choosing two significant extensive variables related to a problem, identifying a constant rate for intensive variables, and using provided information and relationships to either discover an additional value for one extensive variable or to compare two values of the intensive variable computed from the data as a comparison problem. CV is a method of regulating the dependent and independent variables that affect the continuity of the circumstances while testing the hypothesis. PBT refers to a situation in which one is interested in the number of times a specific procedure is carried out, leading to a particular outcome when repeated several times under the same conditions. Predictions in scientific research are made possible by CT, which refers to people’s cognitive patterns for assessing the degree of mutual or reciprocal links between variables. HDR is the quality of reasoning that organises and generates answers to problems at all stages of life.

3.5 Data analysis

The SRTM was administered to the students in the physics course to assess their proficiency in employing scientific reasoning. Assess the SRTM’s psychometric performance using Rasch analysis. Rasch analysis is used to guide the creation of rating scale instruments due to multiple causes. The justification for using Rasch analysis is that the data is ordinal and cannot be assumed to be distributed linearly. It is also employed for various additional purposes, including its ability to assess the instrument’s effectiveness in various ways (e.g., evaluating instrument performance using a Wright Map). The Winsteps computer application is used further to analyse the data (Linacre, 2012).

This research assessed the SRTM instrument’s quality by employing Rasch analysis to examine (a) dimensionality and fit and (b) item and person reliability. Table 2 displays the criteria for unidimensionality values. Table 3 provides the categories for the person value, item reliability, and Cronbach’s alpha value.

Table 2. Category of unidimensionality value.

Value of raw variance explained by measures	Description
X < 20%	Invalid
20% < X ≤ 40%	Enough
40% < X ≤ 60%	Valid
X > 60%	Excellent

Table 3. Values category of item and person reliability, and Cronbach’s alpha.

Item and person reliability		Cronbach’s Alpha
Value	Description	Value	Description
X < 0.67	Weak	X < 0.50	Weak
0.67 < X ≤ 0.80	Enough	0.50 < X ≤ 0.60	Enough
0.80 < X ≤ 0.90	Good	0.60 < X ≤ 0.70	Good
0.90 < X ≤ 0.94	Very Good	0.70 < X ≤ 0.80	Very Good
X > 0.94	Excellent	X > 0.80	Excellent

Following that, a Wright map analysis is to ascertain whether there are any gender-based distinctions in scientific reasoning. The Wright map is a logit scale representing the items and the person’s reliability. The map’s right side shows the distribution of the SRTM item difficulty levels. Conversely, the left side shows the distribution of student proficiency levels, providing each indicator’s scientific reasoning for student performanc.

The final analysis entails a qualitative description of the student’s reasoning in the concepts of force and motion. It also facilitates an investigation of the disparity between students’ reasoning and scientific reasoning regarding open-ended results. For example, students will receive an explanation of scientific concepts when they provide answers and reasons for incorrect concepts. Each indicator will be described using scientific reasoning.

4. Results

4.1 The quality of SRTM

4.1.1. Dimensionality and fit

Each question in the instrument assesses a specific concept to analyse the fit and dimensionality of the SRTM, thereby enhancing researchers’ confidence. Also, it analyses the MNSQ Outfit and Infit and the ZSTD (Outfit Z-Standar) statistics. O’Connor et al. (2016) employed a range of MNSQ from 0.5 to 1.5 and ZSTD from −2.0 to +2.0, and the results showed that all items fit (see Table 4). Moreover, the unidimensionality value is used to calculate the empirical raw variance, elucidated by procedures for calculating the validity value. The results show that the 33.6% of the raw variance explained by the measures may be attributed to unidimensionality. Further, the SRTM demonstrates a high level of validity when classified according to Table 1.

Table 4. Rasch item statistics for each SRTM construct.

Item number	Total score	Total count	Measure (Logits)	Model S. E. (Logits)	Infit MNSQ	Infit ZSTD	Outfit MNSQ	Outfit ZSTD	Pt measure correlation
PPT1	31	36	−1.57	0.52	0.97	0.02	0.58	−0.41	0.39
PPT2	24	36	−0.15	0.41	0.96	−0.14	0.81	−0.51	0.52
CV1	33	36	−2.22	0.63	1.26	0.68	1.71	0.91	0.06
CV2	34	36	−2.69	0.75	1.11	0.38	0.87	0.31	0.15
PBT1	13	36	1.60	0.41	1.11	0.66	1.04	0.22	0.47
PBT2	20	36	0.50	0.40	1.11	0.62	1.07	0.36	0.46
CT1	22	36	0.18	0.40	1.03	0.21	1.03	0.20	0.48
CT2	24	36	0.15	0.41	0.67	−1.87	0.55	−1.49	0.67
HDR1	18	36	0.81	0.41	0.90	−0.50	0.81	−0.73	0.59
HDR2	4	36	3.69	0.64	0.98	0.12	1.47	0.74	0.42

4.1.2. Item and person reliability

Additionally, Rasch’s approaches evaluate the item, subject, and Cronbach’s alpha reliability values. The person and item reliability value indicates the degree of consistency in students’ answers and the quality of the SRTM items (Sumintono & Widhiarso, 2014). On the other hand, Cronbach’s alpha evaluates the reliability and depicts the connection between a person and an item. The SRTM instrument had an item reliability of 0.91, a person reliability of 0.57, and a Cronbach’s alpha of 0.57. Regarding the criteria in Table 2, it can be inferred that the item reliability of the SRTM was very good, whereas the personal reliability was weak. It means that the consistency of the students’ answers was poor, although the SRTM items were of high quality.

Table 3 of the Cronbach’s alpha value indicates that the interaction between person and items in the SRTM is within the acceptable range. The main factor contributing to the SRTM’s weak reliability is the different backgrounds of the first-year physics students from various senior high schools. Several students attended public senior high schools, while others were from private schools that offered customised instruction. Awofala and Lawani (2020) stated that different instruction may strongly affect a student’s foundation of scientific knowledge and reasoning skills.

4.1.3. Wright Map interpretation

The Wright Map (or the item-person map) visually depicts Rasch’s analysis results. It is to depict the connection between the items on a test and the ability level of the test-takers. Figure 3 displays a map where the left represents the students’ abilities, while the right represents the item difficulty. Iramaneerat et al. (2008) determined that higher logits are indicative of students with more excellent abilities (on the left side) and items that are more challenging (on the right side). We can enable the variable map to determine whether the items align with the student’s ability.

Figure 3. A Wright map for the Scientific Reasoning Test of Motion (SRTM) data collected from first-year Student University students attending physics courses.

The test items are assigned an average of 0 logits (Iramaneerat et al., 2008). Based on the variable map, most students score below the mean of the test items. Only a few students with more remarkable aptitude are assigned a value of +3.69 logits, while a few less capable students are assigned a value of −1.33 logits. These logits’ values are determined based on the highest and lowest metrics. Thus, 23 out of 34 students, equivalent to 67.65% of the student population, possess adequate scientific reasoning abilities, as they can answer the questions and find the required items.

Figure 3 shows that the HDR 2 indicator, with a logit scale of +3.69, is the most challenging item for students. It is followed by the PBT 1, HDR 1, PBT 2, and CT 1 indicators, which have logit scales of +1.60, +0.81, +0.50, and +0.18, respectively. In comparison, students find CV 2 the simplest or the most well-understood, with a logit scale of –2.69. Following CV 2, the indicators of CV 1, PPT 1, CT 2, and PPT 2 are relatively easier. It means that PPT and CV are the most effective indicators of a student’s proficiency in scientific reasoning. The scientific reasoning concepts students struggle with are HDR, PBT, and CT.

4.2 Rasch analysis-based gender inequalities in the scientific reasoning

The results of analysing the scientific reasoning of participant’s genders for each indicator are displayed in Figure 4. Figure 4 shows no difference between genders in the CT1, CT2, PPT2, and PBT2 indicators. In other words, male and female students demonstrate equal levels of scientific reasoning. Male and female students have different abilities in scientific reasoning as measured by the indices CV1, CV2, PPT1, HDR1, PBT1, and HDR2, respectively. Male students perform at a higher level of proficiency in PPT1, PBT1, and HDR2 indicators than female students, as seen by a higher DIF measure. This indicates that males possess greater aptitude for completing these tasks than females. However, female students outperform male students on the CV1, CV2, and HDR1 indicators because they have higher DIF measurements, which gives them an advantage when answering questions requiring scientific reasoning.

Figure 4. DIF based on gender.

4.3 Evaluation of each scientific reasoning indicators

A qualitative analysis of the scientific reasoning used in each indicator is to ascertain how students’ answers align with the existing scientific concepts. The SRTM items are organised based on scientific reasoning indicators. Specifically, items 1 and 2 are used to measure PPT, items 3 and 4 are for CV, items 5 and 6 are for PBT, items 7 and 8 are for CT, and items 9 and 10 are for HDR.

4.3.1. Proportional thinking (PPT)

Table 4 displays the findings of the student’s scientific reasoning patterns on the proportional reasoning indicator, with the items PPT1 and PPT2 receiving logit scales of −1.57 and −0.15, respectively. The results show that PPT 1 and PPT2 are more understandable than other indicators of scientific reasoning. PPT is a reasoning pattern that assesses students’ aptitude for establishing connections between variables using data, equations, or figures. The first and second items question about uniformly accelerated motion. Students must select the appropriate graph for the second item based on the provided ticker timer data. The results show that both male and female students can interpret the ticker timer data, demonstrating that the distance between points is progressively increasing and that objects are undergoing uniformly accelerated motion. However, a few students have struggled to determine the exact graph representing uniformly accelerated motion. Figure 5 displays an example of a student’s answer.

Figure 5. An example of a student’s answer on item 2.

4.3.2. Control of variables (CV)

The third and fourth items (see Figures 6 and 7) are questions about analysing the correct experimental design for investigating the impact of the track board’s slope on the travel time objects and its effect on the applied force on the object’s average speed. These items assessed students’ aptitude in identifying experimental variables. The results showed that students accurately discerned the relationship between manipulation and response variables. It can be seen that students are aware that the track board’s slope affects the duration of the car’s trip, although their reasoning may not align with scientific concepts, and they may have misconceptions. In Figure 6, the students demonstrated a lack of understanding that the experimental design was intended to validate Newton’s Second Law, which postulates that a physical object’s acceleration rate is proportional to the net force acting upon it and inversely proportional to its mass. As more force is applied to an item, its acceleration rises. This phenomenon can be expressed by the equation $W=\mathit{\text{mg}}\mathrm{\text{sin}}\hspace{0.17em}\theta ,$ which describes the object’s motion, not $N=\mathit{\text{mg}}\mathrm{\text{cos}}\hspace{0.17em}\theta .$ In other words, students have not been able to apply Newton’s second law in a real-world context.

Figure 6. An example of a student’s answer on item 3.

Figure 7. An example of a student’s answer on item 5.

4.3.3. Probabilistic thinking (PBT)

Probabilistic thinking (PBT) is a student’s skill in predicting the opportunities generated when repetition occurs (Erlina et al., 2018; Ludwin-Peery et al., 2020). The fifth and sixth items measure students’ ability to predict the position of objects that experience free fall motion and the direction of a ball’s motion when its string breaks after being rotated vertically. Figure 2 shows that PBT 1 and PBT 2 have logit scales of +1.60 and +0.50, indicating that these two items were challenging for students to complete. A sample of a student’s response to item number 5, which assesses the PBT 1 indicator, is shown in Figure 7. The findings reveal that students could correctly estimate the items’ positions in free fall motion. Students were able to use the free-fall equations of motion correctly but were unable to implement these equations into non-routine problems. Nasir et al. (2021) contend that undergraduate students poorly master non-routine problem-solving skills in their first and last years.

4.3.4. Correlational thinking (CT)

Correlational thinking is the student’s aptitude for creating mutual relationships between variables (Bao et al., 2022). The seventh and eighth items measure students’ ability to determine the interrelationships of variables in projectile and free fall motion. Figure 2 shows that the logit scale values acquired to see both male and female students typically find working on the CT 2 (Free Fall) questions simpler than the CT 1 (Projectile) topics. The possible reason is that students master the concept of free fall more than the concept of projectiles. In item CT 2, students determine the relationship between the time variable and the object’s weight in free fall. The finding demonstrated that the students already knew that all moving objects in free fall would experience the same gravitational acceleration, namely g = 9.8 m/s². In the CT1 item, students are asked to choose variables related to the initial velocity in the horizontal direction (v_ox), and students tend to answer that there are no influencing variables (see Figure 8). Few students can determine the variables that affect the initial velocity of objects in projectile motion. Female students can memorise projectile equations of motion but have not been able to implement these equations into non-routine problems. According to Lazonder and Harmsen (2016), guidance is necessary for inquiry learning to assist students in solving non-routine questions.

Figure 8. An example of a student’s answer on item 7.

4.3.5. Hypothetical-deductive reasoning (HDR)

Hypothetical-deductive reasoning (HDR) is students’ ability to formulate hypotheses from general concepts to specific ones. The ninth and tenth questions test students’ ability to plan experiments to ascertain the variables influencing an object’s travel time in a free-falling motion (HDR1) and the factors influencing an object’s horizontal fall in a parabolic motion (HDR2). Based on the logit scale that was acquired, HDR2 has the most extraordinary logit scale (+3.69), indicating that the item that students find to be the most difficult, followed by HDR 1 with a logit scale (+0.81), shows that the item that students find to be the easiest.

Figure 9 displays a sample of a student’s response to question number 10 (HDR2). Their responses showed that students still have misconceptions regarding the concept of projectile motion, where they have not been able to compile a hypothesis about the relationship between the height of an object (h₂) and the horizontal distance travelled by an object. Students had a hard time grasping the idea of parabolic motion. It is consistent with other research that explains how students frequently have misconceptions regarding projectile motion (Erlangga et al., 2021; Hidayatulloh et al., 2021; Levin et al., 1990).

Figure 9. An example of a student’s answer on item 10.

5. Discussion

This research provides a framework for analysing students’ scientific reasoning, particularly at the undergraduate level, and how gender affects reasoning patterns. Table 4 shows that SRTM exhibits promising psychometric features regarding the Rasch analysis. The SRTM items, which assess students’ reasoning aptitudes in force and motion, generally perform as expected. The SRTM instrument also has the requisite dimensionality, as evidenced by the fit analysis, which indicates that each item assesses a unifying construct. This finding supports the SRTM’s reliability as a measurement tool for scientific reasoning. Researchers can confidently use this instrument in similar investigations or interventions to assess the mentioned abilities. It also has implications for future studies.

Two outcomes of the Rasch analysis presented in Figure 4 depict how gender influences scientific reasoning patterns. First, gender does not influence correlational reasoning (CT) patterns. However, there are differences between male and female proficiencies in scientific reasoning, especially in specific markers such as control variables (CV), probabilistic reasoning (PBT), proportional reasoning (PPT), and hypothetical-deductive reasoning (HDR). These results align with those studies which confirmed that there are gender differences in scientific reasoning ability, such as Sagala et al. (2019), Kersey et al. (2018), and Nieminen et al. (2013).

Analysing each scientific reasoning indicator also provides essential insights into students’ knowledge and misconceptions regarding various scientific reasoning concepts. The results of Wright’s map analysis (Figure 3) indicate that the logit scales for PPT 1 and PPT 2 are −1.57 and −0.15, respectively. These results demonstrate that students readily comprehend the proportional thinking indicators. Students have grasped the concept of accelerated motion with constant acceleration when associated with motion material. Students can interpret the data on the ticker timer in an appropriate graphical form. It is because of the often use of multiplicative relationships in their everyday lives. The ability to comprehend relationships between quantities and their proportions is a hallmark of the concrete operational stage, which is when Piaget claims that proportional thinking usually manifests. This is reinforced by Gagné’s hypothesis, which highlights the importance of cognitive functions including rehearsal, elaboration, and attention in achieving proficiency on tasks involving proportional reasoning. According to Misnasanti et al. (2017), proportional reasoning ability involves the relationship between multiplication and the ability to apply concrete operations in a formal context, which can be developed using students’ existing experiences. Figure 4 portrays a discrepancy in performance between male and female students on the PPT 1 indicator, with male students exhibiting a more excellent proficiency than their female counterparts. It was discovered that males were more adept than females at developing and applying theories (Nieminen et al., 2013).

In Figure 3, the logit scales for CV 1 and CV 2 are −2.22 and −2.69, respectively. These figures indicate that students have mastered the control variables indicators. The CV is the ability to construct an adequate experimental design, identify appropriate variables, interpret valid data, and understand the many components’ role in the experimental system (Lazonder & Egberink, 2014; Van Vo & Csapó, 2021). According to Piaget’s theory, people may conduct systematic experiments and comprehend cause-and-effect linkages when they have the ability to manipulate variables, which appears during the formal operational stage. According to Gagné’s theory, mastery of control over variables requires both practice and feedback. Moreover, Figure 4 indicates that female students are better at identifying experimental variables than male students, representing a significant difference in scientific reasoning ability in the lens of CV2 and CV 1 indicators. These results are similar to Tairab’s (2015) findings, which demonstrated that more female than male students could provide correct answers and reasoning to the sub-idea “selecting appropriate experimental settings.”

The Wright map analysis result in Figure 3 shows that the logit scale for PBT 1 and PBT 2 are 1.60 and 0.50, respectively. This result demonstrates that the problems encountered by the students in solving PBT1 indicators are complex. PBT 1 problems pertain to students’ understanding of free fall motion. Students are assigned to predict the position of an object in free fall at the second. Nevertheless, students tend to rely solely on mathematical calculations to determine distance, with less emphasis on the position. As indicated by Piaget’s theory, students might still be improving their probabilistic reasoning skills, particularly in difficult situations like figuring out where an object will fall in free fall. This validates Piaget’s theory that adolescents continue to develop their capacity for abstract reasoning throughout adolescence.

It is evident that male students exhibited a higher DIF in the performance of the PBT1 indicators than their female counterparts (read Figure 4). It indicates that males are more proficient in these tasks than females. This research’s outcomes are pertinent to the study by Sari et al. (2017), who posited that male students exhibited multi-structural probabilistic thinking in completing probability tasks, whereas female students demonstrated unstructured probabilistic thinking. Similarly, Valanides (1996) emphasised that males outperformed females in probabilistic reasoning. However, both male and female students have equal abilities to solve question 6 or the PBT 2 indicator.

The logit scale for CT 1 and CT 2 in Figure 3 indicates that students still had difficulties solving problems related to CT1, which requires them to determine the quantities regarding the initial velocity in the horizontal direction in projectile motion. According to Hidayatulloh et al. (2021), students hold numerous misconceptions about projectile motion in the context of graphic and visual representations. In order to effectively address correlation-related challenges, students may need to acquire cognitive processes including problem-solving and decision-making, as defined by Gagné’s Information Processing Theory. On the one hand, there is no discernible difference between male and female students in CT 1 and CT 2, as illustrated in Figure 4. On the other hand, Yenilmez et al. (2005) found that females scored higher on controlling variables and correlational reasoning. Hence, further qualitative research is required to ascertain the potential causes of these findings.

In addition, the student population encountered significant HDR difficulties. This finding aligns with research by Erlina et al. (2018) and Novia and Riandi (2017), which found that students experienced considerable difficulty with HDR. In considering the need to construct and test hypotheses, Piaget’s theory indicates students may still be developing their capacity for abstract and organized thought. The HDR1 and HDR2 indicators demonstrated significant gender-based differences. Male students performed a higher DIF measure for the HDR2 indicator than their female counterparts, suggesting that males may possess a more excellent aptitude for completing this task than females. Conversely, female students performed better than male students on the HDR1 indicator, conferring them a competitive advantage in addressing problems requiring scientific reasoning. The findings of Radulović and Stojanović (2019) also showed no statistically significant correlation between gender and HDR. Future research is recommended to elucidate the underlying causes of these discrepancies and investigate strategies to foster scientific reasoning skills equally among male and female students.

6. Conclusion

A Rasch analysis indicates that the SRTM questions, which assess students’ reasoning abilities in force and motion, exhibit favourable psychometric features. Each item effectively assesses a single underlying construct and demonstrates high levels of reliability with minimal dependency. Findings of a Wright map uncover that HDR, PBT, and CT are indicators of scientific reasoning that present significant challenges for students, whereas PPT and CV are more readily accessible. The results of the Rasch analysis showed differences in scientific reasoning between male and female students on the CV, PPT, HDR, and PBT indicators. However, no significant differences were observed for the CT indicators. Regarding the qualitative analysis, students’ answers showed misconceptions about projectile motion and difficulties in applying Newton’s second law to practical issues. It highlights the necessity for a learning strategy to improve students’ understanding of motion and force concepts.

It must be acknowledged that this research has significant limitations. Due to the limited sample size and the inadequate analytical strategy, the results may not represent the larger population. Rasch analysis can provide helpful insights in some situations, but it may restrict the depth of understanding of the data. It is known for its ease of use and capacity to transform ordinal data into interval-level measurements. However, it has the potential to oversimplify the intricacies of the underlying components under investigation, potentially missing subtle patterns or interactions. Therefore, it is recommended that future research use a bigger and more varied sample size and investigate complementary analytical techniques to provide a more comprehensive and reliable analysis.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Noly Shofiyah

Noly Shofiyah is a PhD student in the natural science education study program at Universitas Negeri Surabaya (Unesa). Her main research interest is 21st century skills such as scientific literacy, collaboration skills, and scientific reasoning, as well as innovative learning such as inquiry, PjBL, and STEM.

Nadi Suprapto

Nadi Suprapto is a Professor at the Departement of Physics, Faculty of Mathematics and Natural Science, Universitas Negeri Surabaya (Unesa). His research interests in the physics literacy, physics education, 21st Century teaching and learning, assessment, cultural studies, curriculum, and philosophy of science.

Binar Kurnia Prahani

Binar Kurnia Prahani is a Associate Profesor at the Departement of Physics, Faculty of Mathematics and Natural Science, Universitas Negeri Surabaya (Unesa). His research interests in the physics literacy, physics education, 21st Century teaching and learning, computer science, and Critical thinking.

Budi Jatmiko

Budi Jatmiko is a Professor at the Departement of Physics, Faculty of Mathematics and Natural Science, Universitas Negeri Surabaya (Unesa). His research interests in the physics literacy, physics education, 21st Century teaching and learning, computer science, and thinking skills.

Desak Made Anggraeni

Desak Made Anggraeni is a PhD student in the natural science education study program at Universitas Negeri Surabaya (Unesa). Her main research interest is Education, Thinking skill, e-learning, Technology in education

Khoirun Nisa’

Khoirun Nisa’ is is a student at the Departement of Physics, Faculty of Mathematics and Natural Science, Universitas Negeri Surabaya (Unesa). Her research interests in assessment, physics education, and cultural studies.

Appendix:

An example of SRTM

A.1 PBT 2 item

A.2 CV 2 item