653
Views
0
CrossRef citations to date
0
Altmetric
Information & Communications Technology in Education

Effects of classroom response system on the achievement and knowledge retention of the students in mathematics

ORCID Icon & ORCID Icon
Article: 2323364 | Received 30 Nov 2023, Accepted 21 Feb 2024, Published online: 11 Mar 2024

Abstract

Classroom Response System (CRS) is a promising educational tool that can create an interactive learning space. In high school, less is known about its impact on learning mathematics. Most studies on the integration of CRS in teaching and learning use internet-dependent tool. This paper aimed to determine the effectiveness of the CRS that uses Bluetooth technology in improving the mathematics achievement and knowledge retention of the students. A quasi-experimental design and longitudinal study with post-test and delayed post-test was employed in this study. The assessment and re-administered assessment results of the CRS and Traditional Classroom Instruction (TCI) groups were compared. Data were analyzed using an independent sample t-test and Welch t-test, and the Mean Percentage Scores (MPS) were calculated to determine the achievement level. Findings revealed that there was a significant difference between the assessment of the CRS and the TCI groups. Moreover, the re-administered assessment results suggest that students exposed to CRS have better knowledge retention compared to the TCI. Furthermore, the MPS showed that CRS exhibited higher mastery level of the learning competencies over the TCI group. This study provides valuable insights into the integration of CRS in mathematics teaching and its consequential effect on achievement and retention.

1. Introduction

1.1. Background of the study

Educational technologies are becoming powerful tools in transforming and promoting active classroom climate to support student learning (Baako & Abroampa, Citation2023; Christopoulos & Sprangers, Citation2021; Haleem et al., Citation2022). Numerous studies have shown that technology-supported learning environments positively impact the students’ achievement (Nagel, Citation2009; Poçan et al., Citation2023; Sitthiworachart et al., Citation2022), motivation and engagement (Francis, Citation2017; Higgins et al., Citation2019), and improves knowledge retention (Shaikh & Algannawar, Citation2018). Most of the students today are considered digital natives (Kiryakova et al., Citation2014) and the teacher should take advantage of their love of technology to refocus education (Rosen, Citation2011). Tech-savvy students learn best when they interact with both technology and peers. Maximizing students’ participation promotes meaningful learning (Stowell & Nelson, Citation2007). Hence, 21st century teachers should use alternative teaching strategies that work better than traditional classroom instruction (Hassidov, Citation2017; Kohn, Citation1999). One of the viable solutions to maintain a dynamic learning space is to use a classroom response system (Premkumar & Coupal, Citation2008; Shaikh & Algannawar, Citation2018).

A Classroom Response System (CRS) is a platform that can be used to poll students which can instantly collect responses to questions posed by instructors (Deal, Citation2007; Firsing et al., Citation2017; Hammil, Citation2011; Wood, Citation2004). In CRS-facilitated instruction, students are polled and given instant feedback on their answers (Deal, Citation2007). Students can respond to the questions anonymously, which makes CRS more appealing. Responses are sorted out and presented in a bar chart indicating the number of students who selected each option. Questions flashed on the screen can range from multiple-choice, true or false, or even open-ended questions. This platform provides a real-time assessment of how well the student understood the instructions delivered and how well they perform against other students.

The CRS, also known as the ‘student’s response system [SRS]’ or ‘audience response system [ARS]’ or simply ‘clickers’, gained popularity in tertiary education since the 1960s (Deal, Citation2007). Since then, there has been a paradigm shift in education from teacher-centered approach to student-centered approach. Kozanitis and Nenciovici (Citation2022) underscored that CRS is among the active learning approaches frequently used in humanities and social sciences that leads to higher achievement compared to traditional classroom instructions. On the other hand, Freeman et al. (Citation2014) reported that active learning through the use of CRS in science, technology, engineering, and mathematics (STEM) yield to greater students performance over the traditional teaching approach. Thus, many teachers including students have embrace the use of CRS in classes in all disciplines (Herrada et al., Citation2020).

Numerous benefits of CRS were mentioned in a literature review conducted by Aljaloud et al. (Citation2015) which highlighted that CRS facilitates learning and develops a deeper conceptual understanding; it increases students’ achievement; improves instructional methods; enhances short-term and long-term memory; and encourages students to do self-reflection. Similarly, Wang and Tahir (Citation2020) literature review on CRS underlined positive effects on student performance, classroom dynamics, and student and teacher receptions. In the most recent exhaustive multidiciplinary litereature review on CRS, Herrada et al. (Citation2020) stated that CRS is a ‘pedagogical resource’ that is favorably regarded by the students and teachers because of its impact on learning, engagement, and interactivity. Although there were challenges in integrating CRS in classroom mentioned by teachers and students like unstable internet connection, time pressure for giving answer, and, time to prepare the material (Wang & Tahir, Citation2020), the benefits of CRS outweigh all these challenges (Herrada et al., Citation2020).

Aside from the affordance of CRS to foster active learning and provide prompt feedback during formative assessment, CRS has been found to be useful in all class sizes. Caldwell (Citation2007) affirms that even in a large classroom, CRS can boost the achievement of low-performing students. Also, Zullo et al. (Citation2011) reported that CRS use in smaller group with at least 15-40 students has an equal positive impact on learning in larger classes. Regardless of class size, research have shown that active learning generates greater achievement than the traditional classroom instruction (Freeman et al., Citation2014; Kozanitis & Nenciovici, Citation2022). Moreover, the CRS class perform significantly better in assessment compared to those students attending traditional lectures (Garcia-Lopez & Garcia-Cabot, Citation2022; Shaffer & Collura, Citation2009) and the CRS group has a ‘more consistent level of comprehension’ than the traditional class (Poulis et al., Citation1998, p. 441).

The use of CRS promotes a deeper understanding of the materials which, in turn, improves knowledge retention (Hammil, Citation2011; Radosevich et al., Citation2008; Wang et al., Citation2014). Investigations on the impact of CRS on knowledge retention were mostly carried out in higher education. In the study of Owen and Licorish (Citation2020), undergraduate students claimed that CRS enhanced their knowledge retention. Although most of the students perceived that CRS improved their knowledge retention, there was no assessment made to quantify the retention rate after some time. On the other hand, Radosevich et al. (Citation2008) reported a statistically significant difference in the retention test of the CRS and traditional groups given six weeks after the same exam was administered. Similarly, Pradhan et al. (Citation2005) found that medical students who were exposed to CRS fared better in the post-test and ‘showed a 21% improvement between the pretest and posttest scores’ than those students attending the traditional lecture with only a 2% improvement. In addition, Liu and Stengel (Citation2011) found that CRS improves students’ knowledge retention and examination performance in mathematics-related courses. The researchers emphasized that feedback plays a big part in improving student retention (Liu & Stengel, Citation2011). In contrast, Doucet et al. (Citation2009) found that the retention rate on tests given after a year was not significantly different between groups even though the final exam results of the CRS group were higher than the traditional class. The researchers argued that the CRS’ impact on long-term retention is still unclear. Shapiro et al. (Citation2017) claimed that CRS can only promote fact retention, but not conceptual understanding.

1.2. Statement of the problem

Keeping students’ attention and enhancing student-teacher interaction during mathematics instruction while targeting greater mastery of learning competencies poses an enduring challenge to high school mathematics educators across the globe. Mathematics teachers can use technology to support conceptual development and stimulate problem-solving and critical thinking (Liu & Stengel, Citation2011). According to Gustafsson and Ryve (Citation2022) teachers can use and design CRS tasks to encourage high school students to engage in discussion, analyze problems, and reason out instead of doing rote calculations or memory recalls. Extensive research were conducted evaluating the impact of CRS on achievement and knowledge retention of undergraduate students. To date, there is limited scholarly work on CRS use in secondary education mathematics classrooms but not in higher education (Hammil, Citation2011).

The quality of learning when CRS is integrated in high school mathematics has been observed to be superior to traditional classroom instruction. It has been reported recently that students’ level of satisfaction in using CRS in mathematics is significantly higher (Curto Prieto et al., Citation2019). Using CRS in mathematics does not only improve engagement and increases participation but can potentially influence students’ mathematics achievement (Homewood et al., Citation2008-2009). Several studies, such as those conducted by McCumiskey (Citation2010), Hammil (Citation2011), and Wang et al. (Citation2014), have compared the CRS and traditional classes which do not use CRS. These studies consistently found that the class exposed to CRS performed significantly better on the mathematics assessment than the traditional class. Interestingly, CRS have been found to have a positive impact on low-performing students in mathematics (Hammil, Citation2011) and has the potential to improve the short-term and long-term memory of the students (Wang et al., Citation2014). The increase in students’ engagement and assessment scores can be attributed to the fact the CRS can afford real-time feedback which does not normally happen in a traditional classroom instruction (McCumiskey, Citation2010; Wang et al., Citation2014). Moreover, Manuel (Citation2015) confirmed that high school students who received immediate feedback using CRS have better mathematics achievement than those who did not received instant feedback in traditional class.

With the promising benefits of CRS in learning mathematics, less is still known about its impact on learning and knowledge retention in high school mathematics classroom. Analyses of the previous studies revealed that mathematics achievement and knowledge retention were often measured and described using the mean gained scores. Although assessments were designed to assess student’s mastery across the cognitive domains, the achievement level per learning competency was not thoroughly examined. In addition, there was no in-depth item analysis of the assessment results of CRS and traditional classes, which may uncover possible misconceptions and procedural errors exhibited by the students and determine whether students are able to demonstrate mastery of competencies that require low-order thinking skills to higher-order thinking skills. Thus, our understanding of the effects of CRS on the achievement and knowledge retention of students in mathematics is very limited. This study not only contributes to understanding of how CRS can be used to enhance student achievement but also provides a deeper analysis of knowledge retention skills of students in both CRS and traditional classes. To our knowledge, there is also a lack of scholarly work about the CRS being used in the Philippine classroom setting.

On the other hand, most of the CRS technologies used in various studies required internet connectivity and remote transmitter or clickers which are expensive (Deal, Citation2007). Today, many applications are available on all major mobile platforms that support classroom response systems for free, one of the exceptional CRS applications is the Blicker. The Blicker is a Bluetooth-based CRS application which can be used to promote active learning without needing to connect to internet (Tan, Citation2016). This mobile application was used for this study since it is meant for schools in developing countries with no internet access and there is no study at present that evaluates its effectiveness.

This study adds to the current discussion about the CRS benefits by using a novel classroom response system that only uses Bluetooth. By comparing two classes, the Classroom Response System (CRS) group and Traditional Classroom Instruction (TCI) group, more can be understood about its impact on the academic performance of the students in secondary mathematics. With these identified literature gaps, the researchers were prompted to conduct further investigation on the effects of the classroom response system on the students’ mathematics achievement and explores whether the classroom response system can enhance retention skills.

1.3. Purpose of the study

This study aimed to examine the effect of the classroom response system on the achievement and knowledge retention of students in mathematics. Specifically, it sought to answer the following questions:

  1. Is there a significant difference between the assessment results of the students exposed to the Classroom Response System and those of the Traditional Classroom instruction?

  2. Is there a significant difference between the re-administered assessment results of the students exposed to the Classroom Response System and those of the Traditional Classroom instruction?

The following hypotheses for this research were established as follows:

Hypothesis 1 (H1): CRS group has greater achievement level than TCI group.

Hypothesis 2 (H2): CRS group has better knowledge retention than TCI group.

2. Literature review

2.1. Why use blicker?

'Blicker (which stands for ‘Bluetooth’ + ‘Clicker’) is a revolutionary classroom response system that uses the new Bluetooth Low Energy Standard for student and teacher interaction’ (Tan, Citation2016). Unlike other applications that support classroom response systems such as Plickers, Kahoot!, Poll Everywhere, Mentimeter, clickers, and the like, the Blicker requires no internet connection. The new Bluetooth Low Energy technology made Blicker a battery-friendly app, it enables smooth Bluetooth pairing, and eliminates the use of the internet. The Interactive Studio who developed this application aimed at eliminating all the disadvantages of the existing classroom response system and combining all their benefits (Tan, Citation2016). For teacher and student interaction to happen, the Blicker for Teacher and Blicker for Student must be downloaded and installed in the teacher’s laptop and student’s device, respectively. The Blicker for Teacher comes with more than 20 different teaching tools which can be used to promote an active learning environment, such as spinning wheel, randomizer, broadcast emails, MCQ Response Mode, and even automated attendance. On the other hand, Blicker for Student can be downloaded freely and installed in the student’s mobile devices and use it to response anonymously to the polls posted before them. Tan (Citation2016) further claimed that this tool can promote participation and increase productivity and is suitable for schools in developing countries.

Most studies on CRS made use of such tools that require internet connections or a handheld transmitter. These types of CRS pose challenges to teachers and students alike which can hinder smooth interactions between students and teacher. Wang and Tahir (Citation2020) pointed out that unstable internet connection is one of main challenges when using CRS in classroom. Thus, using Blicker in mathematics instruction provides new insights about pedagogical advantages of this tool.

2.2. Use of classroom response system in a high school mathematics classroom

Studies on the use of CRS in secondary education mathematics classrooms are inadequate. In this section, similar studies on the use of CRS in the high school mathematics classroom were explored to substantiate the present study on the effects of the classroom response system on the students’ academic performance.

Homewood et al. (Citation2008-2009) reported that the use of CRS in mathematics positively impacts engagement and can potentially influence students’ mathematics achievement. In the collaborative inquiry, CRS were employed in grade 9 applied math class and were assessed at the end of the semester. The assessment results show that the mean score is comparatively higher than the previous semester class who took the same exam but did not use CRS. Although it cannot be seen as casual and substantial, the extensive use of CRS can improve the students’ academic performance. The inquiry team suggests that CRS can be effective when using for assessment for learning.

McCumiskey (Citation2010) found that the class exposed to the CRS performed higher than the group who did not use it by about 12.1% in the unit test in Geometry. CRS was used at the end of every lesson to provide opportunities to practice math skills over the two units while the students in the traditional instruction were given the same problems and solved it on a piece of paper then feedback was given. McCumiskey highlighted that the increase in assessment scores and engagement can be attributed to the fact that the CRS can provide many re-teaching opportunities, it can afford immediate feedback, instructions can be altered instantly, and it save instructional time. Furthermore, the researcher recommended that CRS should be incorporated into review lessons or warm-ups to measure students’ prior knowledge before starting a lesson.

Hammil (Citation2011) affirmed that low-performing students in mathematics greatly benefited from CRS. In this study, the CRS was used in both Geometry and Algebra II classes and compared their assessment results with the two traditional Geometry and Algebra II classes. Results revealed that ‘the geometry class did see a difference in test scores when comparing the CRS to the traditional classes for the B, C, and F grades’ (Hammil, Citation2011, p. 29). This indicates that struggling students in geometry class who were exposed to CRS were performing well than those students in traditional class. Meanwhile, there was no difference in the student achievement in Algebra II classes. Hammil argues that the assessment results are inconclusive and may be affected by some other factors and suggests further study of CRS' effect on student learning.

According to Wang et al. (Citation2014), classroom response system improves short-term memory and long-term memory of students through real-time feedback. CRS not only promotes active learning but enables instant feedback, which does not normally happen in a traditional classroom instruction. The scholars claim that instantly giving feedback is effective in improving student learning in mathematics. Hence, the class exposed to CRS performs significantly better on the assessment than the traditional class (Wang et al., Citation2014).

Likewise, Manuel (Citation2015) examined the effects of immediate feedback using CRS in mathematics achievement in pre-calculus subjects in the senior high school. Students who received timely feedback using CRS exhibited higher mathematics achievement compared to those students who did not receive immediate feedback. With the overwhelming impact of CRS on student achievement, the researchers emphasized the need to discover further the extent to which CRS can benefit and enhance traditional learning approaches.

2.3. Impact of classroom response system on the student’s achievement and knowledge retention

Research on the effectiveness of CRS in enhancing student’s achievement and knowledge retention showed compelling evidence why such pedagogical resources are gaining favourable receptions across educational settings. Among the greatest strengths of CRS is its capacity to provide immediate feedback which boosts academic achievement (Caldwell, Citation2007; Cantero-Chinchilla et al., Citation2020; Mula & Kavanagh, Citation2009; Turan & Meral, Citation2018; Wang & Tahir, Citation2020) and enhance short-term and long-term memory (Owen & Licorish, Citation2020; Aljaloud et al., Citation2015).

Deal (Citation2007) presented a paper that summarized existing studies on the use of CRS in the classroom and its impact on students’ achievement. One of the notable results on CRS's effect on students’ achievement is the study conducted by Hall and his group at the University of Missouri that compared the grade distribution for the semester with CRS to a previous semester without CRS. ‘The percentage of students earning A’s increased from 23% to 40%, and the percentage of students receiving C’s or D’s in the course decreased from 34% to 21%’ (Deal, Citation2007, p. 4). While they acknowledge a lack of specific control measures to assure consistent grading standards and account for student ability across semesters, they report that grades were substantially better in semesters with CRS.

Caldwell (Citation2007) affirms that CRS improve the academic performance of students even in a large classroom. The use of CRS ‘increased the number of A’s earned by 4.7% and decreased the combined proportion of students earning D’s, F’s’ (p.13). These suggest that active engagement can boost the achievement of students who are at risks of failing. Thus, CRS can positively influence the student’s academic performance.

Another study conducted by Poulis et al. (Citation1998) evaluated the performance of students exposed to classroom response system and those of the traditional class. Results revealed that the passing rate for CRS sections was above 80% while for traditional sections was less than 60%. Additionally, ‘the standard deviation was substantially lower in the CRS group, indicative of a more consistent level of comprehension throughout any given class’ (Poulis et al., Citation1998, p. 441).

Improving students’ long-term knowledge retention is a challenge to many educators (Lindsey et al., Citation2014). Watkins (Citation2019) defined knowledge retention as ‘the process by which new information is transferred from short-term to long-term memory’. It involves remembering basic concepts and information and making sense of the knowledge acquired long after the materials were given. According to Edgar Dale’s cone of learning, the average retention rate increases as the students get involve in the learning process (Anderson, Citation2006). Hence, it is empirical that teachers select the appropriate instructional method and tools to promote active learning in order to increase knowledge retention.

CRS enables students to deepen their conceptual understanding and improve their knowledge retention skills. Radosevich et al. (Citation2008, p. 4) reported that there was a ‘statistical significant (t (143) = 5.40, p < .01) difference between the CRS group (M = 48.47) and the traditional group (M = 34.86) on the retention test’ given six weeks after the same exam was administered. Though the scores of both groups decreased, the use of CRS can increase students’ knowledge retention.

Pradhan et al. (Citation2005) conducted a similar study evaluating the influence of CRS on knowledge retention with obstetrics and gynecology residents for six weeks. Study revealed that residents who were exposed to CRS fared better in the posttest and ‘showed a 21% improvement between the pretest and posttest scores’ than those residents attending traditional lecture with only 2% improvement. These results confirm that CRS is an efficient teaching tool in strengthening retention.

Liu and Stengel (Citation2011) studied the impact of CRS on student retention and examination performance in mathematics-related courses and found that the students performed better in the weekly assessment than those who did not use it. The researchers argued that the increased in examination performance of the CRS group was not about that the group was given more opportunities to practice the math skills than the traditional class. The traditional class had the same math exercises as the CRS group only that it was given in the form of homework. Hence, feedback plays a big part in improving student’s retention.

In contrast, Doucet et al. (Citation2009) investigated the effect of CRS on long-term retention and found that the retention rate on tests given after a year was not significantly different between groups even though the final exam results of the CRS group were significantly higher than the traditional class. The results of the study could not confirm a positive impact of CRS on long-term retention. Likewise, Shapiro et al. (Citation2017) argued that CRS can only promote fact retention but not conceptual understanding.

In recent study, Owen and Licorish (Citation2020) reported that students percieved greater achievement and knowledge retention when exposed to CRS instructions. However, it does not accurately quantify the extent of CRS’ impact on knowledge retention. With these inconsistencies, there is a need for further investigation and comprehensive analysis of the effect of CRS on knowledge retention.In light of the current literature, the researchers of the present study hypothesized that students exposed to CRS-facilitated instruction have a greater achievement level than those students exposed to traditional classroom instruction. Similarly, it is posited that the CRS group has better knowledge retention than the TCI group. These research hypotheses will guide the researchers in examining the subsequent effects of the classroom response system on achievement and knowledge retention of students in mathematics.

3. Methodology

3.1. Research design

This study used a quasi-experimental design and longitudinal study with post-test and delayed post-test to compare the academic achievements of the CRS and TCI groups. This design is suitable in evaluating the difference in the academic achievement of students without doing randomization (Cook & Campbell, Citation1979; Cresswell & Cresswell, Citation2018). Thirty (30) grade 11 students were involved in this study. The students were divided in two groups of 15 students prior to the opening of classes for the purpose of limited face-to-face instruction and these groups were randomly assigned to the CRS group and the other to the TCI. The CRS group made use of the Blicker application during mathematics lessons while lecture method was employed in the TCI group during mathematics instructions covering the same learning competencies for simple and compound interests for two weeks. Using multiple-choice questions, students in the CRS group were polled then students were given the opportunity to explain their solutions before the teacher gave instant feedback on their answers. Meanwhile, the TCI group was exposed to the same exercises, and students were selected to present and explain their answers before the class then followed by the teacher’s feedback. Both groups were evaluated at the end of every lesson through a multiple-choice quiz.

The CRS and the TCI groups were assessed using the Simple and Compound Interest Assessment (SCIA) as a post-test to evaluate which group will do better on the assessment. A month after, the SCIA was re-administered (delayed post-test) to both groups to assess whether the use of CRS in mathematics classroom can improve long-term knowledge retention.

3.2. Participants

During the School Year 2020-2021, the school outlined its recovery plan to conduct limited face-to-face instruction. There were thirty-three (33) grade 11 students officially enrolled in this school year, but only thirty (30) students expressed their intention to join the limited face-to-face intervention because three (3) of these students were ‘Balik-aral’ or students who stopped from schooling but came back to school to continue. These three were working students who opted to continue studying during the pandemic through modular distance learning. The senior high school department identified and grouped the thirty (30) grade 11 students into two heterogenous groups, each with 15 students. There were 15 males and 15 females whose ages ranged from 16 to 26 years, with a mean age of 19.2 years and nine (9) of the respondents were 18 years old.

In this study, the researchers considered the existing two intact groups of grade 11 students identified by the school at the beginning of the school year. A convenience sampling technique was employed to select the respondents for this study. This non-probability sampling method is suitable since the samples in each group are established and readily available before the conduct of the study. In addition, Simkus(Citation2023), Cresswell and Cresswell (Citation2018), and Stratton (2021) emphasized that this technique is a feasible way to collect the data to gain deeper understanding of the impact of the teaching intervention in which in this case is generalizable only to the set of samples being studied. In this research, given that the sample size is relatively small (n20), the academic achievement of the students in the CRS and TCI groups can be compared as emphasized in the studies of Kozanitis & Nenciovici (Citation2022) and Freeman et al. (Citation2014) related to active learning approaches using CRS.

On the other hand, each group was then randomly assigned to the control group and experimental group. Moreover, the researchers handled both groups throughout the duration of the study.

3.3. Procedure

Prior to the implementation of the study, the researchers downloaded the Blicker for Students and Blicker for Teacher on Windows 10 App Store and installed on Windows 10 PC tablets and laptop respectively. These PC tablets and laptop were issued to the school through the Department of Education Computerization Program (DCP).

The Interactive Studio (Citation2016) described the technical requirements for the Blicker system to work and where to download the Blicker for Students and Teacher. The Blicker for Students can be downloaded freely from all major mobile platforms while the Blicker for Teacher can be bought from the Windows 10 App Store as well as the Mac OS X app store for about 350 pesos. In terms of the system requirements, the Blicker Teacher is compatible with Windows 10 or Mac OS X and can be installed either on a laptop or tablet pc while the Blicker Student can run on Android, iOS, and Windows 10 mobile devices and tablet PC.

After the necessary applications were installed, the researchers prepared the master list of the CRS group and uploaded it to the Blicker for Teacher application. The purpose of uploading the master list was to set the students’ identities, record their responses when polled, and facilitate real-time feedback. The master list Excel template was acquired through the Blicker for Teacher application. This master list contained the names of the students, their pictures, and the student ID number. The researchers assigned a unique numeric student ID number from 1 to 65535 to each student attending CRS-facilitated instruction.

The researchers used the Multiple-Choice Quiz (MCQ) Response Mode feature of the Blicker for Teacher and uploaded the questions needed in polling the students in the CRS group. All the questions needed for the entire duration of this study were already uploaded and saved in the system for future use.

The researchers also set the Bluetooth visibility of all the Tablet PCs and paired the devices to the laptop. This is done to ensure that all students can effortlessly use and connect the Blicker for Students to the Blicker for Teacher application. All these technicalities were done prior to the implementation of the study.

During the implementation, the researchers handled both CRS and TCI groups for two weeks, following the same content coverage. The CRS group used the Blicker application as a classroom response system, while the TCI group had a lecture method. Students in the CRS group were required to set their identity by entering their assigned Student ID number to connect with the Blicker for Teacher application and interact with the teacher. A typical class started with a review or drill, followed by a presentation and discussion of concepts, independent practice for mastery of skills, and an evaluation at the end of the lesson. The CRS group was given multiple choice poll questions and students were randomly selected to explain their answers then the teacher gave the feedback. The same set of problems was posed to the TCI group and feedback was given right after the students explained their answers before the class. Both groups were given multiple-choice questions in the evaluation. The same experimental setup was used in the study of Hammil (Citation2011) and McCumiskey (Citation2010).

After the learning engagement, the Simple and Compound Interest Assessment (SCIA) was administered to both groups to measure students’ mathematical achievement on Simple and Compound Interest under General Mathematics. The students took the assessment for one hour and answered it on a separate sheet where they shaded the letter that corresponds to their answer.

A month after the assessment was given, the Simple and Compound Interest Assessment (SCIA) was re-administered to both groups to assess whether the classroom response system can improve the retention skills of the students.

The assessment and re-administered assessment answer sheets were digitally scanned and checked using ZipGrade, ‘an application which turns a mobile device into an optical scanner for grading multiple choice assessment’ (ZipGrade, Citation2021). This application was used to facilitate fast checking of the answer sheets.

3.4. Research instrument

Simple and Compound Interest Assessment (SCIA) is a 30-item multiple-choice assessment intended to measure students’ mathematics achievement on the topic of simple and compound interest under General Mathematics. A table of specifications was constructed to ensure that all learning competencies of simple and compound interest were represented in the 30-item SCIA and aligned with the Most Essential Learning Competencies (MELC) as prescribed by the Department of Education (DepEd). Three mathematics curriculum experts from Basilan Schools Division carefully validated the SCIA. The researchers conducted a pilot test of the SCIA to twenty (20) Grade 11 students in one of the senior high schools in the Lantawan District. An internal consistency reliability test was carried out through R and found that the SCIA obtained a Cronbach’s Alpha value of 0.71, which indicates that the assessment is acceptably reliable (Institute for Digital Research and Education Statistical Consulting, Citation2020). Moreover, the item analyses revealed that the SCIA is classified as average with a difficulty index of .263.

3.5. Data analysis

The assessment (post-test) and the re-administered assessment (delayed post-test) results were tested for normality and homogeneity. The Shapiro-Wilk tests on assessment and re-administered assessment results of TCI and CRS showed no significant departure from normality. In addition, the homogeneity of variances of the assessment and re-administered assessment results of the CRS and TCI groups were computed via Levene’s F test. Levene’s test for assessment results was not significant, indicating homogeneity of variances. Hence, an independent sample t-test was employed to test the significant difference between the assessment mean scores of the CRS and TCI. However, the assumption of homogeneity of variances was not met for the re-administered assessment results. Hence, a Welch t-test was used to compare the mean scores of the re-administered assessment results of the students exposed to the CRS and TCI. de Winter (Citation2019), Delacre et al. (Citation2017), and Bluman (Citation2012) emphasized that independent t-test and Welch t-test are appropriate statistical treatments when testing the mean difference between the two independent samples taken from an approximately normally distributed populations with equal and unequal variances, respectively. In this study, analyses were carried out using R statistical package.

The researchers conducted item analyses and computed the Mean Percentage Scores (MPS) to measure the achievement or mastery level of the Grade 11 students on the given assessment and re-administered assessment. Based on Department of Education (DepEd) Order No. 160 series of 2012, to determine the quality of learning outcomes from the assessment results the descriptive equivalent of achievement or mastery level is specified in terms of the Mean Percentage Score (MPS). The calculation of MPS is being used up to the present time by the DepEd to inform progress and achievement level of the students in various learning disciplines in the K-12 Basic Education Curriculum. The MPS per item in the given assessment and re-administered assessment results were calculated and then summarized by learning competency. The MPS per competency and the overall MPS of the assessment and the re-administered assessment of the CRS and TCI groups were compared. The overall MPS was calculated by dividing the mean score by the total number of items in the SCIA and then interpreted using the descriptive equivalent of achievement or mastery level below (DepEd Memorandum no. 160 series of 2012, 2012).

3.6. Ethical considerations

The researchers secured the informed consent and permit to conduct a study given by the Office of the School Principal and the Office of the Schools Division Superintendent of Basilan. Also, the researchers were granted research ethics clearance for the implementation of the study by the Western Mindanao State University – Research Ethics Oversight Committee. Moreover, minimum health protocols and social distancing measures were strictly followed all throughout the conduct of the study as prescribed by the DepEd and the Ministry of Basic, Higher, and Technical Education (MBHTE)-Basilan Schools Division.

4. Results

This section contains detailed presentation of data analyses and the results of this study. The descriptive statistics of the assessment and re-administered assessment results are presented followed by the findings of each research question and the achievement indices and item analyses.

4.1. Comparison of the assessment results between CRS and TCI

The assessment results of the CRS and TCI groups were compared and analyzed to determine the impact of the classroom response system on the student’s achievement in mathematics.

shows the comparison of the summary statistics of the assessment results of the CRS and TCI groups.

Table 1. Summary statistics of the assessment in the CRS and TCI groups.

As displayed in , the descriptive statistics of the assessment results were relatively higher in the CRS group than the TCI group. In the given 30-item assessment, both the minimum and maximum scores of CRS group were greater than the minimum and maximum scores of the TCI group. The assessment mean score of the CRS group was higher than the TCI group. Comparing the means, it showed that the students taught using the classroom response system scored higher in the assessment than the students who were taught in the traditional classroom instruction. However, the standard deviation of the scores of the CRS group was higher than the standard deviation of the scores of the TCI group. This showed that the scores in the CRS group were more dispersed from the mean compared to the TCI group, suggesting a varying degree of consistency in conceptual understanding among students. When the Mean Percentage Scores (MPS) were computed, it was found that the CRS group had more questions answered correctly than the TCI group by 32.67%.

The researchers performed a Shapiro-Wilk test for normality since the sample for this study is relatively small. The Shapiro-Wilk test results showed no significant departure from normality were found; W (15)=0.90058,p=.09714 for TCI and W (15)=0.9456,p=.4458 for CRS. Hence, a parametric test was used. Additionally, the assumption of homogeneity of variances was tested and satisfied via Levene’s F test, F(28)=1.5648,p=.2213.

An independent t-test was carried out at .05 level of significance to test whether the difference in the mean scores in the given assessment is statistically significant. presents the analyses of the assessment results of the CRS and TCI groups.

Table 2. Independent t-test between the assessment results of the CRS and TCI groups.

As shown in , there was a highly statistically significant difference in the assessment results between the Classroom Response System (M = 19.2, SD = 4.5) and Traditional Classroom Instruction (M = 9.4, SD = 3.2); t (28) = 6.9132, p < .001 (two-tailed). Hence, there was a significant difference in the achievement level between the CRS and TCI groups. Additionally, a one-tailed t-test was performed and found that students in the CRS group performed significantly better on assessment compared to the TCI group, p < .001. This result suggests that the CRS group has greater achievement level than TCI group.

presents the achievement indices of the different competencies for simple and compound interest of the assessment results of the CRS and TCI groups.

Table 3. Comparison of the achievement level between the assessment results of the CRS and TCI groups.

indicated the CRS group consistently outperformed the TCI group in the assessment across the different learning competencies by margin of 32.67%. Also, the achievement indices revealed that CRS group performed better in competencies that required higher-order thinking skills in solving problems involving simple and compound interests than the TCI. This finding suggests that CRS-facilitated instruction has positive impact on mathematics achievement.

The researchers conducted item analyses to examine the responses of the CRS and TCI groups in the assessment. In the 30-item SCIA, across the cognitive domain of revised Bloom’s taxonomy there were two (2) items for remembering, two (2) items for understanding, eight (8) items for applying, twelve (12) items for analyzing, three (3) items for evaluating, and three (3) items for creating. The item analyses on the assessment results of both groups were compared and analyzed.

Ten (10) students in the TCI group found items 1, 2, and 3 easy but struggled on item 27. The cognition levels of items 1 and 2 is on remembering and item 3 is on understanding. These items were intended to measure students’ competency on how to illustrate and distinguish between simple and compound interest and determining the frequency of conversion. In item 27, the students need to compute and solve problems involving compound interest and maturity value. For the remaining items in the assessment, students in the TCI group demonstrated low performance. This result suggests that students in TCI group struggled to demonstrate mastery of the competencies that required low-order thinking skills to higher-order thinking skills.

On the other hand, all the students in the CRS group have answered item 13 correctly while most of them have difficulty solving items 4 and 17. The level of cognitive complexity of item 4 is understanding while items 13 and 17 are classified under analyzing. These items require students to demonstrate how to compute the unknown rate in simple interest and solve compound interest. Only 2 students answered item 17 correctly which asked, ‘What is the interest rate per annum of a ₱10,000 for 10 years that accumulate to ₱20,000?’ Most of the students selected the option A (0.1%) which show that students demonstrated the correct solution but did not change the decimal into percent correctly. In the remaining items, most of the students in the CRS group demonstrated higher mastery level, indicating that CRS use in mathematics classroom is effective in improving academic achievement.

4.2. Comparison of the re-administered assessment results between CRS and TCI

The CRS and TCI groups took the same assessment one month after it was administered. The researchers examined the re-administered assessment results to compare the knowledge retention skills of the students in the CRS and TCI groups.

shows comparison of the summary statistics of the re-administered assessment results of the CRS group and TCI group.

Table 4. Summary Statistics of the re-administered assessment in the CRS group and TCI group.

As shown in , the descriptive figures of the CRS group in the re-administered assessment were higher than the TCI group. Both the minimum and maximum scores of CRS group were higher than the TCI group. Conversely, the standard deviation of the scores in the re-administered assessment of the CRS group was larger compared to the TCI group. This implies that there was greater variation in terms of conceptual understanding among the students in the CRS group over the TCI group.

The re-administered assessment mean score of the CRS group was relatively higher than the TCI group. Consequently, the MPS of the CRS group was greater than the TCI group by about 24.9%. This indicates that students in the CRS group had more correct answers than the TCI group in the re-administered assessment.

The researchers performed a Shapiro-Wilk test to determine if the re-administered assessment results are normally distributed. The Shapiro-Wilk tests for both TCI and CRS re-administered assessment results showed no significant departure from normality were found; W (15)=0.90233,p=.1033 for TCI and W (15)=0.95981,p=.6891 for CRS. Thus, a parametric test was employed.

To determine if there is a significant difference in the re-administered assessment results of the students exposed to the CRS and those of the TCI, a Welch t-test was employed at .05 level of significance since the Levene’s test indicated that the homogeneity of variances assumption was not met, F(28)=7.9646,p=.008679.

shows the statistical inference using the Welch t-test on the re-administered assessment results of the CRS and TCI groups.

Table 5. Welch t-test between the re-administered assessment results of the CRS and TCI groups.

As seen in , there was a statistically significant (t (17.961) = 5.1492, p < .001; d = 1.88) difference between the Classroom Response System (M = 14.3) and Traditional Classroom Instruction (M = 6.53) on the re-administered assessment. The effect size for this analysis (d = 1.88) was found to exceed Cohen’s (1988) convention for a large effect (d = .80), suggesting a substantial difference between CRS and TCI groups. Therefore, there was a significant difference in the long-term knowledge retention skills of students exposed to CRS and TCI. Moreover, a one-tailed Welch t-test was done, and results indicate that students in the CRS group performed significantly better on re-administered assessment than the TCI group, p < .001. This finding suggests that CRS enhances knowledge retention of the students.

The researchers calculated the MPS per learning competency of the re-administered assessment to determine the level of achievements of the students in CRS and TCI groups. shows the achievement indices of the different competencies for simple and compound interest of the re-administered assessment results of the CRS and TCI groups.

Table 6. Comparison of the achievement level between the re-administered assessment results of the CRS and TCI groups.

revealed a huge difference in the MPS of the re-administered assessment of both groups, favouring the CRS group. Across the seven learning competencies, the students in the CRS group consistently outperformed the TCI group in the re-administered assessment even one month after the same assessment was given. Moreover, the achievement indices showed that CRS group achieved greater mastery of the learning competencies and demonstrated higher-order thinking skills in solving problems involving simple and compound interests than the TCI group. With this result, it can be claimed that CRS improves not only achievement in mathematics but also knowledge retention.

The researchers carried out item analyses for both the re-administered assessment results of the CRS and TCI groups. Fourteen (14) students in the TCI group found item 1 easy but struggled in almost all questions particularly items 8, 20, 23, and 27 where no one got the correct answers. This revealed that the mastery level of the students in the TCI group in the re-administered assessment was low. Also, this item analysis found that students could not apply the concepts, analyze and evaluate simple and compound interest problems.

In contrast, the CRS group does not only remember the concepts but also able to apply, analyze and evaluate problems in simple and compound interest. All the students in the CRS group found items 1 and 2 very easy but demonstrated very low mastery in items 26 and 28. Only two (2) students were able to answer items 26 and 28 correctly. Students in the CRS group struggled to distinguish scenarios involving simple and compound interest and sketch the growth of the investment in simple interest.

The scores in the re-administered assessment of the two (2) students in the CRS group have dropped by half while another student has dropped over half compared to their previous scores in the assessment. However, only one student in the TCI group scored less than half of the score in contrast to the assessment result. The item analyses of the re-administered assessment result of the CRS group revealed that these three students shared common procedural errors in solving problems involving simple and compound interest. These procedural errors as described by the students include incorrect conversion of decimal to percent when computing for the unknown rate; only the interest was calculated and did not add the result to the principal to get the maturity value of the investment in simple interest; and error in computing the compound interest when the principal was not subtracted from the maturity value.

For instance, in items 4 which asked, ‘What is the interest rate per annum of a P10,000 for 10 years that accumulate to P 20,000?’, the students calculated the rate which is 0.1 but failed to convert it to percent correctly. Most of them answered 0.1% and 1% instead of 10%. Another procedural error in solving the problem was found in item 8, that asked ‘What is the maturity value of a loan for P15,000 at 10% simple interest for 5 years?’ All of the students responded 7,500 instead of 22,500. This shows that students only computed the simple interest but not the maturity value of the investment in simple interest. In item 19 which asked, ‘How much is the interest of P50,000.00 investments at 2.5% compounded annually for 10 years?’, students computed the maturity value of the investment worth 64,994.23 but did not subtract the principal from the maturity value to get the interest.

These analyses showed that the students in the CRS group know the concept and procedure on how to solve the problem yet ended up choosing the wrong answer because of some procedural errors. Unlike the students in the TCI group, results showed that the students only remember the concept but not the process of solving a problem.

It is also noteworthy to mention that the re-administered assessment score of one student in the CRS group had increased by one (1) compared to the assessment result. The student had answered the same item on the assessment and was able to solve one additional problem in the re-administered assessment correctly. This shows that CRS has the potential to improve knowledge retention.

5. Discussion

The present study offers a novel viewpoint on the benefits of CRS in improving student’s achievement and knowledge retention in secondary mathematics classroom. In terms of the impact of CRS on mathematics achievement, results reveal that the students exposed to CRS-facilitated instruction has greater achievement level compared to the students attending traditional instruction. This finding coincided with previous research studies that students in CRS-facilitated instruction perform comparatively higher in the mathematics assessment than those students in traditional classes (Hammil, Citation2011; Homewood et al., Citation2008-2009; McCumiskey, Citation2010; Wang et al., Citation2014). The increased in student achievement in the CRS group can be attributed to the power of the CRS to provide instantaneous feedback which does not normally happen in traditional classroom (Manuel, Citation2015; Wang et al., Citation2014). Feedback plays a significant role in formative assessment and using CRS help students assess their own learning in real-time which directly influence student learning (Cantero-Chinchilla et al., Citation2020; Saleh et al., Citation2019). Curto Prieto et al. (Citation2019) pointed out the CRS promotes self-evaluation in mathematics. Additionally, McCumiskey (Citation2010) emphasized that CRS enable teachers to see how well the entire class understood the math lessons and give reteaching opportunities in cases where students have conceptual and procedural errors in solving problems in mathematics. Evidently, CRS can increase student learning. However, previous studies on CRS only presented a general overview of its impact on academic achievement based on the mean gained scores. There was no comprehensive analysis of the assessment results across the different cognition levels describing the achievement level exhibited by the students.

In this study, item analyses of the assessment results were carried out to identify student’s achievement level per learning competency. The in-depth analyses of the achievement indices based on the MPS show compelling indications that CRS group exhibits greater mastery of the learning competencies over the TCI group. The CRS group demonstrated Average Near Mastery (AVR) of the learning competencies while the TCI group displayed Low Mastery (L) in the assessment. Although the overall MPS of the CRS group is still below the DepEd’s national MPS target of at least 75%, the use of CRS in mathematics instruction can help improve mathematics achievement. Clearly, the difference in the MPS of the assessment results of the CRS and TCI groups show a huge margin in terms of the level of achievement of the students.

In terms of consistency on conceptual understanding, results indicate that students in the TCI group have a more consistent conceptual understanding than the CRS group. Although the TCI group demonstrated consistency in terms of conceptual understanding, still the students in the CRS group fared better in the assessment than the TCI group. This contradicts the findings of Poulis et al. (Citation1998) that students in the CRS group have a more consistent level of comprehension than the students in the TCI group.

The in-depth analyses of the assessment also enabled the researchers to determine if students in the CRS and TCI groups were able to demonstrate mastery of competencies which require low-order thinking skills to higher-order thinking skills. Interestingly, findings reveal that CRS developed student’s higher-order thinking skills reflected on the assessment results which show that students in the CRS group are able to solve mathematics problems across the different cognition levels than the TCI group. This implies that CRS is an effective tool in increasing students’ achievement in mathematics.

Another important aspect of this research is the impact of CRS on knowledge retention. It was found that CRS group has better knowledge retention compared to the TCI group. This result corroborates the findings of Pradhan et al. (Citation2005), Radosevich et al. (Citation2008), Liu and Stengel (Citation2011), and Aljaloud et al. (Citation2015), in which the use of CRS can potentially increase the student’s knowledge retention. Parallel to the arguments of Liu and Stengel (Citation2011), the increase in knowledge retention is not about that students in the CRS group were given more problem-solving opportunities since TCI group got the same math exercises, but it is all about the instant feedback opportunities afforded by the CRS. Similarly, Wang et al. (Citation2014) support the claim that the CRS’ capacity to provide instant feedback creates a deeper understanding of materials which enhances the short-term and long-term memory of the students. On one hand, the result of this study did not conform with the findings of Doucet et al. (Citation2009) that found no significant difference in the retention skills of the CRS and TCI even though the CRS scored higher in the assessment given a year after the treatment. This could be due to the difference in the length of the time in giving the retention test to students. Nonetheless, Doucet et al. (Citation2009) research and the current study reported that CRS group scored higher in retention test than the TCI group.

It is also significant to pinpoint that the scores of the students in the re-administered assessment in both CRS and TCI groups decreased a month after the assessment was administered. This is similar to the observation noted by Radosevich et al. (Citation2008). Additonally, the standard deviations of the re-administered assessment results revealed that TCI group demonstrated consistency in terms of conceptual understanding than CRS group, contrary to the report of Poulis et al. (Citation1998). However, this variance in term of consistency in conceptual understanding does not necessarily equate to poor performance among the students in the CRS group. The TCI group may have consistent conceptual understanding but exhibited lower retention test scores than the CRS group. This implies that CRS enhances knowledge retention.

Unlike other studies that evaluated the impact of CRS on knowledge retention, the current study conducted analyses of the re-administered assessment results and determine the achievement level per learning competency. Findings show that the CRS group exhibited Average Near Mastery (AVR) of the learning competencies while the TCI group demonstrated Low Mastery (L). Across all learning competencies, the CRS group consistently demonstrated greater mastery compared to the TCI group by 25.9%. Consequently, students in the CRS group are able to solve problems involving simple and compound interest which require higher-order thinking skills than the TCI group. The re-administered assessment results expose that TCI group can only solve problems with lower cognition levels while CRS are able to solve problems across the different cognition levels. Hence, this study asserts that CRS in mathematics instruction has the potential to improve knowledge retention. With these findings, mathematics teachers can likewise integrate a Bluetooth-based CRS when teaching mathematics in high school to enhance student learning.

6. Conclusion

This paper aimed to evaluate the pedagogical effectiveness of CRS in increasing student’s achievement and knowledge retention. The results reported here highlight substantial evidence for the CRS’ integration in secondary mathematics classrooms as it induces higher achievement level and knowledge retention among the students, as compared to the traditional classroom instruction. The superior performance of the students in the CRS group in the assessment and re-administered assessment is influenced by the CRS’ ability to give prompt and real-time feedback. These findings are consistent with the previous studies on CRS use in mathematics. Another distinctive contribution of this study is the in-depth analyses of the assessment and re-administered assessment results quantifying the achievement level of students in the CRS group and TCI group across the different learning competencies. This provides a comprehensive understanding of the effects of CRS on mathematics achievement and knowledge retention, which was not done in previous investigations. The results indicated that the CRS group exhibited greater mastery of learning competencies over the TCI group in both assessments. Students in CRS and TCI groups demonstrated Average Near Mastery (AVR) and Low (L) mastery in the assessment and re-administered assessment, respectively. Further, the findings of the item analyses spot that the CRS enables students to develop higher-order thinking skills. These research findings will add to the existing body of knowledge related to use of CRS in mathematics and the benefits of Bluetooth-based CRS like Blicker.

The study implies that CRS is a valuable pedagogical resource which can be integrated in high school mathematics classroom not only to promote active learning but also improve academic performance and increase the short-term and long-term memory of the students. Moreover, mathematics educators are encouraged to use the Bluetooth-based CRS to promote meaningful learning in mathematics.

7. Limitations and future research

The limitations to the generalization of these findings which could be further addressed in future research include relatively smaller class size and short content coverage. Due to the smaller number of grade 11 students enrolled for the said school year, the study was carried out with 30 students only of which 15 students were in the CRS group and the other 15 students were in the TCI group. On the other hand, the simple and compound interest topic is a two-week lesson in general mathematics. It is recommended to conduct similar study to a larger group of students and different grade levels. Additionally, a similar study can be conducted to investigate the pedagogical effectiveness of classroom response system in improving mathematics achievement and knowledge retention of the students by covering a unit or a quarter lesson.

Acknowledgments

The authors would like to express their profound gratitude to the Western Mindanao State University (WMSU) – College of Science and Mathematics, the Ministry of Basic, Higher, and Technical Education (MBHTE) - Basilan, and Tairan National High School for the support given throughout the conduct of this research.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are openly available in Zenodo at https://doi.org/10.5281/zenodo.7852444.

Additional information

Notes on contributors

Jayson Virtucio Alviar

Jayson Virtucio Alviar is currently a PhD student in mathematics education at the University of the Philippines Open University. He earned his bachelor’s and master’s degrees in mathematics education at Western Mindanao State University in 2016 and 2022, respectively. At present, he serves as the Division Mathematics Coordinator for secondary at the Ministry of Basic, Higher, and Technical Education (MBHTE) – Basilan Schools Division and concurrently holds the position of Senior High School and Research Coordinator at Tairan National High School. His research interests include technology-enhanced mathematics teaching, philosophy, theory, and history of mathematics education.

Anabel Enriquez Gamorez

Anabel Enriquez Gamorez is an Assistant Professor at the Department of Mathematics and Statistics at Western Mindanao State University. She earned her Master of Applied Statistics (MAS) and PhD degree in Mathematics at Mindanao State University – Iligan Institute of Technology (MSU-IIT) in year 2014 and 2020, respectively. Her research interests include graph theory, topology, applied and pure mathematics, and current trends in mathematics education.

References