Using an Online Student Response System in Small Group Teaching: A Pilot Study

Abstract

Student response systems have shown to be an effective tool in addressing some of the challenges of teaching and learning in large classrooms. Traditional student response systems are based on clickers—handheld devices used to transmit student responses to a receiver. In contrast, the EduMECCA SRS captures student responses via smart phones and tablet PCs that can be connected to the internet. This paper is focused on the use of the EduMECCA SRS in small group teaching. A pilot study was conducted in the Applied Computing department of the University of Buckingham where the EduMECCA SRS was used in a number of courses of the BSc in Computing programme. Student feedback indicates that the SRS has had a positive effect on student learning and students’ experience in small classrooms. However, practical issues may deter the widespread use of the SRS system within the School/University.

Keywords:

• EduMECCA SRS
• clickers
• student response systems
• enhancing student learning experience

1. Introduction

Student engagement and feedback on learning are important characteristics of a successful learning environment. However, maintaining learners’ attention throughout a lecture and providing real-time feedback is a challenge to lecturers, especially in large classrooms—students are unwilling to participate due to shyness or loss-of-face and practically it may not be possible to engage with all students in the class. The recent increase of participation in higher education could further erode the prospect of effective dialogue between instructors and learners, resulting in an unsatisfactory student experience and achievements below expectation (Russell, 2008; Draper & Brown, 2004).

Information and communication technologies have become an integral tool that facilitates many aspects of learning, teaching and assessment in higher education. One of the technologies used in large classroom teaching is student response systems (SRS), also known as electronic voting systems (EVS) or audience response systems (ARS). There is a significant body of literature that supports the use of SRS technology for reasons varying from simply as means of “spicing up standard lecture classes with periodic breaks” to enhancing students’ understanding of the subject (Caldwell, 2007; Beatty, 2004; Russell, 2008; Cliffe, Davenport, Vos, Parm, & Hayes, 2010).

Traditional response systems are based on purpose built clickers—handheld devices used to transmit student responses to a receiver via infrared (IR) or radiofrequency (RF). Typically, these clickers are handed out to students before each lecture or given on loan for a term/semester. In contrast, the recently developed EduMECCA SRS (EduMECCA SRS; Thorseth et al, 2011) captures student responses via mobile devices such as smart phones and tablet PCs that can be connected to the internet.

This paper is focused on the use of the EduMECCA SRS in small group teaching. The aim of the study is to: a) understand the effectiveness of SRSs within the context of small group teaching and b) to identify practical challenges of using an online SRS where students respond to questions by login onto a website using their own mobile devices.

The rest of the paper is organised as follows: section 2 gives a brief background to the use of student response systems in general; section 3 describes the EduMECCA SRS tool used in this study; methodology and an evaluation of the study is presented in section 4 followed by a discussion of practical issues in section 5; Concluding remarks are given in section 6.

2. Background

Recent studies have investigated the use of student response systems in large-group teaching. Caldwell (Caldwell, 2007) presents a survey of current research on the use of clickers in large classrooms and summarises best-practice tips to help new users of clickers to plan and embed the use of clickers in lectures. More recent work by Cliffe et al (2010) and Brander (2010) highlights the positive effects of voting systems on students’ learning and their learning experience as well as large group participation. Existing literature suggests that the benefits of student response systems are cross-disciplinary (Brewer, 2004; Elliott, 2003; Brander, 2010).

More recently, Lu et al (Lu et al, 2010) and Stav et al (Stav et al, 2010) have introduced a new web-based SRS for PC and mobile devices and used the system in large group teaching and e-learning environments with successful outcomes.

In essence, student response systems have shown to be an effective tool in addressing the main challenges of teaching and learning in large classrooms—difficult to have a meaningful engagement between lecturer and learners or to gauge learners’ comprehension of the lecture—lack of student participation due to shyness, loss-of-face or simply because of limited opportunity.

The situation is somewhat different in small group teaching where groups of 10–25 students are in a classroom. There is opportunity for both lecturer and student to engage in a dialogue, pose questions and receive immediate feedback. However, lack of participation due to shyness or fear of embarrassment remains a barrier to effective engagement. Small group teaching also presents with a scenario where vocal students tend to interact more with the lecturer than the others. In this context, student response systems could provide an equal opportunity and anonymity to students. This study is about the use of student response systems in such a context.

3. EduMECCA SRS

The Student Response System (SRS) is developed as part of the EduMECCA project (EduMECCAProject; EduMECCA SRS; Stav et al, 2010; Thorseth et al, 2011), which is part of the European funded Lifelong Learning Programme. Unlike traditional EVS that use clickers and purpose built infrastructure, the SRS uses widely available mobile devices and mobile services to collect student responses in a convenient way.

3.1 SRS functionality — the online tool

The functionality of the web-based tool is similar to any other EVS tool; a MCQ is presented separately (e.g. include in lecture slides); the lecturer has the option to create a variety of response/vote types on the SRS website. For example, A, B, C, D; YES, NO, I don’t know; or TRUE, FALSE, I don’t know. Students in the class use a session code generated by the system to login to the particular class/lecture. Students access the SRS website using their smart phones, tablet PCs or laptops and select the appropriate answer(s). Once a question has been answered, the lecturer can: a) create a new vote; b) save responses; or c) view a summary of all the results. Figure 1 shows a screen shot of an example vote pattern for a multiple-choice question. For a detail description of the SRS tool, see (Stav et al, 2010; Thorseth et al, 2011).

Figure 1 Screen shot of pooled responses for a mock question

4. SRS in Practice

The EduMECCA SRS was used by this author in two first-year courses (Fundamentals of Computing and Software Engineering) and two second-year courses (Software Project Management and Image Processing) of the BSc in Computing programme at the University of Buckingham (note that the second year is the final year of the two-year degree programme). Class size varied from 12-18 students—some courses were taken by students who study computing as a minor discipline.

4.1 Example questions

Questions were presented on a word document rather than embedding them in lecture slides. Following are two example questions:

Table

Productivity measured in LOCs/time varies with: A. Management skills B. Type of programming language C. Associated risk

The likelihood of completing activity B by day 35 is A. 80% B. 85% C. 90% D. 95%

The SRS was used at one or more of the following stages of a lecture:

• Prior to starting the lecture — questions to review key topics covered in the previous lecture; questions relating to the current lecture (the students may not be familiar with the topics) to highlight the key areas that is to be covered by the lecture

• During the lecture - 25–30 minutes in to the lecture or before starting the next hour to maintain student attention, to understand the level of comprehension

• End of the lecture — questions to highlight key points of the lecture; reflect on understanding before and after the lecture (pose the same questions that was presented at the start of the lecture).

Pooled results for each question were displayed to the students (see Figure 1 for a mock example). Engagement takes place based on these results—if a significant number of responses were incorrect, the question or the relevant topic is discussed immediately.

4.2 Effectiveness of the SRS in small group teaching

A main aim of this pilot study was to assess the effectiveness of online SRS technology in small classroom teaching. Due to the small number of participants, qualitative feedback is used as the primary measure of effectiveness. Student feedback was collected in two ways: 1) small survey, specific to the use of SRS in the Software Project Management course; 2) students’ comments relating to the use of SRS given as part of general course feedback at the end of a course.

Eight out of the 14 students that took the Software Project Management course responded to the survey. Detail survey responses are presented in Figure 2. Highlights of the findings are:

• An equal number of respondents ‘disagreed’ or ‘very much disagreed’ with the suggestion that “review/summary questions before/during/after lectures are a waste of time”;

• Half the respondents said “I am one those in the class who always answer questions” whilst an equal number of the remaining respondents said “I answer only if the lecturer asks me of it if my turn” or “I want to answer, but not in public”;

• Only 3 out of the 8 respondents were happy to answer in public whilst the remaining 5 preferred to answer anonymously;

• All respondents agreed that the online SRS system is a useful tool and that it integrates well with the lecture;

• Except for one respondent who neither agreed nor disagreed, all other respondents agreed that the online SRS system enhances the learning experience;

• Six respondents ‘very much disagreed’ with the suggestion that “the online SRS is a waste of time”. Another ‘disagreed’ and the remaining respondent ’neither agreed nor disagreed’.

• A respondent wrote “it [online SRS] is a good system cause it helps students to remember what has being taught previous and also helps to know how much of understanding of topic and where to improve upon”.

Following is a selection of student feedback (relevant to the use of SRS) given as feedback for a course:

• “The introduction of student response system was a great idea” (Software Engineering, Autumn 2010)

• “Useful evaluation tool (student response system)”; “I like Moodle and student response system”; “The srs [SRS] student response system is a very useful tool in gauging the understanding of materials covered by the lecturer” (Image Processing, Spring 2011)

• “The SRS system was a good way to involve the class”; “[Good points:] Use of online voting system to enhance our study in each lecture”; “Every lecture has a review for last week’s study” (Software Project Management, Summer 2011)

Figure 2 Survey results for student feedback on the use of online SRS.

In essence, this pilot study of using an online SRS in small-group teaching shows: a) students in a small classroom find review questions before—during—after a lecture enhances their learning experience; b) students would find it helpful if they could respond to questions anonymously; c) students may not engage with the lecturer if there were not encouraged to do so; d) the online SRS facilitates interactivity, increases accessibility and removes barriers to participation even in a small classroom.

5. Practical Issues

Whilst the benefits of students response systems is clear, there are some practical challenges that needs to be addressed, especially when using an online response system such as the EduMECCA SRS. Below is a list of these issues experienced by this author.

• Universality - not all students has or brings a suitable device that can be connected to the internet. However, all students should be able to respond to questions and should not be felt left out of the process. This could be a significant barrier to the adoption of the EduMECCA SRS. Providing appropriate devices to students is a costly exercise. However, this may not be a long-term problem as ownership of mobile computing devices is rapidly increasing.

• Availability - availability of WiFi coverage or affordable 3G connectivity for speedy access to the SRS website is essential. If WiFi connectivity is not widely available, some courses may not benefit from the SRS system; inconsistent practice could lead to negative student experience and frustration.

• Reliability - uninterrupted connectivity, website refresh time; reliability issues leads to frustration and time wasting. A test run of the system could identify such reliability issues. IT Services needs to be involved in the process to ensure that a reliable WiFi service is available.

• Technical issues — some mobile devices take longer to connect to network and access the website; leads to frustration and time wasting.

• Distractions - allowing students to use their mobile phones in the classroom may tempt them to check their emails, text messages or updates on social networks; this could be seen as a barrier to adoption. However, this is easier to monitor and control in a small group than in large group teaching.

• Course management — it requires a significant effort by lecturers to prepare and embed questions in their lectures. Furthermore, setting up 3–5 questions on the SRS and waiting for responses could take up to 15 minutes (especially if there are technical or reliability issues). However, the more recent version of the online SRS, which includes a desktop application for the lecturer, is easier to setup.

6. Conclusions

The pilot study has shown that the use of a SRS has had a positive effect on student learning and students experience in small-group teaching. Student response systems increase participation, helps students understand the lecture and indicate where further effort is required. However, there are some practical issues that need to be addressed to achieve the full potential of the online SRS. Some of these practical issues may resolve naturally with improvements and wider availability in wireless technologies and increase proliferation of mobile computing devices.

7.

Acknowledgements

The author would like to thank Dr Joan Lu and Dr John B. Stav for permitting the use of the EduMECCA SRS.