Tips for using mobile audience response systems in medical education

Abstract

Background

With growing evidence on the benefits of active learning, audience response systems (ARSs) have been increasingly used in conferences, business, and education. With the introduction of mobile ARS as an alternative to physical clickers, there are increasing opportunities to use this tool to improve interactivity in medical education.

Aim

The aim of this study is to provide strategies on using mobile ARS in medical education by discussing steps for implementation and pitfalls to avoid.

Method

The tips presented reflect our commentary of the literature and our experiences using mobile ARS in medical education.

Results

This article offers specific strategies for the preparation, implementation, and assessment of medical education teaching sessions using mobile ARS.

Conclusion

We hope these tips will help instructors use mobile ARS as a tool to improve student interaction, teaching effectiveness, and participant enjoyment in medical education.

Keywords:

• mobile audience response systems
• active learning
• medical education
• ARS

Introduction

Active learning, meaningful and rapid feedback, and implementation of a repertoire of pedagogical strategies that were formerly impractical outside of small seminars are just a few of the benefits of audience response systems (ARSs).¹ ARSs were initially implemented with physical clicker devices, but with the advent of mobile technologies, there are increasing opportunities for learners to engage in an educational session by using their laptop, tablet, or phone. Mobile ARS also opens the door to new question types, including word and text answers, matching questions, ordering questions, and even click-on-target picture questions. However, many lecturers struggle with an approach to implement this new technology into their current teaching methods. As with any technology or teaching strategy, poorly implemented mobile ARS can actually make your educational session worse. However, when properly used, this tool can open the gateway to improve learner interaction and different models of education. Although not to be taken as a complete strategy, the authors of this article believe that the 12 tips provided in this article will help the lecturer approach mobile ARS with confidence and to offer some pearls of wisdom to the experienced ARS user.

Tip 1: use mobile ARS as a tool to support pedagogical approaches, not as the focus of your teaching strategy

Keep in mind that mobile ARSs themselves do not improve learning – they are tools used to help apply pedagogical strategies. ARSs have become a common feature of the modern lecture; however, challenges persist in their implementation and in maximizing their learning value. Perhaps for this reason, the current body of literature suggests that although students are enthusiastic about ARS, data on learning outcomes are only modestly positive.^2–4 However, learning results appear to be dependent on how ARSs are implemented.⁵ Technology is fun and exciting to use, but one must be mindful of the learning curves for the instructor and the student. The key tip is that ARSs themselves do not improve learning – they open the door to the use of pedagogical strategies that were previously not possible. Consider ARS as an exciting addition to the lecturer’s toolkit!

Tip 2: ensure your educational setting is ready for mobile ARS

One of the most exciting aspects of mobile ARSs is that they may be used anywhere and anytime that the students are carrying a mobile device, which seems to be always! There is no longer a need for students to remember to bring specific physical clickers or for instructors to distribute them in class. Mobile ARSs do, however, require some type of wireless connectivity and sufficient bandwidth to function efficiently and effectively. Systems with students responding by short message service/text response will require adequate mobile phone connectivity, which may be variable in different areas of the medical school or hospital and dependent on the mobile service provider. Some systems require Internet connectivity for submitting responses (via Apps or web browsers), and a trial run can help ensure your classroom setting has enough Wi-Fi bandwidth for the number of students in your educational session. Most mobile ARSs allow the instructor to display results via the Internet. If the presentation computer is connected to the Internet via Wi-Fi and displaying the results seems to slow down with more students using the Wi-Fi, the instructor may try hard wiring into the Internet. Many podium desktop computers are connected in this manner.

Tip 3: plan for enough time to ask and discuss questions during your session

Asking a question, waiting for responses, displaying responses, and discussing the answer all take time. For simple multiple-choice questions, we wait ~30 seconds to collect responses or until a certain number of responses have been collected. Waiting >1 minute stagnates the rhythm of the teaching session. Regardless, the whole process of asking one question can often take ≥2–5 minutes, depending on the complexity of the question and depth of discussion. An effective ARS question centered on deep understanding may require 5–10 minutes of class time to complete if peer discussion and explanation of answer choices are undertaken. This can be further lengthened if people are new to using the mobile ARS technology. The literature recommends asking three to four questions in a 60-minute lecture,¹ but this means that asking just three questions may occupy ≥15 minutes of valuable teaching time. Many lecturers worry about loss of content volume given these considerations. ARSs do appear to reduce the overall content volume provided in lecture, mitigated by better understanding of content covered.⁶ Is it worth it? Our experience would suggest it absolutely, but it certainly requires a shift in teaching strategy.

Shift your approach from “sage on the stage” toward a “guide on the side” learning experience. It is helpful to consider how closely your current approach to teaching mirrors well-founded principles of learning theory. Using an ARS can allow you to make richer use of these principles. One key tenet is that knowledge is constructed as opposed to transmitted. For instance, when a student is asked a question about a new topic, even if they answer incorrectly they are primed for learning and show meaningful gains in knowledge.⁷

Mobile ARS may also be combined with other techniques, such as think-pair-share. An instructor asks a question, encourages the students to discuss the question with their neighbor, and then awaits for students’ responses. This paired discussion does increase the length of time for asking a question even further. However, the benefit of added layers of interaction and the enlivened sounds of educational discovery are such that, in our local context, we now use this technique almost every time when we ask a question in class.

Tip 4: use mobile ARS to introduce new concepts or to review previous learnt concepts

ARSs allow the professor to incentivize preclass preparation by commencing lectures with questions based on assigned readings. This allows the lecture to shift focus from topic coverage to addressing challenging concepts, illustrative cases, and development of critical thinking skills. Other strategies include the distribution of lecture scripts before class to ensure content coverage and to provide an organizational framework for students, with the bulk of the lecture spent working through illustrative examples and difficult concepts instead of copying down presentation slides.

Spaced repetition is a powerful but often underused technique for promoting long-term knowledge acquisition.⁸ A successful approach for ARS lectures is to divide a lecture into a series of minilectures, each centered on a discrete problem or concept. The underlying premise of this approach is that students learn best with problems to solve rather than facts and knowledge to be memorized.³ One lecture format is to commence with a pretest question to prime students and elicit background knowledge, provide a “minilecture” for 10–20 minutes and conclude with an ARS question to assess understanding and discuss student’s misconceptions as dictated by the class performance. Additionally, dividing a lecture into shorter segments helps to maximize the attention span of students.^9,10 Mobile ARS can be particularly helpful in review sessions.^11,12 The value of discussing both the right and the wrong answers and how to arrive at each cannot be overestimated.

Tip 5: expand beyond recall-type multiple-choice questions

Every question should have a clearly identifiable pedagogic goal, rather than simply a topic to address. Examples include eliciting background knowledge and misconceptions, distinguishing between closely related concepts, and extending theories to novel contexts. Instead of asking simple recall questions, instructors can try more challenging questions that engage students in critical thinking and debate. Seize the opportunity to ask challenging questions not suitable for an examination that may generate multiple defensible answers, include imperfect or insufficient information, encourage debate among the class, and simulate the subtleties of real-world conditions.¹³ Consider a sequence of related questions that extend a concept into different contexts, interactions with other concepts, or into more sophisticated applications. This technique methodically builds comprehension while making efficient use of lecture time and giving consideration to students’ ability to process new material. Creating good questions is admittedly a difficult and time-consuming task that seeks out existing question databases or considers collaborating with colleagues. We have enjoyed great success in putting the challenge of creating questions to students! This encourages a completely different approach to the material and has often led to the creation of excellent questions for future use in teaching and examinations.

Many mobile ARSs allow for question types beyond multiple-choice questions, including word answer, numerical answer, matching, and ordering questions. Some programs even allow an image to be displayed and for the student to “click” (using a mouse or a touch device) on a target area of the image. We have found it is best to start with multiple-choice questions to gain initial comfort and confidence. Following this trial, if the session would benefit by different question types, expand to word answer questions or possibly click-on-target questions, which can be especially useful for questions involving anatomy or diagrams. Even when trialing other question types, however, we have repeatedly found ourselves returning to the single best answer multiple-choice question – for its ease of use, efficiency, focused discussion, and minimal disruption of the educational session.

Tip 6: establish expectations for responding to questions

Moving beyond multiple-choice questions to types, such as word answer, can be fun and enriching to the educational session. However, when opening responses to free text, there is the possibility for errant responses from participants. Humor in the classroom, and in questions, can be an effective and engaging part of teaching, but instructors must be cautious about unfocused or even unprofessional responses from students (especially in an anonymous setting) that may derail the teaching session.^14,15 If using word answer questions, we recommended explicitly stating at the start of the session your expectations for contributory and professional comments only.

Tip 7: expand beyond individual “question and answer”

Mobile ARS can work wonderfully with the concept of the “flipped classroom”, whereby students complete prereadings and prestudying before the session, arrive to the session prepared with some basic concepts, and develop their understanding further during the session through more active and interactive learning strategies.¹⁶ These strategies include small group learning, team-based learning, problem-based learning, peer-based learning, and case-based learning.¹⁷ Each of these models has unique perspectives but shares the commonality of promoting interactive learning, which can be greatly facilitated and augmented by the use of mobile ARS. Again, one simple strategy is to employ the “think-pair-share” technique with mobile ARS.

Beyond these strategies, mobile ARS may provide additional functions to medical curricula. When responses are linked to students’ user accounts, automatic grading functions can provide instant formative feedback to students. Grading functions may also be used for participation marks in a course. Some mobile ARS programs include an attendance function that may be used for mandatory academic sessions or a tournament function for instructors interested in “gamifying” their educational session with some friendly competition between students.

Tip 8: prepare and practice beforehand

Using mobile ARS can be a significant change both for the instructor and the student. Not only it is a different pedagogical approach but also there are technical requirements such as mobile signal reception, Wi-Fi bandwidth, connecting the computer to the projector to ask questions and display results, user registration for the instructor or students, and enrollment into the correct course. As with any initial foray into a new teaching strategy or technology, we recommend testing the mobile ARS beforehand in the specific lecture room you will be using with a few students to proactively discover and solve any issues that may arise.

Good ARS questions challenge students to go beyond factual recall and test their mental comfort zone. The lecturer can assist students in adapting to these new challenges with direct discussion of overall goals and strategies for learning with students. As well, training of students on mobile ARS will allow them to become comfortable with its use before it is fully implemented in their learning. This may be done at the start of the course or in an introductory lecture. Students are much more engaged when they are aware of why they are asked to perform a certain task. At a basic level, this strategy raises student’s attention and engagement in the lecture. At a more sophisticated level, metacommunication facilitates the development of critical thinking skills through explicit discussion of problem-solving strategies, extension of concepts to other contexts, incorporation of background knowledge, and other core reasoning skills.⁵

Tip 9: have a backup plan

Despite all of your preparations, practice, and student instructions, mobile ARS may not simply function effectively on the day of your session. The technological aspect may fail through Wi-Fi connectivity, Internet bandwidth, cellular reception, or user registration. Sometimes, you may find the practical process of asking questions and discussing the answers may impede the flow of that particular lecture. Regardless of the reason, if instructors find mobile ARS not functioning effectively in their session, they should change strategies by having students simply raise their hands to answer questions, or, if the questions and discussions themselves are impeding the lecture, removing the questioning component of the session altogether.

Tip 10: analyze the classroom data to assess results, analyze questions, or perform research

The ARSs provide an exceptional window into the student’s learning experience and offer the capacity to gather feedback on your lecture and improve its effectiveness. Each question allows for immediate feedback on the class’s understanding of the preceding concept, with an opportunity to address gaps in comprehension. Consider challenging your students with questions drawn from material presented at other points during the course or program to encourage contextual learning. Explicit feedback questions on new teaching techniques may be posed to solicit student’s opinions and suggestions on the teaching strategies themselves.

Tip 11: set up support

If you are trying out a mobile ARS, most companies will have phone and web support available, including real-time online support. Websites of many companies will also have online support forums. Look for other instructors in your department or faculty with experience using mobile ARS to guide you through the planning and delivery of the session. Many universities have support available through faculty development offices that offer increasingly sophisticated support for transitioning to ARS-based teaching. Use these resources to your advantage to aid in retooling your lectures, using your institution’s particular ARS system, and troubleshooting technical issues.

If you are promoting mobile ARS within your medical school or department for other instructors to use, consider additional layers of support. One strategy that we have found extremely effective is to designate a few students in each medical school class as “ARS experts”. These students can help instructors to log in to the system and use the mobile ARS. We created a few generic faculty-wide instructor logins (that the student ARS experts could activate), so that instructors could simply arrive to the lecture and use the mobile ARS without requiring individual logins and passwords for each instructor, some of whom may only deliver one or two lectures during the course. Working with the audiovisual technicians to provide on-site support to instructors may also be beneficial. These personnel are often present already at the start of the lecture, performing activities such as setting up the microphone for the instructor. They are thus well positioned to provide support for setting up mobile ARSs for the session.

Tip 12: get started using mobile ARSs

Mobile ARSs can add wonderful interactivity, higher level learning activities, and great fun to educational sessions. We would definitely encourage instructors to try out this increasingly available e-learning tool. There are many systems available with different feature sets and different payment models. A few popular ones are Top Hat (www.tophat.com), Poll Everywhere (www.polleverywhere.com), and Socrates (www.socrates.com), though many others exist. Most systems provide free instructor trial accounts, although they may have limited features or may be only active for a limited time. You can try out different systems by using your computer as an instructor and a separate web browser, computer, tablet, or phone as a student. Try it out with colleagues or with a few students to see what works for you.

There are a few considerations when choosing a mobile ARS. What type of payment model works for your situation – payment by the student, instructor, or institution? Will students be required to log in? Will mobile ARS be used simply for classroom interaction or will you use the system for grading as well? Do you require additional features, such as attendance or special question types? Do you require a presentation tool application (allowing the mobile ARS program to “float on top” of other concurrently running programs such as PowerPoint)? Regardless of which system you choose, we recommend starting off with simple multiple-choice questions, then perhaps word answer questions, and then expand to other question types such as click-on-target once you have developed increasing comfort using the mobile ARSs. Short message service/text responses generally seem to be simpler and quicker for participants to use rather than mobile Apps or web browsers. Start with small and simple use of ARSs, gain some confidence with the tool, and then build-up your success to expand to different question types and teaching strategies.

Conclusion

Mobile ARSs can be great tools to increase student interactivity, deliver real-time feedback to instructors, and provide valuable information to the curriculum. These systems may be used throughout the spectrum of medical education: in undergraduate medical lectures, residency teaching sessions, grand rounds, and continuing professional development. It should be recognized, however, that they are dependent on technology, different teaching strategies, and instructor comfort. As well, the adoption of ARS may require a dynamic curriculum that allows for appropriate introduction and student training. By following these tips, we hope instructors will find using mobile ARS to be interactive, effective, and fun.

Acknowledgments

The authors thank the undergraduate medical education Committee on Online Learning (Keevin Bernstein, Bryan Payne, David Wesst, Joanne Hamilton, and Sandra Schönwetter) for guidance on the implementation and delivery of mobile ARS, and Stephanie Mowat for assistance in the preparation of this article.

Disclosure

The authors report no conflicts of interest in this work. The authors alone are responsible for the content and writing of the article.