Diverse Perspectives on a Flipped Biostatistics Classroom

ABSTRACT

“Flipping” the classroom refers to a pedagogical approach in which students are first exposed to didactic content outside the classroom and then actively use class time to apply their newly attained knowledge. The idea of the flipped classroom is not new, but has grown in popularity in recent years as the necessary technology has improved in terms of quality, cost, and availability. Many biostatistics instructors are adopting this format, but some remain unsure whether such a change would benefit their students. One potential barrier to adopting a flipped classroom is the common misconception that only a single approach is available. Having adopted the flipped approach in their own courses, the authors participated in an invited panel at the 2014 Joint Statistical Meetings held in Boston, Massachusetts entitled “Flipping the Biostatistics Classroom.” A theme emerged from the panel's discussions: rather than being a one-size-fits-all approach, the flipped biostatistics classroom offers a high degree of flexibility, and this flipped approach can—and should—be tailored to instructors' specific target audience: their students. Several of these varied approaches to the flipped classroom and practical lessons learned are described.

KEY WORDS:

• Active learning
• Flipped classroom
• Hybrid/blended/inverted course design
• Video-recordings

1. Introduction and Background on the Flipped Classroom

This article is not intended to espouse the advantages of the “flipped” classroom. Evidence to this effect is plentiful in the literature across many disciplines (Hamdan et al. 2013). Rather, this article describes several varied approaches to using the flipped approach in a biostatistics classroom. The intended audience includes biostatistics instructors who have considered adopting a flipped approach but perceive barriers to doing so in their specific contexts. From the authors' diverse perspectives—spanning multiple institutions, departments, student characteristics, course schedules, etc.—we wish to convey how the flipped approach can be modified to each instructor's individual needs and environment. We provide practical considerations and advice across a number of different pedagogical aspects.

The authors participated in an invited panel at the 2014 Joint Statistical Meetings, held in Boston, MA. The panel was entitled, “Flipping the Biostatistics Classroom,” and was sponsored by the American Statistical Association's Section on Teaching Statistics in the Health Sciences. Panelists were similar in that all had experience using the flipped classroom approach as instructors of (bio)statistical content.

1.1 Literature on the Flipped Classroom in Biostatistics

The flipped classroom is gaining in popularity as evidenced by a recent survey of faculty from higher education institutions in the United States and Canada, as well as a small number of institutions abroad (Faculty Focus 2015). Seventy-five percent of respondents (representing a wide swath of disciplines) had tried flipping some element of instruction, and only about 5% indicated that they would not do it again (Faculty Focus 2015). The rising popularity is also evidenced by the timing of first attempt at flipping the classroom reported by the survey respondents: of the adopters, 32% had first tried flipping the classroom within the last year, 15% first tried one year ago, 17% first tried two years ago, 7% first tried three years ago, and 29% had tried more than three years earlier.

As the popularity of flipped and online course delivery has grown in recent years, so has the published research describing these approaches and comparing them to traditional course delivery. For flipped statistics classrooms, the majority of published research has been focused on undergraduate statistics education, particularly introductory statistics courses. In this context, some authors have found evidence of improved student attitudes in flipped courses compared to traditional delivery (e.g., Carlson and Winquist 2011; Wilson 2013, Peterson 2016). Less clear is the impact on student performance, with some studies showing better outcomes in the flipped design, while others have shown no difference between flipped and traditional delivery methods. The superiority of the flipped design was found by Peterson (2016), who reported improved short-term knowledge (i.e., higher final exam scores), as well as by Winquist and Carlson (2014), who showed improved long-term retention of knowledge (median of 1.7 years after completing the course). However, other authors have not found improved student outcomes under a flipped format versus traditional delivery methods (Gundlach et al. 2015).

Notably, there has been considerably less focus in the literature on flipping the classroom specifically for the teaching of biostatistics and at the graduate or professional level (where a considerable amount of biostatistics education takes place). What little published research there has been in these contexts has been narrowly focused. Hund and Getrich (2015) described replacing in-class software demonstrations with video tutorials that are viewed outside class time. In their small sample (n = 16), the videos were positively perceived. In a recent paper (Schwartz 2014), one panelist described his experience flipping the biostatistics classroom for nursing PhD students. Beyond these two articles, we were unable to find any other published work on flipped classrooms specifically in biostatistics. The objective of our work is to address an identified gap in the literature, based on our review, with regard to a dearth of publications detailing the flipped format as it pertains to the field of biostatistics education.

1.2. Overview of the Flipped Classroom

The “flipped” classroom reverses the traditional approach to lecture and student work. Specifically, didactic lectures are provided outside the class meeting time (typically as pre-recorded videos), whereas class time is largely devoted to hands-on activities. Importantly, it differs from a fully online format (in which there may be no in-person interaction between the students and the instructor) in that the flipped format maintains the in-class component as an essential component of the course. Indeed, this in-class interaction is key to the success of the flipped format, as this is where the active learning for students occurs. Gundlach et al. (2015) provided further elucidation on distinguishing features among the traditional, flipped, and fully online formats.

Pre-recorded lectures offer many advantages. For example, students may view the lectures at an optimal time for their preferences and schedules; for some, it could be early mornings, while for others, late at night, or somewhere in between. Furthermore, the video-recording allows for students to pause, rewind, and re-watch lecture material as necessary, and—on some platforms—to slow down or speed up the video playback.

The in-class activities can take a variety of forms, depending on the nature of the course. A partial listing of relevant activities might include critique of the scientific literature, analysis of real data using statistical software, or working on an exercise that would traditionally be assigned as homework. These should be carefully developed to align with the skills those students need to acquire. Open-ended exercises (such as posing a research question with a relevant dataset, but without structured step-by-step guidance) can develop confidence in approaching challenging problems without prescribed textbook solutions, hence mimicking the real-world approach to research. This type of exercise requires higher level synthesis by the students and represents a complex feat that is often difficult to address comprehensively in the confines of the lecture-based classroom, where a mere “telling” of the difficulties may not be fully appreciated by the listening students.

This flipped approach can be particularly useful in disciplines where the learning is enhanced through doing rather than by merely hearing. While didactic learning is essential, deeper learning occurs through practice. So, the flipped biostatistics classroom emphasizes the learning of biostatistics though practicing problems, performing related exercises, and analyzing real data. The activities help students to attain the mastery they need to address novel research questions.

The flipped environment allows instructors to monitor students as they execute these skills in the classroom. In this way, they can intervene to correct misperceptions in real-time. Otherwise, such a diagnosis may not be possible until the following class period, after the students have attempted to complete their homework.

Finally, a flipped approach has the added value of providing students the opportunity to cultivate essential job skills. As a primary example, small group work and discussion conducted in the classroom should develop teamwork and effective communication skills. These “soft” skills are critically important elements for collaborative research teams that involve the practice of biostatistics.

It is important to note, however, that there are some challenges associated with the flipped format. One such challenge is the substantial amount of time and effort an instructor would need to invest to convert an existing course that is in a traditional format to a flipped format. Another challenge is student resistance. We have found that a small proportion of students prefer the traditional format and do not gain an appreciation for a flipped course. Yet another challenge of note is the reliance on technology that may or may not be easily accessible at an instructor's particular institution.

1.3. Overview of Active Learning

Active learning is widely acknowledged to offer advantages in student learning outcomes across various disciplines (Hamdan et al. 2013) and is a critically important component of the flipped classroom model. Our collective experience suggests several advantages of active learning for the teaching of biostatistics in particular.

First, active learning allows students to learn from one another. Having small groups of students possessing complementary strengths and weaknesses can provide fertile ground for the simulation of multidisciplinary teams commonly found in the workforce. These groups can mobilize and leverage their resources to solve the challenge presented by an in-class activity. While students with weaker preparation clearly stand to benefit from the explanations and perspectives of their peers with stronger backgrounds, the latter group also gains the opportunity to deepen their understanding of the course material. It is often stated that one learns material at a much deeper level when required to teach or explain it (Fiorella and Mayer 2015, chap. 8).

Second, active learning benefits instructors by allowing a direct and immediate view of how well students comprehend the course material. An instructor may circulate throughout the classroom while students are working in pairs or small groups—eavesdropping on the group discussions or looking over their shoulders—which instantly reveals what areas students have mastered and which ones may require additional remediation.

This remediation can take many forms in the flipped classroom. For example, if it is widespread throughout the class, the instructor may pause the group activity and offer a focused “just-in-time” lecture. Or, the instructor might video-record a similarly short lecture on that topic after the class period for the students to view on their own prior the next class period.

Third, active learning permits more one-on-one interaction between the instructor and each student than might be possible in a traditional classroom setting. While circulating around the classroom, the instructor can provide in-depth explanations to individual students and small groups, some of whom might fear asking their questions in front of their peers.

2. Collective Experiences in Flipping the Biostatistics Classroom

Our panel discussion revealed that the flipped classroom is customizable and suitable for teaching biostatistics across a broad array of student populations and contexts. Represented on the panel were instructors of undergraduate students (including those with a premedical school focus, those studying public health, and biology majors), doctoral students (specifically, those pursuing a PhD in Nursing), and physicians (who were enrolled in a Master's degree program in clinical research).

One important emphasis that was evident across the panelists' discussions was the high degree of tailoring that is available to an instructor. While it may sound at first as though there is a single approach to implementing a flipped classroom, implementation can actually take on a wide variety of forms.

In fact, we advocate that a flipped approach can—and should—be tailored to instructors' specific target audience: their students. We found that the structure of the flipped classroom and the types of in-class activities that worked best were not universal and depended heavily on the type of learner and what aspects of biostatistics they needed to gain from the course. Rather than being a one-size-fits-all approach, the flipped biostatistics classroom offers a high degree of flexibility, as elaborated in the following sections.

2.1. Types of Students and Their Varying Needs

One panelist's experience dealt with undergraduate students taking a statistics course to fulfill a pre-medical school requirement. These students include majors from a variety of disciplines. A second panelist teaches undergraduate students majoring in human biology. Many of their students are intimidated by mathematical content and have had little prior statistical background.

Another panelist's course is intended as a “core course” for undergraduate students in the general field of public health, though not for biostatistics majors. These undergraduates are heterogeneous in terms of their mathematical training and comfort level with statistics. The course prerequisite is a “data analysis” general education course, that is, introductory statistics. In reality, however, students enter the course with a range of backgrounds, from students who have never taken a college-level statistics course (perhaps placing out via Advanced Placement credits) to students who have completed three or more statistics courses. A major benefit of the flipped design is the emphasis on hands-on activities with the instructor present, since many of these students may tend to struggle to complete even relatively basic computer lab activities on their own. In addition, public health is a collaborative field, and the flipped design encourages peer-to-peer interaction in the classroom through active learning. These students seem to value the personal attention afforded by the instructor during in-class activities, a benefit that may be rarely found by undergraduates at a large public institution.

Two other panelists use the flipped approach in graduate programs for health care providers or researchers. One panelist uses the flipped approach for PhD students in Nursing. These are students who are training to be nurse scientists and need to be statistically literate, though they may ultimately have access to a professional biostatistician to assist with their future research studies. The course is required for them and is typically taken in the second semester of their first year of coursework. It covers the topics of correlation, regression, and analysis of variance. The flipped approach to this course is thoroughly described in Schwartz (2014). The other panelist uses the flipped approach for physicians earning a Master's degree in clinical research. Most students are physicians who are beginning careers in academic medicine. Classes are designed to make students more statistically literate. Statistical training includes a required sequence of two courses taken in the first year of the program, covering topics such as data visualization, study design and planning, and regression modeling; each topic is enhanced through hands-on data analysis using statistical software. Using a flipped classroom for physician learners has been described by Samsa et al. (2012).

While the above discussion details the student groups with whom the panelists have personal experience, the flipped classroom could potentially be used for any class that includes an in-class component (i.e., not an entirely online course). A flipped classroom allows for customization to the unique demands of each student population through extensive in-class activities that pertain directly to the needs of these students for their future career activities. Removing the need to lecture during the class period frees up class time to add a substantial amount of active learning activities; a flipped approach hence offers an evidence-based advantage over traditional classrooms that lack such an active learning component.

2.2. Formats

In addition to targeting different populations of students, these courses vary in many other dimensions, such as the frequency and length of class meetings.

One course is four credit hours and meets once per week for a duration of 4 hr. The first of those hours is devoted as a computer laboratory, with a graduate student teaching assistant leading that portion. After a short break, the instructor leads the in-class activities for the remainder of the class period. One advantage of this structure is that it allows the students an entire week during which they are expected to view the video-recorded lectures before the next class. Additionally, a three-hour block allows the instructor to plan a more in-depth activity or, alternatively, multiple shorter activities.

Another flipped course meets twice per week, for 90 min per class. The first weekly session is converted to an optional office hour, where students may choose to drop in if they require supplemental assistance. The instructor believes this approach actually brings more students into her office than would be the case under a traditional format, thus increasing the amount of one-on-one instruction that that she provides. The second weekly class session is then used for experiential, hands-on learning activities. The audience for this course is practicing physicians: given their full schedules with limited flexibility, it is critical that the flipped nature of the course not add to the students' total time demands. Students are expected to watch 1.5–2 hr of online lectures per week in lieu of a mandatory second class meeting.

A third alternative involves two 75 min class periods per week. Both of these weekly sessions are similar in structure. There is an additional 75 min computing lab, which is led by a teaching assistant.

A fourth is a three credit flipped course having three meeting times per week, for 55 min each. However, one of these meetings is converted to office hours led by a teaching assistant, so that there are only two in-person meetings per week. The rationale for this is that the time for the third meeting is substituted with the time for students to prepare for class through viewing each week's video-recorded lecture.

A fifth alternative is a three credit course that meets twice a week for 90 min classes. Students are expected to attend both sessions, which are a mix between demonstrations led by the instructors and active learning exercises conducted by the students; each is organized around course modules. Students are expected to watch pre-class video-recordings covering basic content prior to coming to the first in-class session for each module. The instructor-led demonstrations build on the basic content from the video-recordings and cover content at a deeper level. The in-class activities then provide students with an opportunity to apply their new knowledge and to uncover any lingering misconceptions. The instructors for this course found that a “fully” flipped approach was not optimal for their students, given their lack of mathematical and statistical training. Instead, a “mostly” flipped approach was adopted, one that included a demonstration session that helped to bridge the gap between the students' lack of a quantitative background and the training style to which they were accustomed as physician learners (“See One – Do One – Teach One”).

The wide range of class formats represented by the experience of the authors helps to illustrate the wide applicability of the flipped classroom and helps to foster instructors' creativity in adapting the flipped format to whatever course structure in which they may find themselves. Whether the class meets once per week or more frequently, instructors can develop relevant activities to be completed during those class periods that can deepen their students' knowledge of the material in highly relevant ways. We suggest that there is not an optimal number of class meetings nor length of class period for a flipped course, in the same way as there is no single optimal design for a course taught with a traditional format. Strategically designing in-class activities, how long they may take, and whether a particular student population would be able to complete multiple activities (and take away enough from each without fatigue by the end) are important considerations. However, these considerations would also apply to a traditional format, since student fatigue (and loss of attention) can set in during long lectures.

2.3. Video Recordings

As previously mentioned, video recordings are primarily used to present the didactic lecture material to be viewed independently by the students. The format of these recordings also varies widely among the panelists.

Some may be professionally produced with the assistance of campus recording studios and other resources. This level of editing can greatly enhance the video-recorded lectures. For example, readability of statistical tables can be improved through zooming in or the highlighting of appropriate rows or columns of these tables. Instructor's mistakes during the recording (e.g., mispronunciation of a word, loss of their train of thought) can be edited out to avoid re-recording of the entire session. Also, creative touches, such as graphics, can be added.

Another panelist has used her campus' information technology resources to create video-recordings using a lightboard, which is a large piece of glass placed between the presenter and a camera (Figure 1). Adding light infusion, day-glow markers, and video-mirroring capabilities essentially provides a see-through chalkboard, where lecturers write content without needing to turn their backs to the learners or blocking what they write with their bodies. See www.youtube.com/watch?v=ANbBenxdi8I for one of the authors demonstrating use of this equipment. While this professional level of quality can add much value to the lectures, a notable trade-off is loss of flexibility in scheduling recording sessions and ease of editing.

Figure 1. A lightboard in action.

Another approach used by some panelists is to produce video-recordings using appropriate software in their offices. If an instructor has pre-existing Microsoft PowerPoint slides, Articulate (Articulate Global, Inc.) is a PowerPoint plug-in which allows audio-recording over PowerPoint slides. The instructor can incorporate animations, such as arrows and boxes, to mimic the instructor pointing to material projected on a screen in a traditional in-class lecture. One major advantage of this particular method of producing pre-recorded lectures is the ease with which students can re-watch specific content. A table of contents is automatically generated for each recording, enabling students to navigate easily to specific topics for which they are looking. Additionally, recording is usually done slide-by-slide, which forces the instructor to be cognizant of the length of time spent on each slide, and helps maintain content delivery in shorter, more easily digested segments. The ability to edit these recordings is another important benefit. One instructor recently changed course textbooks, and editing only the slides that used material specific to the prior textbook turned out to be quite simple, and certainly less time consuming than re-recording an entire lecture. Another option to consider is Camtasia software (TechSmith Corporation), which also provides editing capabilities.

Other instructors use hardware, such as tablets, that capture the instructor's writing to simulate writing on a blackboard or whiteboard. Computer software can synchronize the instructor's voice, image (if desired), and his handwriting to produce a comprehensive video-recorded lecture. Editing may be less seamless with this type of recording, so one option is to record a greater number of short lectures, such that the chance of needing to edit is reduced, or the notion of re-recording the entire lecture is less daunting.

Some panelists first write the entire transcript for the lecture—slide-by-slide—and then narrate their slides to create their video-recordings. This approach was motivated by a requirement to provide transcripts for accessibility purposes, but also allows multiple instructors to collaborate on the content even if only one does the actual audio recording. Other panelists simply “lecture” from their notes without fully scripting their video-recordings. Closed captioning can be added afterward.

While some instructors may create podcasts that can be downloaded by the students, our panel has tended to have students log on to the course website to stream the video-recordings. This implies that students would require access to an active internet connection.

One instructor assigns five to seven recordings per week, each lasting 5–30 min. Each video is accompanied by one to five quiz questions, which can be automatically graded via course management software (such as Sakai or Blackboard). This helps to ensure the students are viewing the lectures appropriately to gain the requisite level of comprehension of the material prior to progressing to the next lecture.

While many of these video recordings are didactic lectures, others are interactive where the student is expected to pause the video and solve a problem, prior to resuming the video-recording to view the instructor as she walks them step-by-step through the solution. Still others are demonstrations, which might consist of a software tutorial or a critique of an article from the relevant scientific literature.

Another demonstration approach used by a panelist (who also teaches a writing course) is to edit a 500-word essay in real time using tracked changes or to walk students step-by-step through a real data analysis. The instructor begins with data cleaning and checking, moves to data visualization, and proceeds with data analysis through fitting the final statistical model. Another instructor uses a similar approach when introducing new techniques using the R software.

A valuable use of video-recordings is to serve as a solution sketch to quizzes, homework problem sets, or exams. This approach can be particularly helpful in allowing students to follow (i.e., hear and see) the instructor's approach and line of reasoning in solving problems rather than just receiving a listing of correct answers.

One instructor has broken lectures into varying numbers of video-recordings, with some tending to be longer than others, depending on natural segmentation of that week's material. Interestingly, student feedback indicates that while some students prefer the longer videos, others like viewing a larger number of video-recordings having a shorter duration.

In addition to using video-recorded materials, an instructor may continue to assign readings from the course textbook or other relevant materials. Having these readings can appeal to some students, and it can be more efficient for students to skim their readings than to scan video if they are looking for a particular piece of information.

A summary of different uses of pre-recorded video content is provided in Table 1.

Table 1. Variations on types of video-recordings used by the expert panelists in their flipped biostatistics classrooms.

Didactic modules with or without embedded quizzes

Software tutorials

Interactive problem-solving videos (i.e., students pause the video to solve a problem before watching the solution)

Step-by-step demonstrations of real data analyses

Critiques of published articles

Interviews (such as with statisticians or journal editors)

Solution sketches

• Quizzes

• Homework

• Examinations

The use of video-recordings to deliver the key course content is one area where the flipped classroom is distinct from a traditional approach. While offering many advantages from both the instructor and student perspectives (e.g., the video recordings may be re-used over the course of several semesters; students may pause and rewind lectures as needed), instructors should take care not to underestimate the amount of time needed for this task.

2.4. Physical Layout of Classrooms

The structure of classrooms where flipped courses meet can also vary. One instructor uses a lecture hall with rows of tables for her classes having more than 30 students. Another uses a small classroom with rows of tables and chairs that can be re-arranged to allow for flexible grouping arrangements. Yet another uses a moderately sized classroom with auditorium-style seating for 50 students; the chairs swivel, facilitating the students' ability to collaborate with students in front of, behind, and on either side of them. A final variation uses a 30-seat computer lab, since the majority of in-class activities—including exams—require students to use computers. One limitation to this format is that the course is strictly capped at 30 students per section.

The key consideration for all of these classroom layouts is to think about how to facilitate student collaboration in that environment. For example, when there are fixed rows of tables, students can easily assemble into pairs or their small groups within the rows—or turn around to face classmates in the row behind them—for small group work. Moveable tables can be rearranged to accommodate group work. If activities involve computer work, and the class does not meet in a computer lab, then students will need to bring laptops, and thus having sufficient access to electrical outlets will require consideration.

One classroom layout that may seem daunting for use with the flipped design would be a large auditorium capable of seating a very large number of students (perhaps 100 or more). Though none of the panelists have actual experience teaching under this configuration, we suggest that it is adaptable for in-class activities and, thus, for the flipped design. One appealing idea for this layout is to group the rows of seats into threes(i.e., triplets), and then have students sit in the first and second rows of each triplet (assuming the class size is sufficiently small relative to the room capacity). This provides an empty row in between each pair of occupied rows. Within these pairs of rows, students may work with peers beside and behind or in front of them, as the case may be. The empty rows then allow for the instructor or teaching assistants to circulate freely throughout such a classroom that might otherwise be deemed as being too congested for such movement. One challenging aspect of this classroom layout in regards to in-class activities is that students may be crowded with their laptops. However, as students are increasingly using technology even in traditional courses (e.g., to type notes), this would seem to be less of a barrier.

Often, instructors do not have flexibility in choosing their classroom, but our experiences in a diverse set of classrooms lead us to believe that virtually any space could be creatively adapted for a flipped class; many considerations discussed here would also apply to a traditional format that involves some degree of in-class activities. And, just as a traditional format can accommodate many different classroom layouts, the flipped classroom is equally well-suited for a variety of classrooms. In fact, creative solutions, such as those mentioned above, can allow instructors to overcome challenging layouts so that a flipped format can be successfully applied across the gamut.

Certainly, in-class activities may be implemented in traditionally taught classes on a less-frequent basis, but since a flipped design often places a strong emphasis on peer-to-peer interaction, we recommend careful thinking about how to accomplish this interaction in a particular classroom. To the flipped classroom's advantage is the recent trend seen in some universities of replacing static furniture in classrooms to furniture that facilitates the types of active learning we describe in this article. For example, one of the authors teaches in a building with classrooms having wheeled chairs with a platform underneath to stow student belongings, so that varying configurations of seating (e.g., small group circles, directly facing pairs, rows) could easily be attained—or even modified during the course of a single class period —quickly and efficiently.

2.5. Class Sizes

Flipped courses in biostatistics can vary widely in their enrollment numbers, and this approach can be successfully applied to “small,” “medium,” or “large” classes. Our collective experience ranges from six doctoral students to 50 undergraduate students, with intermediate class sizes of 15–20 undergraduate students, 30 undergraduate students, and more than 40 graduate students. However, we are also aware of flipped biostatistics courses with approximately 120 enrolled students.

While a single instructor may be able to monitor smaller class sizes by themselves, having one or more teaching assistants can facilitate the oversight of larger class sizes, where there would be a large number of small groups. By design, most activities in the flipped classroom involve students working together, and we have observed in our classrooms that students helping one another actually can reduce the need for instructor intervention. Thus, even in a large class of more than 100 students, it is feasible for a single instructor (or perhaps an instructor plus one or more teaching assistants) to monitor the room by asking students to raise their hands when their group requires assistance. If budget constraints do not permit the hiring of teaching assistants, instructors could consider some creative options to recruit students to volunteer to assist. This could include inviting students who succeeded in a previous offering of the course to serve in such a role. These students would gain valuable teaching experience they could include on their résumé, would hopefully receive mentoring from the instructor (and a positive recommendation letter), and could possibly even receive course credit if this type of educational training is part of their program's curriculum.

The flipped approach need not be confined to classes of a certain size. As described in this section, small, medium, large, or very large classes can be successfully flipped. Of particular note is the fact that the advantages of active learning can be applied to allow for increased student engagement, even in large to very large classes. From our diverse perspective, flipped courses are not any better suited or worse suited for specific class sizes than traditional courses would be. For example, in a class with 100 students, the instructor burden would simply be shifted: instead of having 100 homework assignments to grade in a traditional course, the instructor might be facilitating in-class completion of a similar graded assignment in a flipped course.

2.6. Practical Considerations for Flipping a Course

In this section, we provide practical considerations to instructors considering the flipped approach for their courses. Foremost is the considerable upfront investment of time and effort required on the part of the instructor to flip a course. One panelist estimates that each hour of video-recording may take approximately 10 hr to create, though this number would be lower for other panelists. This includes preparation, recording, and the review and editing process. It also takes time to create online homework assignments and quizzes, if these components are incorporated into one's flipped course, as well as re-designing existing assignments and lecture examples to serve as in-class exercises that foster active learning.

However, an important perspective on this upfront investment is that once these materials are created, they can ideally be re-used for multiple audiences and classes across multiple sections and semesters. Though the short-term investment may be intense, the long-term time-savings can be even more beneficial. The same materials could even be used for multiple classes; for example, by designing the material to accommodate an advanced course as well as a more basic one. The more advanced mathematical material (such as proofs or calculus-based modules) could be prepared as separate modules/recordings that would be included as part of the advanced course, but would be removed for the more basic version of the course, without losing continuity across the course material.

We suggest that instructors should carefully plan and design their initial flipped courses with this long-term view in mind so that the flipped course can be implemented flexibly. For example, material that is not time-sensitive might be preferred to topics that could become quickly outdated. Additionally, one would want to avoid mentioning dates or current events that would become less relevant over time. For example, instead of stating a specific due date in a recording, one could refer to the “due date listed in the syllabus.” Care should also be taken when referencing a text, as future editions might not contain the referenced material on the same pages, in the same chapter/section, or perhaps at all.

Furthermore, an important point is that a course need not be flipped all at once. Given the amount of time required to flip an entire course, the idea of flipping one lecture or module per semester might have appeal to some instructors. Taking this approach, the course could attain a fully flipped status over multiple semesters.

On a related note, one of the panelists has flipped a single guest lecture for a graduate-level course for biostatistics majors. The course as a whole is not currently using a flipped format (though the instructors are considering moving in this direction). The video-recording for this guest lecture provided didactic material on the topic of sample size estimation and statistical power. Related in-class activities were incorporated to provide the students with hands-on experience with sample size and power calculations, including how this might be presented in the scientific literature, as well as hands-on exposure to relevant software. Response to this approach has been positive, and the author has recently reprised this flipped guest lecture with a minimal amount of effort and preparation. Students appreciate the opportunity to use class time to participate in relevant activities, to interact with classmates, and to complete what might otherwise be an assignment to be completed on their own time after class. Additionally, they have access to the instructor while doing this work, in case they have questions or require clarification.

We also note that instructors may be able to locate and repurpose already existing material. This would include material posted online (e.g., YouTube, textbook publisher websites, statistical software websites). If one is able to select well-tailored existing materials, this would represent a large savings of time and effort. However, we caution that if this material is not quite the right fit (e.g., slightly different use of notation), this could pose significant problems for the class that would require time and effort to backtrack and to rectify. In addition, such web-based materials may change over time—or even completely disappear—so care must be taken each time a course is offered to ensure that the materials still exist in the desired form.

3. In-Class Activities

This section details the use of in-class activities, which is an essential component of the flipped classroom. However, we should note that while some traditional courses may also incorporate some degree of in-class activities as an active learning component, the flipped approach of moving the lecture outside class time is effective in freeing up each class period to allow for a substantial amount of active learning during that class time. A traditional course would need to include the lecture during the class period, so a smaller proportion of class time (and possibly none) can be devoted to such activities. Our use of in-class activities varies according to the needs of the students, and exercises can be tailored to the expected statistical activities in which the students would be engaged for their chosen professions.

The in-class experiences we have implemented into our flipped classes include reviewing and critiquing the use of statistics in the scientific literature, as well as the proper reporting of those statistical methods. Examples include having students spot problematic issues in the literature and assigning students to act as a journal reviewer to submit a report using a simulated peer review format.

Other activities that we have used include solving pencil-and-paper problems; these can range from simple practice problems that review basic concepts through challenging problems which stretch the students' thinking. The class time can also be used to review homework exercises, including the possibility of having students take turns to present their homework solutions to the class.

Software laboratories can also fit nicely into the framework of a flipped classroom, either on a regular basis or as needed. While we have direct experience using SAS, JMP, and R in our classrooms, other statistical packages (e.g., SPSS, MATLAB, Minitab, Excel) would also be suitable. Activities can be designed to emphasize certain statistical concepts, such as selecting and conducting the appropriate statistical analysis given a relevant dataset and an appropriate research question.

Step-by-step training in data analysis with or without the use of statistical software is also a viable approach. Early in the course, students could follow a highly structured set of instructions with an accompanying worksheet that the students would complete and submit. As students' sophistication in data analysis increases, the approach may progress to increasingly unstructured exercises, where students are required to make their own discoveries and address research questions regarding the data.

Yet another experience is to have students work in groups on a class project that spans the entire semester. For example, such a project might start with selecting a research topic and relevant research questions. It could later progress to designing a survey, distributing it online, and collecting data. At the appropriate time, the students could perform the data analysis. Finally, the project would culminate in the creation of a scientific poster (in groups) and a paper (individually).

Another approach is to assign a term project for which each student incorporates the application of the semester's material to analyze a real dataset. In contrast with structured data analysis that constitutes much of that course's previous assignments, this unstructured analysis is often a useful exercise to provide unforeseen challenges to the students, such as recoding missing data values, considering inclusion/exclusion criteria, and using data that may not be quite in the format that the student would like (e.g., income given in ranges rather than as continuous values). These issues provide motivating fodder for deeper understanding of statistical challenges that exist in real-world applications. This is especially true for graduate or professional students in the health sciences who will likely have a specific focus and may appreciate analyzing data related to his/her chosen field of specialty.

We note that assigning term projects like those described above is certainly not unique to a flipped classroom; many “traditional” biostatistics courses use data analysis projects as an assessment tool. However, a flipped format would afford class time for students to complete some of this work. An advantage to this approach is that the students have immediate access to the instructor for questions, and the instructor can monitor group progress and group dynamics.

However, one should take precautions when using this approach, as students could possibly work in an impermissible manner with others outside their groups. At the other end of the spectrum, providing group-specific datasets would present other logistical challenges for the instructor, particularly finding a sufficient number of appropriate datasets as well as the need to understand the findings from each one. One noteworthy method that represents a compromise is to select a common dataset for the class, but then to select multiple random samples from it, one for each group. To facilitate similar, though not identical, results across the groups, a substantial majority (say, 75–90%) of the observations could be common to all datasets, with only the remainder being subject to random sampling. This would prevent plagiarism issues across groups but would simplify evaluation by the instructor, as the group submission should generally follow the same patterns, without being identical.

While not exhaustive, Table 2 lists various types of in-class activities that instructors may consider using in their biostatistics courses. Although these could also be assigned as homework exercises for out-of-class work under a traditional format, the flipped structure allows for instructors to facilitate these activities and to monitor them as the class time unfolds to ensure the students are gaining the appropriate knowledge and experience.

Table 2. Listing of in-class activities used by the expert panelists in their flipped biostatistics classrooms.

Detect statistical problems in published papers

Perform a simulated peer review of a manuscript

Solve pencil-and-paper practice problems

Solve out-of-the-box, challenging problems

Review homework exercises (including student-led review)

Analyze data using statistical software according to structured instructions

Analyze data using statistical software to address open-ended questions

Reproduce tables and figures from published papers (given the corresponding dataset)

Work on term projects in groups (including data collection, data analysis, write-up, and presentation)

4. Discussion

This article is intended to provide a broad overview of multiple aspects on flipping a biostatistics classroom. Our panelists brought diverse perspectives along many dimensions in regard to how a flipped approach could be implemented.

Rather than a single approach, we emphasize the highly customizable nature of the flipped biostatistics classroom. While one instructor's implementation may not suit another's particular needs, we argue there is sufficient flexibility inherent in the flipped approach to allow its adoption—in part or wholly—in virtually all environments and applications.

One theme that was repeated across the panelists was the overwhelmingly positive student feedback. In addition to potentially superior student outcomes (Winquist and Carlson 2014), the students themselves endorsed the approach. In particular, they appreciate the flexible nature of the flipped classroom, the fact that they can watch the videos on their own time and at their own pace, and the opportunity to practice their newly acquired skills in a monitored classroom setting.

As strong evidence in support of the students' acceptance and preference for the flipped biostatistics classroom, the authors have regularly received positive course evaluations directly related to the flipped approach, including the active learning component. See Table 3 for selected pertinent comments, grouped according the themes, including some negative or mixed ones. While our panelists found that the majority of comments related to the flipped classroom format were positive, it is important to note that there will be students who do not embrace the flipped format and claim a preference for a traditional lecture-based approach. One instructor found that across 126 evaluations (i.e., all students who responded to flipped course evaluations for the 2014-2015 academic year), there was only one negative comment regarding the flipped classroom format (included in Table 3). Similarly, another instructor found that among 55 evaluations comprising all students who completed course evaluations over three course offerings, only one was negative regarding the flipped course design (also included in Table 3).

Table 3. Selected student comments regarding the use of the flipped classroom for their biostatistics course.

Positive Comments

Theme: Active Learning

“The use of class time to strengthen concepts learned in our online lectures really worked for me.”

“I went into this class dreading biostatistics but I actually enjoyed a lot of the work we did in class.”

“This is the most practical statistics course I have taken. Great course!”

Theme: Video-recorded Lectures

“It was a nice change from sitting through boring lectures where instructors simply flip through powerpoint slides and talk.”

“This is a great way to get this information, especially given the natural way the content lends itself to small chunks. I also liked that I could rewatch segments and skip segments that I was already comfortable with. This course and the content it covers lends itself well to flipped classroom.”

“I absolutely love the flipped classroom model. I find it very helpful to be able to watch the lectures at my own pace.”

“[This] may be my favorite course ever. No joke–I'd watch the stats lectures or do stats homework when I needed a break from the coursework for my other classes. The flipped classroom design was effective; it kept the students engaged.”

“Flipped classroom format worked well for this class. I liked to be able to watch the videos in my own pace and I learned quite well. However, I would not have wanted to have this class format for a social science class where instructor interactions means so much more.”

“I loved it! I was able to gain a lot of information in the videos, which I don't think we would have had enough time to go over in class. I liked that I was able to pause the videos and learn the information on my own timing, unlike in class how you have to learn it the first time the instructor explains the concept or you have to go learn it on your own outside of class if you did not understand it the first time.”

“I liked it a lot because if I miss a concept I can always go back to the videos and rewatch lectures.” “I loved the flipped classroom format and I wish more of my classes did it. I was able to pause and rewind the videos when I didn't understand concepts. I was also able to watch the videos at my own convenience.”

Theme: Overall Approach

“I loved the flipped-classroom learning style used in this course… I learned more than I ever predicted I would be able to over the course of one semester!”

“The instructors should be commended for the way this course is organized and taught. They do an excellent job teaching to their audience—they are not trying to turn us into statisticians, but instead are trying to give us a working knowledge of statistics to be able to critically review the literature and organize and carry out research projects—which is exactly what they should be doing for this program.”

“The multi-faceted approach is excellent: lectures, hands on examples and activities, course videos and quizzes. Since I am in clinic seeing patients, performing database/clinical research and taking this course it was extremely helpful to have so many ways to review my understanding (including for ex, re-watching a lecture I already attended). Thank you!”

“Sometimes instructors for a topic like statistics project the attitude that ‘this is a difficult topic, not everyone will do well’. [The instructor] clearly goes out of his way to make material in a difficult topic accessible. The flipped classroom was a wonderful way to learn this material and obviously took extra effort on his part.”

“I actually enjoyed this format a lot. Initially I was a bit thrown off, but I was glad that I had the opportunity to understand and keep up with the material at my own pace. I think it may be one of my favorite teaching styles, and one of the most effective I've seen in a long time.”

Mixed/ Negative Comments

Theme: Perceived Extra Work

“Not a fan of the flipped classroom. It makes me feel like there is 2x the work and because the videos are online it makes it hard to concentrate on the presentation.”

“There were weeks that I feel like I spent too much for my time working on this class and not enough on my others, but having said that, the content itself was difficult so the time commitment was necessary.”

Theme: Removal of Interaction during Lectures

“I think that I would rather have in class lectures because you can ask questions right away.”

“Honestly I ended up not liking this format for biostats. I feel that biostats is a subject that I would have learned better in person and would have felt a lot more comfortable with the material.”

Theme: Course Pacing

“Parts of the course dragged and at times it felt like we were moving at the pace of the slowest student, but I learned a great deal nonetheless.”

“It was easy to get behind in watching videos, but that was mostly my own fault.”

“I liked the flipped approach, but would prefer the whole course was loaded on the website at the beginning of the semester rather than module by module - especially the videos.”

To address the themes of these negative comments, we note that the perceived extra work may or may not be accurate: for those students who may need to watch the video-recordings multiple times, the time expenditure could in fact be greater than what it would be for a traditional classroom lecture. However, we would argue this additional time is a necessary investment, as such a student would likely miss important points of the lecture as it was delivered in-class, and thus potentially risk misunderstanding subsequent material. Also, since we include graduate and professional students as the target population for some of our courses, our expectation is a greater level of maturity that should help in time management, as reflected by the comment indicating that one might “get behind” but also taking responsibility for that shortcoming. While it is true that a student does not have the opportunity to interact with the lecturer during the lecture, we assume the student still has the opportunity to contact the instructor prior to the next class to obtain the necessary clarification, whether it be through an electronic message, a phone call, or office hours. A worst case scenario would be for the student to request clarification at the beginning of the next class, prior to engaging in the in-class activities to ensure proper application of the material.

Finally, the comment that the “course dragged and at times it felt like we were moving at the pace of the slowest student” certainly could have also arisen in a traditional course as well. Indeed, one author previously using a traditional format had experienced a question asked by a student more than halfway through the class period that reflected a lack of understanding for a basic concept; the rest of the students waited while the instructor circled back to material covered previously. We suspect that in the flipped design, in which students interact with each other on a more regular basis, the shortcomings (e.g., slow place) of fellow students may be more obvious, perhaps as progress on group work may be slower than if a high-achieving student were working on his/her own. While the flipped design might accentuate this, we would argue that learning to work with people of differing abilities is a skill necessary for the “real world” and thus the flipped design is developing students' “soft” skills, perhaps without them even realizing it.

In addition, several panelists have collected data that indicate that a majority of students prefer the flipped classroom to the traditional classroom. On small post-course surveys given to two of her classes, one instructor found that 77.8% (21/27) of students agreed or strongly agreed with the statement: “I think that using class-time for in-class activities helped me to learn this topic better than a traditional lecture based course would have.” Of the remaining six students, five neither agreed nor disagreed and only one (3.7%) disagreed and indicated a preference for the traditional lecture-based course. Though in our experiences most students indicate a preference for the flipped classroom, an instructor using this format should not be surprised to have some exceptions to this rule.

Data external to our collective experiences validate the flipped format as well. In a recent survey of 1,089 college faculty (Faculty Focus 2015), more than 70% of respondents indicated that flipping was a positive experience for themselves, whereas only 7% rated it as a negative experience, and 22% rated it as neutral. Additionally, 65% rated flipping as a positive experience for their students, whereas only 9% rated it as negative, and 26% as no better or worse than a traditional format. In terms of student benefit, 75% of respondents believed that flipping increased student engagement, and just over half (55%) believed that the method improved student learning as well. The faculty overwhelmingly cited the time commitment required for flipping as the biggest drawback.

Many of our students do not possess a strong quantitative background, and the flipped structure empowers them to see that they are fully capable of comprehending and “doing” biostatistics. They gain this valuable experience through engaging in the classroom activities, individually or with their peers, under the watchful eye of the instructor or teaching assistant. We have found that most students are willing to engage with the activities and thereby learn the material; however, as mentioned above, a few may be resistant to exert this type of effort. We would not be surprised to find that these students who resist the flipped format and do not fully participate in the learning activities would likely report less satisfaction with the approach and would be less prone to experience success in the course.

Our panelists' experiences were specifically in teaching biostatistics, and we have focused this article accordingly. However, we believe our observations should be broadly generalizable to many statistics classrooms. One characteristic that distinguishes between biostatistics and statistics is the former's focus on applications in the health sciences; indeed, our students come from these fields. Additionally, biostatistics tends to focus on graduate education (though not exclusively), while statistics education would extensively encompass both undergraduate and graduate training. However, we see no compelling reason why this pedagogical approach would not be more widely applicable to other disciplines.

Pacing of the material is an important advantage of the flipped classroom. In one regard, the flipped approach allows the instructor to exert control over the amount of content covered in each class period, as students are assigned to view a certain amount of content prior to class. This is an advantage relative to the lecture-based approach where a confused student may ask a question that sidetracks an entire class period, putting the class behind schedule. In addition, the flipped approach allows for flexible pacing within a class period that accounts for variation in student mastery of the content. A basic set of activities should be completed by all students, but optional, higher level exercises may also be made available for those who finish early. Alternatively, the idea of having stronger students serve as peer teachers to those in their small group who might not be grasping the material has great value, since a deeper appreciation of the material is often gained when forced to explain it to others (Fiorella and Mayer 2015, chap. 8).

The timing for the accessibility of the course materials is another issue to consider. While generally in favor of the flipped approach because of the flexibility in viewing and re-viewing the course content online, some learners may voice a desire to have access to all course materials, particularly the video-recordings, at the outset of the course (see related comment in Table 3). While there are merits to both approaches (i.e., releasing the entire course at once versus releasing content module by module), ultimately in our courses we have found the gradual release of materials to be preferred. This provides the instructor with greater flexibility to adapt course content to the current class (e.g., adding an extra in-class exercise on basic ANOVA topics if a class as a whole is struggling with this concept). An alternative that would permit flexibility to accommodate students' busy schedules would be to release the material pertaining to the entire following week's class periods. Students can then plan their time and balance their scheduling across courses or other obligations.

We advise careful choice of technology that is consistent with one's educational purpose; instructors should strive to use technology as an aid to student learning. However, one would not want its use in the classroom to become a distraction or hindrance, as can be experienced when internet connectivity is lost, technology does not perform as expected, or students have an inconsistent experience (such as when software is not compatible across various computer platforms or operating systems). Also, an instructor should be aware that certain technology may not be supported by one's information technology unit, which is problematic when encountering technical difficulties. Additionally, technologies can become quickly outdated such that reliance on certain products may become obsolete or lose functionality over time.

We hope the lessons from our panelists provide a compelling argument for the utility of the flipped classroom when teaching biostatistics to a wide variety of student populations. Realizing the flexibility of the flipped approach, instructors can tap into their creativity to ensure the course is equipping the students with the skills they will need to be successful in their chosen professions. The panelists have found using the flipped approach to be a gratifying experience that frees them to interact with their students at a deeper level than might otherwise be possible under a traditional format.

Acknowledgments

This material is based on an invited panel presentation given at the Joint Statistical Meetings in Boston, MA in August 2014, entitled “Flipping the Biostatistics Classroom.” The authors express their appreciation to two anonymous reviewers, an Associate Editor, and the Editor, who provided helpful suggestions that strengthened this article, such as the suggestion of using “75–90% similar” datasets for group projects. They are also grateful to their students who have been willing to learn under a flipped format that may have been previously unfamiliar to them.