Flipping an Introductory Biostatistics Course: A Case Study of Student Attitudes and Confidence

Abstract

Flipped classrooms have become an interesting alternative to traditional lecture-based courses throughout the undergraduate curriculum. In this article, we compare a flipped classroom approach to the traditional lecture-based approach to teaching introductory biostatistics to first-year graduate students in public health. The traditional course was redesigned to include video lectures and online quizzes which the students were expected to complete before coming to class, followed by a short in-class lecture and time working on applied statistics problems in class. We compared the opinions of the biostatistics field and confidence applying biostatistics methods of 46 students who took the flipped course to 52 students who took the traditional, lectured-based course offering. We found similar end-of-semester opinions and levels of confidence between students in the flipped classes and those in the traditional, lecture-focused classes, though students in the flipped course reported very high satisfaction with the model.

KEYWORDS:

• Active learning
• Flipped classroom
• Online education
• Public health
• Video lectures

1. Introduction

The use of technology in introductory statistics courses has lagged far behind the use of technology in applied statistical analysis (Cobb 2007). Statistics instructors face difficult decisions regarding the trade-off between the amount—and level—of technology to introduce into the course, and content that must be adjusted or removed in order for appropriate emphasis to be given to all parts of the course. Flipped courses give one possible, albeit partial, solution to this problem: they have the potential to increase applied technology in the classroom while still allowing accordance with the GAISE recommendations for statistics education (Aliaga et al. 2005), such as a focus on statistical literacy and thinking, conceptual understanding, and in-class active learning.

The flipped course is an attempt to introduce students to course material outside the classroom so that face time with the instructor can be spent on deeper learning and application, “flipping” the locations of where these two processes traditionally take place. While flipped courses do not necessarily require a technological component, the push to include more technology in the classroom, or to use technology to reach nontraditional students, has made the flipped classroom a fashionable option. For a simple example of a flipped classroom without technology, consider a literature class in which students read the text prior to class, then come prepared for a class discussion on themes, plot development, or other aspects of writing. The flipped class follows naturally in this setting—literature instructors would rarely recommend spending class time reading a novel together before having students leave class to compose a response on their own. In fields such as statistics, the out-of-class introduction to the material can come, for example, from reading the textbook or watching video lectures or instructional tutorials.

1.1. Technology in Statistics Education

Online resources for introducing statistical concepts have existed in various forms for some time, for example, Claremont Graduate University's Web Interface for Statistics Education (WISE) (Romero et al. 2000). Such tools and applets have branched out into a wide array of material in the years since. Baglin (2013), however, distinguished between technology for understanding statistics and technology for doing statistics, introducing the integrative model of skill acquisition in the context of statistical software within the statistics curriculum. Baglin suggested that students’ prior statistical knowledge and course expectations may temper the degree to which technology can be introduced for “doing,” as such a process increases the cognitive load for novice statistics students. Overloading students with both new technological and new statistical knowledge might be akin to “trying to learn to use a word processor while also learning to read and write in a given language,” with potentially discouraging and deleterious effects on student engagement and outcomes (Baglin 2013).

Hund and Getrich (2015) described attempts to include video tutorials of statistical computing procedures to supplement in-class labs in first and second biostatistics courses for graduate students in public health. While their experience is not described as a flipped course, the use of tutorial videos is of interest to instructors considering this method of presenting a flipped course. Students appreciated the videos, particularly the ability to watch the videos whenever and where ever they could, and many believed they would reference the videos in the future. Importantly, the students did not feel that the videos would serve as an appropriate substitution for class time, as there were noted benefits to face time with the instructor.

1.2. Active Learning

One of the benefits of introducing active learning into the statistics curriculum is the sustained opportunity for students to work in formal and informal groups (Aliaga et al. 2005). Appropriately designed group work allows students to learn from each other and develop communication skills around the statistical process (Horton 2013; Hund 2015). Curran et al. (2013) described a peer-led team learning model. The study found improved exam scores for students in the team learning model relative to those not in the model, with greatest improvements in those students who performed poorly on the first exam of the course. In addition, students in the team learning program found statistics to be less difficult and generally had improved attitudes regarding statistics relative to their traditional counterparts.

In their analysis comparing a flipped course to a nonflipped active learning course in general education biology, Jensen et al. (2015) provided support for the hypothesis that the key to driving learning in the flipped setting is the inclusion of active learning rather than the flipped model of having the instructor present for the application stage of learning. They found that exam scores were similar in the two courses, and in both courses, students rated their time with the instructor as most important to their learning, whether that was at the content acquisition phase or the application phase.

1.3. Student Perceptions from Flipped and Active Learning Statistics Courses

Previously published summaries of flipped introductory statistics courses include Carlson and Winquist (2011) and Wilson (2013). For current purposes, it is worth noting that both Carlson and Winquist's and Wilson's flipped courses required reading as opposed to online videos as preparation for class. Carlson and Winquist (2011) investigated student attitudes toward statistics, finding that although students in the flipped course tended to find statistics more difficult than those in the traditional comparison group, these students also liked statistics more and felt more confident in their own statistical abilities. Although they did not measure student learning outcomes, Carlson and Winquist argued that positive attitudes may lead to improved academic behaviors as students understand the importance of statistics and believe statistics to be a less-daunting undertaking. In a follow-up study (Winquist and Carlson 2014), the researchers found that students experiencing the flipped class had higher statistics retention up to two years after completing the course, relative to students in the traditional lecture course. The researchers attribute this difference to the “generation effect,” the belief that generated material is retained at a higher rate than read or otherwise provided material.

Wilson (2013) described her class for undergraduate psychology majors as “half- or three-quarters flip,” with a short lecture at the beginning of the in-class meeting, followed by active group work. Wilson noted that with the Internet, the classroom and instructor are no longer the sole sources of knowledge, and that instructors become relied upon more for guidance than exposition. Wilson's students noted the increased amount of outside preparation, particularly in reading the textbook. This effort was rewarded, however, with better performance on the department's standardized assessment, compared with students from past traditional courses. Horton (2013) also described an active learning approach to an upper-level mathematical statistics course. Though much of the content of such a course is obviously beyond introductory statistics, Horton reached similar conclusions that students realized more before-class preparation was necessary and felt they obtained a deeper understanding of the problems with which they were tasked.

In the current article, we describe an initial attempt to modify an introductory biostatistics course taken by graduate students in public health by incorporating video lectures to be watched before coming to class, opening class time for more conceptual topics and group work. In Section 2, we describe the course, both in its previous and flipped incarnations. Sections 3 and 4 detail the study methods and results, respectively, and Section 5 closes with concluding remarks and next steps.

2. Revising the Introductory Course

At the investigators’ institution, the introductory biostatistics course is taken by nearly all students in the first year of the master of public health (MPH) program, usually in their first semester. There are no prerequisites or assumptions made about students’ statistical training. This necessitates that the graduate-level introductory course is similar to an undergraduate course in depth, though more content is covered. The only students who do not take this course are students concentrating in biostatistics, who are expected to have a stronger math and statistics background and take a theory-based biostatistics course during their first semester.

To fit with within the college's structure and assure some level of homogeneous student experience, there are similarities across all introductory biostatistics sections. Each section consists of 20–30 students and meets once a week for 150 minutes throughout a 15-week semester. All sections use the same textbook (Triola and Triola 2006) and are expected to cover the same material, using IMB SPSS (IBM Corp 2013) as the main computational tool. Course content includes basic probability rules, descriptive statistics, inference for the mean in one- and two-sample settings (independent and paired sampling), one-way analysis of variance, correlation and simple linear regression, inference for a single proportion, ${\displaystyle {\chi }^2}$ tests for goodness of fit and independence/homogeneity, McNemar's test, and confidence intervals for odds ratios and risk ratios, as well as brief introductions to logistic regression and some nonparametric methods. Each course also includes small group presentations of the statistical findings from a public health research article and concludes with a final exam. Regular homework assignments and quizzes throughout the semester keep students up to date with the material and give the instructor assessment feedback.

2.1. Traditional Course Sections

Though each instructor has freedom in presenting the material and hence, their own style, the traditional course was lecture-based. Students would spend varying amounts of time working in class on brief worksheets covering material just presented or going through SPSS steps to run the procedures just discussed. In general, this was less than 30 minutes per week. While the weekly time spent on active learning was relatively low, 3 of the 15 weeks were dedicated “lab” days during which students would be given an applied assignment requiring them to use SPSS to answer multiple questions from a relatively large dataset using all of the techniques they had been exposed to thus far. Students generally worked in pairs throughout any given lab and completed the lab as homework due the next week.

2.2. Flipped Course Sections

In teaching the traditional introductory course, a number of complications arose. Primarily, 150 minutes is a long time for an introductory statistics lecture. Also, trying to supplement the lecture time with more active learning in class led to a time crunch, making it difficult or impossible to cover the required material thoroughly. Second, while teaching the second course in the statistics sequence, which focuses on linear models, it became apparent that many students were not comfortable using SPSS to analyze data, even though they had just completed a full-semester course which used the same software package. Flipping the course by placing lectures online and giving students more SPSS time in class presented a potential solution to both of these problems.

The introductory course was revised so that students would watch instructor-created lecture videos and take short quizzes online before coming to class. Most weeks required watching three videos (Week 5 had four videos, Week 12 had two). With some exceptions, each video followed the same structure of introducing students to a single new statistical method, its assumptions and mechanics, and how to run the test using SPSS. In an attempt to follow recommended practices (Guo et al. 2014), videos were kept to approximately 10 min, though individual videos ranged from 5.2 to 14.7 min (mean 9.8 min). Figure 1 displays a bar plot of the length of each video, separated into lecture (PPT) and tutorial (SPSS) content. Video lecture time ranged from 2.3 to 12.6 min (mean 7.5 min). Ten videos had no tutorial content; in the remaining 23 videos, tutorials ranged from 1.9 to 5.6 min (mean 3.3 min). In their analysis of videos from edX courses, Guo et al. noted a difference in the watching patterns of lecture and tutorial videos. These video types were not distinguished in the current course, as all tutorials were given within the same video as the lecture on the topic. A list of topics covered in each video is given in Appendix A (the Appendices are available in the online supplementary information).

Figure 1. Video length divided into lecture (PPT) and tutorial (SPSS) time.

Every video was followed by a low-stakes online quiz. Szpunar et al. (2013) showed that brief, regular recall tasks such as quizzes or tests have numerous benefits, chief among them reducing mind-wandering, encouraging note-taking, and helping alleviate anxiety for larger, cumulative exams. Each online quiz consisted of three multiple choice or matching questions and required students to recall basic information from the video; no synthesis of knowledge was expected at this point. Question types were chosen so the quizzes were able to be graded automatically through the online course management system and students received immediate feedback. After taking the quiz, students were able to see the problems they got wrong and a general comment from the instructor. They could then attempt each quiz a second time before the due date, allowing them to re-watch the video before the second trial. The quiz performance throughout the semester accounted for 10% of the students’ overall grades.

In class, lecture would last approximately one hour. With the basic mechanics for each test already introduced through the videos, lectures were able to focus on meeting GAISE requirements such as placing inferential techniques in the context of real-world problems and statistical thinking and literacy. Some topics covered explicitly included random sampling versus randomized assignment (Cobb 2007) and cause and effect, statistical versus practical significance, and the underlying issue of power (Utts 2003). Though these are not uncommon topics to cover in a traditional course, they are often given short thrift due to time constraints. In the flipped classroom, the entire lecture could be devoted to explanations and examples of these issues. Lectures often naturally evolved into discussions between the instructor and students, or between the students themselves, as students tend to find the less-mathematical concepts more intuitive (Chance and Rossman 2001) and feel more confident in speaking out in class.

A specific example of the utility of these discussions could be found while covering one-way ANOVA. With Bonferroni comparisons introduced in their own video, the lecture material centered around the reasoning for multiple comparison adjustments. What students found most interesting and relevant was not the use of multiple comparisons as post hoc tests, but rather in comparing many outcomes across treatment and control groups, for which many applied published examples can be found. This led students to bring up issues such as data-dredging and publication bias, which had not been planned to be covered. The extra time allowed by placing the mechanical steps of the ANOVA and Bonferroni tests online allowed time for a much deeper discussion of the true depth of the multiple comparisons problem that likely would not have occurred if the topic was left for the last few minutes of class, after students had sat through a lecture on adjusting alpha levels and p-values.

At the conclusion of the lecture, students were given the remaining time (usually 1–1.5 hr) to work on their weekly assignment. Students were encouraged to work together on the assignment and completed the assignment outside of class as homework to be turned in online before the next class meeting. Each cluster of tables sat approximately six students, though students usually worked in smaller groups, coming together to the full table to ask questions and check answers with each other. Thus, Roseth et al. (2008)'s suggestion of 2–5 students per group tended to follow naturally.

3. Methods

In the fall of 2014, there were four sections of introductory biostatistics. The two sections taught by the first author were taught using the flipped format, while the two traditional sections were taught by two different instructors. Students registering for this course were first-year MPH students, who registered before arriving on campus, and senior undergraduate public health majors. In neither case did students have the opportunity to have a previous course from any of the three instructors, though the undergraduate students may have had some prior expectations due to word of mouth.

During the first meeting of the semester, all students were informed of the study and asked to fill out a “First Day Survey.” A similar “Last Day Survey” was given during the last meeting of the sections. First and last day surveys were linked through a student code. Sample sizes for each section are given in Table 1. The last day survey is given in Appendix B—the first day survey includes the same items except for the omission of the final set of questions under the heading “Course Experience.” This study was approved by the University's Institutional Review Board.

Table 1. Sample sizes for pre-and post-course surveys.

	Flipped A	Flipped B	Traditional A	Traditional B
Pre-course surveys collected	27	25	32	22
Post-course surveys collected	25	21	30	22
Loss to followup (%)	2 (7%)	4 (16%)	2 (6%)	0 (0%)

Differences in demographics and baseline statistical perceptions were compared using Cramer's V effect size measure and Fisher's exact tests. Similar methods were used to assess students’ changes from course start to completion between the flipped and traditional classes. Post-course responses from the “Course Experience” section were also compared, using Cohen's d and Wilcoxon rank sum tests for numerical variables. As this is treated as an exploratory analysis, discussion will focus on effect sizes with p-values available as supplementary information. Using standard rules of thumb, effect sizes between 0 and 0.1 are considered negligible or nonexistent, 0.1–0.3 small, 0.3–0.5 moderate, and greater than 0.5 large.

4. Results

After matching first day surveys to last day surveys, there were a total of 98 respondents. One student was dropped from the flipped class group due to excessive missing data in the baseline questionnaire, leading to a total of 97 respondents (45 from flipped sections, 52 from traditional sections). There were no notable demographic or attitudinal differences between the flipped and traditional students at baseline. From Table 2, we see approximately 70% of participants were female (not uncommon in public health programs) and 60% had some previous statistics course. Of those who had taken a statistics or research methods course, the most common computational tools used were Excel (31%) and graphing calculators (25%). SPSS—the package used for this course—had been used by about 20% of all students with previous statistics experience.

Table 2. Student demographics and previous experience: counts and percentages.

	Flipped (n = 45)	Traditional (n = 52)	Effect size^a	p-Value^b
Gender			0.02	0.828
Male	14 (%)	15 (28.8%)
Female	31 (%)	37 (71.2%)
Current Degree Program			0.19	0.246
BS	11 (24.4%)	22 (42.3%)
Accelerated BS/MPH	3 (5.2%)	2 (3.8%)
MPH	30 (66.7%)	27 (51.9%)
Other^c	1 (2.2%)	1 (1.9%)
Previous Statistics Course			0.04	0.835
Yes	27 (60.0%)	33 (63.5%)
Previous Research Methods Course			0.11	0.374
Yes	11 (24.4%)	18 (34.6%)
Calculation Tool^d			0.37	0.221
By hand/basic calculator	2 (6.7%)	5 (13.2%)
Graphing calculator	5 (16.7%)	12 (31.6%)
Excel	11 (36.7%)	10 (26.3%)
Minitab	2 (6.7%)	0 (0.0%)
SAS	2 (6.7%)	0 (0.0%)
SPSS	5 (16.7%)	9 (23.7%)
Stata	1 (3.3%)	0 (0.0%)
R	2 (6.7%)	2 (5.3%)

NOTES:

^aCramer's V.

^bFisher's exact tests.

^cOne MS. Biomedical Engineering (flipped) and one PhD (traditional).

^dPercentages are with respect to the number of responses to this question. Zero responses for options JMP and Other (not shown).

Student responses to their views of the utility of biostatistics and confidence in understanding and applying biostatistics are shown in Table 3. Students were very aware of the necessity of biostatistics in public health research and practice, as over 90% of students responded Agree or Strongly Agree to these questions. They were less convinced of the relevance of biostatistics to their own career goals (under 80%), or to the lives of the average citizen (53%). Just under half of the students felt confident in understanding statistical results in the media and public health research. Confidence in performing statistical methods ranged from a low of 7% Agree/Strongly Agree (ANOVA, n = 7) to a high of 40% Agree/Strongly Agree (one-sample t-test, n = 39). Given that over 60 students reported having a previous statistics course, these numbers are indicative of the difficulty of retaining statistical knowledge, as noted by Berenson et al. (2008) and Tintle et al. (2012), among other places. There were no notable differences between the flipped and traditional sections, with even the largest difference, confidence in performing and interpreting chi-squared tests, consisting of a relatively small effect size.

Table 3. Counts and percentages of students answering “Agree” or “Strongly Agree” to each question on baseline questionnaire.

	Flipped (n = 45)	Traditional (n = 52)	Effect size^a	p-Value^b
Views of biostatistics
Relevant to PH research	43 (95.6%)	50 (96.2%)	0.06	0.663
Relevant to PH practice	42 (93.3%)	49 (94.2%)	0.06	0.703
Relevant to my career	33 (73.3%)	42 (80.8%)	0.11	0.344
Important for average citizen	23 (53.1%)	28 (53.8%)	0.04	0.840
Confidence in biostatistics
Interpreting claims in media	29 (64.4%)	28 (53.8%)	0.09	0.415
Understanding results of PH research	18 (40%)	24 (46.2%)	0.07	0.543
Performing one-sample t-test^c	16 (35.6%)	23 (44.2%)	0.10	0.410
Performing independent-sample t-test	9 (20.0%)	17 (32.7%)	0.15	0.172
Performing paired-sample t-test	8 (17.8%)	12 (23.5%)	0.07	0.617
Performing analysis of variance	4 (8.9%)	3 (5.8%)	0.06	0.703
Performing chi-square test for indep	12 (26.7%)	25 (48.1%)	0.23	0.036
Performing linear regression	13 (28.9%)	18 (34.6%)	0.07	0.523
Performing logistic regression	7 (15.6%)	6 (11.5%)	0.05	0.767

NOTES:

^aCramer's V.

^bFisher's exact tests.

^cQuestions asked whether students felt confident performing and interpreting the results from each type of analysis.

Table 4 gives the number and percent of students whose responses increased from first day baseline to last day follow-up as well as responses to questions about students’ course experience. There were no notable differences in changes in students’ attitudes after the flipped course relative to the traditional course, with only a few questions reaching the level of a nonnegligible effect. There were, however, notable changes in students’ course experience. Students in the flipped class reported spending more time preparing for each class (mean 2.5 hr vs. 1.7 hr, d = 0.45) and more time working on homework outside of class (mean 3.3 hr vs. 2.7 hr, d = 0.32). Extra time preparing for class makes sense due to the videos and quiz required before attending class. On the other hand, time spent on homework is surprising since the flipped class students regularly had at least an hour in class to begin their homework assignment and homework assignments from all classes came from the same source.

Table 4. Attitudinal changes and course experience.

	Flipped (n = 45)	Traditional (n = 52)	Effect Size^a	p-Value^b
Views
Relevant to PH research	13 (28.9%)	11 (21.2%)	0.08	0.484
Relevant to PH practice	13 (28.9%)	15 (28.8%)	0.01	1.000
Relevant to career	19 (42.4%)	15 (28.8%)	0.13	0.210
Important to average citizen	16 (35.6%)	14 (26.9%)	0.09	0.511
Confidence Understanding
News media	21 (46.7%)	30 (57.7%)	0.12	0.311
PH research	29 (64.4%)	34 (66.7%)	0.02	0.836
Confidence Performing and Interpreting
One-sample t-test	34 (75.6%)	38 (73.1%)	0.01	1.000
Independent samples t-test	37 (82.2%)	41 (78.8%)	0.02	1.000
Paired samples t-test	39 (88.6%)	42 (82.4%)	0.05	0.790
ANOVA	40 (88.9%)	49 (94.2%)	0.12	0.298
Chi-square test for independence	31 (68.9%)	32 (61.5%)	0.06	0.673
Linear regression	32 (71.1%)	38 (73.1%)	0.04	0.823
Logistic regression	33 (73.3%)	40 (76.9%)	0.06	0.645
Course Experience
Hours spent preparing for class	2.5 (2.13)	1.7 (1.42)	0.45	0.016
Hours spent completing homework	3.3 (2.00)	2.7 (1.80)	0.32	0.046
Topic Learned Most			0.24	0.061
Theory	3 (6.5%)	11 (21.2%)
Reasoning	18 (39.1%)	12 (23.1%)
Analysis	24 (52.2%)	27 (51.9%)
Topic Learned Least			0.27	0.033
Theory	38 (82.6%)	32 (61.5%)
Reasoning	2 (4.3%)	10 (19.2%)
Analysis	4 (8.7%)	8 (15.4%)

NOTES:

Views and confidence: counts and percentages of students with increased score from pre-course. Course experience: means and standard deviations (in hours) for numerical variables, counts and percentages for categorical variables.

^aCramer's V for nominal variables, Cohen's d for numerical variables.

^bFisher's exact test for nominal variables, Wilcoxon rank sum test for numerical variables.

In assessing course experience, students were also asked to order three broad topics, statistical reasoning, statistical analysis, and statistical theory, in order from what they felt they personally had learned the most about in the course to what they had learned the least. The two course structures were fairly similar, with only small effects reported (V = 0.24, 0.27, respectively). Further investigation showed that the responses from the flipped class were nearly identical to one of the traditional classes, but the second traditional class was notably different from the other three courses (results not shown). This finding may indicate a substantial difference in the content focus across the sections, though further analysis was not undertaken due to prohibitively small sample sizes at the class level.

Figure 2 shows the changes in percent of students agreeing or strongly agreeing they feel confident in interpreting and performing statistical analyses. As there were few notable changes between the flipped and traditional classes, either at baseline (Table 3) or with regards to change from beginning to end of semester (Table 4), the figure shows data for the two class types combined. The percent of students originally reporting lack of confidence who, at the end of the course, reported confidence ranged from 69% (logistic regression) to 95% (one-sample t-test), with a median of 87.5% across the questions displayed in Figure 2, and all changes observed were statistically significant (McNemar test p-values < 0.001). As this course is designed to be part of a professional degree for nonbiostatisticians, an expressed goal of the course is for students to be able to read and understand public health research. We see a notable change in this response, with fewer than half of the students reporting confidence in doing so prior to the course, but over 95% feeling capable afterwards.

Figure 2. Changes in student confidence from beginning to end of course.

Though there were no substantial changes in attitudes and confidence regarding statistics from the flipped class students, there were still encouraging signs. In post-course IDEA evaluations, submitted by over 60% of flipped class students, the course format was consistently noted as a strong point. Students liked watching the videos before coming to class in exchange for shorter lectures. This response may be due in part to the current format of meeting only one time per week. Students also appreciated the time working on assignments in the presence of the instructor and teaching assistant. The most common suggestions for improvement were more thorough feedback on the homework assignments and, not surprisingly, more real-life examples. Similar to other literature (Horton 2013; Wilson 2013), students noticed this course took more individual effort than the traditional courses. For the most part, students saw benefit in the trade-offs, as there was only one negative comment specifically about the flipped format.

5. Discussion

We adapted an introductory biostatistics course designed for first year masters of public health students to a flipped course, which included online video lectures and quizzes and in-class work time. In order to assess its impact, student confidence, on a five-point Likert scale, was acquired at the beginning and end of the flipped courses and compared with responses from traditional, lecture-based approaches in courses concurrent with the flipped courses. Though student responses increased over the course of the semester, we note no noticeable differences in student's perceived efficacy in performing basic statistical methods or understanding statistical claims in media and public health research by course format.

This study provides only a snapshot into a very specific instance of course redesign and comes with a number of limitations. With respect to the redesign itself, the course was still restricted to fit within the structure of the college, which meant the course could only meet one time per week, had a required list of topics, and a required textbook. Adhering to such requirements limits the total amount of flexibility available and may decrease the utility of the flipped structure. Outside of the limitations to the course restructure, this study has a number of methodological limitations. Primarily, there is strong confounding in that both flipped courses were taught by the same instructor; the two traditional sections were taught by other instructors, making it impossible to reliably attribute any differences to the class structure. Also, the sample size is relatively small, with fewer than 100 total subjects completing both the pre- and post-course surveys, and the study sample comes from a very specific program. As such, the results may not be applicable to a broader range of introductory statistics students. In particular, it may be that graduate students are more open to teaching methods which require more pre-class preparation and individual accountability. Though there are many required similarities across sections, there is not a common exam, so assessing content knowledge on a standardized scale was impractical in the given setting. The focus on student attitudes and confidence sought to work around this issue (Carlson and Winquist 2011), as questions such as those posed in the surveys take much less class time than a content exam.

Although no statistically significant improvements were found, the flipped course still holds as a reasonable alternative to a traditional, lecture-based format. In future iterations of the course, we plan to balance textbook problems with more fully developed assignments based on the previously discarded labs used in the lecture setting, in addition to adding a larger writing component (Moskovitz 2011). We may also experiment with a more structured work time, as a small but consistent number of students regularly did not take advantage of the class time afforded by the flipped format. A more structured environment may keep these students more engaged and further improve the thought process going into statistical analysis for the other students. In addition, a mid-quarter review revealed that students wanted a review of the videos during class. At first glance, this may seem to lessen the necessity of the pre-class videos, but we found that a very brief overview of the videos (less than 5 minutes, only addressing details from specific student questions) tended to satisfy most students’ needs. After a first trial, the flipped class method seems a promising alternative to traditional lecture-based biostatistics courses for public health students.

Supplementary Materials

Appendices referred to in this article can be accessed on the publisher's website.