Flipped Calculus: A Study of Student Performance and Perceptions

Abstract

Flipping the classroom refers to moving lectures outside of the classroom to incorporate other activities into a class during its standard meeting time. This pedagogical modality has recently gained traction as a way to center the learning on students in mathematics classrooms. In an effort to better understand the efficacy of this approach, we implemented a controlled study at a small liberal arts college. We compared two sections of the entry-level course applied multivariable calculus I, with one section taught in a traditional lecture-based format and the other taught as a flipped classroom. During our study, we collected and analyzed data related to student performance, as well as perceptions of the approach and attitude toward mathematics in general. Students in both classes scored similarly on graded components of the course, and the majority of students were comfortable with the format of each section. However, some student perceptions and study habits differed.

Keywords:

• Flipped classroom
• applied calculus
• screencast
• teaching with technology
• student perceptions

1. INTRODUCTION

In an era when the value of a college education is under close scrutiny, and when there is more emphasis than ever on assessment, making maximum use of class time is of utmost importance for college instructors. To this end, one important pedagogical advance has been the development of active learning modalities. Active learning refers to pedagogies that put responsibility on learners by engaging them in activities other than simply listening. A recent meta-analysis considered 225 published experimental studies of active learning in science, technology, engineering, and mathematics (STEM) disciplines [8]. The meta-analysis concluded that average exam scores were 6% higher in active learning sections, and that students in classes with traditional lecturing were 1.5 times more likely to fail a course than peers in a class with active learning. The increased exam score and decreased failure rate effects were found to be most pronounced in small active learning classrooms (with fewer than 50 students).

Technological advances in the past several decades have enabled new modes of active learning. Some of these important technological advances include high-bandwidth internet, cloud computing, video-sharing websites, and broadcast methodologies.

One synergy of effective active learning pedagogies and technological advances is the so-called flipped classroom, a pedagogical modality that involves moving the lecture component of a course outside of the classroom to incorporate other activities during class meeting time. In this paper, we describe a controlled study that compares two sections of a popular introductory-level mathematics course (both taught by the same instructor) using a traditional lecture approach in one section and a flipped approach in the other. Our data reveal that students performed similarly on graded components of the course, and the majority of students in both sections were comfortable with the format of the course, although each section indicated a desire to include components of the format from the other section. Students in both sections also reported that they found benefit in resources created for the flipped section but were made available to both sections. However, students in the flipped class seemed to have a higher comfort level with performing computations using a computer software package, a central component of the course.

The rest of this paper is organized as follows. In Section 2, we describe the background on the flipped classroom pedagogy and discuss a few key entries into the new and burgeoning literature on flipped classrooms. Section 3 provides details on our study and presents the main results. We discuss both student performance (based on graded course items) and student perception (based on a structured discussion and on an end-of-course survey). Finally, in Section 4 we conclude and offer observations about our study as well as instructor perspectives.

2. BACKGROUND ON THE FLIPPED CLASSROOM

The design of a flipped classroom can vary widely depending on discipline, institution, course, and even instructor. However, a common, shared approach is that most of the transmission of course content is moved outside of the classroom, either by reading a text, viewing course notes, exploring online materials, using an intelligent tutoring system, or watching screencasts of lectures or annotated slides. In turn, more assimilation and practice with the content occurs during class time [17]. In a flipped course, class sessions might be devoted to discussing challenges uncovered in students’ pre-class preparation, working through problems, and engaging in collaborative learning with peers. Various flipped models include quizzes to assess student pre-class preparation, in-class clicker (personal response system) questions, just-in-time teaching, working at the board in pairs, lab assignments, problem-solving in groups, peer discussion, student presentations, and doubtless, countless others [3, 9, 11–14, 17]. Flipping can be applied in many different modes, for example, as a one-time class to teach a single topic, as a series of lessons, or as a design of an entire course [18]. Recently, studies in STEM disciplines and economics have looked into the flipped approach. A common thread in many results is that while students go through an initial phase of adjustment to a flipped approach, many view the flipped (or alternatively, “inverted”) approach favorably in the end.

Work in [11] describes how the inverted model can appeal to all types of learners since more than simply a lecture occurs during class time. The model also allows for more personal interactions, both between students and their professors as well as their peers [19]. Students also have immediate access to help while working through difficult problems and can clear up misconceptions early on [4].

In contrast, some feel that “flipping is simply a high-tech version of an antiquated instructional method: the lecture” [1]. This drawback largely centers on classrooms where the majority of class time is spent doing homework problems and not engaging students in other learning activities. It is also not always clear if flipping actually has a direct effect on learning [2]. Another concern with flipping is that if students do not actually prepare before class, class sessions will not be effective [13].

However, incorporating active learning into classrooms has shown positive gains in terms of both student performance and perceptions [5, 8, 10, 11]. Thus, it is desirable to better understand the effect of flipping a classrom. We now describe three small-scale, quasi-experimental studies in the mathematical sciences – that compare the flipped classroom model with a traditional lecture and help to frame our own study.

In a single unit of a second-semester calculus course – namely, a unit on the applications of the definite integral – the author of [13] flipped one section of the course (the experimental section) and compared it to a non-flipped control section. Performance on the first exam, which covered prior units taught to both sections in a traditional lecture format, was similar in the experiment and control groups. However, the second exam covered the unit in which one of the course sections was flipped. The median score was nearly seven points higher for students in the experimental section. Of all students in the flipped section, 77.9% prepared before every class by watching the assigned lecture videos. Perceptions of students in the experimental section were investigated as well, with the majority of students indicating a preference for the flipped format.

An entire introductory statistics course was flipped and compared with a traditional lecture course in [16]. This mixed methods study used field notes, interviews, and focus groups to consider student perceptions of both courses. Students in the flipped course generally felt less satisfied with the structure of the course, yet they were more amenable to cooperative learning and innovative teaching methods than their counterparts in the traditional course.

Finally, [12] compared a flipped section to a traditional section, but this time in a linear algebra course. Analysis of student performance across three course exams and a final showed that students in the flipped course saw more gains on the last two exams as they became more accustomed to the flipped classroom environment but performed similarly to the students in the traditional course on the final exam. Analysis of an end-of-term survey revealed that the majority of students had a positive attitude about the flipped course. The survey also revealed a higher comfort level of talking with peers amongst students in the flipped classroom as compared with the traditional one. This may contribute to developing a social network, as well as to a higher perception that linear algebra is important to their careers.

3. METHODS AND RESULTS

In order to investigate student performance and perceptions at our own institution, a selective liberal arts college in the Midwest with approximately 2000 students, we conducted a study. The goal of the study was to compare two sections of the innovative course applied multivariable calculus I (AMC I), each taught by the same instructor but using two different pedagogical methods: a more traditional lecture approach and a flipped model.

AMC I is an entry-level mathematics course, yet covers curriculum from both a single and multivariable perspective. The original concept of the course was to solve a curricular challenge for disciplines such as biology and economics at liberal arts colleges. These majors typically require a single semester of calculus, and yet the traditional calculus I (or even calculus II) course provides little that is directly applicable to these other fields. AMC I has no prerequisite other than high school precalculus, and yet it can also accommodate students who have seen calculus in high school but are not comfortable moving on in the calculus sequence. Such students will find exposure to material not typically found in the high school calculus curriculum. AMC I is appropriate as a terminal mathematics course, as a precursor to a course in statistical modeling (which was created in concert with AMC I), or as a transition to applied multivariable calculus II. AMC I provides a focus on modeling and computation using the free software R [15], and these approaches underlie the main topics covered in the course: functions as models, scale and estimation, dimensional analysis, single and multivariable differentiation and integration, optimization, and modeling with differential equations [7]. During the 2013–2014 academic year, over 200 students enrolled in eight sections of the course, largely consisting of biology, economics, chemistry, and social science majors and only a handful of mathematics, computer science, and physics students.

The two sections of interest were back-to-back, and the instructional model was randomly chosen so the earlier section was taught as a more traditional lecture and the latter as a flipped course. Students were not informed prior to enrolling in the course of the instructional method to be used. There were 23 students in the traditional course and 22 in the flipped course. As the semester progressed, students were aware that the two sections were being organized in different manners but no students opted to change sections because of the instructional method. Each of the sections met for 60 minutes three times a week.

During the traditional lecture class, the instructor presented the material (e.g., definitions, concepts, basic facts, and example problems) at the board, engaging students in questions and brief opportunities to discuss concepts or problems with their peers. Typically, lecture lasted for about 45–50 minutes and the remaining time was devoted to demonstrations of computations in R or class activities such as worksheets and labs. A list of goals to be discussed in each class period was displayed for the majority of each class period. After almost every class, students were assigned an online checkpoint quiz consisting of two to four questions re-assesssing the main concepts covered in class. These quizzes included both computational and conceptual questions, and the students in the traditional class were asked to complete them by midnight. These quizzes were administered in Moodle, an open-source learning management system.

Students in the flipped classroom were instructed to watch screencasts prior to coming to class. These screencasts were short – typically, under 15 minutes – and served as the basis for each of the lectures in the traditional class. Screencasts consisted of a movie of a presentation-type slide along with a soundtrack of the instructor talking. The presentation slides were produced with the Beamer package. Using the Screenflow screen capture software package, an instructor recorded a screecast by viewing the slides on a computer, explaining them, and using a Wacom Cintiq pen tablet to point to items on the slide and, at times, annotate them with additional notes. Students watched the screencasts through YouTube and could also download a pdf file of the slides as marked up by the instructor. The final slide of each screencast was the same list of goals displayed during the traditional format class.

Before almost every class period, students were required to complete the same online checkpoint quizzes that students in the traditional class took after class. During each of the flipped class periods, the first 5 to 10 minutes were devoted to discussing the goals of the screencast. If there was any confusion with regard to these goals, the checkpoint quizzes, or other concepts pertaining to the lesson, further discussion ensued. The remainder of class time was devoted to structured, hands-on activities. Such activities included solving sets of example problems and presenting solutions at the board to classmates, working on worksheets and labs in groups, solving further practice problems from the text, contemplating and solving homework problems with partners, and demonstrations of computations in R. The instructor interacted with each student during every class session while circulating around the room, addressing questions, and guiding them in their progress.

Both sections shared as many attributes as possible. Each section was encouraged to read the textbook and other assigned readings before class. The screencasts were made available to both of the sections. The same problems presented in the lecture section were used as example problems to be discussed with peers in the flipped classroom. Each of the activities such as the worksheets and labs were a component of both sections. The main difference in this regard is that the traditional class had far less time to discuss these activities with their peers and rarely completed them in class, although students were encouraged to complete them outside of class. The flipped class, on the other hand, had much more time to discuss with peers and regularly completed these activities in class. In addition, both sections were encouraged to do practice problems, but the flipped class occasionally had time to work on these problems in class. Additional resources such as screencasts demonstrating commands for computations in R as well as R Markdown notes developed specifically for the course were made available to both sections. An online discussion forum, Piazza, which allowed students an opportunity to ask and answer questions, reflect on material, connect content to other disciplines, gain more interaction with peers and the instructor outside of class time, and foster a sense of community was utilized in each section.

The graded components of the course were consistent as well. Students in both sections completed the same checkpoint quizzes with the only difference being that the flipped section completed the quizzes after class and the traditional section before class. They were also graded on attendance and participation on Piazza. The course consisted of seven content units. In each of the units, both sections were also assigned the same online homework assignments, in which students were encouraged to work together, although students in the flipped section class often worked on these problems during class. In addition, for each unit, both classes completed timed, online unit quizzes administered in Moodle in which students were not allowed to work with their peers (in fact, doing so would be considered academic dishonesty) but were allowed to use available resources and up to two attempts to complete each quiz. The homework and quizzes included both conceptual and computational questions in a variety of formats (multiple-choice, true/false, numerical answer, and free response). Nearly every assignment included questions that required computations to be done in R. The last component of the course was a written, open-note, proctored final exam which was the same for both sections.

3.1. Comparing Student Performance

As a baseline, students in both sections completed the Calculus Concept Inventory (CCI) on the first day of class [6], before any calculus content had been covered in the course. The means of this measure as well as the means of the checkpoint quizzes, the unit quizzes, and the final exam are displayed in Table 1, which–along with all other tables in this paper–can be found in the Appendix. The medians of each of these measures are displayed in Table 2. Finally, the associated p-values from two-sample t-tests are displayed in Table 3. A sampling of questions from the CCI appeared again on the final exam.

Note that in each category the mean and median scores are quite similar, and the p-values indicate no significant difference, except in the case of the checkpoint quizzes. Upon further inspection, it was noticed that nearly 10% of assigned checkpoint quizzes were not taken by students in the traditional class as opposed to 5% in the flipped class. Recall that with a deadline of midnight, students in the traditional class needed to complete checkpoint quizzes after class, whereas students in the flipped class were required to take their quizzes before the start of class. Removing these “missed” checkpoint quizzes from the data yielded a p-value of 0.11, indicating no significant difference in the means of the two classes.

3.2. Comparing Student Perception

To better understand student attitudes and perceptions regarding the format of each class, two diagnostics were implemented. The first was a Mid-Course Interview (MCI) administered by two faculty outside the department in the middle of the semester. The interview follows a standard procedure as designated by the college. It centers on four questions. First, each student ponders these questions individually. Then, students form small groups to identify common trends within the group. Finally, students reconvene as a whole, identify central themes, and then vote by raising hands whether or not they agree with each statement. Statements where approximately 95% of students raised their hand receive three stars.

The questions themselves as well as most of the three star statements for both classes can be found in Table 4. Where applicable, similar statements were matched in the table. From the students’ perspectives, both classroom models seemed to be well-received. Even though the screencasts were designed for the flipped classroom, students in both sections found them to be a useful resource. They also indicated that they would like more time devoted to learning and practicing R, and the flipped classroom acknowledged that class time devoted to working on R was beneficial. The traditional classroom indicated that there was often not enough time to do in-class activities and that spending class time working on problems, such as the flipped class, would be beneficial. Both sections expressed concerns with regard to using the online discussion forum Piazza. Finally, both sections acknowledged that their participation was key to improving the class.

The other diagnostic to capture student perceptions was an online, anonymous, 73-question survey administered at the end of the semester. Nineteen students in each section completed the survey.

Over 73% of students in both classes agreed that each course was organized effectively, and the weighted averages on a five-point Likert scale referring to the question “Class time helped me better understand course material” were equal. However, there were significant differences between the two classes using the Mann–Whitney U test on the five-point Likert scale on some questions. Using the alternative hypothesis that students in the flipped classroom thought the professor spent too little time lecturing yielded a p-value of 0.0297, indicating significant support for the alternative hypothesis. There was no significant difference between the two classes when asked if the professor spent too much time lecturing, though. However, using the alternative hypothesis that the traditional class did not think enough class time was given to complete activities in class yielded a p-value of 0.00397, again indicating significant support for the alternative hypothesis.

Echoing the perceptions revealed in the MCI, the majority of students in both sections (79% in the traditional and 89% in the flipped classroom) agreed that the screencasts helped them learn course material. The daily checkpoint quizzes – originally designed to encourage students in the flipped classroom to watch the screencasts before class – were found by over 73% of students in each class to be a useful way to keep up-to-date with course material.

A central theme of the course is to show the benefit of computational software when solving applied problems. Over 89% of students in the flipped classroom as opposed to 68% in the traditional classroom indicated that being comfortable with computation would help them earn a living. There was a significant difference between the two classes (p = 0.0275) in responding to “R helped me understand course material,” with the weighted average of the flipped class being nearly a point higher than the traditional class. This may stem from the fact that students in the flipped classroom had more time to practice with R during class time and thus, felt more comfortable with the software.

The last few questions on the end-of-term survey were free-response questions. Although there were both positive and negative comments about the format of the course in each section, a majority of students in the flipped classroom responded that the format of the course was a strength, for instance:

I feel the strength in the course is how we get a taste of the subject by watching screencasts before getting more in depth and clear about it in class.

Several students in the traditional class indicated they liked the format of the class as well but wished there were more opportunities to work on problems. One student in the traditional class even went so far as to say:

I somewhat thought it was unfair that the other class was formatted in a better way … practicing the applications is more important … and I really think that doing it in class with the presence of the professor and peers is beneficial.

4. OBSERVATIONS AND CONCLUSIONS

Our main conclusions and observations are as follows.

1. The flipped model is much more than simply moving lectures out of the class and homework into the class. The variety of activities that can be incorporated into the flipped classroom can appeal to many types of student learners and add dimension to any course.

2. Students in the flipped section prepared for class time by both watching the videos and completing the checkpoint quizzes. In fact, they completed checkpoint quizzes more regularly than students in the traditional section.

3. Students in the flipped section did not mention or complain that they had to do more work outside of class by watching the screencasts. In fact, with the occasional time spent working on homework in class, they likely spent approximately equal time working on material outside of class as students in the traditional course.

4. Developing the screencasts and online checkpoint quizzes are time-consuming endeavors, but the majority of students in both sections agreed that they were valuable resources.

5. Although there were some concerns about using Piazza in both sections, it allowed for an opportunity to discuss material outside of class time. For the flipped section, it allowed students to ask questions about the screencasts prior to class, which was a concern for students participating in the study of [13]. Also, for the traditional section, Piazza helped to create a sense of camaraderie with peers and an opportunity to discuss material that was not possible during class.

6. More students in the traditional section attended office hours and preceptor help sessions than in the flipped section, likely because more student questions were answered during class for students in the flipped course.

7. The flipped model allowed for more time devoted to practicing with R as a class. This seems to have come through in student responses related to computation both with regard to understanding course material and importance for future careers.

8. There is a trade-off between class activities and lecture. Students in both sections seemed to like the format of each class. However, there was an indication that students would have liked some aspects of the other section’s format. Students in the flipped section indicated that there was too little lecture. Students in the traditional section indicated a preference for more time devoted to learning R and problem-solving. Also, they indicated there was not enough time devoted to hands-on class activities. Note that 78% of students in the traditional section indicated that if they did not complete an activity in class they also did not complete it outside of class. Thus, students in the traditional section were missing out on valuable practice, whereas the flipped model allowed more time for these activities.

9. There was little difference in performance between the two sections.

Given that the learning outcomes were similar between the two sections, it is important to observe the effect flipping the course did have on student learning. The flipped model certainly allows more time for different modes of active student learning such as working on, discussing, and presenting to peers on a variety of problems as well as incorporating computational software into a course. The body of evidence is growing that such endeavors support students in their learning.

From the point of view of the instructor, the flipped class felt much more relaxed and less rushed than trying to cover the content in the traditional course. Students seemed to be more engaged during the flipped class. They developed a learning community through discussions with their peers. By the end of the semester, they became very self-sufficient and really took the learning into their own hands. However, the personal interactions between students and the instructor were more prevalent than in a traditional lecture class and were very rewarding.

As more research is done on the flipped classroom environment, we gain more insight into the impacts this approach may have on student learning and perceptions. Further work could include conducting a similar study over multiple semesters and tracking how student performance in subsequent courses compares for students in each of the two sections. No one study, such as our own or the others previously mentioned, can single-handedly determine whether a flipped model is more effective than a traditional lecture approach. However, these studies along with those yet to come–across various disciplines, institutions, and courses–provide more understanding of the flipped pedagogy. Flipping affords opportunities in the classroom that simply are not possible in a traditional lecture-based class.

ACKNOWLEDGEMENTS

The authors would like to thank Rachel Levy for discussions related to the study on flipped pedagogy at Harvey Mudd College as well as Erik Larson and Alicia Johnson at our own college for advice on survey design and analysis of the data.

APPENDIX

Table 1. Mean score on course assessments. CCI out of 22 points, checkpoint (CHPT) quiz average (36 quizzes), unit quiz average (seven-unit quizzes), and final exam percentage

Grade item	CCI	CHPT quiz	Unit quiz	Final exam
Traditional	9.09	84.73	88.18	73.13
Flipped	10.50	90.72	88.93	73.43

Table 2. Median score on course assessments. CCI out of 22 points, checkpoint (CHPT) quiz average (36 quizzes), unit quiz average (seven-unit quizzes), and final exam percentage

Grade item	CCI	CHPT quiz	Unit quiz	Final exam
Traditional	8.00	91.00	89.00	77.00
Flipped	11.00	91.75	90.43	77.00

Table 3. Two-sample t-test p-values on course assessments. CCI, checkpoint (CHPT) quiz average (36 quizzes), unit quiz average (seven-unit quizzes), and final exam percentage

Grade item	CCI	CHPT quiz	Unit quiz	Final exam
p-value	0.191	0.0485	0.705	0.948

Table 4. The four questions asked during the MCI as well as representative responses receiving broad consensus from students in a given section. See text for more details of the MCI process

Traditional	Flipped
What is working in the class to facilitate learning?	What is working in the class to facilitate learning?
The lectures are good, clear, and concise–great energy.	We like the way the class is set up.
Screencasts before or after class are helpful to either prepare for class or reinforce what was said in class.	Watching screencasts before class and then working/talking about problems in class where we can get help.
Daily checkpoint quizzes are working.	[The instructor] constantly walks around asking people if they need help; this is very useful.
Resources in Moodle are very helpful.	Working on Rstudio together as a class is very helpful.
	We like the free work time to work on the homework.
What is impeding learning?	What is impeding learning?
When we do in-class activities, we are seldom given enough time to complete them.	Piazza posts focus on quantity rather than quality.
We don’t have enough examples of R; more would be more beneficial.
We spend time talking about definitions and concepts that are in the book. We could spend that time on problem sets or other activities that would be more beneficial.
What could students do to improve this class?	What could students do to improve this class?
We are too timid. We could participate more.	We could participate more.
We could be better about asking questions, discussing them.	Come to class prepared.
What could the instructor do to improve this class?	What could the instructor do to improve this class?
Devote more time to learning R. Do not make Piazza mandatory.	Spend more time with Rstudio.

Additional information

Notes on contributors

Lori B. Ziegelmeier

Lori B. Ziegelmeier (B.S., B.A., M.S., Ph.D. Colorado State University) is an Assistant Professor at Macalester College. She has a deep interest in engaging students of all levels in the beauty of mathematics. She does so by incorporating innovative teaching strategies into the classroom, integrating activities such as a calculus fair to demonstrate class projects, and participating in and organizing outreach programs such as Math Circles for middle school students. Lori’s research is in geometric and topological data analysis.

Chad M. Topaz

Chad M. Topaz (B.A., Harvard; Ph.D., Northwestern) is a passionate educator of undergraduate applied mathematicians. Currently a Full Professor at Macalester, he has won the college’s Rossmann Excellence in Teaching Award as well as the Sorgenfrey Distinguished Teaching Award from the UCLA Department of Mathematics. Chad’s research, benefitting from the involvement of many students and supported by the NSF since 2006, is in the areas of applied nonlinear dynamics, pattern formation, and biological swarming.