Teaching Introductory Statistics Online — Satisfying the Students

Abstract

This paper describes the components of a successful, online, introductory statistics course and shares students' comments and evaluations of each component. Past studies have shown that quality interaction with the professor is lacking in many online courses. While students want a course that is well organized and easy to follow, they also want to interact with the professor and other students. Interactions in this course took place through small group discussions, emails, weekly announcements and graded exams. The course also contained lecture slides with audio prepared by the professor. As the variety and quantity of interaction increased, student satisfaction with the amount of interaction with the professor increased from 75% the first year of the course to 99% the fifth year. Overall satisfaction with the online course increased from 93% the first year to 100% the fifth year.

• Course design
• Online versus traditional learning
• Statistics education

1. Introduction

With more people coming back to college to pursue a degree while working full time, and with an increase in the number of students with children, interest in online education is growing. Early studies of online education concentrated on student performance. Studies were designed to show that whether students received their instruction online or in the traditional classroom setting their level of performance was the same. The majority of the many papers published showed that students in online courses performed as well as those in traditional courses when comparing test scores and grades. Russell (1999) cited many of these papers in his publication, The No Significant Difference Phenomenon. A list of abstracts can be found at www.nosignificantdifference.org. Most of the studies that Russell mentioned, however, were not experimental studies but surveys, and none dealt with an online statistics course.

Most of the published research regarding online statistics courses deals with either video conferencing or a combination of online teaching and traditional teaching referred to as a hybrid course (Stephenson 2001, Yablon and Katz 2001, Utts, Sommer, Acredolo, Maher, and Matthews 2003, Ward 2004). All of these studies show similar student performance between the online course and the traditionally taught course. In particular, both Utts, et al. (2003) and Ward (2004) compared an introductory statistics course that used the same instructor and the same textbook for both the traditional version of the course and the hybrid version. Although class sizes varied, 200 traditional versus 80 hybrid and 56 traditional versus 22 hybrid for Utts, et al. (2003) and Ward (2004) respectively, neither study found any difference in student performance when comparing test scores and grades. In Utts, et al. (2003), the hybrid class used CyberStats, a commercial introductory online statistics course, for its online materials. Ward (2004) used a combination of interactive worksheets, applet demonstrations, review sheets, and PowerPoint reviews of textbook materials.

Student satisfaction with the course was also assessed by both studies. Utts, et al. (2003) reported that the students in the traditional class were more satisfied with the course's organization, pace and expectations. There was no difference in satisfaction with the course or with the instructor. In contrast, Ward (2004) found no difference in student attitudes toward course content, organization, requirements, or grading and evaluation. Even more interesting was that students in the hybrid class had a more positive attitude regarding the instructor's knowledge, presentation of subject matter, academic motivation and overall rating of instructor. It appears that the presentation and quantity of the online materials, as well as how much and what type of interaction there is with the professor, affect the students' satisfaction levels. This idea is not new. Many studies have supported the concept that student-to-professor and student-to-student interactions are important elements in the design of an online course (Fulford and Zhang 1993, Kearsley 1995, Sherry 1996, Picciano 2002). These studies found significant differences between online and traditional classes regarding students' satisfaction with instructor's explanation, instructor's enthusiasm, instructor's openness to students, and instructor's interest in student learning. None of the studies, however, focused on a statistics class.

Summers, Waigandt and Whittaker (2005) compared an online statistics course (19 students) to a traditional statistics class (14 students). The same materials and professor were used in both classes. The online materials consisted of readings in the textbook and an online discussion. They found no significant difference in grades between the courses. Students enrolled in the online section, however, were significantly less satisfied with the instructor, class discussion, quality of problems and evaluation, and grading.

In 2005, an improved meta-analytical study of research on distance education was carried out by Zhao, Lei, Lai, and Tan. They used a more systematic way of choosing and examining the studies than Russell (1999). Only articles published in a journal and with a complete list of references, for example, were included. Their search identified 421 journal articles from 1966–2002 that contained empirical data comparing a distance education course to a face-to-face course. Two-thirds of these studies showed results favoring online learning and one-third favored face-to-face learning. The purpose of their paper was to identify factors that affect the effectiveness of distance education. Interestingly, instructor involvement was the most significant moderator among all identified factors! None of the studies focused on mathematics or statistics.

Noticeably lacking in all of these papers is a detailed description of the online materials provided to the students, how they are presented and used, and the types of interaction that occurred between the students and the professor. The purpose of this paper is to show that an online course in statistics can include instructor interaction in a way that will please the students. In addition, such a course can be created from materials previously used in a traditionally taught course. These materials might include lecture notes, hands-on activities, group discussions, quizzes, homework assignments, and exams.

This paper will describe the different components of an online course that was built from materials from a traditional course, but prepared in a way to benefit online learning and online logistics. Students' comments on each part of the course will be summarized. These comments were collected over five years (10 semesters) and were responses to one of the following questions on the end of the semester evaluation form: “What suggestion do you have for improving this course?” or “Please make any comments you would like about the effectiveness in promoting your learning of the professor, course content, assignments, grading, teaching methods, or other aspects of this course.” All comments were voluntary and anonymous. The comments were typed and separated into positive and negative comments by an adult education specialist. I was provided with the typed list of comments and had no way of identifying who made the comment. I will discuss how I used the students' comments to change aspects of the course to increase student satisfaction.

After several years of making changes to the course, I believe that I have come up with a format for a successful online introductory statistics course. This paper describes the format and contents of the course, and discusses which aspects of the course appear to improve student satisfaction with the amount of interaction with the professor.

2. The Components of the Course

2.1 Background

In 1999 I was hired by a large university to create an online introductory statistics course. I had no previous experience with online courses. It took me about six months working ten hours per week, with the help of an adult education specialist and two technical people, to create all of the materials. An online course takes more time to create than a traditional course, but requires less time during the semester. The extra hours spent in developing the course are well worth the flexibility an online course offers the professor and the students. After presenting the course at several conferences, I came to realize that what I had learned about an online statistics course was actually of interest to and could be of help to others who might be beginning the process.

2.2 Course Design

The course was an entry level master's course titled Principles of Statistical Inference. The only prerequisite was a basic algebra course. The students consisted of undergraduates (juniors and seniors) (7%) and graduates (93%). Roughly one-third of the students were residential students and two-thirds were distance education students who never came to campus and lived from an hour away to as far away as India. All of the students were pursuing some type of degree in public health, but none were biostatistics majors. All of the distance education students were pursuing either a certificate or a master's degree in public health. Enrollment ranged from 15 to 75 students per semester. The course was designed to be completed over a 15-week period. All materials were provided online except for the textbook, Introduction to the Practice of Statistics by Moore and McCabe. Graded assignments included online small group discussions and take home exams, and will be discussed in Section 2.8 and Section 2.10, respectively. The professor was available by email everyday and all emails were answered within 24 hours.

The course was originally created using the course management software WebCT, but later was switched to Blackboard. Use of software makes it easy to organize the materials and make them accessible to the students. A well designed course makes it easy for the students to find and use the materials in the correct order. To create the online course, existing lectures were divided into seven units based on the major topics in the course (e.g., survey sampling, inference from one sample). There were originally eight units, but it was hard to schedule enough time for each unit in a 15-week semester. Each unit was divided into three to six lessons. It was helpful to have each lesson follow the same design because once one lesson was completed everyone knew what to expect for the rest of the lessons. To follow good pedagogy it was suggested that each lesson consist of a list of objectives, assigned readings, assigned problems, a tutorial with quizzes, and an activity, if available. In addition, the students had an online syllabus that contained sections describing the teaching methods and materials provided by the course, including suggestions on how to use the materials to master the content. This fundamental lesson design, as well as repeating the format from one lesson to the next, was intended to provide most of the events of instruction articulated by Gagne, Briggs and Wager (1992). These events include:

1. informing the learner of the objective‐ carried out with a detailed syllabus and list of objectives presented for each unit and each lesson

2. presenting materials - carried out by using the same format for every lesson

3. eliciting performance - carried out by quizzes, activities, practice problems, discussion forums and exams

4. providing feedback about performance correctness - carried out with graded and returned exams and answer keys, small group discussions, emails and posted announcements

5. assessing performance - carried out with exams and small group discussions

6. enhancing retention and transfer - carried out by using and expanding upon the same basic statistical information over and over and applying it on exams and in discussions

For more information on how this course followed Gagne's model of instruction, see Hannon, Umble, Alexander, Francisco, Steckler, Tudor and Upshaw (2002). In response to the organization of the course, students often posted comments such as “The course was very well-organized, and nicely broken down into manageable steps” and “The way the course was taught, the assignments and the course content were extremely effective” on the end of the semester evaluations. As seen in Table 1 in Section 3.1, student satisfaction with the course was 90% or higher every year among those students who filled out an end of the year course evaluation.

2.3 Tutorials

The most time-consuming aspect of the course was creating the tutorials or lecture part of the course. I recommend tutorials consisting of visual slides with corresponding audio and written notes. Using audio reduces the need for wordy slides that resemble a textbook. Written notes that accompany each slide compensate for poor audio and can be easily edited later. Tutorial slides can be created using PowerPoint. A qualified technician can upload the files and streamline the corresponding audio using software like Real Player. This allows students to move forward or backward among the slides, always getting the correct audio. Alternatively, the professor can upload the PowerPoint slides and the audio files separately. Students can then view the PowerPoint slides as they listen to the audio. Three student reviewers decided that 30 slides is the maximum number a student can absorb at one sitting and that a tutorial should last no more than 30 minutes. Eleven of the tutorials used in this course are available to the public under the Biostatistics section at www.sph.unc.edu/nccphp/training/training_list.

Not all of the online statistics courses mentioned in the Introduction contained online lecture materials. Some courses used only java applets or other interactive activities not created by the professor. Others, like Summers, et al. (2005), used textbook readings and online discussions. Some textbooks offer online tutorials that can be accessed by the students (MathZone, for example), and these could be used in place of creating new materials. CyberStats, a commercial introductory online statistics course, is another option. After teaching an online statistics course for many years, I believe tutorials that are created by the professor help the students feel more connected to the professor. As one student wrote, “The online lectures were the real meat of the course.” Perhaps combining some personally created slides and materials along with prepackaged materials like CyberStats or MathZone would be the easiest way to create a course for professors who do not already have a set of typed slides.

2.4 Written Notes

The written notes that go with each slide of the tutorial can be the written version of the audio, contain additional information not in the audio, or provide a second explanation of the materials using a different angle. The notes for this class were a mixture of all three. The original version of the course did not have notes, but as the students complained that they did not always have access to the audio and the audio was not always clear, notes were added. Based on students' comments on the evaluation forms and by simply asking them, I discovered that the notes were very important to the students and did not cause them to skip the audio as was originally hypothesized. One student wrote: “The most helpful section was that where the course notes were printed along with the outline [slides]. It is helpful to see and hear the words at the same time.”

2.5 Audio

For an online course, the professor will probably be the one who records the audio for each slide. When audio was first recorded for this course, it was a very new concept in online learning. As the amount of audio in the course increases, so must its quality, because students can only tolerate so much static and background noise. For this course, the audio improved as the quality of the microphone improved and the amount of background noise was reduced. I originally recorded the audio in a recording booth, but now I record right at my desk. Affordable and easy to use audio software can be used to remove background noise and to delete or add sections. In addition, it helps to have some training in audio recording. I found that I mumble sometimes and my voice drops off at the end of the slides. Once I was informed of these bad habits, I was able to eliminate them in future recordings. I believe that audio is a key component of a successful online course. Students are more likely to feel connected to the professor if they hear the professor's voice in every lesson.

2.6 Self-Help Quizzes

Stopping points may be incorporated into the tutorials so that the students can pause the tutorial to take a quiz. These quizzes are separate entities from the tutorial and can be assessed at anytime. I used the quizzes as practice materials and did not grade them. Both Blackboard and WebCT have the capabilities to create quizzes that can be graded immediately following submission and/or have an answer key pop up after the student completes the quiz. In this course, all the quizzes were Word documents and included open-ended questions with a “model answer.” The quizzes were one of the most popular components of the course. As seen in Table 1, 97% of the students who completed an end of the semester course evaluation reported favorably on the quizzes. One student wrote: “The quizzes in each lesson helped me a lot. I was able to learn from my mistakes before having to take an exam that counted for credit.”

2.7 Activities

The purpose of an activity is to provide the student with a hands-on-experience that parallels what he or she would experience in a classroom setting. Online activities can include a professor's own creation, a java applet, or an activity on another website, and can be added to any lesson. The only difference between using an activity in an online class, as compared to a traditional class, is that the students must do the activity completely on their own. As a result, very specific written instructions, as well as a clear purpose and set of objectives, are necessary. This course had only three activities that were interactive. The other activities were exercises where the student worked out the answer with paper and pencil. I originally feared that the students would find the set up of the course boring and would need more activities or flashier ones. Over ten semesters, however, there were only three students, out of 292 who filled out an evaluation, who mentioned the need for more exciting or interactive materials.

2.8 Group Discussions

Successful large group and small group discussions (e.g., six or fewer students) are the hardest part of a traditional course to include in an online course. Both large and small group discussions can easily be incorporated using Blackboard or WebCT, where postings and replies to postings are allowed and show up one right after the other. The student simply clicks on whatever posting he or she wishes to read. Each posting displays the topic line and who posted it. To incorporate online discussion, some rules of online etiquette, or “netiquette,” may need to be introduced, and the facilitator has to work harder because he or she has to point, click and read rather than just listen. I have learned that an interesting and controversial topic, with very detailed directions and very specific questions, is the key to having a good discussion. Trained teaching assistants are very helpful as additional facilitators. It is hard to offer quality feedback to more than three groups at a time.

For the first three years, this course did not have a required discussion forum, but did have an optional large group discussion forum that all students could access at any time. A newspaper article or a summary of a study was posted for each unit, and the students were asked to post answers to a set of related questions. Participation was very low. A small percentage of the students provided comments indicating that a small group discussion would increase interaction and lessen the feeling of being isolated. Thus, at the start of the fourth year (7^th semester), two small group discussions were added in an attempt to improve student satisfaction regarding interaction with the professor and the other students. The small group discussions were required, and posted comments were graded.

The small group discussions followed a format I copied from a co-worker who had researched online group discussions. Each student read a synopsis of a study and a list of questions, and then participated in an online discussion with five or six other students. For full credit, a student had to make two major contributions (or four smaller ones) over five days. One or two students volunteered to be group leaders, and they wrote up a summary of the discussion. The summary requirement was dropped after two semesters because it did not appear to add anything to the learning experience, according to me and the students. Over the last two years the course was offered, the students provided comments regarding the small group discussions. Some comments were positive, stating that the discussions helped them grasp the material better, improved their logic, or were just fun. Other comments were negative. Some students found the discussions to be awkward because the students lived in different time zones or because the speed of their internet connection was too slow. From these comments, it appears that the effectiveness of online discussions in a statistics class is still debatable. The biggest factors affecting their success may be the topic of discussion and the quality of the questions.

2.9 Statistical Software

Students can be taught to use statistical software in an online course if the software can be made available to every student. Excel was used in this course because it was deemed to be inexpensive and widely available. The students learned Excel through a set of tutorials that I created in Word. The tutorials contained pictures of Excel spreadsheets and command windows with written instructions, and covered descriptive statistics, graphs, t-tests, correlation statistics, and simple linear regression. I created my own tutorials for two reasons. I found it difficult to understand the Excel manual, and I wanted the students to focus on only the commands that were necessary for the course. Most of the students had never used Excel to compute statistics, but they had no trouble mastering the commands. Some textbooks include an Excel manual on a CD or provide access to a manual on the internet. In addition, manuals for other software, such as Minitab and SPSS, may be available. The professor will need to become familiar with these materials to decide what works best for his or her course.

2.10 Examinations

The take-home exams were Word documents, containing both open- and closed-ended questions, that the students downloaded, typed in their answers and emailed back within three days (except for the first year when the students returned the exams via postal mail). Requiring typed answers was not a problem because the exams did not require the students to write out equations or formulas (although some students would cut and paste the formulas into their exams from the tutorials). Students showed their work with actual numbers, and the few symbols that were used could be inserted with Word. The downside to using open-ended exam questions was the amount of time it took to save them, grade them, resave them, and email them back to the students. Close-ended exam questions can be created and graded using Blackboard or WebCT.

The students were tested often in order to keep them on track. It was necessary to provide very detailed answer keys for each exam because the exam could not be “discussed” in class. Comments on each individual exam were provided as a way to increase the students' satisfaction with the amount of interaction with the professor. The students appreciated the regular unit examinations, as opposed to fewer exams that covered several units of content, because it helped them master the materials in smaller pieces. As one student wrote, “Many unit tests were much better than two big tests, it helped me track my learning as I went.” Another student wrote, “The frequent exams kept me from falling behind.”

2.11 Supplementary Course Materials

In addition to the main components of the class (objectives, readings, tutorials, exercises and discussion forum), there was also a list of helpful websites that the students could use. The websites contained online textbooks, activities the students could perform online, and practice exercises with feedback. In addition, there were two sets of videos that paralleled the topics in the course and were available at two local university libraries. By providing a variety of materials, the chance of reaching every student was increased. Also with variety came repetition, which further helped the students. This was confirmed by many students who wrote comments such as, “I like seeing the information multiple times. It helps to maintain the information in memory” and “You must do all the readings, activities, exercises, tutorials etc. I don't think I would have understood as much without all of the various forms of learning, the repetition of ideas, etc.”

3. Student Evaluations

3.1 Description

At the end of each semester, the students filled out an optional, online student evaluation form that examined students' general satisfaction with the course, the contribution of each teaching method to students' learning, and students' self-assessed achievement of the overall course learning objectives, and had space for general comments. The closed-ended questions had five categories: strongly agree, agree, disagree, strongly disagree, and does not apply. The category “does not apply” was used because the same evaluation form was used for five online courses and not all courses had materials or experiences that applied to every question. This last category was excluded from calculations. The results over ten semesters were tabulated and are provided in Table 1.

3.2 Results

Out of a total of 435 students, 292 (67%) filled out an online course evaluation form. Each year, 90% or more of the students who filled out an evaluation were satisfied with the course overall. Satisfaction with the amount of interaction with the professor increased from 75% the first year to 99% the fifth year, as exams were emailed back with written comments, the professor started emailing the students as a group every week and posting weekly announcements, and with the addition of required small group discussions. After the first year, ratings were always high (90% and higher) for the content and design of the course. The question regarding the way the course was taught had very high ratings (95% and up) for every year except for the first and fourth years. The first and fourth years had the largest proportion (50% or more) of residential students. This may suggest that students who choose to take the course online will be happier with the format than those who are “forced” to take it online, which is to be expected. All of these results are limited by the low response rate. Since the evaluations were anonymous, there is no way of knowing whether the students who did not fill out an evaluation form are similar to those who did.

Table 1. Percent of Students Who Agreed or Strongly Agreed with Each Statement on the End of the Year Online Course Evaluation.

Statement	First Year	Second Year	Third Year	Fourth Year	Fifth Year
Satisfied with course	93	90	94	90	100
Satisfied with amount of interaction with professor	75	81	94	87	99
Activities fit well together	90	95	96	94	99
Exams helped my learning	96	95	96	96	100
Way course taught helped my learning	89	Not Asked	98	81	97
Web-based lectures helped my learning	86	92	94	90	98
Quizzes helped my learning	86	92	100	100	98
Discussion groups helped my learning	Not used	Not used	Not used	75	90
# of responders	28	63	50	92	59
Enrollment	38	107	84	121	85

Note: Percent of students who responded Agree or Strongly Agree out of four choices: strongly agree, agree, disagree and strongly disagree. Excludes “Does Not Apply” responses.

3.3 What the Students Did Not Like About the Course

This paper has emphasized what the students liked about the course, but not everyone was happy. Negative comments fell into four main categories: the need for notes to go with the audio, displeasure with the speaker's speed and mumbling, the need for more interaction with other students and with the instructor, and feedback on exams. In response to these comments, I added notes to go with each lesson, added the small group discussions to increase interaction, and also started emailing all the students once a week, as well as posting an announcement on the announcement page about the materials being covered that week. In addition, I started emailing back the graded exams with typed comments on each question and posted very detailed answer keys with step-by-step calculations. It appears that the students appreciated the extra materials and interaction and the improved feedback on the exams.

The last two semesters that I taught the class, I received no negative comments regarding the audio or the lectures. One student asked for more interaction with the professor, and one asked for more feedback on the exercises in the textbook. Four students wanted more comments from the TA or professor in the discussion forum. Three students would have liked more activities or experience with real life data. One student wanted more quizzes, and one wanted more examples in Excel. The other 48 comments were positive and mentioned things that students liked about the course.

4. Conclusion

To summarize the course structure and materials, I quote one of my students: “I believe this is my 4^th class completely online. This class, its organization, and your (the professor“s) involvement have far exceeded the others. I have been very impressed with the organization and your hands-on involvement (rather than delegating to a TA).” Organization and involvement are the two key components for a successful online course. As reported in Utts, et al. (2003), when a traditional statistics course was compared to a partially online course (hybrid course), the students performed the same, but the students taking the hybrid course wanted more interaction with the professor and more time to ask questions and go over problems. I think that hybrid and online courses can be very successful, but one cannot underestimate the importance of the professor's involvement with the students. For most students, there must be enough interaction with the professor to keep the course from being an independent study course. For a completely online course, small group discussions, feedback on the exams and weekly online communication with the professor appear to provide enough interaction. The process of creating an online course can actually promote good organization, the second component of a successful course, because it forces the professor to think about how the course is structured and how the students use the materials. In the online format, the course materials are all presented at one time for anyone's scrutiny, giving the professor the opportunity to view the entire course from the student's perspective.

Acknowledgments

I wish to acknowledge, with appreciation, the invaluable assistance of Karl Umble and Ansje Burdick in the original development of this onlline course. In addition, I wish to thank the Associate Editor and the referees for many extremely useful comments.