Creating a Student-Centered Learning Environment Online

ABSTRACT

Shortly after we created a face-to-face, student-centered, general education classroom at the University of Kentucky, we were asked to create a similar environment online. In this article, we describe what we did and what we learned in the process. This is not a comparative, controlled study, but an anthology of examples and reflection. Our aim is to share experiences that other instructors might find useful when constructing their online classes.

KEYWORDS:

• Active learning
• Peer grading
• Peer instruction

1. The Redesigned Course

A statistical reasoning course has been taught at our university for over thirty years. In fact, the University of Kentucky was one of the first in the nation to do so, thanks in large part to the vision of then-faculty member Dr. Dennis Haack, who authored what is arguably the first ever textbook for teaching statistical reasoning. Our introductory level course covers most of the same topics as any introductory statistics course, but emphasizes the reasoning necessary for intelligent consumption of statistical arguments over computations. It has long been the primary class that students use to satisfy an inferential reasoning general education requirement. For approximately 25 years, each class consisted of six sections of approximately 24 students per section. Students would meet together with the primary instructor for two class hours per week and break out into a recitation with just their section for the third hour of the course each week. Until 2010 the time spent with the primary instructor was in a lecture format (typically, a PowerPoint presentation) and the time spent in recitation with a teaching assistant was for discovery and discussion. In 2010 we piloted a new student-centered format for this course and in 2011 all sections—encompassing approximately 3000 students per year—transitioned to a similar format.

People mean different things by “student-centered.” Borrowing the words of STEM (science, technology, engineering, and mathematics) educator Robert Talbert, our goal for a student-centered classroom is to change the mindset of the student from a “renter” to an “owner.” In a classroom of renters, fees are paid and the management is expected to deliver. In a classroom of owners, students are forced to realize that it is their responsibility to engage, absorb, and retain. The instructor's job is to prepare an environment in which that can happen.

One of our primary motivations for adopting the new format in 2010 was the structure and staffing of the recitation. As a rank-one research institution we, like many of our colleagues around the nation used graduate students to staff our recitations. This was trickier in our concepts course, though, than it was in our traditional first-year methods course. In the former, recitations were designed as discussion sections, not problem sessions, and it was up to the teaching assistant to reinforce the core concepts with activities and discussion. More often than not, new graduate students come to our program with outstanding mathematical resumes, but they are sometimes as challenged by the conceptual material as the undergraduates are. It would be unfair to say this is the fault of the graduate students. They are unlikely to have ever actually discussed, for example, what a p-value really is and what added intellectual value it brings to the badging of a statistical result, as opposed to how to compute it in various contexts. In the end, even though we worked hard to script and supervise the recitations, we weren't doing a stellar job delivering many of the core concepts within that structure.

It was, in a way, irrational that we were leaving the most important, or at least most difficult part of the class pedagogy to the teaching assistant. To change this practice, we had to reimagine what we would do in the classroom and how we would do it. We ended up having to shift much of the rote parts of the traditional lecture outside the class time, first to videos, and then later to a choice of videos or short, focused readings that the students have to prepare before coming to class. In turn, the recitations were structured to both capture the talents of the teaching assistants and to meet the needs of the student. We started using them for computations (e.g., working with proportions) that would emerge as critical for understanding harder ideas (e.g., Simpson's Paradox, or sensitivity and specificity), and for practice with other tools (e.g., Excel and CamScanner) that we need in class. The professor retained the responsibility for delivering the deep ideas, either through in-class activities or some other means, but she could focus on the ideas, since some attention has already been devoted to vocabulary and simple computations that often limit the students’ abilities to successful grasp those ideas.

Still yet, the primary motivation for our redesign was to increase student engagement. We wanted to shift the main responsibility for learning to the student, and away from the professor and her PowerPoints. We had become concerned about how dependent students were on us to serve up content that they would then passively write down and take away. And we were sobered by multiple research studies suggesting that students remember very little of what we teach them beyond the first few weeks after the semester ends. The use of activities to engage students and improve recollection of course material is supported by Cherney (2008) and it continues to be in the news, evidenced by recent events at the University of Vermont (Cornish, 2017).

As mentioned above, to make room for our pedagogical changes, we had to move some of our content out of the classroom. Initially, a series of video lectures were created using Camtasia Studio and PowerPoint with audio narration developed from transcripts. Both the transcripts and ADA-compliant versions of the videos are still made available to students. The use of content videos is discussed and supported by Choi and Johnson (2005, p. 215). They found that

There was a significant difference in learners' motivation in terms of attention between the video-based instruction and traditional text-based instruction. In addition, the learners reported that the video-based instruction was more memorable than the traditional text-based instruction. This study implies that context-based videos in online courses have the potential to enhance learners' retention and motivation.”

To better facilitate engagement in our new classroom environment, Rayens wrote and published an activities book (Rayens, 2017) for use in the course. The book is structured around more than 100 daily exercises, a handful of slightly larger projects, and two extensive capstone projects. Many of the workbook prompts require the student to access additional reading materials from the web in order to complete the activities. Typically, instructors have some of these completed as homework, and have the remainder completed in class, either in groups or as part of some orchestrated classroom dynamic. In almost all cases, time is set aside for students to report out on what they have done. This happens many different ways, depending on the instructor and on the physical classroom space. Until recently, the content was only available in the content videos, but the book was recently adapted to contain mini-chapters based on the video transcripts. This addition came from listening to students, many of whom liked the video instruction, but some of whom missed a more traditional tactile source for the content. To encourage students to watch the videos or do the short readings, instructors routinely ask students to complete a worksheet or take a short quiz associated with the content.

In short, the fundamental changes we made to our face-to-face class can be summarized as follows:

·	The harder deliverables were moved back into the hands of the professors.
·	Space was made in the traditional lecture time by off-loading routine content to videos or short readings.
·	Students were forced to take responsibility for their own learning by preparing the content outside of class and participating in a student-centered environment inside of class.

This is not a profoundly new way of thinking about statistics education. These kinds ofpedagogical changes are supported by Woodard and McGowan (2012) who describe the redesigning of an introductory course to incorporate the Guidelines for Assessment and Instruction in Statistics Education (GAISE). That paper discusses the GAISE recommendations for introductory college courses. They are:

1.	Emphasize statistical literacy and develop statistical thinking.
2.	Use real data.
3.	Stress conceptual understanding rather than mere knowledge of procedures.
4.	Foster active learning in the classroom.
5.	Use technology for developing conceptual understanding and analyzing data.
6.	Use assessments to improve and evaluate student learning.

Our face-to-face course was developed with these kinds of recommendations in mind.

2. Creating the Course in an Online Environment

We were not quite through our second semester of piloting the above-mentioned changes to our face-to-face environment when our college administration asked us to offer the same class online. They requested that we include as much of our new student-centered focus as we could. As far as we could determine, this had never been done before, and the challenge to take this course online was initially daunting. We did not want a course that was essentially a distance learning course that revolved around static PowerPoints, and allowed students to be isolated from any kind of actual classroom dynamic. But it was not clear how we could capture what we valued most about the face-to-face course and deliver nearly the same online. We knew something about the learning styles, expectations and general needs of online students from Mupinga, Nora, and Yaw (2006, p. 186). In their paper, they write: “… the top three expectations of the online students were communication with the professor, instructor feedback, and challenging online courses.” While communication and instructor feedback alone do not necessarily equate to a student-centered course, they do describe an online environment that is not static. This was part of what we wanted. We added personalization and daily involvement to communication and timely feedback, and took these as the design principles used to develop our course. Ultimately, we developed an online structure with the following characteristics:

·	A “moving window” which controls the pace for classroom activities, exercises, prompts and content and required on-going student engagement.
·	A current “daily message” from the instructor.
·	Periodic, required video responses from students for selected prompts.
·	Real-time facilitation of discussion, response, and interaction.
·	Peer grading, both to facilitate timely feedback and to keep students in the learning loop on all homework prompts.

Previous work in online education both supports the choices we made and the need for them. Tudor (2006) discussed teaching an introductory online statistics course and stresses the importance of interaction between instructor and students. The pedagogy and methods she used are similar to those in our paper, with notable differences including the pairing of the workbook and the videos, moving window of availability, daily videos and the discussion of student assessment. Mills and Raju (2011) provided a detailed summary of online courses up to 2009, and it is clear from their work that many of these course stop well short of the kind of course we attempted to construct. In what follows, we will briefly elaborate on each of the bulleted course characteristics mentioned above. We will end with a reflection on what we feel worked best and what did not work so well.

2.1. Moving Windows for Assignments

When developing our course we tried to anticipate a student preconception that "online" is synonymous with correspondence. Many students view a correspondence course as one that presents them with all of the assignments at the beginning, and allows them the freedom to turn assignments in anytime during the duration of the course. It turns out that anticipating this preconception was a worthy concern. The first time the course was taught was during a six-week summer session we had many students with those expectations. Other students requested to be unavailable for large portions of the course. We had requests for as many as 3 weeks off from the 6-week term, with promises that they would make up the work once they came back. The reasons varied, with family vacations and military training being the two most common reasons. However, we felt that if students were going to be able to have anything close to the immersion experience we were trying to create in our face-to-face classroom, then we would need to gently coerce a sustained exposure to material. We tried to build this in by creating and using a moving window for assignments.

Here is an example of how many of our instructors have implemented the moving window. The topic for the day, along with instructions and requirements for any associated activities, readings, or homework prompts, appears on the course management website at 12:01 a.m. and disappears at 6 p.m. the following day. All work associated with that topic, including homework or outside readings, have to be completed and submitted during this 42-hr window. This allows students plenty of flexibility to access and work on material during a time of day that best fits their schedules, but at the same time requires them to keep a pace similar to the in-class version of the course. Although the details vary among instructors, it is common to have an electronic folder appear on the course management system that contains all of the material for the next 42-hr time period, and then have it disappear once the deadline passes. The choice of a 42-hr window was arbitrary. Any window that provides adequate time to allow for different work schedules, while being short enough to mimic the sustained exposure in our face-to-face class would work similarly. In retrospect this idea seems very simple, but it was in some ways fundamental to the tone of the course and the forced immersion experience.

Figure 1 shows an example of what students see when they access work for our course in our course management system. The figures in this article show excerpts from when we used Blackboard™ as our course management system. We now use Canvas™, but the basic management strategies remain the same. The 6-week online course is naturally partitioned into three subject modules, with each module lasting approximately 2 weeks. When students log into the course management system, they have access to “Daily Assignments” (or similarly named) folder. Within this folder, students then can access folders for each of the three modules. Once the student opens a module folder, they are shown assignments that are currently accessible. Notice that within the Daily Assignments, the availability of each assignment is displayed. We have found that it is quite useful to have these dates and times in multiple places since assignments will expire as the moving window closes.

Figure 1. Student view of daily assignments.

Figure 2. Student view of daily work folder.

Managing the availability of the folders can be simplified by setting folder expiration dates and times beforehand, provided your course management system allows that option. Below is the instructor's view of the available work folder, which shows that information on the subfolders. By placing expiration dates to control the visibility and accessibility of a folder, we do not have to place an expiration date on each item in the folder. This provides an easy way to control student access to the assignments and greatly reduces the work for the instructor, since each folder is likely to contain several assignments.

We have experimented with placing actual timers in the folders once they are opened. That is, while the expiration date and time are automatically set in the course management system, an explicit timer can also be positioned in the folder itself and synchronized with the time that the folder will close. This timer is the first item the student sees when the folder is opened. These (literal) timers are relatively easy to embed using html code. Immediately below the timer, students see a brief description of the work for that day, a description of each assignment, and the instructions for each assignment. Once the timer at the top of the page stops, the folder automatically closes and students can no longer access the assignments. Figure 2 shows an example of what the students see.

Figure 3. Organization of module threads.

While having assignment folders disappear at fixed times helps keep the students on pace, the fact remains that they may legitimately want access to the contents at some later time. To accommodate this need, some of our instructors just create a separate link on the course management system that acts as an archive for any material the student may want to reference later. That archive would contain all videos, notes, and assignments that the student would have previously encountered. Of course with most course management systems, students are able to access all previously submitted materials from within the grade center.

Some instructors construct informal video guides that explain this folder structure and the moving window idea. We have linked three of those below from an early iteration of our online course. Again, these kinds of videos are often provided on the first day of course

·	http://screencast.com/t/hOfWY2lw06v;
·	http://screencast.com/t/NwYaoRePSFd;
·	http://screencast.com/t/qjUEXKd2.

The above videos were created using JING, which we will say more about later. Additional information about JING can be found at https://www.techsmith.com/jing-features.html.

2.2. Daily Messages from Instructor

We also very much wanted our online class to be personal. We wanted the students to feel like they knew the instructor and, in some small way, have a sense of community similar to what we try to construct in the face-to-face version of the class. So we built into our design a “daily message” from the instructor. This message briefly introduces the topic which will be covered that day, explains how that material connects with what was just finished the day before, and what kinds of activities and exercises to expect in the next activity window. For most of us, these are videos we make and have appeared at midnight, which is the beginning of the next moving window. Some instructors mix videos with typed messages, but the point is that they are freshly created, a way of having the instructor greet the class at the beginning of each moving window. Some instructors embed the recorded video on the course management site, while others just post a YouTube link on that site. No matter, the purpose is to create a daily communication with the instructor that also serves to explain what the students were expected to learn and achieve for the day. Rose (2009, p. 1), found that

…student perceptions of this instructional strategy (Instructor-made videos) were overwhelmingly positive, with all of the students surveyed expressing satisfaction with this method of instructor-learner interaction. Open-ended answers revealed that students felt more connected to the instructor when able to view instructor-made videos.

It is cumbersome to give the reader a feel for the daily message in a print medium such as this. However, the following link to one on YouTube that will give you the basic idea of what we did: https://www.youtube.com/watch?v=lO4OBx3w2zk. Remember, there may not be anything particularly profound said in any videos. Rather, their importance is that they are present each morning when the 42-hr window opens, creating a connection between the student and the instructor.

The videos are not elaborate productions in any sense, but rather short videos of the instructor casually talking into a camera on their computer or tablet. The message is a personal, new-and-not-recycled greeting from the instructor, explaining what is about to happen and how it relates to what has just happened in the last window. In the video presented above, the instructor gives a brief introduction to the work for that day, and emphasizes the importance of that work in the larger context of the course. This particular video introduces the vocabulary words parameter, population, statistic and sample. The instructor gives an example and also explains that the tasks at hand will focus on these words. Without the video, students may not understand the important role those words will have for the rest of the course.

As mentioned above, not all the daily messages were on video, though that is the medium many of us feel is best for them. Some were just notes from the instructor. For example:

Daily Message 7/31

Since we are heading into our final week of class I want to thank everyone for all of your hard work. You have all been great, I have been more than impressed with everyone. I know taking an online class has its own set of challenges, but you have met them.

I know this last module has been challenging and is the hardest of the three. Make sure to take the time to look over all of the answer keys.

In order to help you prepare for the Exam and Beyond the Class 3, I am going to start a thread on the discussion board titled "Module 3, Review material" in this thread leave me anything you are having problems with and would like me to clarify. This evening I will prepare a video to address any topics you would like help with. When leaving messages try to be a specific as possible as to what you would like me to discuss, also if there are any problems that you do not understand (after looking at the key) I can work through those and explain the answers. Just like the other Exams you will have a review sheet the day before the exam.

We are approaching the home stretch so keep up the hard work!

Once again, the point is that the contact is daily and personal. Our goal is to create a paced, real-time, connection between the instructor and the students. We want to reinforce to the students that the instructor is actively engaged in the material and partnering with them in the daily progress of the course. We do feel that it is important that the daily messages are not made too far in advance. The fresher the better. We have found that a near real-time production can address issues the students had with the material the day before, and allows for the inclusion of pop culture references as they happen. For example, during the presidential debates in the fall of 2016, the use of statistical concepts were repeatedly misused, and one of our instructors surfaced these mistakes in her daily videos, which gave the online course a dynamic element.

2.3. Video Responses to Prompts

When we first began to envision the online version of this course, we knew we wanted to be able to judge the depth and integrity of student responses to prompts. In a face-to-face environment, we can do that in a variety of ways, some as simple as giving quizzes or tests under tightly controlled circumstances. This has long been a problem for online classes and continues to be. Lockdown browsers and ethical agreements meet some of the need. More recently online video and audio proctoring services have become widely available and aside from the extra expense for students have real potential. We took a simpler approach, and try to address this concern by requiring certain prompts be answered by way of video submission. That is, students are asked to explain a certain concept or computation and film themselves doing that. We typically place restrictions on the length of the response (e.g., no more than 3 min) and control within reason the video type. We also require that the video be practiced more than once so that the final product is somewhat refined and organized. Typically, we construct a rubric in advance, and use that to grade the video submission. Two examples are given below. We are not able to link to actual student responses because of privacy issues.

Example Prompts

1.

This prompt addresses parameters and statistics. Your job is to create a 3-min video in which you do the following. Make sure you post that video on our course management site by the due date. • Give an example that illustrates a parameter, sample, statistic, and population. • Explain the difference between a statistic and a parameter. Which one do we typically know? • Explain the role that a statistic and a parameter have in the use of a confidence interval (CI).

2.

This prompt addresses correlation and causation. Your job is to create a 3-min video in which you do the following. Make sure you post that video on our course management site by the due date. • What is correlation? How does it differ from causation? • What is the correlation coefficient? Use three pictures to help with your answer. You need to show the pictures in the video. • Give an example when the correlation and causation are likely the same. • Give an example when the two are likely not the same. Explain why they are likely not the same.

What started out as a way of imposing a simple quality control check on the students’ work became much more. It has created a kind of personal buy-in for students that we did not anticipate. Indeed, by and large the students really like responding this way, once concerns over the mechanics of how to make the videos fade. Students now have a way to deliver a personalized message to the instructor, just as the instructor is delivering a personalized daily message to the class. So far, we have gotten videos from farms in rural Georgia, vacation homes in the Carolinas, and dining rooms from several states! We have students showing us PowerPoints, writing on paper, on dry erase boards, and just talking. One student even used sidewalk chalk and did her presentation outside on a stone surface, while her beagle bayed his disapproval in the background.

Students typically use either a webcam or their phones to record their videos, and most of them choose to then upload the videos to YouTube and submit the corresponding url for their assignment. It should also be noted perhaps that these videos are no harder to grade than short papers. In some ways they are notably easier since when it is clear that a student knows what she is talking about and has spent time with the prompt, it is easy to slide quickly through the video looking for key parts identified by the rubric. Rarely, do we have to listen to the entire 3-min video to judge the quality of the response. To be honest, they are often a lot of fun to grade.

2.4 Discussion Boards and Online Instructor Access

Students in the in-class version of our course have the opportunity to work in small groups as well as having real-time access to the instructor. We want students in the online class to have similar opportunities that would still offer flexibility of schedule and location. We attempt to address this with the use of discussion boards and virtual office hours.

Discussion boards are far from novel, but we ended up with a few interesting and perhaps unique twists in our implementation. Our primary goal is to have a way for students to ask questions so that other students can see both the question and the answer, and to avoid duplication of questions. Many of our instructors insist that all questions about the course material be posted in the discussion board. Questions about grades or other personal matters need to be emailed directly to the instructor. The following is specific to one instructor, but typical of what others do. When a student initially enters the discussion board, they have five forums available to them as follows.

·	Introductions—This forum is for introducing students to each other in the class. Students were asked to write a couple sentences about themselves, including hometown, hobbies, year in school, favorite movie, etc.
·	Questions on material from Module 1—This forum is for asking, answering, and archiving questions about Module 1.
·	Questions on material from Module 2—This forum is for asking, answering, and archiving questions about Module 2.
·	Questions on material from Module 3—This forum is for asking, answering, and archiving questions about Module 3.
·	Issues with Quizzes—This forum is for asking questions about quiz.
·	Miscellaneous—This forum is for anything that does not fit in the five above.

The introductions forum is used the first day of the course, on which students are required to post an introductory message to the discussion board. As noted above, students are asked to include some personal, though not invasive information. We feel this accomplishes two things. First, when students introduce themselves to each other and to the instructor, this adds to the personalized feel of the course. Second, requiring this initial post forces a first-day familiarization with how to access the discussion board and create a post.

The forums for questions over material from the three modules are where the majority of content questions are posted. Once a student enters one of the modules, she see a post created for each day of the course. See Figure 3 for a partial view of what this page looks like. Threads are fixed and, to minimize confusions, students are unable to create new threads. Students click on the link corresponding to the day's material that they have questions about.

Figure 4. Comparison of grades in in-class vs. online.

Figure 5. Comparison of DEW rates in-class vs. online.

Once selected, they are able to ask a question or respond to other students’ questions. Students can access any part of the discussion board during the course, even if a module is already over. The instructor monitors these forums and follows up as necessary when peer answers are either in short supply or incomplete. Many of our instructors will respond to an emerging common misconception by posting a short, customized tutorial video.

The “Issues with Quizzes” forum is used to address any issues with the quizzes. Students are asked to leave the date of the quiz with a brief description of what they were confused about, what might have been misstated, or was otherwise unclear. They are forbidden to post any answers to the quizzes. This is useful to us as instructors because if there is a mistake on a quiz, students can see that it was already brought to the instructor's attention via this forum. This saves the instructor from having to answer multiple emails about the same issue. Students are directed to look at the forum first if these kinds of questions arise.

As noted, using discussion boards is hardly new. One well-known challenge to using them is the potential difficulty for the instructor to keep up. There is no perfect answer to this. The more interactive an online class is, the more work, potentially, it will be for the instructor. In our case, we do the following and have found that the workload is quite manageable.

·	Limit need for trolling—Once a student posts to any of the threads on any of the forums, the instructor receives an email alerting them that there was a new post. This way the instructor doesn't have to constantly troll the course management site. Some of us route these emails to special folders so that we can check them when we have the time. We have tried requiring postings and even having the course management system automatically grade participation. The upside is that all students have to get involved. The downside is that the instructor gets a lot of email alerts and many of the postings are not urgent. After all, if the course management system is grading postings automatically it is only able to give credit for a student accessing and posting something. It has no way of assessing whether the post is in any way substantive. So our inclination has been to have the discussion boards be voluntary, a way of creating a useful though imperfect version of interaction students may be used to in a face-to-face class.
·	Have preset times for responding—While students are told not to expect immediate responses to postings, we often set particular times of the day when students know postings will be read and responded to. The pace varies according to instructor. We have some who like to respond within an hour and others who agree to respond only once or twice a day. Keep in mind that even in the face-to-face version of the course students unfortunately do not ask that many questions. Our online experience has not been so different. Rather, our goal was to make this online experience different than the stereotypical, wherein the instructor may not be available at all, or rarely.
·	Let peers take some of the load—Questions are not always or even most often about some subtle content issue. The same is true in our face to face course. Many questions concern generic issues such as where to find a certain definition in the readings or videos, what the second sentence in the third prompt means, or how to use Excel ™ to create a scatterplot. Peers are very good at answering these types of questions. Again, at the risk of over-making this point, instructor personalities vary a lot here, even in our small group of online instructors at Kentucky. Some are more comfortable than others allowing peers to handle questions without too much oversight, others not so much.
·	Keep responsibility on the questioner when appropriate—For many questions, the answers are simple: look in the syllabus, review the previous reading, Google on “scatterplots in Excel™,” etc. This not only keeps the workload more reasonable for the instructor, it is in our view the right thing to do for the student. As much as is possible we try to keep the pressure where it belongs.

As noted above, having students post to the discussion board reduces the need to answer the same question over and over. To help students become comfortable with the discussion board, we include information about the discussion board in both the syllabus and information given on the first day of the course. If students email us with questions that are not somehow private, we often just ask them to post the question on the discussion board and look for an answer there. If we happen to see a student trying to address a harder content item (e.g., “why is a p-value not the same thing as an alpha level”), we intervene, either with an encouraging message affirming that the response was correct or offering advice if not. In any case, it is best just to make sure the students realize the discussion board is about as reliable as peer discussions in class. They need to realize that while the interactions are useful, peer responses do not come with guarantees and the ultimate responsibility for knowing a response is correct rests with the student.

An interesting note is that students by and large do not fish for answers to the homework. In our experiences so far students use the discussion board as intended and ask genuine questions, though most are about execution and procedure and not about subtle content. Just below, we have included an example interaction on one of the discussion boards. We have repeated the student syntax exactly. The “3.14” refers to a particular exercise in the required class workbook.

Student 1

3.14

On Problem 2 are we supposed to try and solve it for the fpr? How do we know the phat and Po are?

Student 2

RE: 3.14

I think you're just supposed to say what the Ho and Ha are and then write what the FPR would be since the results are not statistically significant like we did the other day.

Instructor

RE: 3.14

[Student 2] is correct. Good Work.

This is a pretty typical example of the discussion board interaction, and a good example of a peer helping out.

We also use online office hours so students can have face-to-face contact with the instructor. Different instructors use different methods, and we do not micromanage what instructors use as long as they provide some version of this personalized contact. That having been said, most instructors here use Adobe Connect or Skype. Adobe Connect is a web-conferencing software that allows an instructor to set up meeting times with individuals or groups of students. With Adobe Connect, instructors are able to present students with materials, as well as have typed, audio, or video discussions. Instructors have extensive control over the interaction, with an ability, for example, to have a private conversation with a student if the student wants, or the ability to silence a student who might be attending the session just to complain. Admittedly, we have never had to do the latter! Our university has a license to use this software so any student who signs up for our courses can access it for free. As soon as the instructor initiates an Adobe Connect session, students are notified and can join if they want. Skype also offers an easy, familiar tool for students and instructors to connect face-to-face. Indeed, students tend to be more familiar with Skype and some of us find it easier to use than Adobe Connect despite the fact that Skype has less features and the instructor has less control over the communication and interaction.

Initially, we all offered preset office hours where we would hold an open session in Adobe Connect and students could join during that time. Students would typically not come at these fixed times and instead often requested to meet at another time. Of course this is not surprising since similar things occur with office hours in all of our face-to-face classes! The upshot is that some of our instructors just stopped holding fixed hours and hold virtual sessions on demand instead. The pattern we see is that most routine questions get answered by the discussion board. Students needing help beyond the discussion boards typically just contact us for a one-on-one virtual session.

This may seem daunting at first, since, in theory, one could end up with an impossible number of requests. We have not had that happen. In fact, the demand is very similar to what we see with our regular classes. Rather we only see one or two virtual sessions here and there requested. Let us not lose sight of the bigger point, which is that we wanted to make sure the contact was offered and available in an online setting, much the way it is in a face-to-face class. If for some reason dozens of students started requesting individual sessions we would have to change our approach and go back to the fixed hours. Consistent with encouraging more student responsibility, that may be the best thing to do anyway. Again, our own instructors vary on how they choose to handle the online contact.

It should also be noted that other technological methods may be better (e.g., Google Hangouts), but we have not yet tried them. We encourage you to go with what you are most comfortable with, provided you allow some kind of personal contact this way. We have had lots of good feedback from students about how this helped connect them to the instructor and the course. One of authors was even stopped on campus the year after teaching this course online and greeted by an online student who recognized him from the previous summer. That is quite unusual for an online class. We are sufficiently enamored with the idea here at the University of Kentucky that we have even built it into a new online applied master degree we recently started. These sessions are billed as “Meet the Expert” sessions and they are proving to be very popular with the graduate student population as well.

2.5. Peer Grading

The workbook required for this course emphasizes short answer student responses. While these short answer responses are beneficial to student learning, they also tend to create a challenging grading burden for instructors. Many of our instructors for this course use peer grading, which can be especially easy to adapt to an online class. We have used two automated methods of peer grading. One is called “Calibrated Peer Review™,” often just referred to as CPR, which is a utility out of UCLA. For each homework assignment, students are electronically calibrated, assigned peer documents to grade, and then required to grade their own assignments as well. Each step of this process is scored automatically as a part of the overall assignment grade. Of course, we had to build all the calibration materials and rubrics for our course into the UCLA sponsor site. There is a learning curve for instructors and students to familiarize themselves with CPR, and some students struggle initially with the complexity, but the payback on that time investment can be excellent. The peer review tool on Canvas is also an excellent choice. While this tool has far fewer bells and whistles than CPR, it is much easier for students and instructors to use. That can be a good trade-of and either can be used as seamlessly online as in a face-to-face setting. We currently use both here at the University of Kentucky.

Why do we like peer grading? Even if we could manage to grade huge numbers of submissions ourselves, it became clear to us long back that students rarely look at the results of that effort. In that sense we felt like the learning loop was not being closed. We want students to know what was valued about the assignment and to see suggestions for what reasonable answers might have been. By using rubric-driven peer grading, we are able to gently force an encounter between the student and a rubric and, hence, more effectively communicate what was valued about the assignment. This is part of the assignment in some of our classes and full credit for the assignment is not possible if the assigned peer review is not completed. Indeed, we consider the closing of that learning loop as a critical part of the homework assignment.

We would encourage you to not think of peer grading as only a means of coping with large numbers of students, though that is a good outcome in itself. Gunersel et al. (2008) suggested that using CPR actually helped students develop writing and reviewing skills in biology. Pittard (2014) discusses CPR in the context of the course in question, and more information can be found on the CPR home page: http://cpr.molsci.ucla.edu/Home.aspx.

3. Student Assessment

Hodgson and Pang (2012) make the case for providing students multiple venues of assessment, and we agree. For our course, students are assessed multiple ways. They have daily graded assignments and tests over larger quantities of material. They are also peer-assessed on homework, and sometimes self-assessed, depending on the instructor and the peer-assessment tool used. Most of us divide their course grades into three sources: Daily Work [the Beyond the Numbers (BN) assignments], Beyond the Classroom work (BCs), and Exams. The weight for each category, and what is actually included in each category, depends on the individual instructor. Daily Work usually includes the daily activities students perform, including quizzes and the BN worksheets, some of which may be peer graded. BC assignments are mini-projects and these are discussed in more detail below. Typical BN worksheets and BC Worksheets can be found at http://www.statconcepts.com/look-inside. Exams are given two–three times during the term, and usually consist of a multiple choice portion and an open response portion. The open response portion typically requires students to create either a text or video response.

We purposely structured our course so that students complete work daily, and receive some type of feedback every day. Our goal is not to make the course more work than a face-to-face class, but rather to make it more like a face-to-face class than a correspondence course. We were also mindful of the importance of keeping the scope within the practical limits of what we as instructors were able to do. Most of us simply use online quizzes for the daily feedback. These are easily constructed in advance and deployed on our course management system. While it depends on instructor, most just have these quizzes be multiple choice quizzes that are graded electronically. These quizzes can take a variety of forms. Sometimes we adapt questions from a homework assignment into a quiz format, and sometimes we use these auto-graded quizzes as checks on whether videos were watched or readings were completed. In all cases, we believe this helps keep the student involved in the course on a daily, or nearly daily pace, and keeps the instructor engaged with the students on a level that is manageable with respect to workload.

At our university we typically have a two-person team teach this course online. For the online course, the instructor consists of a primary instructor and a grader. We should emphasize that the way we have set up the course, graders are very useful but not necessary. Having a grading assistant simply adds another pair of eyes to the discussion board, provides another person to answer emails, and perhaps another person to help with the organization of the learning management site. What makes the online course different is not really dependent on us having a grader. If we have to work without a grader, or with an inexperienced TA, then we adapt just the way we all do in a face-to-face environment. As mentioned, electronically graded quizzes are very helpful in this regard, as is peer grading, and grading with rubrics in general. Some of our instructors will grade for effort only, and some may select only certain items to grade. These are the same compromises we might have to consider in a face-to-face class, and may be necessary in an online class if the enrollments are not capped to be similar to the face-to-face environment. While we do not endorse any particular compromise approach, we do want to point out that we feel strongly about the use of peer grading for increasing the student-centered experience, and we hope the student learning, while substantially reducing the grading load for the instructor.

With the use of peer grading, auto-graded course management assessments, and video submissions, we are able to both provide timely assessment and increase the students’ involvement in their own course. These assessment strategies also keep the grading load at an acceptable level for us. This is very consistent with our mantra that one of the primary duties for an instructor in a student-centered environment is to prepare that environment in such a way that the student is gently forced to take responsibility for her own learning. While this creates more work up front for the instructor, that work tends be more manageable (and less time sensitive) and tends to be more focused on creating assessment plans for the entire class, as opposed to scoring individual papers for class members.

4. Reflection

We are generally very pleased with the course we have created. It has been used as a model for other online courses in our college, and—as mentioned above—for our new online master in applied statistics. On the one hand, much of what we have described above points simply to design challenges that we would all face when creating any online class in virtually any subject. The larger point is how we have tried to address those challenges while at the same time maintaining some of the innovative qualities of a good face-to-face environment. In addition, we have described strategies we are using in an attempt to capture online some of what we feel is special about our student-centered offline environment.

We have taught this class online now for approximately six summer terms, five winter terms, and two fall terms. Therefore, we have had ample time to reflect and calibrate based on what we hear from the students and experience as instructors. An anecdotal reflection on what we think is working and not at this point would include the following.

·	Students in our face-to-face version of this class are not always happy about the extra work that comes with a student-centered environment. Our online students largely feel the same way about the workload. In truth, this is not so much about student workload—that is very manageable—as it is about consistent, ongoing student engagement. Our moving window design forces that engagement much like the natural structure of a three-day-a-week face-to-face class would.
·	Our instructors are generally very positive about the effect of the daily messages and the moving window. They feel that this gives them a personal connection with many of the students and forces the students to stay involved in the class on a regular basis. One instructor noted that “the moving windows for assignments forces the students to treat the class as they would a face to face summer class. So instead of trying to take two or three massive bites of information they are forced to process smaller bites at a time.”
·	There is still concern about cheating. But this is not specific to our class, rather is a concern about all online classes. One instructor lamented that if a student were determined to cheat then all he would need to do is provide his access ID to another student who could take the entire class for him. Another instructor uses an online proctoring system because of similar worries.
·	The discussion boards are generally seen as better to have than not have, though many of us feel that once you are in the online environment it is simply very difficult to create the kind of personal connection that you can have with as student face to face.
·	Students who are unhappy with the course are mostly unhappy with the moving window, which instructors are the most positive about. Some students are not willing to accept the dynamic nature of the course and are upset that if they stop participating for two weeks that they are then not allowed to go back and make up material they missed.

At some point, experienced instructors simply have to gravitate toward what they feel is best. Felder and Brent (2017), responding to similar student pushback in any student-centered environment, implore the instructors to stay the course:

This is no time to give up. To counter students’ resistance, try to persuade them that you are neither playing a game nor performing an experiment, but teaching in a way known to help students learn more and understand better.

This is just the approach we are taking. We try to explain to the students why we have the course structured the way it is, why this is a better product for the money they are spending, and how it is in their best interest to have a student-centered experience even online. To be fair, however, most of our students seem to appreciate how the course is structured and are cognizant of the attempt to make it the best course for them that we can.

We do not have any possibility to compare the way we offer this class online to some other way of offering it. Indeed, when the course was brought online in 2011, it was the first time it was ever taught online in any fashion. It is interesting, however, to compare the grade distributions and the DEW and pass rates for this class and its in-class counterpart. The DEW rate is a measure of success used by the University of Kentucky and it is the percentage of students who get either a D, an E (failing grade), or withdraw from the course. Figures 4–6 are for the same instructor, over several semesters in-class and several teaching the online class.

Figures 5 and 6 offer similar comparisons for DEW and pass rates. The error bars exhibited are simple 95% error bars. Specifically, the CIs are as follows.

Figure 6. Comparison of pass rates in-class vs. online.

CIs for DEW

1.	Online 0.1453–0.2316
2.	In-class 0.0723–0.1365

CIs for PASS

1.	Online 0.8631–0.9302
2.	In-class 0.9113–0.9623

Likewise, a two-tailed z test comparing DEW rates produces a z of −3.1445 and a p-value of 0.00168. This is consistent with the CI. A two-tailed z test comparing pass rates produces a z of 1.9213 and a p-value of 0.05486, which in this case is also consistent with the CI perspective.

Clearly, the online class is not performing quite a well as the face-to-face version of the same class. However, research shows that the differences between two such offerings are usually much greater than we are seeing here. There are several sources suggesting that a higher dropout rate for online courses is expected (Jaggars and Bailey 2010; Lee and Choi 2011), with some suggesting that the online dropout rate can be two–three times higher than in-class (Blackner 2000; Zavarella 2008). This suggests that, although our withdraw rate is higher in the online class, we are still doing better than previous studies would have predicted.

We are continuing to think about how to make the online version of this course better. At the present we are trying to determine the best, most cost-effective proctoring software to build into the course offering. Students already pay a course fee that is designed to support infrastructure, but since these online students are no physically sitting in our classrooms using our computer equipment we want to try and divert those fees to partially underwrite state-of-the art proctoring support. We are currently looking at several different brands, some of which monitor eye movement using the student's web camera, some record the student's full movements, others have an actual person doing the monitoring. We have experienced these being used successfully at other places on our campus (e.g., engineering) and at other universities (e.g., Miami of Ohio).

Finally, we are also working with a vendor to move the book to a new and innovative online format. In this new format, students would have to purchase a key code to have access to the book, but all answers to activities could then be created online and exported to system-personalized pdf files for peer grading or general submission to Canvas. This would help us as instructors know that all students have the book, something which is impossible to know with online students. And having system-authenticated pdf files created by the online book management software could help with integrity issues.

5. Contact Information

The authors are excited about what they have learned from this online experiment and would be very happy to engage in further discussion with any reader interested in knowing more about our experiences. Our contact information is given on the opening page of the article.