Do Online Discussion Groups Enhance Students’ Analytical Ability and Recall of Factual Knowledge?

ABSTRACT

This research examines whether online discussion groups enhance students’ ability to analyze political problems and recall factual knowledge. The study compares four sections of the author’s introductory world politics course—two that employed online discussion groups and two that did not. The results revealed little difference in the performance of the sections with and without the online groups. These findings suggest the need to pursue further research focusing on how learning is measured, how discussion group size affects outcomes, how providing students with a study guide detailing course objectives impacts learning, and whether alternative forms of active learning, such as research projects or problem solving exercises, might better develop the capacity for analysis.

KEYWORDS:

• Discussion groups
• interactive learning
• online learning

Notes

It should be noted that the courses analyzed by Pollock and Wilson (2002) and by Wilson et al. (2006) had additional online course components that went beyond the use of ODGs.

While previous research reports that ODGs are related to improved factual recall, it is not always clear how such recall was measured.

The Institutional Review Board for Human Subjects Research at Miami University (Ohio) approved this research. Miami University is a publicly funded residential institution in southwest Ohio with approximately 17,000 students. U.S. News ranks Miami #79 among national universities. Miami accepts approximately 79% of its applicants. Among Miami’s students, 25% received a composite ACT score of 24 or lower, while 25% received a score of 28 or higher.

By section, the enrollments in each class were Spring 2014: 37, Fall 2014: 38, Fall 2015: 35, and Spring 2016: 52.

The author used the coin toss to determine which pair of classes, Spring and Fall 2014 or Fall 2015 and Spring 2016, would employ ODGs.

Bryant (2005, 274) recommends providing detailed class instructions and holding individual student meetings to guarantee that students can participate appropriately online. It should be noted that the author communicated immediately with any student who failed to engage in the required weekly ODG activities.

Hamann et al. (2009, 9) employed ODGs of 10 students but suggest the need for research that looks at larger or smaller groups. Hence, the smaller group sizes in this analysis.

Clawson et al (2002, 714) and Hamann and Wilson (2003, 547) recommend providing a weekly prompt to promote and guide online discussion.

Over the course of the semester, this weekly participation format meant that students who participated fully would write about 30 posts and would read approximately 120–150 posts from other students.

While awarding participation credit is common, the amount granted varies greatly. For example, Webb et al. (2004, 96) gave students 3% credit, while Ellis et al. (2009, 307) allowed ODG activities to count for up to 40% of their students’ final grade. Indeed, one might regard offering participation credit as an integral component of the ODG teaching technique since without it students tend not to engage in ODG activities. Of course, this means that when analyzing student performance in ODG and non-ODG sections one confronts the possibility of an interaction between the amount of participation credit awarded and the effects of the ODG exercise upon student performance. That is, a superior outcome for ODG students (which, as will be seen below, this research did not find, rendering the problem moot in this case) may result either from the ODG exercise or from the extra incentive given to ODG students. This problem may be resolved in part by conducting a series of studies in which the level of participation credit varies. If participation credit is the key to student performance, then one should detect a relationship between level of credit and student performance. While such research is beyond the scope of the current analysis, it is an interesting area for future inquiry. The current research should therefore be regarded as but a single step toward understanding the effects of ODGs and of granting credit for extra activities. It should be noted that when evaluating weekly posts, students received one point for an initial post that provided a complete discussion of a concept or problem (depending upon the nature of the weekly assignment) and a second point for a post that clearly indicated that the student had read the posts from the other members of her/his ODG and was providing a substantive response to one or more of those posts.

To assess how comparable the ODG and non-ODG sections were, the author calculated the students’ average overall class GPA. The data used included every grade every student earned in every class with the exception of the author’s world politics class. The average overall GPA both in the ODG and in the non-ODG sections was 3.1. All GPA data were obtained from the university registrar.

To guard against biased grading, 15 of the examinations from each of the exams in each of the classes in the study were intermixed with examinations from previous classes and graded again. In all cases, the exercise produced the same grade as was originally assigned. Examples of the factual knowledge answers are available from the author upon request.

It is important to remember that the comparisons of student grades herein are based on how well students performed on assignments relating to the recall of factual knowledge and the analysis of a political problem, as described in the preceding two paragraphs. The overall course grades, which in the ODG sections included ODG participation credit, were not compared. Hence, the comparisons between the performances in the ODG and non-ODG sections were not affected by the credit the ODG section students received for engaging in ODG activities. To avoid bias, the same type of intermixing of exams that was described in the previous Note was employed with this exercise with the same results. Examples of the analytical essays are available from the author upon request.

Each student’s overall GPA was calculated for all college courses taken to date with the exception of the author’s world politics course. For students in their first semester in college the calculation of overall GPA was based on the grades earned during that first semester, excluding the author’s course.

An A grade meant that the student participated in her/his ODG to the extent that she/he received at least 90% of the available ODG points, a B meant the student received between 80 and 89% of the points, a C was assigned to those getting 70 to 79% of the points, a D involved 60 to 69% participation, and an F meant the student participated less than 60% of the time.

The program used herein to set up the chat rooms for the discussion groups did not have the capacity for determining how often students visited their chat room to read posts. Hence, it was not possible to conduct the same detailed analysis performed by Hamann et al. (2009).

Average grades were calculated by assigning a value of 4 to a grade of A, a 3 to a B, 2 to a C, 1 to a D, and 0 to an F.

The z test for statistical significance was employed instead of a t test because the sample sizes were relatively large and because the z test does not require that the data be normally distributed. Of course, the data in this analysis are not from a random sample, which both the z test and the t test require. Therefore, all significance tests here should be treated as an approximation. All tests were two-tailed. See Hodges et al. (1975, 185–188) and Bowen and Weisberg (1980, 134–138) for discussions of significance tests.

Chi square allows one to compare two or more distributions to assess whether they are independent or appear to come from a single population (see Hodges et al. 1975, 219–37; Kay 1991, 134–46).

To assess whether a more nuanced grading system would produce another result, the z score and chi-square tests were calculated for a grade distribution that included + and − grades. The results were not significant at the p < .05 level. In addition, to determine whether ODGs might have varying benefits for students at differing times of the academic year, the Spring 2014 and Fall 2014 recall and analysis grades were disaggregated and analyzed as above. There was no significant difference between these sections.

For a discussion of this issue, see Du et al. (2005, 209) and Meyer (2006, 86).

Clawson et al. (2002) serves as an example of such an effort.

Additional information

Notes on contributors

John Rothgeb

John M. Rothgeb Jr. is a Professor of Political Science at Miami University in Oxford, Ohio. He is the author of several books on International Political Economy and World Politics and has published several dozen articles on domestic and international conflict, alliances, development, interdependence, academic tenure, and teaching techniques.