Success is an Open Book: Online Diagnostic Tools and Learning Outcomes in Introduction to American Government Courses

ABSTRACT

This project examines the utility of a particular course enhancement, Cengage’s Aplia, and more generally interactive tools designed to facilitate reading in the introductory American government course. Using two control and two treatment sections of the course (one section each for two instructors) during the Fall 2013 term, we measured student performance, engagement, and interest in the subject matter. While there was no statistically significant benefit regarding interest level, use of Aplia resulted in improved performance and engagement in the course. We conclude that, subject to further study, the Aplia diagnostic quizzes offer tangible benefits in teaching the introductory American government course and offer further insights regarding these benefits.

KEYWORDS:

• American government
• engagement
• online
• reading

Notes

MindTap is Cengage’s more comprehensive online course supplement package, of which Aplia is one component. This study examines Aplia only, which is available as standalone product as well as part of the MindTap package.

The mean grade point average (GPA) for all sections of the course, for the past several years, is approximately 2.0. Over the past two years our department, which includes political science, history and philosophy, has tracked student engagement—attendance, use of supplemental instruction, contacts with the instructor, accessing online review materials (if available)—in our general studies courses and has found the most telling factor to be attendance: The mean GPA of students with three or fewer absences per term has been approximately 2.3; the mean of students with more than three absences has been 0.99.

This concern is well documented in K–12 education, as noted by Williams and Ortlieb (2014).

Information about Aplia and sample course Web sites by discipline can be viewed at http://www.aplia.com. Aplia allows instructors to add their own content to the site and the site’s gradebook, enabling the site to serve as a primary course management tool. To date neither of us have used the site in this fashion.

We did not require graded Analyzing Elections as part of the study; these exercises were either not used or were available within a menu of online active-learning exercises.

Aplia provides three options for scoring the Problem Sets. The default option allows students up to three attempts for each question, with the final score being the average of the total score divided by total attempts. Two other more lenient options are available: (a) “Do No Harm” and (b) “Keep the Highest.” The “Do No Harm” method allows students to answer up to three versions of a question wherein only scores equal to or higher than subsequent attempts will be considered in the average. Under the “Keep the Highest” method, students can answer up to three versions of a question, and the final score is the highest score of the attempts.

The study coincided with our department’s adoption of a Cengage textbook for the course; due to our rental program, the university purchased several hundred copies of the book, each packaged with a free Aplia access code.

One of us did have slight differences in the control and treatment sections: the control section used an online Constitution and state/local government quiz, plus one additional active-learning exercise to create a balance in workload between the sections. The mean score for these additional assignments were within 0.5% of the course mean in the control; thus, they had no discernable impact on the data presented here.

Due to institutional requirements and Family Educational Rights and Privacy Act regulations, we were limited in our ability to examine or report individual student performance relative to Aplia participation and/or self-reported engagement. We were additionally prevented from accessing student GPA information, which would arguably have been useful. Less than 5% of students in our Introduction to American Government courses are political science majors. Additionally, we can provide a profile of students at our university: 12% of our students are of minority background, 10.9% of our students are international, and 37% are from other states. The average ACT score for incoming freshmen is 22.8 and the average high school GPA is 3.35.

While it may have been advantageous to conduct the study over multiple semesters, the authors considered it to be unethical to withhold the Aplia “treatment” from students in future control groups after discovering its effectiveness.

It is important to note that the course grade averages for the treatment groups were recalculated without the Aplia quiz grades. This recalculation was necessary to remove any effects on overall averages of the Aplia quiz grades themselves.

A copy of the survey is included as Appendix 1.

As noted above, Aplia provides three options for scoring the Problem Sets: “Average,” “Do No Harm,” and “Keep the Highest.” Given our observations from the study, both authors have since adopted the “Do No Harm” method.

Additional information

Notes on contributors

Ryan Reed

Ryan Reed is an Assistant Professor of Political Science at Bradley University in Peoria, Illinois. He earned his PhD from the University of California, Davis, in 2012. His primary field of scholarship is political theory. His research focuses on social contract theory and issues of rights surrounding sexual orientation and gender. His recent publications include “Are the Kids Alright?: Rawls, Adoption, and Gay Parents” in Ethical Theory and Moral Practice and “September's Children: Does Liberal Neutrality Fail Gay and Lesbian Children?” in Politics, Groups, and Identities.

Daniel E. Smith

Daniel E. Smith is an Assistant Professor of Political Science at Northwest Missouri State University. A former practicing attorney, he teaches courses in political science and criminal justice. His primary research interests include judicial politics, jurisprudence, Constitutional Law, emphasizing individual rights, equal protection, and election law. His work has appeared in History of Political Thought, and his work on congressional elections has appeared on multiple occasions in The Roads to Congress series (Lexington Press). He has also done extensive work developing judicial simulations for undergraduate teaching.