Targeting Conceptual Understanding: How to Improve Learning and Course Retention in Research Methods Courses

Abstract

In the summer of 2015, I conducted a Scholarship of Teaching and Learning (SoTL) experiment in my quantitative methods for political science class. The experiment tested if, during instruction, asking conceptual questions improved student learning more than asking recall questions. This article provides evidence that instruction with conceptual questions leads to better performance on daily quizzes, but not better performance on the course final. The teaching methods of this course led to an improved retention rate, where zero students dropped the course (in a course that historically has the highest drop rate in the major). I argue that the mechanisms that lead to these successes are higher student metacognition, an enhanced ability for the instructor to identify student misconceptions, improved class discussion, and an improved growth mindset for all students.

Keywords:

• Research methods
• clickers
• course retention

Notes

Notes

1 Clickers are remote controls brought by every student into the classroom. Instructors can then ask multiple choice questions to every student. Every student then has the opportunity to select their response. The instructor then sees every student’s response and has the option of sharing a histogram with the class.

2 Because LO9 was randomly selected into the treatment group, only the conceptual question as actually used in class. The recall question presented here is merely to illustrate how I discern the types of clicker questions.

3 Our data only indicate the number of students who dropped the course after the third week of class. Thus, it is likely that the true drop rate was considerably higher, as students typically leave courses in the first two weeks.

4 Clickers were only used during instruction. Quizzes and the final exam were taken with pencil and paper.

5 The final exam was worth 40% of the final grade. Quizzes, participation with clickers, and a research paper including a multivariate regression were each worth 20% of the final grade.

6 There were two learning objectives regarding skills in STATA. I did not test these skills in quizzes or the final exam. Thus, the students’ mastery of 28 learning objectives are included in this study.

7 The unit of observation is student-question. Thus, for each quiz question, there are approximately 26 observations (how many ever students were in class to take the quiz that day). The sample size for the treatment group (quiz questions from units that were taught using conceptual clicker questions) is 356. The sample size for the control group (quiz questions from units that were taught using recall clicker questions) is 359.

8 The standard deviation for concepts that had been taught with conceptual clicker questions was 0.37, while the standard deviation for concepts that had been taught with recall clicker questions was 0.43. Quiz questions could be graded either 0 or 1.

9 The unit of observation is student-question. Thus, for each exam question, there are 30 observations (the number of students who took the exam). The sample size for the treatment group (exam questions from units that were taught using conceptual clicker questions) is 300. The sample size for the control group (exam questions from units that were taught using recall clicker questions) is 240.

10 The standard deviation for concepts that had been taught with conceptual clicker questions was 2.67, while the standard deviation for concepts that had been taught with recall clicker questions was 2.25. Exam questions could be graded anywhere between 0 and 10.

11 The Cohen’s d and Hedges’s g estimates are both 0.23 (a similarly small effect size compared to the differences in quiz scores).

12 The standard deviation for concepts that had been taught with conceptual clicker questions, excluding the Question 1 outlier, was 2.59, while the standard deviation for concepts that had been taught with recall clicker questions was 2.25. Exam questions could be graded anywhere between 0 and 10.

13 When excluding exam question 1, the sample size for the treatment group is 270. The sample size for the control group remains 240.

14 The teaching assistant was responsible for the following: (1) grading all quizzes and exams, (2) attending class, and (3) assisting students while the class completed in-class practice assignments. The teaching assistant did not hold any sections or do any instruction independent of the course instructor. Furthermore, the teaching assistant was unaware of the hypotheses of this study, preventing biased grading of exam and quiz questions.

15 When teaching this class in a 3-hour section; I had three learning objectives per session with one or two clicker questions during the instruction of each concept. In a 50-minute, three-times-weekly course, I teach one learning objective per day with one or two clicker questions during each session.

16 For this to be successful, incorrect options should be intentionally designed to catch misconceptions. For example, different calculators treat negative numbers differently (in some you must put negative numbers in parentheses, but not in others). Thus, if the question is −10², “−100” would be a good incorrect option. For students who incorrectly select “−100,” you can make sure that they know how to properly use their calculator and the basics of arithmetic with negative numbers.

Additional information

Notes on contributors

Nathan J. Combes

Nathan J. Combes is an Assistant Professor of Political Science and Public Administration at Columbus State University. He teaches research methods, comparative politics, international relations, American government, and public administration. His research agenda focuses on pedagogy in political science, African politics, and public health.