![Sample our Politics & International Relations journals, sign in here to start your FREE access for 14 days]({"type":"image" , "src" : "/sda/5947/TOC-Subject-Sample-2021-Politics-International-Relations.jpg"})
Abstract
This research examines whether explicit course-level learning objectives (LOs) affect students’ perceptions of courses and ability to recall factual knowledge and analyze political problems. The study compares four sections of the author’s introductory world politics course – two that were provided with the explicit learning objectives and two that were not. The results revealed no difference in students’ performances when it came to recalling facts and analyzing problems. Analyzing student evaluations, however, indicated that students who received learning objectives perceived the grading of their assignments as fairer than did students who did not receive LOs, indicating that LOs help students feel comfortable in a course. This suggests the need for research examining how LOs interact with such educational techniques as learning teams and weekly writing assignments. Future research also might explore how LOs affect students in courses employing online and other delivery methods.
Notes
1 This definition is a rewording of one provided by Osueke et al. (Citation2018, 1). Allan (Citation1996, 93) and Prideaux (2000, 168) note that the educational literature employs many terms, including goals, intents, and aims, to refer to similar phenomena. Harden (Citation2002a, 117; Citation2002b, 151) and Maher (Citation2004, 46) distinguish between learning objectives as defined herein and learning outcomes, which are the knowledge and skills students actually display at the conclusion of a course of instruction.
2 Royal’s research was based on surveys of approximately 1,100 U.S. colleges and universities. The examples of institution-wide objectives mentioned in the text were among the most frequently found across the responding institutions. For additional discussions of institutional objectives, see Appendix A, Aziz et al. (Citation2012), and Martin and Mahat (Citation2017).
3 Kelly and Klunk (2003) and Ishiyama and Breuning (Citation2008) conducted surveys of political science departments to determine what learning objectives they set for themselves. The objectives found in the text were among the most often mentioned in both research efforts.
4 Additional discussion of this use of LOs is found in Howell et al. (Citation2003), Torrance (Citation2007), Stehle and Spinath (Citation2014), Mitchell and Manzo (Citation2018), and Osueke et al. (Citation2018). The author wishes to thank one of the manuscript reviewers for pointing out that this type of use of LOs is best described as an intervention.
5 Popham (Citation1973, vi) is among the earliest of the scholars who urges that learning objectives be used as an aid to student learning, labelling such usage “explicit objectives.”
6 Tyler (Citation1949, 16–17) discusses how many educators reacted to the changes in higher education and the scientific advances in the late nineteenth and early twentieth centuries by seeking to devise new approaches to teaching. He notes that when contemplating this challenge they gained insights in part from the analysis that was employed in World War I to determine the critical activities workers needed to know in order to become competent at jobs relating to war production. These early versions of what are now referred to as learning objectives were then imparted to workers as a means for speeding up their training. This approach to teaching was then incorporated into an academic setting as time passed. The work of Mager (Citation1962) and Popham (Citation1973) indicates that even as the use of LOs assumed institution-wide and departmental roles in later years, they continued to play an important role as aids to enhance student learning. For a discussion of the evolution of American higher education, see Cole (Citation2009).
7 Later in the same book, Popham (Citation1973, 109) states that “the pupils home environment can greatly influence … learning” and that this points to the need for “objectives [to] be communicated early in the instructional program” so that students know exactly what they are meant to learn.
8 It should be noted that Torrance’s (Citation2007, 292) research showed that explicit LOs helped students focus on some elements of their courses but also revealed that more subtle elements of the courses did not benefit from the LOs. In addition, Mitchell and Manzo (Citation2018, 466) reported research indicating that students made no systematic use of LOs as they engaged in coursework.
9 For a discussion of the important role both of these types of learning play in the political science curriculum, see Wahlke (Citation1991).
10 The Institutional Review Board for Human Subjects Research approved this research. The institution at which this research was conducted is a publicly funded residential institution with approximately 18,500 students. U.S. News ranks the institution among the top 80 of national universities. The institution accepts approximately 79% of its applicants. Among the institution’s students, 25% received a composite ACT score of 24 or lower, while 25% received a score of 28 or higher.
11 By section, the enrollments in each class were Fall 2016: 57 students, Spring 2017: 50 students, Fall 2017: 57 students, and Spring 2018: 50 students. The similarity in enrollments is due to departmental limits.
12 Campbell and Stanley (Citation1963, 12) note that a problem associated with the design used for this analysis (which those scholars refer to as “Static Group Comparison”) is whether the experimental and control groups are equivalent when the research begins. To determine whether the classes with and without ecplicit learning objectives were equivalent, the author calculated the section-wide average overall GPA across all of the enrolled students. This yielded an average GPA of 3.1 for the 107 students in the Fall 2016/Spring 2017 sections and an average GPA of 3.1 for the Fall 2017/Spring 2018 sections. In addition, the university at which the analysis took place does not allow students to receive credit for a course if they have taken the course previously, have taken a similar course at another university, or received AP credit for the course. Thus, one may reasonably assume that the sections in this study contained students with nearly equal academic abilities and knowledge of the material covered in the course. All GPA data were obtained from the university registrar. It also should be noted that Campbell and Stanley (Citation1963, 9) argue that employing a pre-test to establish equivalence runs the risk of alerting the subjects to the nature of the research and can undermine validity. To avoid this problem, the author chose to establish equivalence by examining the students’ overall GPAs and by relying upon university rules about previous academic experiences in similar courses. While establishing equivalence in this way is not ideal, it seemed to be the best available means for doing so without alerting the subjects to the basic purpose of the study.
13 A copy of the class syllabus with all of the learning objectives for each unit of the course is available from the author upon request.
14 Davis (Citation1993, 4) recommends that the number of LOs be kept manageable. Across the fifteen weeks semester, the students in the LO sections were given approximately 120 concepts to learn (about 8 each week) and 20 analytical problems to ponder (just under 1.5 per week).
15 These grading standards were modelled on those found in Educational Testing Service (2003, 27–28) and Allen and Tanner (Citation2006, 5–6).
16 Disaggregating the explicit objectives from the evaluation standards also points to an area for further research involving a situation wherein some sections receive the specific objectives but not the evaluation standards while others get both the specific objectives and the standards. This would allow one to determine how providing the evaluation standards affect student performance.
17 The reader is reminded that one purpose of learning objectives is to make it clear to students exactly what material they should focus on both when they attend class and when they study. Hence, while the students in all sections of the class were taught the same material pertaining to the learning objectives, only the students in the LO sections were told beforehand by way of the LOs where to concentrate their attention. As was noted above, the scholarly literature suggests that this prior information should lead the students in the LO sections to perform better than the students in the non-LO sections.
18 To reiterate, the factual recall exams did not ask students to define concepts that did not appear on the lists of learning objectives provided to the students in the LO sections of the course. It should be emphasized that the results in this analysis do not pertain to the overall course grade students earned. Instead, they relate to the grades students received for two types of examinations: (1) those that measured how well students could recall the factual information presented in the class and (2) those that sought to determine how well students could apply theoretical material discussed in class and in the readings to evaluate an historical event. The factual recall results in Table 1 pertain to the average grade students earned across all of the factual recall examinations. To determine whether analyzing the exams separately yielded different results, the same statistical techniques were applied to the average grades earned on each exam. That is, the first factual recall exams in the LO and non-LO sections were compared, as was the second such exam, etc. In every case, the results were consistent with those reported in Table 1.
19 The theory the students were asked to apply was selected from among the analytical problem prompts that were included among the learning objectives provided to the LO sections of the class.
20 To guard against biased grading, 15 of the answers from each of the exams or assignments in each section were intermixed with exams and assignments from previous classes that contained the same exam material and were then graded again. In all cases, the regrading produced the same grade as was originally assigned. The reader is also reminded that the comparisons of student grades relate to how well they performed on the factual recall tests and on the analytical problem assignments. The overall course grades were not compared because those grades included credit for other elements of the course that are not related to the current study.
21 An anonymous referee who reviewed this research suggested examining whether the LO and non-LO sections were significantly different as regards the number of students who dropped the course or received a final course grade of D or F. This analysis revealed that one student from the LO sections and one student from the non-LO sections received a final grade of D or F. No students dropped the course during the period under evaluation. Hence, there were no significant differences between the LO and non-LO sections along this dimension.
22 Since there were more than one factual recall tests, each student’s results were combined to produce a single score. These scores and the analytical scores were averaged by assigning a value of 4 to an A, a 3 to a B, a 2 to a C, a 1 to a D, and a 0 to an F.
23 The Z test for statistical significance was employed instead of the t test because the sample sizes were relatively large and because the z test does not require that the data be normally distributed. Of course, the data in the analysis are not from a random sample, which both the Z test and the t test require. Therefore, these significance tests should be treated as approximations. All tests were two-tailed. See Hodges et al. (Citation1975, 185–188) and Bowen and Weisberg (Citation1980, 134–138) for discussions of significance tests.
24 Chi square allows one to compare two or more distributions to assess whether they are independent or appear to come from a single population (see Hodges et al. Citation1975, 219–137; Kay Citation1991, 134–146).
25 To assess whether a more nuanced grading system would produce a different result, the z score and chi square tests were calculated for grade distributions that included + and – grades. The results were nearly identical to those that are reported. In addition, a z score test was made to determine whether there was a statistically significant difference between the LO and non-LO sections as regards the proportion of students receiving a grade of B or better. Again, the results revealed that there was no significant difference.
26 All student course evaluations were administered by the university registrar. The evaluation system called for students to rate each course along several dimensions using a scale that assigned a 4 to an excellent rating, a 3 to good, a 2 to fair, and a 1 to poor. The mean ratings and the standard deviations that are reported were provided by the registrar.
27 See Rothgeb (Citation2013) for a discussion of learning teams.
Additional information
Notes on contributors
John M. Rothgeb
John M. Rothgeb, Jr. is a Professor of Political Science at Miami University in Oxford, Ohio. He is the author of several books on International Political Economy and World Politics and has published several dozen articles on domestic and international conflict, alliances, development, interdependence, academic tenure, and teaching techniques.