Explicit Content: Two Experiments on Bringing Writing Instruction into the Political Science Classroom

Abstract

This paper reports findings from two studies of student writing performance carried out in undergraduate political science courses to test whether low-cost interventions could lead to improvements in writing outcomes. Students enrolled in an introductory comparative politics course (N = 180) and a sophomore-level survey course (N = 101) were randomly assigned to receive short instruction sessions in specific writing skills led by a faculty member or senior graduate student. In the first experiment, half the class received explicit instruction on writing (focusing on two specific skills from a comprehensive rubric that we subsequently coded their papers against) while the other half did not. In the second experiment, all students received general writing instruction, but the treatment group also received detailed instruction in the use of a writing-skills rubric that formed the basis for evaluating students’ performance on written assignments. In both cases, the post-treatment difference between groups was small but showed an improvement in the specific skills taught, while showing no change or slightly decreased performance in related skills that were not explicitly taught. We conclude that limited interventions of the sort we were able to engage in were not sufficient to bring about a meaningful improvement in student performance, but the preliminary evidence suggests that the use of rubrics as a pedagogical tool, combined with repeated small-scale interventions, may be effective if targeted and/or repeated across a course or curriculum.

Keywords:

• Writing
• rubric
• assessment
• experiment
• randomized

Acknowledgements

The authors wish to thank Jenny Bergeron, Emily Clough, Ryan Enos, Jennifer Hochschild, Leslie Finger, Bobbi Gentry, Tom Jehn, Jonah Johnson, Risa Kitagawa, Steve Levitsky, John Phillips, Kirsten Rodine-Hardy, Shanna Weitz, and Cheryl Welch for their support and helpful feedback, as well as the six graduate student scorers.

Notes

1 This mirrors approaches to writing-as-critical thinking in other disciplines; see, for example: Emig 1977; Bangert-Drowns, Hurley, and Wilkinson 2004; Fry and Villagomez 2012; Hebert, Simpson, and Graham 2013; and Arnold et al. 2017.

2 A different set of scorers was employed for each study, but the calibration process was kept the same. All scorers within a given study were current Ph.D. students enrolled in the department, and all were the same gender (all male in study 1, all female in study 2).

3 The small number of students taking the course in study 2 outside their second year (8 of 91) does not permit a meaningful measurement of average differences and is not considered here.

4 In the first study, a number of students were taking this class concurrently, or did not take it until after the course studied here (the latter are excluded from results).

5 The number of students in the upper-level course who had taken Expository Studio (4 of 91) is too small to allow a meaningful measurement of average differences, and is not considered here.

Additional information

Funding

Harvard Initiative for Learning and Teaching (HILT); Harvard College Dean's Office.

Notes on contributors

Colin M. Brown

Colin M. Brown is an Assistant Teaching Professor in Political Science at Northeastern University. His research interests are in migration, citizenship, and political incorporation, particularly in Western Europe. Additionally, he conducts pedagogical research focused on writing in the discipline.

Sarah E. James

Sarah E. James is a Ph.D. candidate in Social Policy & Government at Harvard University. Her teaching and research interests focus on race, social policy, and inequality in American politics. Sarah's dissertation uses an organizational and institutional approach to examine state-level social policies and evidence-based policy making, particularly as they impact persistent inequality. Prior to graduate school, she worked in K-12 education as a high school principal and earned an M.Ed. from Boston University.

George Soroka

George Soroka is a Lecturer and Assistant Director of Undergraduate Studies in the Government Department at Harvard University, where he oversees the senior thesis writers' program. His primary research interests focus on identity politics and regime change in post-communist Europe.