Scalable, scaffolded writing assignments with online peer review in a large introductory economics course

Abstract

Despite widely acknowledged benefits of integrating writing into economics courses, instructors’ costs are often prohibitive. To reduce costs and make writing assignments more feasible, the authors describe multi-part, scaffolded writing assignments developed by an economist and a WAC (Writing Across the Curriculum) specialist, integrated into an 800-student introductory economics course with multilingual students and TAs. Students draft and revise an abstract and later draft and write an op-ed with a convincing economic argument for a general audience. The authors use writing centers and peer review software to provide feedback while reducing grading time, and train inexperienced TAs to evaluate student writing through detailed rubrics and moderated marking sessions. They provide detailed assignment descriptions and an accounting of resources and time needed to grade each assignment.

Keywords:

• Assessment
• critical reading
• peer review
• rubrics
• writing assignments

JEL codes:

• A2
• A22

Notes

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 The International Network of WAC Programs (INWAC) (2014) has a statement of WAC principles and practices, including “WAC refers to the notion that writing should be an integral part of the learning process throughout a student’s education, not merely in required writing courses but across the entire curriculum. Furthermore, it is based on the premise that writing is highly situated and tied to a field’s discourse and ways of knowing, and therefore writing in the disciplines (WID) is most effectively guided by those with expertise in that discipline.”

2 “The nationwide movement of ‘Writing Across the Curriculum’ offers possible help [so that] Economics faculty can with modest effort learn how to make writing integral to their advanced courses within a reasonable time commitment” (Siegfried et al. 1991, 211). Crowe and Youga (1986) were the first to briefly discuss WAC in economics, but the AEA report by prominent economic educators had a much broader impact.

3 Articles by Hansen (1993), Cohen and Spencer (1993), Abdalla (1993), and Davidson and Gumnior (1993).

4 The Department of Economics has 62 full-time tenure-track faculty and seven full-time teaching-stream faculty. Most TAs are economics MA or PhD students, but between one-half and three-fourths of the TAs in our course are graduate students in the 2-year Master’s of Public Policy (MPP) program, which requires a full-year graduate micro/macro course taught by an economics faculty member. While the economics qualifications of these MPP students are not as strong, they are excellent TAs for this economic literacy course—more versed in writing, policy applications, and the experience of learning some economics for its applicability.

5 It is impossible to get a precise number for the percentage of English Language Learners because there is no definition of such a category, and the university does not administer a language aptitude test to all incoming students. We have data on the number of incoming students required to show proof of English language proficiency (TOEFL or other test score), but this applies only to those who did not complete 4 years of high school in Ontario, and does not cover the many students who are more comfortable in a language other than English, whether they are Canadians, permanent residents, or visa students.

6 The Faculty of Arts and Science has seven college writing centers with approximately 30 full-time equivalent instructors and twice as many adjunct instructors. The University of Toronto as a whole has nine undergraduate Writing Centers on three campuses. http://writing.utoronto.ca/writing-centres/

7 Normally, participating WIT departments receive between 280 and 380 hours to supplement course TA hours in WIT courses with individual TAs receiving an additional 5 to 15 hours over their base hours for teaching and grading writing.

8 At the University of Toronto, students must choose either an intensive specialist major, a double major, or a major and two minors. With 40 one-semester courses required for a degree, a specialist major has 24 courses, a major has 14 courses, and a minor has eight courses. Specialist majors generally are preparing for graduate school. Students who take ECO100Y, Introduction to Economics, are eligible for all economics degrees. The unsexy name for ECO105Y, Principles of Economics for Non-Specialists, is accurate in that it allows students to pursue an economics major or minor, but not a specialist degree. Both courses are being transitioned to a micro-macro pair of semester courses.

9 The ECO100Y introductory course required for specialists in economics does not use the abstract or op-ed writing assignments. ECO100Y is more model- and technique-oriented, has a math assessment module, and uses MobLab (moblab.com) for games and experiments in tutorials.

10 Hall and Podemska-Mikluch (2015, 13) have a similar objective for the op-ed assignment they created in 2013 in an intro class of 26 students at a small liberal arts college requiring “that students internalize the economic way of thinking and learn to exercise active citizenry.”

11 According to Cohen and Spencer (1993, 221), “Students face a unique difficulty in clarifying the audience for whom they are writing. In most writing situations, the writer writes to inform a reader who is generally less knowledgeable. In sharp contrast, students are asked to write for a reader (the instructor) who is clearly more knowledgeable. This places the student writer in an artificial and hence difficult situation.” Our writing assignments all specify the audience as the general reading public. Because of the peer review process, the actual audience of fellow students is similar.

12 First drafts submitted on the peerScholar software are not graded. But students who do not submit a first draft cannot participate in the peer assessment, and lose those assessment marks (roughly 20% of each writing assignment grade). Because work-minimizing students might submit effortless first drafts, the assignment instructions include “Your draft must be complete and written in full sentences. You will lose marks for submitting an outline or point form draft.” The combination of incentives and potential penalties (we have never actually subtracted marks on first drafts) has produced reasonable first drafts.

13 According to Phillips (2016, 12), “even after controlling for differences in students’ accounting knowledge and initial case writing performance, peer assessment scores were positively associated with performance … for a second case. These incremental learning benefits arose only by giving feedback to others; subsequent case scores were not associated with feedback received from high-quality peer assessors … These … findings are consistent with prior research that finds actively generating feedback can be more effective than passively receiving feedback in connecting with subsequent academic work (Li, Liu, and Steckelberg 2010; Lundstrom and Baker 2009). Active generation of feedback for others fosters connections with subsequent performance because it engages students in (1) understanding the standard against which students compare their peers’ work, (2) recommending actions and steps to close the gap between the standard and their peers’ current achievement, and (3) reflecting on one’s own work (Sadler 1989; Nicol, Thomson, and Breslin 2014).”

14 http://vision.peerscholar.com.

15 Competitor software includes Turnitin’s PeerMark, Calibrated Peer Review (CPR), and Scaffolded Writing and Rewriting in the Discipline (SWoRD). For comparisons, see Phillips (2016) and Søndergaard and Mulder (2012).

16 Breuer (2017, 39) describes this approach as seeing “behind the linguistic façade, actively making efforts to find out what the writer wants to say and evaluating the content independently of the language quality in which it is executed.” We encourage the students giving peer feedback and the TAs to take this approach, not because we believe grammar and correctness are unimportant, but because there is little evidence that grammar correction is effective. Instead, English language learners need more practice reading and writing in the target language (Truscott 1996). Breuer recommends undertaking linguistic revisions as a separate step to avoid overtaxing students’ cognitive capacities and argues that students are more motivated to revise their texts if they receive useful feedback on their content.

17 Students did use the campus writing centers for help with the writing assignments. They work one-on-one with a writing center instructor for 40 to 50 minutes in what are usually face-to-face sessions (a small percentage of appointments are virtual). In 2016–17, 71 ECO105Y students (out of 615 who finished the course) had a total of 139 appointments at the college writing centers. In 2017–18, 59 students (out of 675 who finished the course) had 135 appointments. For the two years combined, about 10 percent of students sought help. Because most students seeking help went multiple times, writing-center appointments as a percentage of enrollments were 21 percent.