The Role of Homework in Student Learning Outcomes: Evidence from a Field Experiment

Abstract

In this article, the authors describe a field experiment in the classroom where principles of micro- economics students are randomly assigned into homework-required and not-required groups. The authors find that homework plays an important role in student learning, especially so for students who initially perform poorly in the course. Students in the homework-required group have higher retention rates, higher test scores (5 to 6 percent), more good grades (Bs), and lower failure rates. The authors also study the relationship between endogenous homework submission and test performance using instrumental variable estimation and find that homework submission has a large positive effect on test performance.

Keywords:

• classroom experiment
• homework assignment
• student performance

Keywords:

• JEL codes A22
• C93
• I21

Acknowledgments

The authors thank Tisha Emerson, Ozkan Eren, Bob Hammond, Brendan Kennelly, Sudipta Roy, Michael Salemi, Katherine Sauer, Lester Zeager, and two anonymous referees for their helpful comments, and are grateful to Rob Lange and Kristi Merritt for their assistance in obtaining these data. This research has benefited from comments received from seminar participants at East Carolina University, North Carolina State University, University of Nevada Las Vegas, University of Texas–Dallas, Robert Morris University Teaching Economics Conference 2010, ASSA Poster Session 2010, and ASSA Teaching Economics Conference 2011.

This article is based on a paper that was presented at the National Conference on Teaching Economics held at Stanford University on June 1–3, 2011.

Notes

1. For a thorough survey of the homework literature, see Cooper, Robinson, and Patall (2006).

2. Specifically, students with an even last digit of their identification number were required to do homework if the coin-flip outcome was “heads,” while students with an odd identification number were required to do homework if “tails.”

3. Test questions were similar (but not identical) to the homework questions. Using similar questions on tests may cause some to question whether students are learning the material or simply gaining familiarity with comparable questions. We cannot identify the exact mechanism (learning or familiarity) through which homework increases test scores.

4. Initially, the Economics Department provided Scantrons free of charge to all participants; however, due to state budget cuts, free Scantrons were no longer given away in the final semester of the experiment. Students must pay 10 cents per Scantron.

5. Only the multiple choice answers were provided on the answer key without explanation (e.g., Q1: B, Q2: C, etc.).

6. Self-selection bias could potentially be corrected if we had the ability to estimate the propensity to drop the course (see Becker and Powers 2001). We did not, however, have a suitable variable in our data to serve as a valid exclusion restriction in a Heckman selection model.

7. For example, Hanushek (1979, 2005) suggested including past performance to control for the effect of previous inputs. Todd and Wolpin (2003) and Rivkin, Hanushek, and Kain (2005) proposed using fixed effects model with panel data to control for unobserved student characteristics.

8. Additional downward biases arise from (a) students in the homework-required treatment group who copied other students’ homework assignments, and (b) students in the homework not-required group who did not turn in the homework, but used the homework solutions to prepare for exams.

9. This graph appears in a longer version of this article, which is available upon request.

10. We find no significant differences among the treatment and control groups in our three observable ability measures (SAT Verbal/Math and High School GPA) for the students who remain in the course after each test. We are, however, concerned that an unobservable variable, (lack of) student motivation, is causing lower retention rates in the control group.

11. Each test was typically curved so that the class averaged a high C (approximately 78 percent) based on a 10-point grading scale. Also, course grades for homework-required students included a 10-percent weight on homework performance, which complicated the comparison of course grades between control and treatment groups. Nonetheless, we found no significant differences in the distribution of As, Cs, or Ds for treatment and control groups. We did, however, find that the treatment group has a significantly larger distribution of Bs (p-value < 0.05 using a Wilcoxon rank-sum test) and fewer Fs (p-value < 0.01) than the control group.

12. Students who take all four tests do so because they are more likely to pass the course. If we restricted the sample to only students who took all four tests, then we would be imposing a restriction to include only the group of students who expected to pass. This induces a survival bias on the sample because the propensity to take the test differs for control and treatment groups. These results appear in a longer version of this article, which is available upon request.