Impact of a Flexible Evaluation System on Effort and Timing of Study

Abstract

This paper examines results of a flexible grading system that allows each student to influence the weight allocated to each performance measure. We construct a stylized model to determine students' optimal responses. Our analytical model predicts different optimal strategies for students with varying academic abilities: a frontloading strategy for those with high academic ability and a gambling strategy for others. We test the model using data gathered from several sections of an Intermediate Accounting course offered by a Canadian university. We find that most students did make decisions that were consistent with our analytical model. Our results suggest that the flexible evaluation system does not uniformly motivate students of differing abilities and does not encourage most students to maintain effort uniformly throughout the semester.

Keywords:

• Flexible evaluation
• timing of study
• accounting students

Notes

The error term represents other impacts on the grade besides basic academic ability (as proxied by academic average) and effort. These impacts could be the match between the student's learning style and the instruction style, a talent for the particular subject, strong people skills or even just luck. Further, it is not necessary to suppose that the same effects operate on each course component. For example, time management skills might be important in improving the outcome on the final examination, while the ability to adapt quickly to a new instruction method might be more important at the beginning of the course. In our discussion, we will refer to the effect as ‘luck,’ but it should be remembered it is not luck in the sense of random effects.

At the beginning of the course, the instructor provided an example of a prior year's mid-term, so that students could evaluate the level of difficulty in advance of writing the mid-term examination. Students were also advised that the format and types of questions on the final would be comparable in level of conceptual difficulty to the mid-term.

The terms Gamblers and Frontloaders are used to refer to observed groups while the terms Gamble and Frontload strategies are used to refer to strategies predicted by the model.

See Appendix 2 for more details regarding the predictions.

We do not focus on this prediction and other similar ones that involve unobservable variables such as effort and utility. These predictions are more difficult to test given that the variable of interest, although they can be derived directly from the model.

All of our data is taken from the courses taught by two instructors. We compared the results by instructor. While one instructor consistently had lower class participation allocations and scores, there were no other significant differences by instructor on average weightings or scores. Most importantly, the relationships across groups remain the same even when the data is analysed separately for each instructor.

Additional information

Notes on contributors

Parunchana Pacharn

This paper was edited and accepted by Richard M.S. Wilson.

Darlene Bay

This paper was edited and accepted by Richard M.S. Wilson.

Sandra Felton

This paper was edited and accepted by Richard M.S. Wilson.