Development and application of assessment standards to advanced written assignments

ABSTRACT

This study describes the results of a project that focused on developing an assessment rubric to be used as the assessment criteria for the written thesis of accounting majors and the quality of the coursework during the seminar. We used descriptive analysis and the survey method to collect information for the development work and to examine the effect of the rubric on learning. We find that the rubric has a positive effect on students’ understanding, self-assessment, confidence, and integration. We contribute to the extant literature by adding to prior work that has examined factors that can improve students’ learning outcomes. By synthesizing theories on approaches to learning and self-regulation, and combining them with literature on self-efficacy and social/academic integration, we bring conceptual clarity to the elements of learning in a course, which consist of written assignments and the accompanying group work. The paper demonstrates a way to help university students to learn via explicit assessment rubrics, and thus offers novel ideas for accounting educators.

KEYWORDS:

• Assessment
• assurance of learning
• rubric
• written assignments

Acknowledgements

We thank the reviewers and Gordon Boyce (the Associate Editor) for their very helpful comments. We are grateful to Kristine M. Brands for discussing the paper in the 2014 Midyear Meeting of the International Accounting Section of the American Accounting Association (San Antonio). We also thank Louella Moore for discussing the paper in the 2013 Annual Meeting of the American Accounting Association (Anaheim). We also appreciate all other comments received in those seminars. In addition, we thank Johanna Rämö, Kari Lukka, and other seminar participants for their constructive feedback in the faculty seminar of the Turku School of Economics in 2013 (Finland) and the comments received at the 2013 Annual Congress of the European Accounting Association (Paris). The authors would especially like to thank Maija Lampinen, Maire Syrjäkari, Seppo Ikäheimo, Juha Kinnunen, Barbara Kardos, and Matti Rudanko for their help and encouragement during the project’s development. We thank Wayne Losano and Jake Carpenter for language editing.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 AACSB = The Association to Advance Collegiate Schools of Business.

2 We define advanced written assignment to mean assignments that are written after the basic studies at the undergraduate level (e.g. electives in accounting) or which are written at any level of graduate studies. We acknowledge that, in many universities, undergraduate students do not have advanced studies (intermediate studies being the highest category). However, we use ‘advanced’ to signal that we are examining written assignments that require and develop deeper understandings to create new content from students’ existing knowledge following the principle of constructive learning.

3 The format of a bachelor’s seminar is clearly different from other courses at the bachelor’s level. The role of students’ independent work is highlighted. In addition, particularly for thesis-writing seminars, acting as an opponent to other students’ theses and active participation in class are emphasized.

4 In 2007, the Aalto University School of Business was granted an AACSB accreditation as the first business school in the Nordic countries. In 2012, the accreditation was renewed. The accreditation process will subsequently be renewed every five years if the required criteria are met. Aalto University School of Business is one of the six schools of Aalto University that was created as a combination of the Helsinki School of Economics, the Helsinki School of Technology, and the Helsinki School of Art in 2010.

5 The modified version of Bloom’s taxonomy was introduced in 2001 (see Anderson & Krathwohl, 2001).

6 Aggregate final grade is one measure for academic performance, and its positive association with deep learning has been discussed and documented in the existing accounting education literature (Biggs, 1989; Ramburuth & Mladenovic, 2004).

7 Earlier work uses the terms strategic and achieving approach interchangeably (Lucas & Mladenovic, 2004).

8 Academic self-efficacy means general perceptions of academic capability. An example of a representative statement is ‘I have a great deal of control over my academic performance in my courses’. Performance self-efficacy means perceptions of academic performance capability. An example of a representative statement is ‘What is the highest GPA that you feel completely certain you can attain?’ (Richardson et al., 2012).

9 The three forms of integration are defined in Richardson et al. (2012, p. 358). Social integration means perceived social integration and the ability to relate to other students. An example of a representative statement is ‘I find it easy to get to know other people’. Academic integration means perceived support from professors. An example of a representative statement is ‘Professors take a personal interest in helping me with my work’. Institutional integration means commitment to the institution. An example of a representative statement is ‘I am confident that I made the right decision in choosing to attend this university’.

10 Internal versus external control refers to the degree to which people expect that a reinforcement or an outcome of their behavior is contingent on their own behavior or personal characteristics versus the degree to which people expect that the reinforcement or outcome is a function of chance, luck, or fate, is under the control of powerful others, or is simply unpredictable (Rotter, 1989).

11 We acknowledge that the psychosocial context (Richardson et al., 2012) may affect learning outcomes also through an alternative path. For example, the student may achieve a flow-feeling, which directly improves the learning process without affecting her/his perception of task requirements. Moreover, we acknowledge that our model aggregates students’ feelings and does not make a distinction between past feelings and those which occur in the context of the writing course. For the sake of brevity, we do not add these additional dimensions in the model.

12 Further information on the Department of Accounting can be found at the following address: http://accounting.aalto.fi/en/.

13 We used descriptive analysis to justify the need to develop the assessment rubric. However, in Table 1, we report statistics which also use available data for understanding, motivation, and confidence in the time period after the implementation of the rubric to increase the sample size.

14 In the end of the bachelor’s thesis seminar course, we asked for undergraduate students’ motivation and self-confidence regarding the writing of the master’s thesis in the future. We acknowledge that the motivation and confidence towards the writing of the bachelor’s thesis is not totally the same as motivation and confidence towards the writing of the master’s thesis. However, comparing these measures provides a good surrogate for the development of motivation and confidence during the bachelor’s thesis seminar course.

15 For the sake of brevity, we give the final rubric in Appendix C and D. The final rubric is the improved version of the rubric that we tested with the students (please see section ‘Epilogue: the final rubric').

16 In the first survey in spring 2013, the phrasing of the questions varied slightly compared to the following surveys (starting from fall 2013), although the meaning of the questions was largely the same. In addition, this survey was conducted only at the end of the course. In Table 3, we omit those students, and only use participants who handed in both the starting and ending questionnaires in fall 2013, spring 2014, and fall 2014. The mean values for the six questions regarding the usefulness of the rubric in spring 2013 are 4.67, 4.33, 4.44, 4.22, 3.89, and 4.67, respectively. The number of observations is 9.

17 The mean VIF-value of the regressors is 1.61, and all VIF-values are below 2. Thus, multicollinearity probably has only a small effect on the results.

18 By using propensity score matching, we can control the observed student heterogeneity and make some suggestions regarding causality.

19 Open discussions on the advantages of the assessment rubric were conducted on April 2013 and December 2013. They were semi-structured and lasted approximately 15 minutes.

20 We obtained information on the norming of the rubric from the following site: http://www.teachingmatters.org/toolkit/norm-setting-protocol.