Using digital representations of practical production work for summative assessment

Abstract

This paper presents the findings of the first phase of a three-year study investigating the efficacy of the digitisation of creative practical work as digital portfolios for the purposes of high-stakes summative assessment. At the same time the paired comparisons method of scoring was tried as an alternative to analytical rubric-based marking because we believed that it was likely that a more holistic approach to scoring would be more appropriate. Researchers created digital representations of the practical submissions of 75 Visual Arts and 82 Design students graduating from secondary school in Western Australia. These digital portfolios were scored using the two methods, with the scores compared to those officially awarded to the physical forms. It was concluded that the digital representations of the Visual Arts submissions had adequate fidelity for the purpose of awarding high-stakes scores particularly using the paired comparisons method. However, the Visual Arts teachers and students were opposed to digitisation. For the Design portfolios teachers and students were supportive of digital submission, but the structure of the portfolios reduced the reliability and validity of scores, particularly from analytical marking.

Keywords:

• digital portfolios
• summative assessment
• paired comparisons
• Rasch modelling
• visual arts
• design

Acknowledgements

The study was conducted by the Centre for Schooling and Learning Technologies (CSaLT) at Edith Cowan University in collaboration with the School Curriculum and Standards Authority of Western Australia (SCASA) and supported by an Australian Research Council research grant. A team of researchers and assistants in CSaLT contributed to the first phase of the study coordinated by Dr Martin Cooper.