Quantification for complex assessment: uncertainty estimation in final year project thesis assessment

Abstract

A quantitative method for estimating an expected uncertainty (reliability and validity) in assessment results arising from the relativity between four variables, viz examiner's expertise, examinee's expertise achieved, assessment task difficulty and examinee's performance, was developed for the complex assessment applicable to final year project thesis assessment including peer assessment. A guide map can be generated by the method for finding expected uncertainties prior to the assessment implementation with a given set of variables. It employs a scale for visualisation of expertise levels, derivation of which is based on quantified clarities of mental images for levels of the examiner's expertise and the examinee's expertise achieved. To identify the relevant expertise areas that depend on the complexity in assessment format, a graphical continuum model was developed. The continuum model consists of assessment task, assessment standards and criterion for the transition towards the complex assessment owing to the relativity between implicitness and explicitness and is capable of identifying areas of expertise required for scale development.

Keywords:

• complex assessment
• final year project
• thesis
• uncertainty
• validity
• reliability
• expertise
• peer assessment

Acknowledgements

The author gratefully acknowledges the feedback from the staff members of Mechanical Engineering, University of Newcastle, for the developed rubric.

Additional information

About the author

Dr Ho Sung Kim is Senior Lecturer and one of editors for ISRN Materials Science. He is currently a final year project coordinator following a position of Postgraduate Director, in Mechanical Engineering, University Newcastle, Callaghan, Australia. His main teaching and research areas include complex assessment, statistics, mechanics of solids, and composite materials. He has published numerous papers, has assessed various research proposals nationally and internationally and has refereed numerous international journal/conference papers.