The Analysis of Australian Proficiency Test Data over a Ten-Year Period

ABSTRACT

Recent criticism of forensic science has focused on the fundamental aspects of the science, including supporting empirical studies, validation, accreditation, limitations and error rates. Proficiency tests are an essential component of accreditation and can be used to evaluate performance and identify systematic issues within components of the service provision. 3,176 CTS proficiency tests undertaken between 2005 and 2015 by Australian government service providers were collated. These tests represent 43 unique CTS test types, covering 21 disciplines. The results for 262 tests were not submitted; the remaining 2,914 tests involved 32,208 comparisons and 21,187 results which were analyzed to determine the accuracy rates and error rates for each of the 21 disciplines. The results demonstrate that errors exist even though practitioners know they are examining a proficiency test and the tests are reviewed. This study illustrates the need to monitor trends in proficiency test results and also highlights the need for well-designed, relevant, blind error rate studies to determine the approximate error rates for casework.

KEYWORDS:

• Forensic science
• proficiency test
• CTS
• error rates
• Australia

Acknowledgments

The authors would like to acknowledge the work of Allison Roper and Lori Hillier in assisting in the collation of CTS files and data for analysis.