Development and Validation of Instruments to Measure Learning of Expert‐Like Thinking

Abstract

This paper describes the process for creating and validating an assessment test that measures the effectiveness of instruction by probing how well that instruction causes students in a class to think like experts about specific areas of science. The design principles and process are laid out and it is shown how these align with professional standards that have been established for educational and psychological testing and the elements of assessment called for in a recent National Research Council study on assessment. The importance of student interviews for creating and validating the test is emphasized, and the appropriate interview procedures are presented. The relevance and use of standard psychometric statistical tests are discussed. Additionally, techniques for effective test administration are presented.

Keywords:

• Assessment
• Formative assessment
• University
• Science education
• Evaluation
• Assessment design

Acknowledgements

We would like to thank the members of the Carl Wieman Science Education Initiative (CW‐SEI) at the University of British Columbia and the Science Education Initiative (SEI) at the University of Colorado, especially Sarah Gilbert, Laurie Langdon, and Jennifer Duis for their helpful comments. This work was supported by the University of Colorado through the SEI and the University of British Columbia through the CW‐SEI.

Notes

1. We use this statement to discard the survey of people who are not reading the questions: Please select agree‐option four (not strongly agree) for this question to preserve your answers.