Characterisation of Cognitive Difficulty Level of Test Items

Abstract

The study of subjects in science and technology involves high level thinking. Student‐material match at the design stage of a curriculum refers, in most cases, to the general criteria of thinking and learning theories, such as the ability to carry out ‘formal operations’ according to Piaget. The question of relevance may arise when dealing with a specific subject matter, in this context. Checking the student‐material match through an achievement post test enables us to refer to the relevant contents.

A method is proposed to characterise a test item through a content ingredient (Number of Schemes NS required for solution) and the required learner's resources (represented by the Problem Solving Taxonomy PST level). The Cognitive Difficulty Level CDL is then defined as: CDL=NS ∗ CDL.

The validity of CDL as a difficulty index of a test item, has been verified in the case of teaching basic electronics at the high‐school level (tenth and eleventh grades): a decrease has been obtained in students' achievement scores with an increase in the CDL value. Two types of student populations have been involved in the study: high and low achievers. The findings indicate that at the initial stage of learning a new subject, mainly the content factor (NS) affects the difficulty level of a mental task. After the ‘transient time’ has elapsed, both factors, NS and PST (i.e. CDL) affect the cognitive difficulty level. In the later stages, when the complexity of the material increases, the effect of PST becomes most dominant.