Assessing System Thinking Through Different Concept-Mapping Practices

Abstract

System thinking is usually investigated by using questionnaires, video analysis, or interviews. Recently, concept-mapping (CM) was suggested as an adequate instrument for analysing students' system thinking. However, there are different ways with which to use this method. Therefore, the purpose of this study was to examine whether particular features of CM practices affect the valid assessment of students' system thinking. The particular features analysed were the medium (computer versus paper–pencil) and the directedness (highly directed versus nondirected) of CM practices. These features were evaluated with respect to their influence on (a) students' performance in CM and (b) the validity of different CM practices for system thinking. One hundred fifty-four German fourth graders (mean age: 9.95 years) and 93 eighth graders (mean age: 14.07 years) participated in the study following an experimental pre-test–post-test design. Three variations of CM practices were applied: (a) highly directed computer mapping, (b) highly directed paper–pencil mapping, and (c) nondirected paper–pencil mapping. In addition to the CM task, a paper–pencil questionnaire was employed to investigate the validity of the CM practices. Results showed that the computer positively influenced student performance in CM when compared with paper–pencil. By contrast, there was no difference between highly directed and nondirected mapping. Whereas the medium rarely influenced the validity of CM for system thinking, high directedness showed a positive influence. Considering the limitations and benefits of particular CM practices, we suggest highly directed and computer-based CM as an appropriate assessment tool—in particular, with regard to large-scale assessments of system thinking.

Keywords:

• System thinking
• Concept-mapping practices
• Primary and secondary students

Acknowledgements

We would like to thank Prof. Jürgen Rost, Prof. Jan Retelsdorf, Dr. Cornelia Sommer and Dr. Tim Höffler for their comments on an earlier version of the manuscript. We are grateful for the valuable support from Kristine Bern.

This work was supported by Deutsche Forschungsgemeinschaft Grants DFG (SO 937/1-1).

Additional information

Notes on contributors

Kristina Brandstädter

These authors contributed equally to this article.

Jörg Großschedl

These authors contributed equally to this article.