Development and validation of an instrument to measure students’ engagement and participation in science activities through factor analysis and Rasch analysis

ABSTRACT

In this study, we present a new instrument, the Participation and Engagement Scale (PES), for the evaluation of the students’ involvement in STEM-oriented activities. The instrument was administered to about 1000 secondary school students who participated in the activities of the Italian Piano Nazionale Lauree Scientifiche in Biology, Chemistry and Physics. The activities were carried out in a remote modality due to the COVID-19 restrictions. Through an exploratory and confirmatory factor analysis, it was possible to validate a two-factor structure of the instrument: satisfaction towards the activities and value of the activities. The proposed factor structure shows a good model fit, with each of the obtained scales displaying excellent reliability. Construct-related validation evidence was obtained through the Rasch analysis, which allowed further psychometric improvement of the instrument. Convergent validation evidence was established through a correlation with the academic motivation and perceived difficulties scales. Using the proposed instrument, we found no statistically significant relationships between engagement, the different types of science activities attended, and the intention to enrol in a STEM course. Implications of the study for the evaluation of public understanding of science activities in both remote and in-presence modalities are also discussed.

KEYWORDS:

• Engagement
• out-of-school activities
• confirmatory factor analysis
• Rasch analysis

Acknowledgements

The authors acknowledge the kind collaboration of all the students who responded to the instrument questions, the teachers of participating schools, and the help of the colleagues at their departments who implemented the described activities.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

Research funded by Ministero dell'Istruzione, dell'Università e della Ricerca under the Project Piano Nazionale Lauree Scientifiche 2017-2018 DM 1047/29 dic 2017.