Development and Validation of an Assessment Instrument for Course Experience in a General Education Integrated Science Course

ABSTRACT

Identifying instruments and surveys to address geoscience education research (GER) questions is among the high-ranked needs in a 2016 survey of the GER community (St. John et al., 2016). The purpose of this study was to develop and validate a student-centered assessment instrument to measure course experience in a general education integrated science course. A one-shot case study of pre-experimental design (Creswell, 2009, 2014) was used to understand student experiences in a large-enrollment course with digital content integration, including out-of-class video, online presentations, and warm-up questions, as well as in-class video paired with discussion and small group activities. In two sections taught by the same geoscientist, 209 students accessed course content in an online learning management system prior to classroom instruction. We adapted the 36-item Course Experience Questionnaire (CEQ; Wilson et al., 1997) to a Web-based survey. We conducted statistical analysis on the CEQ responses, including item factor analysis, examination of communalities and measures of association, confirmatory factor analysis, and reliability and stability testing through bootstrap resampling. The statistical analyses indicate that the results in this study are comparable to those from other cultural contexts and subject areas. Given the moderate fit of the model and reasonably stable results, we propose that the core indicators of the CEQ constructs, including Appropriate Workload, Clear Goals and Standards, Generic Skills, Good Teaching, and Emphasis on Independence are sufficient to assess students' course experiences in general education science settings. These results provided moderately to strongly valid information that can help instructors in designing their technology-integrated classes, although further study with a larger sample population would be beneficial. From this study we conclude that students' perceptions of their course experiences are closely related to their development of problem-solving and analytical skills, clear course expectations to direct their own study plans, their preference for a motivating instructor, the opportunity to have a variety of learning choices, and their workload. We propose the use of a 25-item version of the CEQ that is appropriate for formative assessment in discipline-specific STEM introductory classes, including geoscience classes.

Key words: :

• course experience
• assessment instrument
• general education

Acknowledgments

This research would not have been possible without the support from a General Education Grant at James Madison University and the persistent collaboration among the researchers from different discipline areas. The authors want to extend collegial appreciation to Dr. Keithia Wilson from Griffith University for her permission to reuse the 37-item Course Experience Questionnaire in this study. The authors also want to acknowledge Dr. Julie Libarkin from Michigan State University and two anonymous reviewers for their constructive comments and suggestions on the manuscript and for insights on data analysis methods.