Characterizing Teaching in Introductory Geology Courses: Measuring Classroom Practices

ABSTRACT

Most research about reformed teaching practices in the college science classroom is based on instructor self-report. This research describes what is happening in some introductory geology courses at multiple institutions across the country using external observers. These observations are quantified using the Reformed Teaching Observation Protocol (RTOP). A scoring rubric created to support consistent application of the 25 items on the RTOP yields very high inter-rater agreement over multiple observations throughout a 3 y period. Using the adapted RTOP instrument, 66 separate observations of introductory physical geology classrooms at 11 different institutions (four associate's colleges, three baccalaureate colleges, a master's university, and three research universities) were collected, and those observations indicate three categories of instruction: (1) teacher-centered, traditional lecture-dominated classrooms (RTOP < 30) with little student talk and minimal student activity beyond listening and note taking; (2) transitional classrooms with some activities involving brief student discussions centered around right/wrong answers; and (3) student-centered classrooms (RTOP ≥ 50) with considerable time devoted to active learning and student communications to promote conceptual understanding. The progression from teacher-centered to transitional and then to student-centered categories is incremental across all subscales of the RTOP instrument except for propositional knowledge (character of the lesson's content and instructor's command of the material), which only increases between teacher-centered and transitional categories. This means there is no single path to an active learning, student-centered introductory geology classroom. Such learning environments are achieved with a holistic approach to all aspects of constructivist teaching as measured by RTOP. If the instructor incorporates small changes in multiple aspects of their teaching from disseminator of knowledge to supporter of student learning, then the transition to a student-centered classroom becomes an approachable process. Faculty can also use the RTOP and rubric to guide course planning, promote self-reflection of their teaching, and assist in the peer evaluation of other's teaching.

Key words: :

• RTOP
• classroom observation
• instruction
• pedagogy
• student-centered learning

Acknowledgments

This research was supported by the National Science Foundation Division of Undergraduate Education grants 0817260 and 1023097 to D. A. Budd, 0914404 to D. A. McConnell, and 1022980 to K. J. van der Hoeven Kraft. Dexter Perkins, Karl Wirth, and Ellen Iverson provided very useful comments on early drafts of the manuscript. We thank the instructors who opened the door to their classrooms and enabled us to conduct this research. We also thank Editor for Research Karen McNeal and two anonymous reviewers for their comments

FIGURE 1: Correlation between RTOP scores in classrooms observed at the same time by both observers. Inter-rater agreement, as defined by the linear regression (long dashed line) correlation coefficient (r = 0.94), is excellent. Short dashed lines are ±10% of a one-to-one correspondence, which is the solid line.

FIGURE 2: Distribution of average RTOP scores and scores of individual observations for all 26 instructors. Observations that covered the same topics (successive classes in one semester) are in open circles; all other observations were of classes that covered different topics (in the same or different semesters).

FIGURE 3: Whisker and box plots for RTOP scores as a function of instructor's (A) gender, (B) institution type, (C) class size, and (D) years of teaching experience (median is 12 y). Whiskers mark 10th and 90th percentiles; top and bottom of the boxes mark 75th and 25th percentiles. Solid line in middle of the box is 50th percentile; solid gray dot is the mean value; open circles are outliers. RU1 denotes research university.

FIGURE 4: RTOP subscale scores versus total RTOP scores for all 66 observations. Subscale 2 has a maximum possible score of only 16 because item 10 is excluded due to its low reliability.

FIGURE 5: Bar charts of subscale scores as a function of instructor grouping. Arrows indicate greatest changes in subscale scores between instructor grouping (i.e., >median difference of 4.0). Subscale 2 has a maximum possible score of only 16 because item 10 is excluded due to its low reliability.

TABLE I: Subscales of the RTOP instrument (from Sawada et al., 2002).

TABLE II: Values of Cronbach's alpha obtained for the RTOP and its subscales when scored with the rubric.

TABLE III: Characterization of participating instructors.

TABLE IV: Population statistics for RTOP scores by demographic subgroups.

TABLE V: Average RTOP scores for each item and subscale by instructional category.^1,2

Notes

¹ The first and second authors developed the rubric and made all observations reported. One is a male with 24 y of teaching experience at a research university and a background in geoscience research, not education research. The other is a female with a background in geoscience education and 13 y teaching at a community college. The classrooms of both, plus the third coauthor, were observed; their scores are included in Table II, and they represent a range from transitional classrooms to student-centered classrooms.