The Classrooms of STEM Graduate Students Are Interactive After a Practice-Based Pedagogical Training Program

Abstract

Graduate students in STEM fields undertook a 9-month training program focusing on practice-based development of active teaching methods. As instructors of record in their own senior-level courses, these trainees were observed on their independently designed classroom sessions using the COPUS classroom observation tool. Compared to a large sample of college STEM instructors, trainee instructors in this program showed more interactive teaching at this early part of their careers. Through their use of a range of active methods, this study suggests that trainees from this practice-based program will begin work in the teaching profession in strong position to maximize outcomes for their students.

Keywords:

• Graduate
• Active learning
• Pedagogical training
• COPUS
• Practice-based

The scientific success of future generations is contingent upon effective science, technology, engineering, and mathematics (STEM) education in colleges and universities. Substantial evidence demonstrates that active-learning principles, which employ teaching techniques that go beyond traditional lecturing, can improve student outcomes (Theobald et al., 2020; Udovic et al., 2002). As a result, there is a growing demand for the next generation of active instructors to transform the current state of higher education STEM (Alberts, 2009; Bradforth et al., 2015; Handelsman, 2006; White et al., 2021; Wieman, 2014). A major challenge, however, arises from the fact that the teaching workforce is almost exclusively trained in disciplinary science with limited pedagogical practice (President’s Council of Advisors on Science and Technology [PCAST], 2012), and traditional lecture practices predominate in North American universities (Apkarian et al., 2021; Stains et al., 2018). In this article, we describe the outcomes of the Science Teaching Experience Program for Upcoming PhDs (STEP-UP), a teaching training program for graduate students that aims to support the use of more active learning approaches and thus improve outcomes for future decades of their actively taught students (Park & Xu, 2022). If these training methods are successful, then they offer a tool for improved STEM education.

STEP-UP follows a new tradition of innovative training programs for future professors (Brower et al., 2007; DeChenne et al., 2015; Deshler et al., 2015; Gilmore et al., 2014; Reeves et al., 2016; Rivera, 2018; Schussler et al., 2015; Ufnar & Shepherd, 2021). While pedagogical training can be diverse in both content and methods, this program used practice-based professional development (Stroupe et al., 2020; Windschitl et al., 2012) intended to help graduate trainees implement active teaching strategies in authentic college classrooms before they entered the professorial job market. In STEP-UP, graduate students complete an unpaid, 9-month training program, which included a pedagogical training course, a writing-intensive quarter with mentor feedback, and a subsequent undergraduate STEM course of their own design as an official instructor of record. Throughout the program, trainees were provided with many opportunities to apply and reflect on newly learned active teaching skills while under the guidance of expert practitioners (Byars-Winston et al., 2019; Henderson & Dancy, 2007). A separate qualitative research investigation focused on instructional choices and perspectives related to inclusive teaching by these trainee instructors; in this article, we focus on the classroom evidence of active teaching techniques as outcomes of STEP-UP.

Active classroom practices are correlated with improved student outcomes (Chi, 2009; Freeman et al., 2014; Weir et al., 2019; Wiggins et al., 2017). However, achieving institution-wide improvement in teaching is not trivial and can require significant inputs of time and resources (Darling-Hammond, 2008; Jones, 2018; Matz & Jardeleza, 2016; Wieman, 2017). Thus, it is critical to target resources so they can have the most impact. Within postsecondary STEM teaching, professors have important interactions with diverse groups of students that can affect retention and success even on the first day of the first course they teach (Seymour & Hunter, 2019). STEP-UP elects to train PhD students, not current faculty, in active teaching approaches in line with calls for graduate training in pedagogy (Brower et al., 2007; Dewsbury, 2017; Love Stowell et al., 2015; Mutambuki & Schwartz, 2018). This early-career training represents potential gains both for students throughout the span of an instructor’s career (Leshner, 2018; Reeves et al., 2016) and for the instructors themselves by instilling useful growth mindsets and cycles of reflective self-improvement (Clarke & Hollingsworth, 2002).

To characterize teaching approaches, we are using direct classroom observations. This allows for data collection that can be compared with broad norms across STEM, using the Classroom Observation Protocol for Undergraduate STEM (COPUS) tool (Smith et al., 2013). COPUS is broadly used and an “industry standard” for undergraduate STEM classroom observation; further, Stains and colleagues (2018) provided a large sampling across authentic North American STEM classrooms. This existing research provides an opportunity to directly compare our graduate trainee classrooms with a broad cohort of course instructors on an objective characterization of active teaching. Our study attempts to quantify the teaching practices that these early-career trainees accomplish after professional development. Using the same tool as the original research by Stains and colleagues (2018), we asked the following two research questions: (i) What teaching practices are our trainees using in their classrooms? (ii) How do trainees of this program compare with current broader teaching practices?

Methods

The cohort observed in this study included 13 PhD candidates enrolled in STEP-UP. These trainees responded to an open invitation to the program and were currently enrolled in graduate program within a wide range of biology or biology-adjacent fields. Those who applied were admitted to the 9-month program on the basis of research principal investigator approval, prior progress toward their STEM research dissertation defense, and lack of formal prior instruction in educational methods beyond work as a teaching assistant. As pre-PhDs, the observed STEP-UP trainees would ordinarily have little direct teaching beyond work as a graduate teaching assistant. The amount of real teaching practice can be highly variable in such roles (Schussler et al., 2015). All trainees volunteered for the program because it aligned with some mix of their prior interest in teaching and/or perceived marketability of teaching skills in future job searches.

To capture the teaching practices used in the STEP-UP classrooms, the COPUS was used (Smith et al., 2013). This tool allows for the rapid and frequent analysis of the activities undertaken by both instructors and their undergraduate students (Cleveland et al., 2017; Connell et al., 2016; Denaro et al., 2021; Gibbons et al., 2018; Maciejewski, 2015; Smith et al., 2014; Tomkin et al., 2019; Weir et al., 2019). COPUS was chosen both for the simplicity of its training (Carl Wieman Science Education Initiative [CWSEI], n.d.) and the prior existence of a large, open-access data set from a North American population sample.

Each of the 13 graduate trainees in STEP-UP were video-recorded for a randomly chosen instructional session during their regular teaching in the final third of the program. All classes observed were primarily small (usually between 20 and 24 undergraduate students) and all were at the senior (4th-year) level for STEM majors. The classroom recordings were independently scored by three observers using COPUS. The observations shared over 91% agreement (average = 96.1%, +/− 2% SD), and thus only one observer’s data were used for all further analyses. Using the methods of Stains and colleagues (2018), accessed via the authors’ Shiny app at http://copusprofiles.org, each classroom’s COPUS dataset was assigned to one of three “styles”: didactic, interactive, or student-centered. For comparing the number of classroom styles against a baseline, a subset of relevant classrooms (i.e., small course size, senior-level course) from the Stains and colleagues 2018 dataset was used. The proportions of styles were compared between the Stains and STEP-UP samples using a Fisher’s exact test, alpha of 0.05, calculated using the XLSTAT program (Freeman & Halton, 1951). All participants and data were protected under institutional research review #STUDY00006242.

To avoid biased data collection, the program director and mentor (author B.W.) did not inspect or analyze the COPUS data prior to assigning styles. Graduate trainees were not prepared for COPUS evaluation or introduced to COPUS categories before their teaching in any way, and these COPUS observations were not used for any individual evaluative purpose in STEP-UP that might tend to prepare them for favorable outcomes with this tool compared with the national sample.

Results and Discussion

Following a 9-month professional training program for graduate students, COPUS data were collected on the classrooms of program participants. To broadly characterize this COPUS data for purposes of comparison, the full time-series data were collapsed into three broad “instructional styles” following the work of Stains and colleagues (2018): didactic, interactive, and student-centered. Generally speaking, didactic classrooms spend more than 80% of time on instructor lecturing; interactive and student-centered classrooms include less lecturing and more time on active student work. In general, student-centered classrooms contain the least amount of lecturing and/or a variety of student activities such as worksheets, though we note that the styles are not a tight linear progression but instead represent clusters of related classroom datasets. Of the 13 STEP-UP classroom observations, three were didactic, nine were interactive, and one was student-centered. As we do not have classroom observations of STEP-UP participants prior to the program, we can use the North American 2018 dataset as a relevant comparison (Stains et al., 2018). As seen in Figure 1(A), the STEP-UP classrooms had significantly more nondidactic teaching compared with the reference sample of similar size and level for North American undergraduate STEM classrooms. This increase in nondidactic teaching is driven by a more than double amount of interactive lecture classrooms taught by STEP-UP participants than interactive lecture classrooms in the reference sample. This outcome is gratifying to the STEP-UP coordinators given that active approaches improve student performance (Freeman et al., 2014) and that the program participants are junior instructors (graduate students) with a long career of impactful instruction ahead of them.

Figure 1. Observed classroom practices of STEP-UP graduate trainees. Note. (A) Distributions of COPUS instructional styles of STEP-UP graduate trainees relative to broader historical data of the same class size and year in college;* data from supplemental materials of Stains et al. (2018) of small 4th-year classes; **Fisher’s exact test p = 0.030. STEP-UP data from COPUS observations as described in the methods. (B) Student activities in each STEP-UP classroom as a proportion of time observed. The 11 COPUS student codes are collapsed into four categories, following Smith et al. (2014).

The particular approaches employed vary (Figure 1(B)), suggesting that instructors make individualized decisions about their teaching “moves” after completing this training program. This is unsurprising (Lo, 2018; Lund & Stains, 2015; McConnell et al., 2021) and is a welcome outcome of the program, as an approach needs to fit the instructor for it to be persistently adopted (Andrews & Lemons, 2015).

The proportion of STEP-UP student-centered classrooms is roughly equivalent to the broader North American population. Notably, the student-centered style is partially defined by the presence of in-class worksheets, which were used in all of the STEP-UP student-centered classrooms. Worksheets may be particularly strong predictors of student outcomes, above and beyond other active practices (Weir et al., 2019; Wiggins et al., 2017), and they are a low-barrier strategy for both individual instructors and training programs.

At the level of individual teaching practices, graduate trainees in STEP-UP implemented a range of active teaching methods, as shown in Figure 2. While some lecturing remains a staple of the overall teaching by STEP-UP trainees, the majority of student classroom time is spent on a variety of activities that involve asking and answering questions, indicating lively discussion in the classroom. The demonstrated range of teaching methods mirrors the wide range of individual and group-based activities that were introduced, modeled, and practiced throughout the STEP-UP cohort training course. The focus for trainees was on creating opportunities for students to do authentic science work, rather than on specific styles of teaching that lead to this work, so it is unsurprising that the resulting teaching included a wide range of COPUS-identified types of active learning. The relatively low prevalence of “clickers” (audience response devices) is unsurprising, as trainees were recommended to instead use lower-tech audience response methods due to a lack of reliable wireless infrastructure (Caldwell, 2007; Mayer et al., 2009; Trees & Jackson, 2007) in their specific classrooms. A wide range of active learning methods may have positive outcomes for students when implemented well (Nguyen et al., 2021; Stains & Vickrey, 2017), and the use of a wide range of active learning methods seen in trainees’ teaching, instead of simple implementation of a specific style, is potentially positive because it indicates that instructors are making personal choices. Personal instructional choices are more likely to be persistent in their long-term approaches (Andrews & Lemons, 2015). We are gratified that the graduate students in this program are implementing active approaches despite having had less direct experience and potentially less opportunities for large-scale change in their curricula and in their first time teaching.

Figure 2. Specific types of instructor and student activities observed in STEP-UP classrooms. Note. Percentages are relative to the total number of activities observed.

STEP-UP is one of several published programs for professional development in undergraduate STEM teaching, and approaches vary widely (Schussler et al., 2015). The STEP-UP goals are in line with calls for graduate student support and for early training to make real change (Reeves et al., 2016; Walsh et al., 2022). Given that COPUS categorization is a recent opportunity (Stains et al., 2018), program assessment using COPUS data from the participants is a relatively new approach. To date, we see three other institutions that have published findings that assess their programs in this way, with the results summarized in Table 1. In those programs, with cohorts largely consisting of current faculty (not graduate students) and having variable program lengths, between 48% and 80% of observed classrooms were active (i.e., nondidactic). The vast majority (92%) of STEP-UP classrooms are using active approaches after a 9-month program. While this is not a statistical comparison, this value favorably demonstrates the success of the program and its participants.

Table 1. Literature-reported COPUS styles following teaching professional development programs.

Program for professional development in undergraduate STEM teaching			Proportion of active classrooms observed*	Reference
Institution type and name	Program name	Program length and cohort
R1 (University of Washington at Seattle)	STEP-UP	9-month program (n = 13 graduate students)	92%	Barker et al. (this article)
Mixed: Two community colleges and one regional primarily undergraduate institution (institute names not reported)	C-Core	4-year STEM faculty development program. (n = 5 faculty)	80%**	Borda et al. (2020)
R1 (University of British Columbia, Canada)	Course transformation	Science education specialist devoting roughly one third of their time over 4+ teaching terms (n = 18 faculty)	72%***	Jones (2018)
R1**** (University of Virginia)	Course design institute (CDI)	Weeklong course design institute (n = 28 faculty + graduate students)	48%	Wheeler & Bach (2021)
	STEM faculty learning community	CDI plus semester- or year-long learning community (n = 17 STEM instructors)	53%	Wheeler (2021)
	New faculty program	CDI + sustained faculty learning community (semester or year) (n = 105 new faculty over the first 3 years of the program)	57%	Favre et al. (2021)
	New faculty learning community	CDI (above) plus semester- or year-long learning community (n = 35 new faculty)	74%	Wheeler and Bach (2021)
Mixed (North American colleges; baseline for comparison)		Observational study (no interventions known/specified; n = 2,008 classrooms)	45%	Stains et al. (2018)

*The COPUS styles of interactive and student-centered were added into a single category of “active classrooms” due to constraints of data reports in these articles.

**COPUS styles from this work are estimated from individual COPUS data reported as % of active vs. passive engagement.

***COPUS styles were collapsed from COPUS profiles from Lund & Stains (2015).

****These data were collected from interrelated programs at this institution and are not fully independent of each other.

Overall, the classrooms observed following STEP-UP are positive, with PhD students achieving professional development goals correlated with desired outcomes. This professional development for precareer graduate students in independently taught courses is in line with national calls for active teaching (Woodin et al., 2010), with strong potential for long-term impact throughout the careers of these current graduate students. The STEP-UP intervention itself was modest in that it fit smoothly within the already busy work life of a PhD student; graduate trainees did not noticeably delay graduation, change funding sources, or need to apply to a separate program. The resulting positive change in this reasonable structure compared with the other programs noted in Table 1, offers optimism about expanding the scope of similar sustainable programs in the future.

It is especially encouraging that program participants—graduate students who are potential future faculty members—are demonstrating evidence-based teaching methods without having had years of experience in the job itself. A similar practice-based program could provide relevant training to current faculty as well. Future research will be needed to determine how, and under what conditions, the STEP-UP practice-based methodology translates from widespread use in K–12 professional development to work with college and university STEM faculty.

Future work will also be needed to identify which adopted approaches the participants persist in using and how they develop their classroom teaching now that they have bridged the gap from didactic to active (Lund & Stains, 2015). Connections between members of the training cohort were fostered before and during teaching, which hopefully formed the nucleus of a community of practice. Communities of practice have been shown to correlate with a higher use of evidence-based teaching practices (Tomkin et al., 2019).

Limitations

The research described has several limitations for analysis. First, this is not a pre-post comparison of improved teaching by the graduate trainees. While none of the participants had prior instruction in educational techniques, it remains possible that the self-selection process introduced bias because they may have developed active teaching approaches even without STEP-UP. Similar groups of study participants have shown pre-instructional COPUS results that were more in line with the national averages (Favre et al., 2021), but we cannot rule out exceptionality of the trainees, nor was there a reasonable way to provide a teaching opportunity to undergraduate students without some guarantee of strong teaching from these first-time instructors. This dataset is limited to a single randomly selected observation day from each instructor, and it is possible that these results represent disproportionately active sessions from an otherwise more passive set of classes.

Conclusion

Using an optional 9-month training program based on practice-based training for science teachers, graduate students in their first independent college teaching opportunity demonstrated teaching outcomes associated with more experienced and highly effective professors in a national sample. As we work toward improving outcomes for all students in science, this study demonstrates the benefits of efforts to transform STEM education and may suggest effective pathways toward having a talented, well-prepared instructor in every college classroom.

Acknowledgments

We thank Casey Self and Becca Price for a prior attempt at COPUS-ing for this research program, Rich Gardner and Nina Salama for administrative help with the creation of STEP-UP, Seth Wiggins for help with statistical analysis, and Kelly Hennessey and Teddy Maley for thoughtful editing. Our graduate trainees are inspirational to us for the service they want to do in the world, and they will inspire and support their students wherever they may go.

Additional information

Funding

This work was supported by a grant from the National Science Foundations through the program for Innovations in Graduate Education Grant #1855841. Human subjects protection for all participants was guided under IRB##STUDY00006242.