Providing Teachers With What They Need: Re-thinking Historic Site-based Professional Development After Small-scale Assessment

ABSTRACT

Supported by an Institute of Museum and Library Services National Leadership Grant, Mystic Seaport Museum educators present how findings from evaluations of their teacher professional development programs challenged them to reconsider what it means to understand and support teachers’ unique professional needs. Included is a discussion of what it means to be teacher educators, including the support museum educators need to engage with that professional responsibility.

KEYWORDS:

• Teacher education
• teacher professional development
• pedagogy
• standards-based evaluation
• Q methodology

Professional development (PD) for teachers at historic sites and museums is often designed by museum professionals with the intention of providing K-12 teachers an all-access pass to the many resources of an institution to inspire them and help them explore new pedagogical ideas. We hope that these workshops and programs will inspire a change in their understanding of subject matter and in their teaching when they return to the classroom. But what are teachers actually taking from our PD programs and what can we do to improve it?

Like many institutions, Mystic Seaport Museum (MSM) has hosted various types of PD programs for teachers for decades. To evaluate the programs, we have frequently employed some sort of traditional Likert-scale tool to give us the basic details of teachers’ reactions to our programming. Essentially, we discovered if teachers liked the programs or not, or if their feelings fell somewhere in-between. The responses typically gave us enough information that we could decide whether to do the program again, but it did not give us the constructive feedback we needed to re-design a program that would better fit the needs of our participants.

This sense that we were not getting the whole story – or actionable data – from our end-of-session surveys is why we joined with Thomas Jefferson’s Monticello and researchers from Teachers College, Columbia University and Tufts University, on an Institute of Museum and Library Services (IMLS) National Leadership Grant Project to build and test an evaluation tool to better understand teacher PD at historic sites.¹

This tool is based on Q methodology, a version of factor analysis that offers a rigorous, quantitative study of people’s points of view on particular topics. This methodology enables researchers to quantify the individual participant’s perceptions and use that information to discover patterns across groups of participants to explain how they extract meaning and useful information from historic sites. The “Q-sort” is a card-sorting process in which participants read a series of statements covering a wide range of perspectives (the “concourse”) and prioritize those statements in relation to each other. This process enables participants to react to the statements based on their values and beliefs. The final step in the process is an interview in which they explain the reasoning behind their choices. Unlike our end-of-session Likert scales, the Q-sort evaluation was not a single-session of data collection. Participating teachers completed the Q-sort before the PD program began (upon arrival on the first day), at the end of the program, and six months after completing the program. Additionally, Q methodology is “a small-sample technique,”² so this tool was developed, in part, to take into consideration the small group size of most historic site-based PD and still render statistically valid findings.

We tested the tool at MSM during the summer of 2017 PD programs, which included a three-day summer residential institute with Connecticut Teachers of the Year and the Mystic Seaport for Educators (MSE) Fellows Program. This article shares what we learned from the Q-sort and the evaluation process, information that the Likert scale did not have the ability to show us.

MSM PD programs

MSM used Q methodology during two very different formats of PD for teachers: (1) the Connecticut Teachers of the Year Institute: a three-day overview of MSM’s site and collections; and (2) the MSE Fellows Program: a 40-hour, summer-long, research-based program.

Program 1: Connecticut Teachers of the Year Institute

During the three-day institute for Connecticut Teachers of the Year, participants learned how to use MSM and its collections in their classrooms through active participation and interaction with a variety of experts, hands-on activities, primary source workshops, and guided exploration of the exhibits and grounds of Mystic Seaport.

Eight teachers participated in the 2017 summer institute. On the first day teachers took tours of the collections vault and the Mystic Seaport grounds. They also took part in a chantey program, watched an overview of the MSE website, and climbed the rigging of the historic ship the Joseph Conrad. They participated in a “Seaport Sleuth” activity where they found objects on our grounds and then presented what they found to the group. On the second day the teachers saw a planetarium show, participated in a primary source workshop, took a tour of a Viking ship, and rode on an historic steamboat. During the second day the teachers also began developing ideas for their lesson plan. On the third day teachers had a tour of the whaling vessel, the Charles W. Morgan, through the lens of Herman Melville’s Moby-Dick, experienced a costumed-interpreter presentation from 1876, and participated in two hands-on activities – whaleboat rowing and carving. Teachers also had more time to develop their lesson plans.

Program 2: MSE Summer Fellowship Program

The MSE Summer Fellowship Program focuses on the research and development of features for the MSE website. This website is an ongoing collaboration between Mystic Seaport and a growing community of educators. The MSE website represents a shift from delivering content in a static, one-way manner to a more participatory learning model that fosters shared authority, merging the expertise of museum staff with the standards-based needs of classroom educators. In the planning phase for this website, Mystic Seaport worked with over 140 educators, parents and students in a series of 56 “co-creation” workshops. Their ideas and feedback shaped the development of the MSE website. All content on MSE has been and continues to be developed by educators and museum staff working together. One method for creating content is through the summer fellowship program. Educators selected for the fellowship spend time in various training workshops, which prepare them to create content. The trainings include collections tours, one-on-one time with subject experts, library and research guidance, a session on the technical details of creating features, and a morning-long writing workshop with a local writing and literature professor. Ten teachers participated in the MSE Summer Fellowship Program during the summer of 2017, and at the end of the summer fellowship, museum staff reviewed and edited the completed projects, transforming approved work into website features.

This program is more intensive³ and self-directed than the Connecticut Teachers of the Year program, as participants create their own experience around what interests them, within the museum collection. They develop their own schedules of research, writing and consulting; and generally spend about 40 hours throughout the summer, crafting their content. They complete their projects with the guidance of museum staff, and many times the fellows explore research opportunities in other museums or institutions if it helps to strengthen their project.

What did the Q-sort teach us about our PD programs?

After carefully reviewing our results with the research team, MSM had some big take-aways from the evaluations and participant interviews that will inform the next cycle of PD programs. These lessons learned will serve as guidelines for planning future workshops, and are summarized below.

Meet the needs of participants

Why are teachers coming to our programs? What do they expect from us? These important questions deserve a considerable attention when planning a PD program. The groups of teachers attending our two PD programs wanted very different things from the programs they attended and had very different expectations about their experiences should entail. One of the precepts that the evaluation project was based on was the notion that teachers are not a monolithic group, but have different needs at different stages of their careers.⁴ This was evident in the outcomes.

The MSE Fellows were seeking pedagogical strategies for their classroom, time to collaborate with their peers, development of their historical thinking skills, and MSM content. Due to the self-directed nature of the program, most fellows indicated that the experience met their expectations. This is probably because the fellows had more freedom to seek out or follow up on aspects of PD that interested them. For example, several teachers shared that they had hoped to spend more time learning to do research and actually doing it – something many of them had not done since college, but were expected to model for their students. At the end of the fellowship, multiple teachers expressed that they achieved this personal and professional goal through the fellowship program.

What we found with the Teachers of the Year was quite different. First, not a single teacher was seeking to learn about content from MSM, but at the end of the institute, they all felt like they left with MSM content knowledge. One reason that teachers were not looking for MSM content was because the Teachers of the Year is part of a larger program, involving many different PD opportunities around the state. Our program is part of the package offered to the finalists. They come to our site because it is an opportunity for PD at a museum/historic site in Connecticut and because it is an opportunity to spend time with their Teacher of the Year peers, and not because of the specific focus of the museum. Teachers of the Year expected to focus on pedagogy. None of the participants felt that they had done this by the end of the institute. This is addressed further in the next section.

End the “content buffet”

Like many institutions striving to make meaningful connections with teachers, we often bombard our PD participants with a little taste of everything we have to offer. We cram the schedules with a variety of subjects, tours, exhibitions, lectures and workshops with very little time for deep exploration, which, it turns out, is the opposite of what teachers are craving. The evaluation of the two different programs made it clearer what format was more effective.

The Teachers of the Year Program operated under the “buffet” approach, offering quick segments of the many different programs at the museum. While this covered quite a bit of content from all across the museum, there was little time for deep exploration and digestion. In addition, this style was often overwhelming for participants and led to fewer final products (as discussed later in this article). Accordingly, the participants noted that while they enjoyed their time at MSM, it did not meet their needs.

Conversely, in the MSE Fellowship program, teachers had time for a deep dive into content. This was exactly what they were looking for and expecting from the program, and of the two groups showed greater satisfaction with the program and integration of materials into their classroom. From this distinction, we took two important lessons: do fewer things in greater depth and let the teachers determine some element of what they are doing.

Teachers need time to debrief and collaborate with peers

In museum teaching, we draw upon various methods for debrief sessions with students in our programs. Techniques like “pair share” or group conversations help our students to create meaning after activities or interactions with content. While we might incorporate some of this into PD for educators, the Q-sort showed us is that we do not use it enough. One of the most underutilized resource in the Teacher of the Year program were the teachers’ peers. This was evident in some of the statements from the Q-sort concourse chosen by participants. Participants prioritized statements indicating the importance of working with peers (e.g. Statement 42: spending time with peers receiving constructive feedback from peer-teachers; Statement 43: giving me the opportunity to build a network of peers who share my content interests; Statement 48: providing opportunities to receive constructive feedback from peer-teachers).

In their interviews, participants also revealed that they valued debrief sessions that gave context after exploring content. They yearned for opportunities to discuss how content could be used in classroom settings with peers and staff. These findings echo what our collaborating institution, Monticello, was also finding in its programming: teachers need time to consider the content and process it with their peers⁵. Conversations with peers is how teachers solve pedagogical problems and translate the content presented at the museum into classroom content. The evaluation revealed that this is essential for teachers’ ability to use the museum materials in their teaching. Building time into the schedule for teachers to talk to each other is not “down time,” but instead a critical part of their professional work.

Final products designed by teachers should be easy to complete

Our evaluation researchers made several recommendations to improve the Teachers of the Year program. In addition to reducing the overwhelming amount of content that we share with teachers, we should revise the “final product” assignment to be simpler and more in line with teachers’ professional work. Until this evaluation, each participant was required to create a lesson plan inspired by what was learned in the institute. However, very few teachers actually turned in lesson plans. This lack of final products most likely stemmed from some of the problems we discovered because of the Q-sort – too many activities, not enough time to process the information, plus the addition of a major task – the creation of a lesson plan – all in three days. It turns out, contrary to what many people think, very few teachers write out formal lesson plans for each of their lessons. Thus, asking for formal lesson plans, unless they are required to do so by their principals/districts, is asking teachers to do something many have not done since their certification programs, which for many, was their undergraduate experiences 20+ years prior. The researchers reminded us that teaching is an oral medium – teachers can talk all day about what they are doing in their classes. Far fewer can, or have the time to write it out in sufficient detail for other people to follow.

For our 2018 program, we opted for a simpler way to share lesson plans – short videos in which each teacher shared an idea from the institute that he or she could use the classroom. This final product, which we call “The Digital Hallway” – after the space where teachers share most of their lesson ideas, in the hallway between classes – requires an outline of the lesson concept, a description of the collections or site materials on which it is based, pulled together in a 2-minute video with the teacher describing how they would use this in the classroom. This more closely models how teachers talk to each other and share lesson materials. The Connecticut Teacher of the Year staff responded very favorably to the video model when MSM staff shared this idea. To encourage us, the Connecticut Teachers of the Year staff (veteran teachers who have been involved with the Museum and attended the PD every year) helped create videos that served as the models off which the attending teachers in could develop their own. The “Digital Hallway” videos debuted on the MSE website during the fall of 2018.

Use content as a vehicle to explore pedagogical concepts

As mentioned previously, a large percentage of our agenda during PD programs involved showcasing content from many different areas of the museum. While participants expressed that they felt like they learned a lot about the museum and the content it offers, what the Q-sort helped us to see was that we should use the content as an opportunity to explore or demonstrate appropriate pedagogical concepts. In short, we need to focus on helping teachers understand not just the content but – and this is critically important – how to teach it. What are the big ideas that this object or document illustrates? How do the materials connect to larger themes or skills that the teachers are developing in their classrooms?

The richness of MSM’s archival collections offers us the perfect opening to demonstrate historical reading and thinking skills. For the 2018 Teacher of the Year Program, we added a “Reading Like a Historian” workshop featuring a handful of our manuscripts that they can see in person at MSM, but also have access to online, via the MSE website. The teachers also explored methods for artifact analysis using our education artifact collection and had time to work with their peers to develop meaningful content and conceptual threads that tie it back to their classroom work. Again, these methods can be taken back to the classroom, and used with resources from the MSE website.

We might be “accidental” teacher educators

One important lesson learned from the Q-sort is that some museum educators may be “accidental teacher educators;”⁶ starting out in other areas of the museum as content experts or interpreters, with little or no training in either formal or informal education. This does not mean that we are not qualified to lead teacher PD programs. It does mean that we need to learn to understand and meet the very particular professional needs of classroom teachers and make sure that those we hire as museum educators have expertise in pedagogy and classroom teaching. While most museum educators have a good understanding of content and know how to work with children and youth, teachers’ needs are different. To serve their needs, many museum educators need more training on the pedagogical methods, theories, and current research about teaching and learning that shape teachers’ professional lives in their schools and classrooms. Having a fuller understanding of the contexts in which they teach will help museum educators meet teachers’ stated needs.

The Q-sort results showed us that teachers wanted demonstrations of some of our hands-on methods using our primary source materials. While modeling how to deconstruct a museum artifact is important, being able to explain how it connects to larger concepts in the historical/archeological fields that they exemplify in a way that teaches teachers both the method and how to transcend the method – by making the process of analysis applicable beyond the specific artifact in question – is critical. For example, the exercise of analyzing and demonstrating artifacts like a chip log or a taffrail log (instruments used by sailors to measure the speed of a ship as it moves through the water), becomes a great way to discuss the importance of speed – whether a clipper ship carrying cargo or a fishing schooner speeding back to port. Putting these tools in the hands of teachers and making explicit links to critical concepts like time as a commodity, or supply and demand, supports teachers’ ability to teach complicated concepts in compelling ways. Time as a commodity ties to the economic standards and helps address questions like “Why do companies charge for shipping?” “Why does shipping something faster cost more than shipping it more slowly?” “Why do we even call it ‘shipping?’” Another example; making and using navigational instruments like a quadrant becomes a way to talk about longitude – another geography concept that can be hard for teachers to explain in a way that captivates students. Using museum materials in this hands-on way – introducing, discussing, and demonstrating pedagogical and historical concepts – creates a real connection with educators, showing them that when the opportunity arises, they can look to museums to provide the primary source resources and the skills they need.

Using the Q-sort results to modify our PD programming

The findings from the Q-sort evaluation are a catalyst for us to refine and enhance our PD sessions. We are revising our summer institute for Connecticut Teachers of the Year to incorporate teachers’ feedback from this study. Specifically, we will reduce the number of activities and amount of content we present and provide more in-depth information and examples of how teachers can use the specific content in their classrooms. We will do this by providing examples of how other teachers have done it in the past, give examples of our own, and we will implement the “digital hallway” concept noted above. We will designate time during the institute to record teachers brainstorming ideas they have generated about how they might use what they are learning in their classrooms. We will also develop and teach a workshop about historical thinking and model how to do it using primary resources from our collection.

In our upcoming summer institutes, we will better contextualize the content we do share with teachers, and provide structured time to process the content and debrief examples of how to best use it in the classroom. As museum educators we have to model what we preach, which is to serve more as facilitators of good conversation using the inquiry process and less of the expert transmitting an overwhelming amount of content especially without allowing enough time to digest it.

The Q-sort evaluation helped us better understand the needs of teachers and what they are looking for in PD programs at historic sites and museums. Museum educators can serve as the bridge between the dense content of our museums and the needs of the classroom teacher.

Disclosure statement

No potential conflict of interest was reported by the authors.

About the authors

Krystal Kornegay Rose is the Manager of Digital & Primary Source Education at Mystic Seaport Museum.

Sarah Cahill is the Director of Education at Mystic Seaport Museum.

Christine Baron is an Assistant Professor in Social Studies and Education at Teachers College, Columbia University.

ORCID

Krystal Kornegay Rose http://orcid.org/0000-0002-3582-0194

Sarah Cahill http://orcid.org/0000-0002-5614-6629

Christine Baron http://orcid.org/0000-0002-2046-9880

Additional information

Funding

This work was supported by Institute of Museum and Library Services.

Notes

1. For full project details, visit the project website http://www.teacherinsites.org/.

2. O’Leary et al., “Q-Methodology as a Research and Design Tool for HCI.”

3. The structure of the fellowship met the criteria for effective teacher professional development in terms of: focus on content matter that is aligned with content participating teachers teach; duration (over 40 contact hours); promotion of active learning (archival research and on-site pedagogical activities); planning classroom implementation (curriculum unit development); and presentation of final products. The number of contact hours is a critical determinant for teacher professional development. However, the configuration of them (e.g. 40 hour week v 40 hours over 6 weeks) has been shown to have no significant effect on HSBPD outcomes. Baron, “Using Inquiry-Based Instruction to Encourage”; Borko, Jacobs, and Koellner, “Contemporary Approaches to Teacher Professional Development”; Clarke and Hollingsworth, “Elaborating a Model of Teacher Professional Growth”; Garet et al., “What Makes Professional Development Effective?”; and Wei et al., “Professional Learning in the Learning Profession.”

4. Borko, “Professional Development and Teacher Learning”; Opfer and Pedder, “Conceptualizing Teacher Professional Learning”; and Vermunt and Endedijk, “Patterns in Teacher Learning in Different Phases.”

5. Baron et al., “Understanding What Teachers Gain”; and Cooper et al., “Teaching Teachers Onsite.”

6. Goodwin, “Globalization and the Preparation of Quality Teachers.”