Understanding what teachers gain from professional development at historic sites

ABSTRACT

Despite decades of formal work with teachers, little is known about what they gain from professional development at and with historic sites. This article presents the first data-set from a 3-year Institute for Museum and Library Services National Leadership Grant project designed to develop a broad-based assessment for understanding what teachers learn in historic site-based professional development. The centerpiece of this project is an assessment tool based in Q methodology. This round of study focuses on the pre-post Q sorts and interviews of 29 teachers in a history-focused institute regarding how they see their work at historic sites affecting their professional development. Contrary to advocates’ assertions, results indicate that only a small number of participants specifically set out to seek historical disciplinary expertise and analysis, while greater emphasis was placed on working with content-area peers, developing pedagogical practices, and the power of place.

KEYWORDS:

• Historical thinking
• historic sites
• pedagogical content knowledge
• peer mentoring
• Q methodology
• teacher professional development

Each year, thousands of teachers engage in historic site-based¹ teacher education and professional development (PD) programs in order to improve their historical content knowledge, pedagogy, and skills (Boyer, Fortney, & Watts, 2010). The format and content of these programs range as widely as the sites that offer them: from international (e.g., UNESCO World Heritage sites at the Memorial and Museum at Auschwitz-Birkenau to Thomas Jefferson’s Monticello), national (e.g., National Endowment for the Humanities Landmarks in American History program), and state-level partnership organizations (e.g., the Ohio History Connection), to university consortia and travel programs (e.g., Five College Center for East Asian Studies) and countless local offerings. Despite this increasingly formal work with teachers, little research exists about the effectiveness of historic sites’ role in teacher education in achieving these stated aims (Marcus, Levine, & Grenier, 2012; Stoddard, 2018).

Part of the problem that persists in the research on what teachers learn from historic places is the narrowness of the range of both the ideas explored and methodologies used to investigate learning in museums, either independently or in conjunction with teacher education. Museum education literature is dominated by practitioner-based works highlighting anecdotal program descriptions or advice on the logistical considerations and constraints of working with teachers and schools (e.g., Fortney & Sheppard, 2010; Gupta, Adams, Kisiel, & Dewitt, 2010; Marcus, Stoddard, & Woodward, 2012; McRainey & Moisan, 2009; Simon, 2010). Quantitative research about learning in museums generally tends toward visitor surveys, which emphasize capturing affective responses to specific events or exhibits (Dudzinska-Przesmitzki & Grenier, 2008; Falk & Dierking, 1995); federal and foundation funding for history museums is considerably lower than it is for art or science museums, thus truncating sustained, empirical evaluative efforts (American Academy of Arts and Sciences, 2014; The White House, Office of Management and Budget, 2014). The most commonly published type of research in this vein is short-term, qualitative studies or small-scale surveys of site-specific program reviews that offer little generalizable information on which other researchers can build upon or from which to develop theory (cf. DeWitt & Storksdieck, 2008; Dudzinska-Przesmitzki & Grenier, 2008; van Hover, 2008).

Although evidence-based literature on teaching and learning in museums grows, most of it has been conducted in science and art museums, leaving social studies educators to infer disciplinary understanding into those findings. For example, most of what we know about field trips is drawn from science² and art museums (e.g., Bitgood, 1994; DeWitt & Storksdieck, 2008; Lavie Alon & Tal, 2015; Tal, Bamberger, & Morag, 2005; Tal & Morag, 2007; Tal & Steiner, 2006). Even within these field trip studies, the focus is on student learning rather than the museum as educational spaces for teachers. The most rigorous examinations of instructional methodologies employed in museums come from art museums (e.g., Bowen, Greene, & Kisida, 2013; Housen, 2002; Luke, Stein, Foutz, & Adams, 2007; Yenawine, 1998). These studies provide broad guidelines for understanding teaching and learning in museums. However, they do not help address disciplinary questions raised by historic sites.

Although disciplinary inquiry-based modeling has been shown to be effective for supporting teachers’ skill and knowledge development and transfer to the classroom (Baron, 2013; Schrum, Kortecamp, Rosenfeld, Briscoe, & Steeves, 2016), most of the literature specific to history education shows that many teacher education programs and tours at historic sites emphasize didactic knowledge-transmission (Hall & Scott, 2007; Moyer, Onosko, Forcey, & Cobb, 2003; Pesick & Weintraub, 2003; Warren, 2007; Zeisler-Vralsted, 2003), which raises questions about the value of interactions with these sites for teacher education when inquiry is not explicitly modeled.

Perhaps most importantly, the types of questions historic places pose to visitors tend to be deeply subjective. Consider the complex semiotic text that visitors to a traditional house museum must decipher: Curated to a single moment in time are the artifacts of a (likely White) family of some note. Embedded within this curated space are the signs and symbols of not only the family’s wealth and social prominence but also the implicit and explicit values of the concerned citizens and organizations that have/continue to engage in the work of preserving and interpreting it. The pristine, no-longer-lived-in setting often renders invisible the work of servants, enslaved or free; laborers; craftspeople; and the larger community who made the family’s lives possible. From the second visitors arrive on site, they begin to take in these messages and assign meaning to what they are seeing, before any staff or interpretive text begins to mediate this experience (Doering & Pekarik, 1996). Even when interpretive programming and media offer nuanced and complex interpretations of the history of these places (cf. Handler & Gable, 1997; Lewis, 2005; Tchen, 1992), often the degree to which one sees past a simple narrative of a benevolent, White, wealthy, upper-class family, depends on one’s position relative to it. The ability to see the curatorial layers of interpretation requires both an awareness of such curation and a willingness to question the authority with which it tells the story (Trofanenko, 2006). This requirement encapsulates the essential problem of evaluation of historic sites: If every visitor begins a visit in a different relative position to a host of complex issues and ideas, how can the impact of interpretive programming be determined by the visit’s end?

The complexity of this problem is amplified when the purpose of such a visit is part of a teacher PD program: How does the programming offered help teachers encourage more critical engagement with the site relative to that starting position and help change classroom practice? What disciplinary content, skills, or dispositions do they bring to and/or learn in these spaces that they can use in their classrooms? How do teachers working together in these spaces co-construct meaning and develop pedagogy? And how can we compare that across multiple PD programs held at multiple historic sites? Thus, we have difficulty capturing cultural discrepancies that are challenged or pacified and otherwise identifying meaningful impact of interpretive programming in historic sites. This difficulty, coupled with the conception of each historic site as a unique singularity, is part of the barrier to understanding what is gained from interacting with the multiple singularities that are historic places.

Although it is true that the range of content, landscapes, structures, events, and collections offered in the programs presented at these places should be explored as unique contexts, there are ways to frame the commonalities across them so that their use and impact can be meaningfully investigated. To that end, research into the work teachers do at historic places must ask questions about learning that transcend individual sites and field boundaries. Further, we must employ research methodologies that can rigorously measure perceptual and attitudinal shifts that result from work at historic sites.

History teacher professional development

As van Hover (2008) stated and van Hover and Hicks (2018) reaffirmed in their reviews of PD in the social studies:

Our knowledge of the professional development of social studies teachers is idiosyncratic and there exists no “big picture” of social studies professional development across the [United States]. A plethora of professional development opportunities exist for social studies teachers, but they vary wildly in format, content, and quality. (van Hover, 2008, p. 355)

Much of the research about teacher PD remains “home grown” (Hill, Beisiegel, & Jacob, 2013, p. 476) and small scale. The absence of large-scale assessments of history/social studies teacher PD³ creates a cyclical problem: without greater research on teaching and learning that occurs in PD programs, practitioners cannot make their programs more effective. The lack of effective programs hamper clearer development of best practices for teaching, learning, and transfer to the classroom. Absence of these structures prevent substantive assessments that tie site-based PD work to classroom instruction and student achievement (U.S. Department of Education, 2011; van Hover, 2008; van Hover & Hicks, 2018).

This research seeks to explore these problems, both in terms of substantive issues of teacher education at historic sites and the need to employ different methodologies for said explorations. This article reports on the first round of data from a three-year Institute for Museum and Library Services National Leadership Grant project to develop a broad-based assessment for understanding what teachers gain from historic site-based PD programs. The centerpiece of this project is an assessment tool based in Q methodology for use at historic sites. Q methodology helps identify a generalizable phenomenon using a systematic and objective procedure when studying human subjectivity. This round of study focuses on 29 teachers in a history-centered institute at Thomas Jefferson’s Monticello and the ways in which they see their work at historic sites affecting their PD.

The tool used in the present study was developed, in part, to provide a way, to start to transcend some of the differences individual historic sites present and systematically look across these historic spaces. To begin to answer the larger questions posed above, the current study seeks to understand why teachers attend historic site-based PD and what impact they perceive that work to have on their development as teachers. From these questions we seek to develop a foundation for meta-analyses that would better inform our understanding of what occurs in historic site-based PD across multiple sites. As Q methodology is “a small-sample technique that can lend statistical validity to the qualitative interpretation of subjective data” (O’Leary, Wobbrock, & Riskin, 2013, p. 1941), the construction of the current Q methodology tool acknowledges the state of history/social studies teacher PD as it exists: unfederated, context-bound, and small-scale.

Conceptual framework

To understand why teachers choose to engage in PD at historic sites and what they take from the process, as well as how to assess it in a meaningful way, requires looking at their choices holistically. For example, many of the pedagogical foundations for museum education rest on constructivist principles. However, grounding this study in a strict constructivist frame would presume that a particular kind of pedagogy is driving teachers’ choices, whether or not it actually is. Instead, to consider the range of professional knowledge, pedagogical content knowledge (Shulman, 1987), responsibilities, and dispositions on which teachers base their decision making, we were guided by the following principles:

1. If historic site-based teacher education programs are to function as integral partners in teacher education, then their work must be guided primarily by the professional standards and expectations of teacher education and teachers’ working lives, rather than the institutional goals of the historic site (DeWitt & Osborne, 2007; Gupta et al., 2010; Hochberg & Desimone, 2010; Zeichner, 2005).

2. Teachers are not a monolithic group. Research needs to consider the full working life of teachers—from preservice induction through to master teacher status—with distinct needs and motivations at each stage. Within these professional stages, research must allow for the particular needs of teachers with different knowledge, practices, experiences, and beliefs, working across grade levels, content areas, and students that they serve (Borko, 2004; Opfer & Pedder, 2011; Vermunt & Endedijk, 2011).

3. If the primary rationale for supporting historic site-based teacher education is access to authentic historical materials and deep disciplinary knowledge, then the content and skills presented must be relevant for classroom use and support teachers in the full complexity of their professional contexts (Garet, Porter, Desimone, Birman, & Yoon, 2001; Hochberg & Desimone, 2010; National Research Council, 2009; Penuel, Fishman, Yamaguchi, & Gallagher, 2007).

Teacher education and content standards

Accordingly, to develop the Q methodology tool, also known as a concourse, we drew upon established standards and frameworks created by teachers and teacher education professional accrediting and affinity groups. The concourse was designed as a series of statements to address four areas relevant to history/social studies teachers’ professional lives as they intersect with historic site-based PD: content knowledge, disciplinary analysis and skill development, pedagogy, and professional dispositions.

To guide our work related to the professional dispositions, pedagogy, and content knowledge deemed essential for teachers, we used the Interstate Teacher Assessment and Support Consortium (InTASC) Core Teaching Standards (Council of Chief State School Officers, 2011) and the National Council for Accreditation of Teacher Education (NCATE, 2008) Professional Standards for Teacher Preparation Institutions. To guide our work related to the historical, economic, geographic, and civic content knowledge and analytical skills development offered at the site, we used the National Council for the Social Studies (2013) College, Career, and Civic Life (C3) Framework for Social Studies State Standards. In addition, we used the Ethical Dimension from Seixas, Morton, Colyer, and Fornazzari’s (2013) Big Six Historical Thinking Concepts, as this element was noted by historic site staff as representative of rationale previously offered by teachers but not fully addressed by other sets of standards (Table 1).⁴

Table 1. Concourse item national standard correlates.

Areas for assessment	Standard set	Standard items
Content Knowledge	National Council for the Social Studies C3 Framework for Social Studies State Standards NCATE Professional Standards for Teacher Preparation Institutions InTASC Core Teaching Standards	D2.H.1, 3–4, 7–10, 13, 16–17 D2.Eco.10–13 D2.Civ.3 D3.1–2 1a, Candidate Knowledge, Skills, and Professional Dispositions; Diversity 4 Content Knowledge 4.o
Historical Thinking and Analysis	National Council for the Social Studies C3 Framework for Social Studies State Standards NCATE Professional Standards for Teacher Preparation Institutions Seixas et al.’s Big Six Historical Thinking Concepts	D2.H.3, 7–10, 16–17 D3.1–2 1a, 1b, 1d, Learner Development; 4, Content Knowledge Ethical Dimension
Pedagogical Content Knowledge	NCATE Professional Standards for Teacher Preparation Institutions	1a, 1b, 1c, 1d, Candidate Knowledge, Skills, and Professional Dispositions; Diversity 4d
Professional Dispositions	InTASC Core Teaching Standards	4o, Content Knowledge; 7o, Planning For Instruction; 9n, Professional Learning and Ethical Practice; 10c, k, r, Leadership and Collaboration.

These standards were each created based on a synthesis of current research, policy directives, and the collaboration and consensus of scholars and educators from across the educational landscape. Although valid critiques exist of each of these sets of standards, in combination, they provide a foundation from which to begin a broad-based assessment of how teachers’ work intersects with historic sites.

In the present study, we seek to understand the following: Why do teachers attend historic site-based PD and what impact do they perceive that work to have on their development as teachers? From these research questions, we can identify program and site elements that support or impede teachers’ growth and better address their self-identified needs.

Research site

There are few individuals who so embody the central conflict at the heart of the American Experiment as the slave-owning author of the Declaration of Independence’s signature line, that “All men are created equal.” Over the past three decades, Thomas Jefferson’s home, Monticello, and The Thomas Jefferson Foundation that operates it, as an institution, have been conspicuous in embracing the full legacy of Jefferson, including, but not limited to, Monticello as a plantation dependent upon slave labor. Particularly in the years since the 1997 publication of Annette Gordon-Reed’s book Thomas Jefferson and Sally Hemings: An American Controversy and the DNA study undertaken by Foster et al. (1998) published the following year, Monticello has worked to restore the plantation landscape and surviving structures and to leverage decades of archaeological, architectural, and documentary research in exhibitions and the educational and interpretive programming, with an emphasis on the stories of the enslaved persons whose lives and work made Jefferson’s life and work possible.

The degree to which these efforts have been successful is open to debate: some visitors seeking hagiographic Jeffersonia find the endeavor off-putting, preferring that there were greater focus on Jefferson’s public accomplishments; some find the restoration of the slave quarters on Mulberry Row and the ongoing restoration of Sally Hemings’s chamber too little, too late; some find the complexity of the site and its persons eye-opening about the American past and its echoes in the present; and for others, it is merely the historical wallpaper against which their family vacations are set. It is well beyond the scope of this research to adjudicate such matters.

However, it is in these contested meanings and tensions that Monticello embodies that it is necessary to study what teachers learn at historic sites. Historic sites are complex constructions of the historical past, preservation initiatives, collective memory, multiple and mixed media spaces, and interpretative layers that both reveal and conceal during a visitor’s embodied experience. As all of these responses to and interpretations of an historic place and persons dwell within and bump up against each other, understanding which one(s) hold sway when teachers are immersed in such a place and what aspects they bring with them into their classrooms becomes of critical import.

Method

Q methodology

Q methodology was first introduced in 1935 as part of a larger effort to develop options in psychometric testing that provided greater insights for understanding particular themes within certain content areas (Watts & Stenner, 2005). Q methodology is a research tool that can be used in a qualitative study to generate quantitative results (Dick, Gleeson, Johnstone, & Weston, 2010; Purslow & Red Owl, 2012; Shinebourne & Adams, 2007; Watts & Stenner, 2005). Traditional methods, such as Likert measures commonly used in surveys, place respondents on a linear scale, whereas Q methodology allows respondents to create individual definitions based on their personal experience and opinions (McKeown & Thomas, 1988). Q methodology is the systematic and rigorous quantitative study of subjectivity (McKeown & Thomas, 1988), and subjectivity is defined here as a person’s communication of his or her point of view on any matter of personal or social importance. In this case, participants in Monticello’s Teacher Institute rank ordered statements pertaining to PD.

The method has advantages over more traditional Likert scale surveys for understanding attitudes, as participants must use “forced choice” which brings their personal values and experiences into the ranking process. In a traditional Likert scale survey, participants would be able to agree or disagree with as many statements as they chose to see themselves in a positive light. However, in Q method, participants need to rank the statements against each other, which potentially works to eliminate some of the bias of self-reporting in these surveys.

Teachers in this study ranked statements three times: once immediately before the Monticello Institute took place, once immediately at the Institute’s conclusion, and again approximately five months later. The experiences and values of each participant guided them in how they chose to rank the statements, and the teachers in this study were able to draw on their own past experience with PD to sort the items in the first round of sorting while bringing in their experiences with the Institute for the second and final sorts. This article reports on the first two sorts.

Designing the tool

The data collection tool in Q methodology is called the Q sort, and it is comprised of statements that participants rank on a normal distribution curve. The statements used in Q methodology are called the concourse (Brown, 1986), and the initial step in developing the tool was to create a concourse that represented a range of perspectives on PD. Having a range of perspectives in this process was critical, as it allowed participants to describe their own diverse experiences with PD.

To create the concourse, the research team and the educational staff at Monticello identified the most salient elements from the different sets of standards to be addressed and, together, developed a draft set of statements. The research team further refined these statements to develop a draft concourse.

To validate the concourse, five experts in museum, history, and teacher education were enlisted to examine the concourse statements. These experts hold staff and faculty positions at museums and universities across the country. They were given a briefing on Q methodology and then asked to examine the draft concourse statements consisting of content, historical thinking, pedagogical content, and professional dispositions to determine if they fit into the study framework. The reviewers also assessed how concisely they defined problems, terms, and goals of teachers who teach social studies and attend PD workshops. In addition, they were asked to consider which perspectives were missing from the concourse. With this feedback, the statement list was returned to the staff at Monticello to ensure that the language utilized in the statements was clear and relatable for teachers. Sixty statements covering a range of perspectives about PD were chosen, giving participants a variety of options to rank while still being short enough for them to complete in a reasonable time.

Participants

Twenty-eight teachers from 20 states and one from the United Kingdom participated in the study (n = 29). They represent the entirety of those accepted to the Monticello Teachers’ Institute (MTI), as no participants opted out of the study. They were divided into two one-week sessions. The teachers had areas of focus in U.S. History, Social Studies, Library Media, and American Studies. The three Library/Media instructors and one Social Studies instructor worked with K–12 students, whereas the remaining 25 participants taught in Grades 5–12. Of the 29 participants, only two participants did not have master’s degrees, and one was in the process of getting it. A third participant had his doctorate.

All of the participants had applied for and been selected to participate in the MTI, situated on the Monticello campus. The focus of the fellowship was to allow participants to use the archival and historic site materials to complete a self-directed project to create lessons for their own use. Participants worked independently or with one-on-one support in the archive. They participated in lectures/discussions and toured the site as a cohort. Therefore, participants should be seen as self-selecting and motivated in a well-resourced program. For a full description of teacher demographics, see Table 2.

Table 2. Participant demographics with pre and post factor loadings.

Parti-cipant	Race	Sex	YT	Grade	Teaching specialty	Highest Degree	Sorter	Loading by factor
								1	2	3	4
1	White	M	8	HS	U.S. History	MEd	1 Pre	.079	−.018	.664X	.664
							1 Post	.277	−.162	.641X	.020
2	White	F	35	MS	U.S. History	MA	2 Pre	.2409	.265	.596X	.298
							2 Post	.331	.154	.558X	.306
3	White	F	12	HS	U.S. History	MA + 15	3 Pre	−.057	.299	.433X	.264
							3 Post	−.021	.298	.574X	.296
4	Hispanic	F	5	MS	U.S. History	MA	4 Pre	.129	.347	.197	.716X
							4 Post	.307	.320	.456X	.379
5	White	M	5	HS	U.S. History	MA	5 Pre	.682X	.140	.248	.349
							5 Post	.737X	−.007	.211	.357
6	White	F	11	K-12	Social Studies	MA	6 Pre	.322	.382	.336	.470X
							6 Post	.534X	.423	.259	−.045
7	White	F	23	MS	Special Education	EdS	7 Pre	.186	.467X	.130	.144
							7 Post	.415	.274	.425	−.099
8	White	M	6	MS/HS	History/Politics	PhD	8 Pre	.429	−.091	.695X	.170
							8 Post	.424	.259	.706X	.069
9	White	F	13	MS	Early American History	MA	9 Pre	.311	.286	.428	.534X
							9 Post	−.054	.202	.655X	.129
10	White	F	15	K-12	Librarian	MA	10 Pre	.571X	.326	.381	2289
							10 Post	.682X	.353	.329	.256
11	White	F	9	MS/HS	American Culture	BA	11 Pre	.579X	.202	.220	.291
							11 Post	.713X	.108	.305	.324
12	White	F	29	HS	U.S. History	EdS	12 Pre	.373	.374	.408	.182
							12 Post	.476	.284	.439	.302
13	Latino	F	29	HS	U.S. History	EdS	13 Pre	−.350	−.157	.349	.301
							13 Post	.506X	.119	.264	.211
14	White	F	14	HS	History	MA	14 Pre	.268	.299	.578X	.518
							14 Post	.323	.168	.4867X	.305
15	Arab	F	26	MS	U.S. History	MA	15 Pre	.119	.316	.139	.342X
							15 Post	.068	.173	.362X	.148
16	White	F	16	UES	All	MEd	16 Pre	.362	−.071	.382	.441X
							16 Post	.363	−.147	.524	.522
17	White	F	9	HS	U.S. History	MA	17 Pre	.324	.205	.576X	.253
							17 Post	.339	.330	.587X	.241
18	White	M	21	MS	Social Studies	MA + 75	18 Pre	−.038	.405	−.098	.519X
							18 Post	.165	.179	.054	.383X
19	White	M	9	HS	American History	MA	19 Pre	.629X	.001	.112	.575
							19 Post	.666X	.176	.007	.544
20	White	F	24	MS	Social Studies	MA	20 Pre	−.104	.288	.747X	.209
							20 Post	.221	.469	.576X	−.012
21	White	F	22	PreK-5	Library Media	EdS	21 Pre	.178	.375	.215	.490X
							21 Post	.204	.653X	.139	.463
22	White	F	22	K-5	Library Media	MA + 60	22 Pre	.147	.170	.715X	.386
							22 Post	.229	.124	.718X	.319
23	White	M	37	HS	Ancient History	MA	23 Pre	.119	.784X	.232	.208
							23 Post	−.089	.864X	.093	.089
24	White	M	8	MS	U.S. History	MA	24 Pre	.489	.567X	.289	.074
							24 Post	.475	.574X	.376	.002
25	Caribbean/ East Indian	F	8	MS	U.S. History	MA	25 Pre	.349	.467X	.276	.346
							25 Post	.377	.115	.541X	.390
26	White	F	16	MS	Social Studies	MA	26 Pre	.499X	.021	.340	.399
							26 Post	.646X	−.008	.355	.362
27	White	F	3	MS	History	MA	27 Pre	.194	.015	.379	.788X
							27 Post	.239	−.020	.446	.713X
28	White	F	5	HS	U.S. History	BS + 30	28 Pre	.166	.395	.243	.690X
							28 Post	.363	.191	.196	.599X
29	White/ Latino	F	5	HS	U.S. History	MA	29 Pre	.104	.072	.310	.755X
							29 Post	.284	−.087	.573	.569

Q sorting process

The Q sort was administered once at the outset and once at the conclusion of each session of the MTI (two identical sessions were offered). This pre-post administration was designed to capture teachers’ initial thinking about their PD program and how, if at all, it changed over the course of the week’s activities.

During the administration of the Q sort, each participating educator was given a set of 60 index cards. Each card had one of the 60 concourse statements written on it. They were first asked to do an initial sort where they placed the cards into three piles of “Most Like Me,” “Least Like Me,” and “Neutral,” based on the prompt: “Professional development at historic sites affects my development as a teacher by ….”

As part of the instructions to participants to determine which statements were “most” or “least” like them, they were asked to consider the notion of flossing their teeth: Although flossing might be something that they knew to be of great value to their health, did they actually do it on a daily basis? Thus, the determination of whether something was Most or Least Like them was based on whether they acted upon it frequently rather than held it as an abstract value.

Next, they were given a worksheet, pictured in Figure 1, on which to rank the different statements on a 60-item forced distribution ranging from + 6 (Most Like Me) to −6 (Least Like Me). Participants began by taking the pile of statements they designated as most like them and choosing the two statements MOST like them to place on the worksheet in the “+ 6” column on the far right. They were then asked to place the next three statements most like them into the + 5 column and so on until all the statements from their “Most Like Me” pile had been placed on the worksheet. This process was repeated with the “Least Like Me” statements, starting with the two statements in the −6 column, and finally with the Neutral statements. Neutral statements generally ranged from −1 to + 1.⁵ Participants began by placing the most neutral statements in the “0” column, and statements that were slightly more like them would veer toward the positive, whereas statements slightly less like them would veer toward the negative side of the ranking sheet. Each statement card had a number on the back, and after the participants were satisfied with how they had arranged the statements on the spreadsheet, they wrote the statement numbers onto the worksheet in the corresponding squares.

Figure 1. Q sort worksheet.

Immediately after the Q sort administration, participants who sorted their statements in ways that were indicative of larger trends, as determined via PQ Method software analysis, were chosen for follow-up interviews in order to gain insight into their thought process. These participants were chosen to ensure a comprehensive understanding of the different attitudes that might emerge from the participants in the study about PD. The interviews consisted of three questions: Why did participants sort and rank their “Most Like Me” and “Least Like Me” statements in the way they did, and what was their experience of doing the sort.

Finally, participants provided basic demographic data about themselves, their years of teaching, level of education, and student demographics. Rather than using preassigned categories, participants were asked to provide their preferred descriptors for gender, race, and ethnicity. These descriptors are used herein.

Data analysis

Q methodology software, PQ method (Schmolck, 2014), analyzes correlations among sorts, eigenvalues, as well as rotated factors, factor loadings, z-scores, and rankings of Q statements in each factor (type or pattern). A centroid⁶ method was used to obtain an initial estimate of potential factors, and a varimax rotation was used to facilitate interpretation of the factor loading values. A varimax rotation allows the researcher to see the solutions that maximize variance explained by the extracted factors and creates a simpler way to understand the factors that are extracted (cf. Watts & Stenner, 2005). If participants ranked the Q sort statements in a similar way, they loaded on a factor. If enough participants loaded the ranked the items in a similar way, the factor would have an eigenvalue of greater than 1.0 (Anderson, Avery, Pederson, Smith, & Sullivan, 1997).

Four factors were retained for interpretation that met the criterion of having an eigenvalue greater than 1. The interpretation was based on factor scores, and each factor that emerged represents a particular view that the participants have regarding PD. A factor score is an average score, or z-score, given to a statement based on all the Q sorts associated with that factor (Brown & Stephenson, 1980). This composite represents how a hypothetical respondent with 100% loading on the factor would have ranked the statements in the Q sort (van Exel & de Graaf, 2005) and is the best estimate of the viewpoint the factor represents (Watts & Stenner, 2012). The z-scores were next converted into a factor array, which is a single Q sort designed to represent the viewpoint of a particular factor (Watts & Stenner, 2012). These arrays, combined with participant demographics and participant interviews, can be used to facilitate a holistic interpretation designed to understand the viewpoint that each factor represents. Differences and similarities between factors were also examined using distinguishing and consensus statements.

Results

Four factors were identified using the Q sort results and the corresponding interviews. These factors identify groups of people who have rank ordered the statements in a similar way. When participants sort items in a similar way, it is referred to as “loading” on a factor. The four factors determined, based on the prompt “Professional development at historic sites affects my development as a teacher by…,” were named Type 1: Historical Pedagogical Content Knowledge (14% of variance; 12 sorts); Type 2: Monticello Focused: Learning from Experts (10% of variance; 7 sorts); Type 3: Monticello Focused: Learning from Colleagues (19% of variance; 20 sorts); and Type 4: Focus on Relevance: Connecting the Past and Present (15% of variance; 13 sorts). Seven confounding sorts (more than two significant or no factor loadings) were eliminated from the interpretation. Distinguishing statements, higher ranking statements, and z-scores are described in Tables 3–7. Also, Table 2 demonstrates the four-factor solution and how the participants loaded on each.

Table 3. Factor Q-sort values for each statement.

No	Q statements	1	2	3	4
1	Providing the true story of American history	−5	−1	−3	−3
2	Helping me establish a clearer chronology of the events of the Revolutionary Era and Early Republic	−4	2	−1	−2
3	Demonstrating how the complex conditions of a plantation economy affected real people	−2	4	3	−3
4	Helping me recognize that Monticello offers a true picture of life in the Early Republic	−2	0	−2	−3
5	Helping me recognize that Monticello represents the authentic work and values of the people who lived and worked there	−1	3	2	−2
6	Providing immediate access to the places and objects related to Monticello to improve my understanding of the people who lived there	−1	5	6	−1
7	Letting me compare life then and now	0	1	−1	−1
8	Helping to simplify what I know about historic events related to Jefferson and Monticello	−1	0	−2	−2
9	Demonstrating how representing the perspectives of a single dominant culture offers a clear national narrative	−5	−2	−4	−5
10	Showing the continued influence of the Declaration of Independence in American life	−2	−1	1	2
11	Helping me to see the superiority of America’s culture to other world cultures	−6	−6	−6	−6
12	Helping me to understand how landscape, architecture, and natural resources shaped life for people at Monticello	−2	4	5	−2
13	Showing how new evidence changes our understanding of historical events and persons over time	4	5	3	3
14	Illustrating how each time we teach a story we change it based on what evidence we emphasize	4	1	0	−1
15	Demonstrating how historical questions are framed by contemporary issues	1	1	0	6
16	Helping me consider evidence from multiple perspectives and develop a reasoned argument about the past	5	4	1	5
17	Providing opportunities to judge people in history against contemporary standards	−2	−2	−5	−3
18	Helping me recognize that much of what I see at historic sites is shaped by professional historians	0	0	−1	−4
19	Making me less confident that I know the whole story of Jefferson and his time	−3	−2	−3	−4
20	Demonstrating that what we don’t know about historical events and persons is as significant as what we do know	2	0	0	2
21	Helping me consider the historical context of people’s motives and actions in the past	4	6	3	5
22	Clarifying that what happened in the past and our interpretation of it is not the same thing	2	4	−3	1
23	Giving me the tools to use new technology in the classroom	−3	−2	0	1
24	Helping me develop critiques of “common knowledge” about historical events and persons	6	2	−1	2
25	Making it clear that we cannot understand the motivation of people from the past in studying history, and shouldn’t try	−6	−5	−4	−6
26	Working with history practitioners helps me develop a more critical eye toward historical evidence	3	2	1	3
27	Helping me question what I think I know about historic events and persons	2	3	1	1
28	Helping me recognize that an historic site can serve as a microcosm of larger historical realities	0	5	1	0
29	Allowing me to see that the “power of place” is critical for informing the historical narrative	0	6	4	1
30	Making me more comfortable with the lack of clear and finite answers in historical study	0	0	−2	−4
31	Helping me to integrate diverse perspectives into my curriculum	3	1	2	5
32	Modeling how to integrate historical content and concepts in ways that draws upon students’ family and community experiences	−2	−1	−1	1
33	Helping me select and develop appropriate strategies for teaching the complexities of history to my students	1	0	5	6
34	Giving me tools to provide my students with multiple perspectives from which to view historical events	4	2	6	4
35	Modeling how to address difficult or controversial topics in the classroom	0	−2	0	4
36	Giving me tools to engage students with diverse cultural backgrounds	−3	−1	−1	3
37	Seeing new history/social studies teaching techniques modeled	−1	1	1	3
38	Helping me simplify historical content for my students	0	0	2	2
39	Helping me design activities that support student learning	2	3	4	4
40	Providing strategies for how to create learning experiences that draw upon students’ communities of origin	−3	−1	−3	0
41	Modeling how to frame substantive historical questions for my students	1	2	0	4
42	Providing opportunities for me to collaborate with and provide feedback to peer-teachers	3	−3	4	0
43	Giving me the opportunity to build a network of peers who share my content interests	3	−3	5	0
44	Providing opportunities to actively reflect on my teaching practice	6	1	2	3
45	Encouraging me to consider how my core values are part of how I select topics for and teach my course content	0	0	−2	−1
46	Giving me the opportunity to build a network of peers and mentors with culturally diverse perspectives	2	−3	2	0
47	Strengthening my ability to use technology featured at historic sites (e.g., web pages) in my teaching	−3	−3	2	0
48	Providing opportunities to receive constructive feedback from peer-teachers	1	−4	0	−1
49	Helping me clarify my own perspective on content and teaching techniques by working with teachers with perspectives different from my own	3	−1	3	0
50	Reinforcing my belief that technology offered by historic sites is too difficult to use in the classroom	−5	−4	−5	−5
51	Providing opportunities to refine my current teaching techniques	5	2	4	2
52	Helping me develop a caring and supportive network of peers	1	−6	2	0
53	Giving me tools to support students with exceptionalities (e.g., physical, language-based, or intellectual disabilities)	−4	−2	−3	0
54	Giving me tools to support multi-lingual students (e.g., ELL)	−4	−4	−4	−1
55	Providing me with refreshing time to spend with peers	5	−4	0	−2
56	Providing me with a space in which I feel safe to make mistakes and express opinions	−1	−5	−4	−3
57	Demonstrating how difficult it is to have discussions with people who disagree with me	−4	−5	−5	−5
58	Providing a break from unsupportive peers with whom I usually interact	−1	−3	−6	−4
59	Helping me develop goals for improving my content knowledge and teaching	1	3	2	2
60	Recognizing that history is a construction	2	3	−2	1

Table 4. Factor 1 distinguishing statements: Historical pedagogical content knowledge focus.

No	Q statements	Ranking
44	Providing opportunities to actively reflect on my teaching practice	6*
24	Helping me develop critiques of “common knowledge” about historical events and persons	6*
55	Providing me with refreshing time to spend with peers	5*
14	Illustrating how each time we teach a story we change it based on what evidence we emphasize	4
43	Giving me the opportunity to build a network of peers who share my content interests	3*
22	Clarifying that what happened in the past and our interpretation of it is not the same thing	2
48	Providing opportunities to receive constructive feedback from peer-teachers	1*
33	Helping me select and develop appropriate strategies for teaching the complexities of history to my students	1
37	Seeing new history/social studies teaching techniques modeled	−1*
58	Providing a break from unsupportive peers with whom I usually interact	−1*
56	Providing me with a space in which I feel safe to make mistakes and express opinions	−1
36	Giving me tools to engage students with diverse cultural backgrounds	−3
23	Giving me the tools to use new technology in the classroom	−3
2	Helping me establish a clearer chronology of the events of the Revolutionary Era and Early Republic	−4
53	Giving me tools to support students with exceptionalities (e.g., physical, language-based, or intellectual disabilities)	−4
1	Providing the true story of American history	−5

Note. *(p < .05), denotes distinguishing statements for this type.

Table 5. Factor 2 distinguishing statements: Monticello focus, learning from experts.

No	Q statements	Ranking
29	Allowing me to see that the “power of place” is critical for informing the historical narrative	6*
28	Helping me recognize that an historic site can serve as a microcosm of larger historical realities	5*
22	Clarifying that what happened in the past and our interpretation of it is not the same thing	4*
27	Helping me question what I think I know about historic events and persons	3
2	Helping me establish a clearer chronology of the events of the Revolutionary Era and Early Republic	2*
34	Giving me tools to provide my students with multiple perspectives from which to view historical events	2*
14	Illustrating how each time we teach a story we change it based on what evidence we emphasize	1
7	Letting me compare life then and now	1
4	Helping me recognize that Monticello offers a true picture of life in the Early Republic	0*
33	Helping me select and develop appropriate strategies for teaching the complexities of history to my students	0
23	Giving me the tools to use new technology in the classroom	−2
35	Modeling how to address difficult or controversial topics in the classroom	−2*
9	Demonstrating how representing the perspectives of a single dominant culture offers a clear national narrative	−2*
46	Giving me the opportunity to build a network of peers and mentors with culturally diverse perspectives	−3*
42	Providing opportunities for me to collaborate with and provide feedback to peer-teachers	−3*
43	Giving me the opportunity to build a network of peers who share my content interests	−3*
48	Providing opportunities to receive constructive feedback from peer-teachers	−4*
55	Providing me with refreshing time to spend with peers	−4*
56	Providing me with a space in which I feel safe to make mistakes and express opinions	−5
52	Helping me develop a caring and supportive network of peers	−6*
11	Helping me to see the superiority of America’s culture to other world cultures	−6

Note. *(p < .05), denotes distinguishing statements for this type.

Table 6. Factor 3 distinguishing statements: Monticello focus, learning from peers/colleagues.

No	Q statements	Ranking
43	Giving me the opportunity to build a network of peers who share my content interests	5*
33	Helping me select and develop appropriate strategies for teaching the complexities of history to my students	5
29	Allowing me to see that the “power of place” is critical for informing the historical narrative	4*
16	Helping me consider evidence from multiple perspectives and develop a reasoned argument about the past	1*
48	Providing opportunities to receive constructive feedback from peer-teachers	0
41	Modeling how to frame substantive historical questions for my students	0
24	Helping me develop critiques of “common knowledge” about historical events and persons	−1*
18	Helping me recognize that much of what I see at historic sites is shaped by professional historians	−1
30	Making me more comfortable with the lack of clear and finite answers in historical study	−2
60	Recognizing that history is a construction	−2*
22	Clarifying that what happened in the past and our interpretation of it are not the same thing	−3*
17	Providing opportunities to judge people in history against contemporary standards	−5*
58	Providing a break from unsupportive peers with whom I usually interact	−6*

Note. *(p < .05), denotes distinguishing statements for this type.

Table 7. Factor 4 distinguishing statements: Focus on relevance.

No	Q statements	Ranking
15	Demonstrating how historical questions are framed by contemporary issues	6*
33	Helping me select and develop appropriate strategies for teaching the complexities of history to my students	6
31	Helping me to integrate diverse perspectives into my curriculum	5*
35	Modeling how to address difficult or controversial topics in the classroom	4*
36	Giving me tools to engage students with diverse cultural backgrounds	3*
32	Modeling how to integrate historical content and concepts in ways that draws upon students’ family and community experiences	1*
22	Clarifying that what happened in the past and our interpretation of it are not the same thing	1
60	Recognizing that history is a construction	1
42	Providing opportunities for me to collaborate with and provide feedback to peer-teachers	0*
43	Giving me the opportunity to build a network of peers who share my content interests	0*
46	Giving me the opportunity to build a network of peers and mentors with culturally diverse perspectives	0*
52	Helping me develop a caring and supportive network of peers	0
53	Giving me tools to support students with exceptionalities (e.g., Physical, language-based, or intellectual disabilities)	0*
54	Giving me tools to support multi-lingual students (e.g., ELL)	−1*
48	Providing opportunities to receive constructive feedback from peer-teachers	−1
30	Making me more comfortable with the lack of clear and finite answers in historical study	−4*
18	Helping me recognize that much of what I see at historic sites is shaped by professional historians	−4*

Note. *(p < .05), denote distinguishing statements for this type

Factor 1: Historical pedagogical content knowledge focus

There were six teachers from the preinstitute sort that can be characterized as having a Historical Pedagogical Content Knowledge Focus. Of these six, five remained on this factor in the postsort whereas the sixth teacher could not be defined by any factor after the institute had been completed. Two additional teachers could be characterized by Factor 1 after the institute had concluded. All of the teachers in this group were White, and two of the eight total teachers were male, six female. The range of number of years of teaching experience was 5–29, with an average of 14.6 and a mode of 9. (See Table 8 for participants and their movement onto or off Factor 1.)

Table 8. Factor 1 loading participants.

Other factor pre	Factor 1 pre	Factor 1 post	Other factor post
	Participant 5	Participant 5
Participant 6 (Factor 4)		Participant 6
	Participant 7		Participant 7 (none)
	Participant 10	Participant 10
	Participant 11	Participant 11
Participant 13(none)		Participant 13
	Participant 19	Participant 19
	Participant 26	Participant 26

Statements chosen for this factor as “Most Like Me” were largely representative of the declarative content-focused C3 statements and pedagogy-focused NCATE and InTASC statements. Statements chosen as “Least Like Me” were primarily drawn from the NCATE professional disposition statements. The content focus for these teachers emphasized developing an accurate sense of the persons and events of the past, including clarifying prior “common” knowledge and “myth-busting” elements of U.S. history that they were responsible for teaching. Further, the idea that they could “simplify” historical information (statement 8) did not mean emphasizing “obvious, cheerful, and stereotypical” (Levstik, 2000, p. 290) aspects of American history. Rather, participants were focused on accurately grasping the conceptual significance of the stories the site told so they could help students comprehend the whole. For example, Participant 7 stated in her post interview: “If I can figure a way to break it down when [students] don’t get it, and figure out what they don’t get, then I can make it more real and they can see the big picture.”

Participants also noted the ways in which the interactions with the site, particularly the complex legacy of slavery, forced them to reconsider whether simplifying history for students meant presenting simplistic versions of historical events. Participant 6 noted:

I realized this week that I’m always teaching the safe side of history, I’m not getting into controversial issues, I have a tendency to kind of want to wrap up all those loose ends with a topic before I moved on. It occurred to me that I was teaching my students that history has finite answers and how dangerous that is because when we think that history is finite, we become complacent and when we become complacent we don’t question, and when we don’t question, there is tyranny … I need to be more okay with the ugly, messy, complicated, unresolved side of history and that it’s okay to bring in those controversial issues with kids and just let it hang there for what it is. Let the students respond to what they’re thinking and that’s okay rather than trying to steer it towards the safe side of the topic.

One of the elements that participants deemed significant in helping them shift their thinking was the repeated exposure to the same site (e.g., Monticello main house) viewed from different perspectives (e.g., House tour, Slavery tour, Gardens and Grounds). Participant 7 also noted:

The first time [I went to Monticello] I knew we didn’t see everything and this time I got to see a lot more. I had the opportunity to stay longer. I learned more and have more questions. No one gets it all the first time. It’s like that’s why you watch movies three or four times. Then you actually know what’s going on. That [immersion] made a big difference, because I know one time we went through the garden tour and didn’t get a whole lot or I got some information and then another time, I heard part of it, so each time through, especially with the different interpretations from the tour guides, you get a different story from each one so you learn a little bit more.

Participant 26 noted that the repeated exposure to the different elements of the historic site, combined with being with other content-area teachers, helped her use the content of the historic site to address the pedagogical and practical needs of the classroom:

I really liked the multiple tours of the house because we went through the house and took several different points of view; every time I went through it, I saw something new. And those a-ha moments were shared with everybody in the group … I’m looking at the specific concept in my lessons and I have all of these wonderful people that I can work with and refine what I do.

Although the members of this factor group remained relatively stable in the pre- and postsorting, in the postinstitute administration, they were more likely to note that the institute provided opportunities to reflect on their teaching practice and spend time with peers with similar professional interests and dispositions. Participants noted that these interactions with like-minded peers were essential for helping them raise questions that they might not consider on their own and problem-solve pedagogical issues for integrating what they learned into their own classrooms.

Factor 2: Monticello focus, learning from experts

Three teachers from the presort can be characterized as having a Monticello-Learning from Experts Focus. Of these three, two remained on this factor in the postsort, while a third teacher also moved to this factor during the postsort. The statements chosen for this factor as “Most Like Me” were exclusively representative of the content-and historical analysis-focused C3 statements. The statements chosen as “Least Like Me” were primarily drawn from the professional disposition-focused NCATE statements. Participants who loaded on this factor were primarily very experienced White, male teachers (average years teaching 17.5, range 8–31, mode 22). One Caribbean female teacher, also experienced (8 years), also loaded on this factor in the pretest, but moved to factor 3 in the post. Of the four factors, this one represented the smallest grouping of participants. (See Table 9 for participants and their movement onto or off Factor 2.)

Table 9. Factor 2 loading participants.

Other factor pre	Factor 2 pre	Factor 2 post	Other factor post
	Participant 21	Participant 21
	Participant 23	Participant 23
	Participant 24	Participant 24
	Participant 25		Participant 25 (Factor 3)

These teachers were characterized by statements that emphasized the value of working with the site staff experts, particularly to the exclusion of working with peers. For example, participant 23 at the outset of the institute stated,

I don’t know that I have in my teaching career the time or the interest to continue developing peer relationships from seminars … . Not knowing in a week the warp and weave of other peers, I don’t know that I’d ask them anything, or that I’ve ever asked them anything.

These participants’ concerns were the most indicative of complex historical thinking and disciplinary learning, both for them and for their students, repeatedly citing the ways they would use evidence to make historical claims. Participant 23, discussing why he chose statements 28 (helping me recognize that an historic site can serve as a microcosm of larger historical realities), 21 (helping me consider the historical context of people’s motives and actions in the past), and 6 (providing immediate access to the places and objects related to Monticello to improve my understanding of the people who lived there) as most like him, noted:

The statements [chosen] are more analytical, asking what the people, as best as we can tell through primary sources and secondary sources, were thinking at the time that the action was happening. History becomes a story laden with historical memory, which is sometimes very accurate and sometimes very inaccurate … Once you begin to read more and ask more questions, you begin to discover from experts how much you don’t know. It’s a rewarding experience to ask good questions and be directed to areas to understand additional information.

Similarly, Participant 24, who had previously worked at an historic site, was particularly aware of the historic site as a construction shaped by professionals (statement 18: helping me recognize that much of what I see at historic sites is shaped by professional historians) and enjoyed discussing with a curator the interpretive changes being made to the physical plant based on new evidence about the location of Sally Hemings’s dwelling. He connected that revision/reinterpretation to the work he had done at the Civil War site where he had worked:

You had to be conscious of upholding different sides of the conflict, and [interpretive media] was very carefully written and presented. As new evidence comes to light, people change, and you try to give people the more historically accurate view while being sensitive [about their previously held views].

This focus on the complexity of the historic site and the historical processes was largely absent from other participants’ descriptions of their decision to engage in on-site PD.

Factor 3: Monticello focus, learning from peers/colleagues

There were eight teachers from both the pre and post sort that can be characterized as having a Monticello-Learning from Peers/Colleagues Focus. In the post sort, three additional participants loaded on this factor, with only one participant dropping off. The statements chosen for this factor as “Most Like Me” were representative of the declarative content-focused statements from C3 statements that emphasized a sense of place and professional disposition-focused NCATE statements. The statements chosen as “Least Like Me” were the historical analysis-focused statements from the C3 Framework. Participants who loaded on this in the pre- and postsorts were primarily very experienced female teachers, with only one male participant (average years teaching 15.5; range 5–35; mode 8). This factor represents the largest and most stable—in terms of people not shifting off of it—of the four factors. (See Table 10 for participants and their movement onto or off Factor 3.)

Table 10. Factor 3 loading participants.

Other factor pre	Factor 3 pre	Factor 3 post	Other factor post
	Participant 1	Participant 1
	Participant 2	Participant 2
	Participant 3	Participant 3
Participant 4 (Factor 4)		Participant 4
	Participant 8	Participant 8
	Participant 9		Participant 9
	Participant 14	Participant 14
Participant 15 (Factor 4)		Participant 15
	Participant 17	Participant 17
	Participant 20	Participant 20
	Participant 22	Participant 22
Participant 25		Participant 25

Although it is not exclusive, one of the interesting aspects about the individuals who moved into this factor in the postsort is that three out of four of them identify as people of color. Given the sensitivity of the topics explored over the course of the institute, particularly regarding the experiences of enslaved individuals, it speaks to the collegiality of the group that there would be such movement.

In their interviews, participants spoke effusively about working with peers from around the country. For example, Participant 2, who loaded on Factor 3 in both her pre- and postsort, noted that she chose statement 43 (giving me the opportunity to build a network of peers who share my content interests) as most like her:

I love the fact that when you come to professional development you meet amazing teachers that come from places different than yours…. After the institute the networking piece moved up for me. Having the Sally Hemings conversation was really powerful for me. I could have gone all afternoon. I loved hearing the different perspectives from colleagues.

Along with this interest in working with colleagues was a clear sense of connection and fascination with the historic site itself. However, unlike participants who loaded on Factor 2, the participants who loaded on Factor 3 were less concerned with the historical processes and evidence necessary to construct the historic site than with the numinous nature of historic sites and the “inspiration” that being on-site provides. Consequently, there was a less critical/analytical questioning of the validity of the arguments put forward by the site in ways consistent with prior studies indicating the trust that teachers, and White individuals more generally, place in historical sources to “get the story right” (e.g., Burgard & Boucher, 2016; Epstein, 1998, 2000; Marcus, Levine, & Grenier, 2012; Rosenzweig & Thelen, 2000). For example, Participant 9 chose statement 29 (allowing me to see that the “power of place” is critical for informing the historical narrative):

The power of place is much more on topic when you are at a historical spot. And when you are at Monticello, you can really see Jefferson as a flawed human rather than a god. So the power of place is so important, and it’s hard to translate this into the classroom, but we should all try.

Throughout their choice of statements and interviews, there were loose suggestions of tying the experience to the classroom or of pedagogy, but unlike participants in Factors 1, 2, or 4, few specifics for classroom application or deep content/process knowledge were offered.

Factor 4: Focus on relevance (relating history to current, real life situation)

Of all the factors, Factor 4 showed the most volatility in terms of participants moving on or off the factor. There were three teachers from both the pre- and postsorts who loaded on this factor. Another seven participants loaded on Factor 4 in the preinstitute sort but not in the post. Only one participant moved onto the Factor in the postinstitute sort. The participants represented the most demographically diverse group of participants. Of the 10 participants, two were male, and three identified as people of color. Participants loading on this factor had the lowest average years of teaching (8.4; range 3–26; mode 5). (See Table 11 for participants and their movement onto or off Factor 4.)

Table 11. Factor 4 loading participants.

Other factor pre	Factor 4 pre	Factor 4 post	Other factor post
	Participant 4		Participant 4 (Factor 3)
	Participant 6		Participant 1 (Factor 1)
Participant 9 (Factor 3)		Participant 9
	Participant 15		Participant 15 (Factor 3)
	Participant 16		Participant 16 (None)
	Participant 18	Participant 18
	Participant 21		Participant 21 (Factor 2)
	Participant 27	Participant 27
	Participant 28	Participant 28
	Participant 29		Participant 29 (None)

The distinguishing statements were also far more moderate (i.e., ranking as a 1 rather than 6) than other factors and addressed a variety of pedagogical and student-focused concerns. Disciplinarily, participants’ statements, and item selection hew closest to the aspect of historical pedagogical content knowledge Monte-Sano and Budano (2013) referred to as Attending to Student Ideas. More broadly, though, their concerns were situated within the desire to engage in culturally responsive pedagogical practices (Ladson-Billings, 1995).

Participants were notably most interested in addressing issues of contemporary relevance and their connection to historical events, which was particularly evident in terms of their interest in the stories related to race and slavery. According to Participant 29 in the post institute sort:

Jefferson has always been a figure in history that I have admired, but I haven’t known as much about the slavery aspect of Monticello. We did a slavery tour that was absolutely fascinating. [The guide] made us think more about slavery; he made us think about race. He talked about the book by Ta-Nehisi Coates, Between the World and Me, which I read in grad school, in rural Alabama, and there was a lot of tension in my classroom when we read that book. My professor tried to have us understand the aspects of privilege, and there was a lot of animosity in the classroom from White middle-class students. And that got me thinking about how there needs to be more discussion, and in my own classroom how we can incorporate all the historical information into the conversations we are having today.

Participant 28 noted that choosing statement 33 (helping me select and develop appropriate strategies for teaching the complexities of history to my students) aligned with what she prioritized in historical study and teaching:

I think that motives, complexities, and actions [in history] are most important to me. We have to acknowledge both sides … These complexities make me think about my students. We have a lot of complex lives. We are multiracial and very diverse, and I want to be able to tie my lessons to be relevant to them. There are a lot of complexities in history that are relevant to them, fortunately or unfortunately.

Participants who loaded on Factor 4 in the pretest were more likely than in other factors to move to a different factor or not load on any factor at all in the postsort. This indicates two critical conditions: First, that these perceptions are diffused through the other factors on which participants load; second, that these culturally responsive pedagogical perspectives, while present, were not a priority for the majority of participants, relative to other perspectives indicated.

Most common statement

In arranging a concourse, the statements participants are given to respond to are intended to represent a range of perspectives about the research question, without pointing toward one particular “correct” viewpoint or opinion (Watts & Stenner, 2005). For this concourse, it meant including views that visitors to the site held about history and historical persons/events, even if these views run counter to the stated mission of the institution. With a site such as Monticello, it meant nodding to the extreme patriotism or jingoism that often draw people to the home of an American president and Founding Father.

In this concourse, one such statement was #11: Professional development will help me to see the superiority of America’s culture to other world cultures. Across all the administrations of the concourse, this statement was the most frequently ranked. Participants held strongly negative views of this statement, most frequently indicating it at the extreme (−6) in the Least Like Me section. Most of the teachers gave very thoughtful answers to explain why they did not believe in the superiority of America’s culture to other world cultures and how detrimental that kind of thinking is in the classroom. Others saw no relevance to the task at hand. As one participant explained, “I’ve never seen America’s culture as superior. It has nothing to do with the institute.”

This finding is not meant to suggest that the participants were unpatriotic or did not hold Jefferson in high esteem; rather, it indicates that their thoughts and feelings about Jefferson, and the American experiment more broadly, were complex and lacking in reflexive jingoism. Indeed, over the course of the institute, participants who held strongly positive views of Jefferson at the outset struggled with how to reconcile the “real” person with the hagiography they had previously been steeped in, but across the participants, the responses to this statement reflected a strong pluralistic stance and a rejection of the notion of an uncritical sense of “superiority” of the nation as reflective of their position or pedagogy.

In the construction of the concourse, the research team, site staff, and review panel struggled to find the right wording to encapsulate this idea. Phrases like “American exceptionalism” were deemed to be too freighted to provide useful feedback. However, as this item skewed so negatively for so many participants, there was consensus among the researchers and site staff that the statement phrasing was insufficient in capturing the sensibility without over-signaling a potentially desired response and should be removed/revised from the concourse in future administrations.

Diversity statements

Included in the concourse were statements drawn from the NCATE standards on diversity (e.g., #53: giving me tools to support students with exceptionalities (e.g., Physical, language-based, or intellectual disabilities), #54: giving me tools to support multi-lingual students (e.g., ELL). These, too, were statements that many of the participants indicated were “Least Like Me.” However, given the example of flossing their teeth to help participants distinguish between statements, participants who indicated that these statements were “Least Like Me” or “Neutral” noted that the reason they did so was because they, for example, tended not to have large multi-lingual student populations or that they sought out PD that specifically addressed those needs, rather than turning to historic sites for that support.

Limitations

As with all assessment projects, there are limitations to what we can learn from using this concourse. The primary limitation is that since Q methodology requires participants to rank concourse items, we are limited to what we can find based on the statements we ask participants to rank. As the concourse was constructed to look at a broad range of content, skills, and dispositions that motivate teachers to participate in historic site-based PD, it was effective in terms of illuminating patterns in which those occur. However, complex questions of what and how participating teachers learn across the different layers of the historic site and how the interpersonal interactions that occur on site affect their professional growth that occur within those layers must be addressed by other means.

Discussion

Given the complexity of the results and the range of interactions involved, the discussion of the results of this study are considered in terms of Space/Place elements and Interpersonal Elements.

Space/place elements

The participants in this study, particularly those loading on Factors 2 and 3, noted that the “power of place” was instructive and essential for understanding why they chose to engage in this kind of PD and part of what they sought to bring back to their classrooms. Therefore, we must consider the historic site itself as a pedagogical element of this PD, which offers both significant challenges and opportunities.

Power of place

The appeal of concourse items in this study indicating the importance of place, coupled with statements that take a relatively uncritical stance regarding the historical information the site presents, should raise concerns for museum and teacher educators alike. As the participants noted, it is difficult to transfer the “non-narrative” aspects (Barton & Levstik, 2004, p. 140) of the power of a place into the classroom. Although the passive acceptance of the information the sites “tell” participants echoes prior research (Marcus, Levine, et al., 2012; Rosenzweig & Thelen, 2000), history educators are better positioned to counter that passivity than ever before.

One of the primary pedagogical challenges of working with historic places is that they are multilayered artifacts that are perceived holistically (Baron & Dobbs, 2015). Unlike documents, there is not always an obvious point to start “reading” the place, or when on guided tours, the opportunity to do so independently. Few outside the ranks of history practitioners (e.g., historians, curators, etc.) have been trained to read place and objects in systematic ways, thereby creating a need to be “told” what they are looking at, by either guides or interpretive media. This lack of historical literacy, along with the perceived authority of the historic sites as purveyors of the “truth” (Rosenzweig & Thelen, 2000; Trofanenko, 2006), further reinforces uncritical acceptance of the stories with which visitors are presented. Before we can encourage or expect more critical appraisals of historic sites, we need to provide pathways that allow individuals to assess the sites as readily as they would traditional texts. Thus, if we follow the path of the research that laid the groundwork for greater use and critical reading of historical documents in the classroom, we have models for how to incorporate the analysis of the historical materials found on site and transfer those methods and materials into the classroom. It is time to apply those lessons to historic sites.

Although there are many ways to divide the layers historic sites present, three primary layers relative to teaching and learning at historic sites are textual (the physical place of the natural and built environments); perceptual (how we perceive the place and its materials using our senses); and interpretive (the meaning we ascribe to the place; see Bell & Dourish, 2004; Callanan, 2012; Carmona, Tiesdell, Heath, & Oc, 2010; Dwyer & Alderman, 2008; Drozdzewski, De Nardi, & Waterton, 2016; Smith, 2006). The textual layer—the peculiar collection and configuration of the built and natural environments, material culture, and ephemera—will always remain unique to individual historic sites. Rather than being a single-author text, historic sites are anthologies of different kinds of texts, with many different authors and amendments, each offering a part of the whole story of the place. We argue that it is in the perceptual and interpretive layers in which those disparate historic materials are analyzed and understood (or not) and that there are considerable opportunities for developing critical pathways for learning and transfer across multiple sites and from sites to the classroom (cf. Barnett & Ceci, 2002; Baron, 2014).

An emerging body of research, rooted in that perceptual layer, considers the cognitive, affective, and/or embodied experiences people engage in when encountering different historic materials. This research offers promising pathways for building analytical practices that bridge the textual and interpretive layers and support the pedagogical development necessary for effective transfer into the classroom (e.g., imagery and verbal cues: Glaser & Schwan, 2015; imagery and text: Baron, 2016; readability of the site: Baron, 2012; Gussmann, Merkt, & Schwan, 2017; sound: Lee, Hicks, Henricksen, Mishra, & Cain, 2015; augmented reality/mobile technologies: Johnson et al., 2017; Price, Jewitt, & Sakr, 2016; the interplay of cognitive and affective dimensions: Endacott & Brooks, 2013; Savenije & De Bruijn, 2017).

The current study points to how critical it is for researchers and practitioners to work together to continue these lines of research across the layers of historic sites and to do the translational work that takes these research findings and tests applicable site-based pedagogical solutions. The net effect of such efforts would not only provide teachers with deeper explorations of the complexities of historic places but also model the use of the effective pedagogical content knowledge and processes that drive teachers’ PD choices.

Teacher education at historic places

The majority of studies and meta-analyses indicating best practices and design standards for teacher PD were not based on social studies PD—either as separate disciplines (e.g., History, Geography) or as a whole.⁷ As a result, recent calls to redesign the ways in which PD is evaluated to focus on specific program features overlook the challenges of teaching and learning history, generally, and the circumstances of historic site-based PD programs specifically. For example, Hill et al. (2013) called for the test of the same content at multiple PD locations. However, unlike STEM subjects or literacy activities that can be decontextualized, history is deeply context-dependent, and these recommendations presume that the locations of the PD programs are not themselves content for consideration.

Even if a second iteration of the MTI program was offered at the historic site of a different Founding Era president and signer of the Declaration of Independence, such as John Adams’s Quincy, Massachusetts, home, the content would diverge from the moment participants arrived on site: In contrast to Monticello’s neo-classical mountaintop plantation, Adams’s house, embedded in the contemporary landscape, is a modest dwelling with low ceilings and a warren of small rooms, typical of New England vernacular architecture. Even if the readings, activities, and presenters were the same, any discussion of either Jefferson’s or Adams’s public or private life would be shaped, minimally, by the physical context in which they lived and the stories that those spaces and their interactions with them tell about them. To ignore these differences and insist on continuity of content is to presume that the spaces themselves stand outside the content under consideration. The results of this study indicate that they, quite clearly, are not. Future research into history/social studies teacher professional education needs to take up the challenges of understanding how immersion in these historical contexts shape teachers’ learning and teaching.

Interpersonal elements

Peer-to-peer interactions

Participants who loaded on Factors 1 and 3, including several who moved off of Factor 4 in the post test, noted that working with peers who share their content interests was an important part of why they chose to attend PD at an historic site. Given that teachers who arrive at an historic site for PD come with their own historical identities, perspectives on the past, and political and epistemological positions on the past relative to the content they teach (Klein, 2016; Knowles, 2017), putting a group of teachers together to work with peers, who also bring their own complex understandings of the past, to help interpret these multilayered spaces and the stories within deserves further attention.

Although there is a growing body of literature on effective “teacher talk” in history classrooms (e.g., Havekes, Van Boxtel, Coppen, & Luttenberg, 2017; Reisman, 2015; Reisman et al., 2017), absent are studies assessing the discourse in teacher-to-teacher discussions exploring how they co-construct meaning about the content, conceptual, or pedagogical understandings either in history education, as a whole, or at historic sites, specifically. Since much of the literature on disciplinary inquiry-based history education emphasizes understanding the constructed nature of and range of histories in play (VanSledright, 2002), this gap is critical in understanding teachers’ disciplinary and professional development.

Site staff-to-teacher interactions

Similarly, the role of the museum educator to teachers requires considerably more investigation. However, these are not the only members of the staff with whom teachers interact. For many reasons, the traditional didactic tour model is likely to remain a feature of visits to historic sites for the foreseeable future. Given that participants reported strong responses to their time with the tour guide who led the Slavery at Monticello Tour as having a very large impact on how they viewed the site and the historic agents whose stories they told, these interactions deserve further consideration.

Prior research has shown that tour guides play a significant role in shaping how slavery is represented in plantation house museums by encouraging participants to emotionally invest in “planter versus the enslaved” narratives, often creating “affective inequality” in those representations (Modlin, Alderman, & Gentry, 2011, p. 5). Considerably more needs to be known about how those narratives shape participating teachers’ curricular choices for projects developed in PD programs and subsequent classroom explanations.

Using Q methodology

The purpose of this study was not just to understand the workings of the MTI program, but also to understand how the use of Q methodology and the concourse illuminates what is occurring in historic site-based teacher education programs. In the absence of a defined research agenda for PD, we do not have a broad-based understanding of what teachers learn from PD or how they connect what they have learned to their working lives. No one research methodology or tool can address all the unanswered questions that exist. However, this concourse offers a tool to help shape the conversation about the ways in which on-site learning and programming contributes to teacher professional growth and development. It is not intended to provide a comprehensive assessment; rather, it is a tool with which to create a foundation to move away from siloed program evaluations toward research that considers larger questions about social studies teachers’ professional lives that takes into account the peculiarities of different historic sites, types of PD programs, and teacher populations. Use of this concourse across different sites and different types of programs can reveal patterns or raise issues that have otherwise been missed in previous investigations.

One such unexpected issue emerged in the discussion of the results of this Q sort with the education staff at Monticello. It is essential to remember that teacher PD at historic sites is presented by professionals in one institutional context for professionals who work in different institutional contexts. Therefore, using guiding principles outlined in the conceptual framework and grounding the concourse in the existing professional and content standards were critically important for not only understanding what teachers prioritize in their own development, but also in bringing the museum educators who present PD programs more fully into the discussion of what it means to be teacher educators.

Although the museum educators who helped develop this concourse have long been familiar with state content standards, the professional dispositional and skill standards (NCATE, InTASC) were eye-opening. Developing statements using terminology such as “pedagogical content knowledge” or dispositional standards related to collaborating with peers highlighted some of the differences in the professional language between university-based teacher educators, museum professionals, and teachers, as well as the resulting misunderstandings that can ensue.

That these issues emerged to shape Factors 1 and 3 were surprising for individuals working in an institution that prioritizes the dissemination of substantive historical content. Unprompted by the researchers, the resulting discussions among the museum educators shifted from acknowledging that “teachers like to talk to each other” during on-site programs to understanding why teachers wanted to talk to each other, how they used those spaces to problem-solve curricular issues, how critical those opportunities were for their professional growth, and how to modify the program to provide better space and support for such activities.

As Goodwin and Kosnik (2013) noted, “It is reasonable to assume that quality teacher preparation depends on quality teacher educators” (p. 334). As teachers are critical to the classroom, so are the museum educators for historic site-based teacher education. Thus, with the increasing interest in teacher education at historic sites, we must begin to explore the role of historic site staff as teacher educators, including their qualifications, training for, dispositions toward, and the manner in which they direct instruction for teachers at the sites (Goodwin & Kosnik, 2013; Menter et al., 2010; Swennen, Jones, & Volman, 2011).

Unlike their counterparts teaching in K–12 classrooms, museum educators generally are not required to obtain a certification as a museum educator or teacher educator, and there are few opportunities for formalized training in this area. Although there is increasing professionalism within its ranks, often museum educators teach with little formal knowledge or training in teacher education, often in isolation from peers or other professional educators and with limited access to resources or time to reflect on their practice (Castle, 2006). Educators frequently begin working in museums with disciplinary expertise relevant to the subject matter addressed in their museum, yet they often do not have the professional training or experience with emerging educational theories and practices or larger conversations about traditional student or teacher education practices (Castle, 2006; Grenier, 2009; Hooper-Greenhill, 2007; Lindauer, 2003, 2007). This absence of formal grounding related to teacher education for museum educators raises critical questions about the substance and quality of support teachers get during historic site-based PD.

As such, fundamental questions need to be addressed about museum educators/history practitioners who take on the role of teacher educator: Do they identify as teacher educators? If so, how are they conceptualizing that role? Do they have the contextual knowledge of the schools, classrooms, and communities their students (i.e., teachers) will be teaching in to effectively support them? Do they model and can they explain how to adapt effective pedagogies to support student learning? Do they see themselves working within the larger social justice agenda of teacher education? Do they see their work with teachers as “accidental” or intentionally oriented toward teacher education? What role does that intentionality play in their preparation and performance? If we could offer education/PD to history practitioners to support them as teacher educators, what would that entail? What kind of support would they most need (Dinkelman, 2011; Goodwin, 2010; Mayer, Mitchell, Santoro, & White, 2011; Vloet & Van Swet, 2010)? Minimally, this concourse and the conceptual framework in which it is grounded offer a place to start these discussions.

Conclusion

Using an assessment tool rooted in Q methodology, this study offers a window into the multiple ways in which history/social studies teachers use historic sites for PD. Although this study begins to illuminate these practices, it raises questions about the pedagogical role of historic places in history/social studies teacher education, including the critical role of site staff and museum educators as teacher educators. As always, these questions require further research, but this study offers a foundation from which to start systematically exploring these emerging questions.

Acknowledgments

The authors would like to acknowledge the contributions to this work of Thomas Jefferson Foundation Staff members Gary Sandling, Linnea Grim, Jacqueline Langholtz, Melanie Bowyer, and Lora Cooper.

Additional information

Funding

This project was made possible in part by the Institute of Museum and Library Services through the National Leadership Grants for Museums program (www.imls.gov; Federal Award Identification Number: MG-10-15-0095-15).

Notes

1. Herein we use the terms “museum” and “historic site” interchangeably as the assignation of one or the other often has more to do with individual institutional customs than any meaningful professional or educational delineations. These institutions include, but are not limited to, history museums, historic houses, battlefields, cemeteries/burial grounds, and so forth whose primary disciplinary focus could be situated within history and the social studies (anthropology, civics, economics, geography, etc.) rather than art or science.

2. Although comparisons to research in science education must be considered within the context of the inequality in funding between these content areas, it is essential to note that the current state of the research follows a decade of significant investment of U.S. federal resources in the form of the Teaching American History grants (USDOE, 2011). Even with a billion-dollar investment requiring the use of historic sites as an essential part of the projects enacted, we know little more about teacher education at and in conjunction with historic sites than we did a decade ago.

3. The Teaching American History program (2001–2011), although offering structures that would have allowed for district-wide studies of professional development for history teachers, was plagued by a lack of clear assessment directives and data sharing expectations and ultimately was cancelled, in part, because it could not show any effect on classroom practice (Ragland & Woestman, 2009; Schrum et al., 2016; U.S. Department of Education, 2011). Similarly, funding stipulations for National Endowment for the Humanities Landmarks in American History and Culture and Summer Institutes for Teachers, the sole ongoing federal humanities-based teacher professional development program, do not include substantive funding for or a coherent plan for program evaluation. It is currently suspended for 2018 (National Endowment for the Humanities, 2017).

4. To see the full set of statements and the standards to which they are tied, refer to the project website: http://www.teacherinsites.org/2016/10/25/the-2016-concourse/.

5. As participants were not required to sort the statements into three equal piles, what they determined to be Neutral versus Most/Least Like Me depended on the individual. It was not uncommon for Neutral statements to bleed into the −2/+ 2 columns. As such, statements ranked at + 2/-2 should be seen as low positive/negative to neutral.

6. A centroid method is a factor extraction technique that is preferred by Q-methodologists. The PQ Method software provides a function to complete this factor extraction to find the best possible factor solutions. The software searches for a “shared pattern or sorting configuration in the data,” and extracts the common variance which then becomes Factor x. The factor loadings are then to the extent that “an individual Q sort can be said to exemplify” the Factor x pattern (Watts & Stenner, 2012, p. 100).

7. A review of these frequently cited articles on teacher PD (Borko, 2004; Desimone, 2009; Garet et al., 2001; Hill et al., 2013; Kennedy, 2016; Penuel et al., 2007) showed that of more than 60 studies/PD programs referenced, only two (Grossman, Wineburg, & Woolworth, 2001; Thomas, Wineburg, Grossman, Myhre, & Woolworth, 1998) focused on the social studies/social studies teachers.