Staff matter: Gender differences in science, technology, engineering or math (STEM) career interest development in adolescent youth

ABSTRACT

We explore the understudied role of program staff in an out-of-school time (OST) program at a large science museum, which may be especially relevant for supporting underrepresented minority (URM) youth’s interest in science, technology, engineering, or math (STEM) careers. Using a sequential explanatory mixed-method design, we surveyed 167 program alumni on their science attitudes, career interests, and memories about how the program compared to experiences at home, school, and with friends. We followed that with 49 interviews with alumni. Findings show that, while in the program, alumni who identify as women reported a much greater increase in their STEM career interest than those who identify as men. Interviews suggest this may be related to different types of staff relationships between the genders. We interpret results through the lens of positive youth development and offer recommendations for OST program providers and researchers.

Introduction

More than 10 million children each year participate in out-of-school time (OST) programs, with 69% offering science, technology, engineering, or math (STEM) learning activities (Afterschool Alliance, 2015). Such programs often occupy a hybrid space between the home, school, and social lives of adolescents. Partly because of the flexibility of this unique arrangement, they can meet certain needs of underrepresented minority (URM) youth through cultivating key elements of positive youth development (PYD) in ways formal education cannot (Fenichel & Schweingruber, 2010; Larson, 2000; McClure & Rodriguez, 2007). Increased URM involvement in the national STEM workforce pipeline is critical to our future economic success (National Academies Press [NAP], 2011) and OST programs offer key support for that pipeline (National Research Council [NRC], 2009).

An important and understudied aspect of PYD is the role of program staff (Larson, Eccles, & Gootman, 2004; Roth & Brooks-Gunn, 2016). Given the importance of other adult-participant relationships, such as parent–child and teacher–student, in PYD programming (Roth & Brooks-Gunn, 2016), it is somewhat surprising that the staff-participant role has not been well explored. Such relationships may be especially relevant for low-income students, students of color, girls, and others who have been historically marginalized (Scales et al., 2016), especially in STEM fields (Chen & Soldner, 2013). Social relationships, including with staff, are among the highest cited positive influences of girls in STEM based OST programs (McCreedy & Dierking, 2013).

The Museum of Science and Industry (MSI), Chicago has run an OST adolescent development program since 2003. This study surveyed and interviewed program alumni to find out how the program may have impacted their career choices and attitudes toward science. Data were analyzed to look for differences between genders and the role of staff in their experiences. The research question was: “What impact did the staff have on STEM career interest and attitudes towards science of alumni of a science museum-based PYD program?”

The article begins with a literature review about OST programs as avenues of PYD, with a focus on the role of staff on participant career choices. Then we introduce the Science Minors and Achievers (SMA) program as a “context for development” (J. L. Roth & Brooks-Gunn, 2016). In so doing, we call attention to a STEM-based OST program that incorporates the critical elements of PYD. Next, we describe our survey and interview protocols and results. Finally, we interpret results through the lens of PYD and offer recommendations for OST program providers and researchers.

Literature review

OST programming and science

Most afterschool programs involve science programming (G. Wright, 2009). One study of 415 afterschool programs in California found that 87% had offered science activities or content during the last year (House, Llorente, Gorges, Lundh, & Mata, 2015). A study of OST program leaders found 90% would like to expand their scientific programming (Chi, Freeman, & Lee, 2008). While science is widely considered a key component of afterschool programs (Afterschool Alliance, 2013) it is often neglected due to a variety of factors, not least of which is lack of staff training in science education (Coalition for Science After-School, 2007; Freeman, Dorph, & Chi, 2009; The After School Corporation, 2010).

STEM-based OST programs can serve as important environments for youth development (Laursen, Thiry, Archie & Crane, 2013; J. L. Roth & Brooks-Gunn, 2016). They have proven “to contribute to young people’s interest in and understanding of STEM, connect young people to caring adults who serve as role models, and reduce the achievement gap between young people from low-income and high-income families” (NRC, 2015, p. ES-1). An Afterschool Alliance (2016) report echoed similar sentiments, relating that “new evaluative data strongly demonstrated the lasting impact that afterschool programs had on students’ ability to connect the importance of STEM to their future success and communities” (p. 2).

Much of the literature regarding the impact of OST programs has focused on aspects such as the physical space (Trost, Rosenkranz, & Dzewaltowski, 2008), social setting (Shernoff, 2010), key program features (Canzano, Anthony & Scott, 2016; Durlak, Weissberg, & Pachan, 2010) and programming content (Mahoney, Parente, & Lord, 2007). Many OST educators and researchers think of OST programs as hybrid spaces where elements of home, society, and school work together to generate knowledge, identity, and discourse (Calabrese Barton, Tan, & Rivet, 2008; Moje et al., 2004; W. M. Roth, 2007). The National Research Council found making purposeful connections between home, school and other settings to be a core criteria for a positive OST experience (2015). Examples of ways in which hybrid spaces are constructed include creating scientific artifacts, providing an environment to experiment or “play” with different and new identities, and supporting the strategic negotiation of staff and youth roles (Tan, Calabrese Barton, Kang, & O’Neill, 2013).

Hybridity has been shown as an effective way to look at how girls, in particular, view science since it addresses issues of identity, knowledge, skills, and goals from a broad sociocultural perspective (Calabrese Barton et al., 2008). It is also a framework well-tailored for OST spaces, because it treats science as a horizontal learning process (across a wide breadth of contexts and spaces) as opposed to a vertical learning process (evolving from novice to expert within a specific domain) (Gonsalves, Rahm, & Carvalho, 2013), thus taking advantage of the broad nature of OST learning environments as spaces related to both home and school. However, studying hybridity takes time since these spaces necessarily evolve slowly as they are created. Hence, long-term, longitudinal and ethnographic observation is needed.

This hybrid nature also provides OST programs with flexibility to meet certain needs of URM communities in ways formal education is not (Fenichel & Schweingruber, 2010; McClure & Rodriguez, 2007). Enrollment in these programs is growing quickly among URM groups (Dahlgren, Noam, & Larson, 2008), but research of OST programs often lump URM groups together and ignore their unique characteristics and histories (Williams & Deutsch, 2016). Thus, there is a need for more research that considers the unique strengths and values of specific URM populations in OST environments (Gonsalves et al., 2013).

OST impact on careers

OST programs have been shown to have a positive impact on science career interests and aspirations (DeWitt, Osborne, et al., 2011; G. Jones, Taylor, & Forrester, 2011; NRC, 2015). When studying the relationship, researchers often focus on the role of STEM-based programming in stimulating science interest as a determinant for future career paths (Dabney et al., 2012; Dierking & Falk, 2003; Krapp, Hidi, & Renninger, 1992; Lent, Brown, & Hackett, 1994). In a meta-analysis of evaluative data from 11 established OST STEM programs, the Afterschool Alliance found that OSTs have a strong impact on youth’s ability to identify as potential scientists (Krishnamurthi, Ballard, & Noam, 2014). Retrospective studies in particular show that children who grow up to work in science careers often report initial science interests in childhood that are sparked by OST experiences (Crowley, Barron, Knutson, & Martin, 2015: p. 297; G. Jones et al., 2011; Maltese & Tai, 2010). Although retrospective studies have offered important links between OST programming, science interests, and career paths, studies that use a longitudinal research design are still needed (Dabney et al., 2012; Rennie, Feher, Dierking, & Falk, 2003). Furthermore, OST STEM programs are not without their issues. Many OST experiences are more impactful on science career interests of male participants than female participants (Maltese & Tai, 2010) and URM populations may have less access to OST resources (Archer & Francis, 2006, cited by DeWitt, Archer, et al., 2011).

OST programming as positive youth development

PYD is a field of adolescent development that describes programs and models whose aim is to promote positive youth behavior outcomes (Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2004; Eccles & Gootman, 2002; Larson et al., 2004). It emerged in response to a deficit view of youth development that has dominated the field for decades (Bowers et al., 2010; Lerner, Brentano, Dowling, & Anderson, 2002). Instead, PYD builds on young people’s strengths and focuses on their potential contributions. In particular, we found the Programs, Activities, Relationships, Culture (PARC) model of youth development (Hirsch, Deutsch, & DuBois, 2011) is particularly aligned with the program being studied (see the following sections). PARC includes four PYD program elements that teens experience in an overlapping and integrated manner in such a way to create an individualized experience (as opposed to assuming all teens in a program have the same experiences). See Table 1 for examples of some activities of the SMA program model mapped onto the PARC model.

Table 1. Comparison of Hirsch et al. (2011) PARC youth development model with examples of SMA key program elements.

PARC element	Associated SMA Key Program Elements
Programs	Morning check-in Floor hours Formal public events
Activities	Sustained contact hours
Relationships	Teen-Staff relationships Relatable participants
Culture	Science Museum espirit de corps and perks

When referring to PYD, scholars and practitioners alike will frequently refer to the “Five Cs,” which emphasize the strengths of adolescents (Bowers et al., 2010). The Five Cs include: competence, confidence, connection, character, and caring. “[It] posits that positive development occurs if the strengths of youth (represented, for instance, by the enormous potential for systematic growth, i.e., for plasticity, within the adolescent period) are aligned systematically with positive, growth promoting resources in the ecology of youth” (Bowers et al., 2010, p. 721). The model treats young people as resources to be developed rather than inherently deficient (Bowers et al., 2010; Lerner et al., 2005; Roth & Brooks-Gunn, 2003). Developmental systems theories inform the model with an emphasis on plasticity (Lerner, 2004). This model and the underlying developmental systems theories that underscore plasticity speak to a “life-span developmental process” that may be labeled as “thriving,” an important developmental concept to PYD that “denotes a healthy change process linking youth with an adulthood status enabling society to be populated by healthy individuals oriented to integratively serve self and society” (Lerner, Dowling, & Anderson, 2003, p. 176).

OST staff roles

Research on adult-youth relationships in OST settings has tended to focus primarily on parents and/or formal teachers (Jones & Deutsch, 2011). However, recently scholarship has begun to interrogate the role of staff in afterschool and OST programs as being distinct from these formal relationships (Hill, 2016). Roth and Brooks-Gunn (2016) surmise that in reports examining program impact on participants, “supportive and sustained” adult-youth relationships are the one consistent factor, leading some researchers to describe them as the “critical ingredient” in effective youth programming (p. 193; see also Gupta & Negron, 2017; Rhodes, 2004). Such studies reveal that young people place great importance on their relationships with program staff. Roth and Brooks-Gunn (2016, p. 9) observed, “[D]uring interviews many adolescents report they attend programs because they ‘like the staff’ or the adults at the program ‘care’ about them”. Supportive relationships are a key element of “active ingredients” that community programs need to use when designing PYD programs (Larson et al., 2004; Rhodes, 2004; Roth & Brooks-Gunn, 2016). As Larson et al. (2004) points out:

The quality of relationships with adults consistently comes up as a critical feature of any developmental setting. Researchers speak of the importance of warmth, connectedness, good communication, and support. Theorists talk about adult who provide secure attachments, are good mentors and managers, and provide scaffolding for learning. Practitioners talk about caring and competent adults. Adolescents may use more evocative terms—like being loving or “cool” (p. 9).

We and others attest that such settings—that emphasize caring staff-youth relationships—index “thriving” (Larson et al., 2004; Lerner et al., 2003; Theokas & Lerner, 2006), which refers to “the active process […] by which individuals shape and engage with their developmental contexts, in whatever context they inhabit, in order to develop a life trajectory of competencies, skills, and behavioral repertoires that are simultaneously beneficial to self and society” (Scales et al., 2016, p. 166; see also Benson & Scales, 2009; Benson, Scales, Hamilton, & Sesma, 2006; Lerner, 2004). In other words, adult-youth relationships are integral components to positive youth development.

Studies have revealed that such relationships are important influences on students’ career selections (DeWitt, Osborne, et al., 2011; G. Jones et al., 2011b). Staff have the potential to change the “culture of power” and “challenge narrow assumptions about who is capable of learning and doing science in a field historically dominated by white, middle-class males” (Aschbacher, Li, & Roth, 2010, p. 564; Calabrese Barton & Yang, 2000). By observing staff that look and talk like them, young people may be able to envision themselves one day occupying similar roles of power and authority (Ginwright, 2007). Indeed, youth program participants “may find some careers as more desirable than others because role models are available to them” (Oyserman, Gant, & Ager, 1995). Hirsch (2005) observed higher participant retention and engagement when program staff reflected the neighborhoods and/or racial/ethnic backgrounds of the young people with whom they work. The stakes are higher for girls in particular because “[t]here is some evidence that strong relationships with family may have a negative impact on girls’ science career selection” (G. Jones et al., 2011, p. 1655; see also Packard & Nguyen, 2003). Family members’ expectations of particular gender roles may inhibit young women from exploring particular career interests. Moreover, Eccles (1987) has proposed that young women and girls may view science careers as an impediment to family life and to maintaining close relationships. Hence, girls’ relationships with OST program staff may actually challenge the negative view of a science career and, instead, reconfigure it as appealing and attainable.

Developmental context: SMA program structure and participants

Located in the South Side of Chicago, MSI is the largest science center in the Western Hemisphere. The SMA program is its adolescent development program aiming to help teens discover new interests in science and technology, develop leadership and communication skills, prepare for college and careers, and learn from and mentor new cohorts of teens while fulfilling service-learning hours. Approximately 140 teens participate in the entire SMA program each year. The program consists of three sessions per year, each of which last ten weeks. In each session, new participants are placed in a group called the Science Minors while returning participants join the Science Achievers.

Youth tend to join the Science Minors program when beginning high school. It focuses on introducing scientific content knowledge and communication training and ends with youth leading interactive science experiences for Museum guests. The program takes place every Saturday from 9 am-12 pm during the session period.

After they complete their first 10-week session, Science Minors have the opportunity to advance to the Science Achievers group by spending an additional ten hours leading interactive science experiences on the museum floor. Approximately 75% of Science Minors go on to do this. The Science Achievers are at the Museum the entire day, from 9 am-4 pm. A typical day for the Science Achievers begins with a morning meeting where participants and staff share personal news and prepare the day’s activities. Then they split up into smaller groups doing activities including inquiry-based and hands-on science experiences, further training for interacting on the Museum floor with guests, college and career preparation activities, leadership training and more. Off-site activities include visits to local colleges, working laboratories, etc.

At the time of data collection, the program had five full-time staff with support from two paid part-time staff who are local college students and former Science Achievers, along with 4-6 regular volunteers. The staff had backgrounds in education, biology, communications, chemistry, astronomy, and youth development with a combined museum education experience of 38 + years. Program staff emphasize family involvement and generate opportunities for family members to learn more about the program and what youth are learning and doing. The first Saturday of every session is an orientation for both new and old participants where parents are strongly encouraged to attend. The last Saturday of each Science Minor session is a “Family Day” (Figure 1) where participants present science content and interactive demonstrations using their newly acquired knowledge and communication skills. At the end of each program year, in June, family and friends are invited to celebrate the Science Achievers at a semi-formal showcase event. During this event, Science Achievers do activities and live science demonstrations in front of an audience comprised of friends, family, and VIPs (e.g., the Museum president, donors and long-time supporters). A more detailed description of the program structure can be found in Cole (2012).

Figure 1. Science minors presenting to friends and family on “family day.”

Consistent with one of the key features of PYD programs, the program is relationship-driven. (Larson et al., 2004). Decisions about the program’s scope of work are often developed around the emergent needs of each participant, rather than being based upon a particular predetermined program theory or imposing a one-size-fits-all model. This attention to and accounting of participant needs, backgrounds, and learning styles has the added benefit of creating a community of practice (Wenger, 1998), which has been shown to be particularly effective in working with young girls (McCreedy & Dierking, 2013) and URM youth (Chinn, 2006). Bell, Lewenstein, Shouse, & Feder (2009) demonstrated that it is important for OST spaces to offer youth an environment where they can consciously reflect upon and create their own role in science.

The young people who participate in the program come from a wide array of backgrounds. They are diverse in terms of race/ethnicity, geographic distribution, family background, socioeconomic status (SES), STEM interest, personality types and more. Program participants hail from all over the Chicagoland area, which includes outskirts of the city as well as suburbs and parts of Indiana. They attend public schools, private schools, charter schools, Catholic schools, Jewish schools, and are homeschooled. Some drive to the program each Saturday in their own car, others travel an hour each way on public transportation. Some hitch rides with parents, who may camp out at the Museum all day watching videos on a tablet device, exploring the Museum and sometimes eating lunch with the staff. Many report this experience was their first time to form relationships with those different from themselves in terms of race/ethnicity, SES, geographic location, etc. while also being their first time to find people like themselves in terms of interest in science and learning.

Methods

The study follows a sequential explanatory mixed-method design beginning with a survey of program alumni, followed by two rounds of semi-structured interviews. The Museum had contact information for 575 SMA alumni, consisting mostly of information on record when they left the program (e.g., residence while in high school). A survey was mailed to all of these addresses with a self-addressed, stamped envelope and a cover letter providing an option for them to complete the survey online in lieu of postal mailing it back. Operating under the assumption that these addresses were mostly associated with their parents/guardians, we planned the survey mailing to arrive the weekend before Thanksgiving so as to be available when students arrived home from college. A postcard reminder was mailed to recipients and scheduled to arrive the day after Christmas. The link to the online version of the survey was also emailed to the email addresses we had on file and was posted to a Facebook group created and moderated by SMA alumni. We offered a $15 Amazon gift card as an incentive and included a statement in the recruitment letter requesting feedback from all participants, regardless of whether they memories were positive or negative. After the surveys were processed, all respondents who indicated they were open to interviews were invited to participate in a telephone interview for an added incentive in the form of a $20 Amazon gift card. All research activities were approved by the Museum’s Institutional Review Board.

Participants

Surveys were returned by 167 alumni. The first phase of interviews included 28 respondents and the second phase included 21 respondents. The average age of a respondent was 21 (SD = 2.4) and ranged from 18-28, indicating a higher response from recent alumni. We believe this was due to having more recent contact information for them. The respondents self-reported as 66% female and 34% male¹. Using the 2010 U.S. Census format for race/ethnicity, they identified as 50% African American, 25% White, 6% Chinese, and 29% other groups (multiracial or “other”). Additionally, 25% reported as being of Hispanic, Latino or Spanish origin with 16% Mexican, 4% Puerto Rican and 5% another Hispanic, Latino or Spanish origin. When asked to categorize their social class while in the SMA program using a five point scale from “Lower Class” to “Upper Class”, 41% chose “Middle Class”, 19% chose “Lower-Middle Class” and 19% chose “Upper-Middle Class”. No respondents chose Lower Class or Upper Class.

Surveys

The printed and online survey had identical content. One section was devoted to demographic information. Another included a selection of items from the Test of Science Related Attitudes (TOSRA) assessment (Fraser, 1981). TOSRA is a commonly used science attitude scale and has been cross-culturally validated by prior researchers using Chicago area students (Khalili, 1987; Welch, 2010). Out of the 70 total items included in the TOSRA, we adopted 15 that we felt were aligned with the goals of the SMA program, mainly related to career interest and informal learning of science (Table 2). The scale was found to be highly reliable, α = .82. Another portion asked alumni to describe their educational and career interests at various times – in middle school, at the start of high school, at the end of high school, at the start of college, at the end of college and what career they ultimately entered. Interests were coded as a STEM or non-STEM career interest according to definitions established by the National Science Foundation (National Science Foundation [NSF], 2013).

Table 2. Alumni responses to attitude toward science items (1 = Strongly Disagree, 5 = Strongly Agree).

Prompt	Means
	Female (N = 110)	Male (N = 57)
I would enjoy visiting a science museum at the weekend.	4.60 (.66)	4.68 (.54)
I am curious about the world in which we live.	4.58 (.82)	4.84 (.37)
I like to listen to people whose opinions are different from mine.	4.38 (.76)	4.34 (.67)
A job as a scientist would be interesting.	4.24 (.84)	4.11 (.92)
I would like to work with people who make discoveries in science.	3.96 (1.11)	4.25 (.85)
Working in a science laboratory would be an interesting way to earn a living.	3.93 (1.10)	3.74 (1.23)
In science experiments, I like to use new methods which I have not used before.	3.88 (1.00)	4.11 (.82)
I would like to do science experiments at home.	3.81 (.96)	3.89 (.94)
I would like to be given a science book or a piece of scientific equipment as a present.	3.52 (1.23)	3.79 (1.22)
I get bored when watching science programs at home or on TV.	1.18 (1.10)	0.72 (.82)
I dislike reading newspaper articles about science.	1.15 (1.16)	0.70 (.94)
I would dislike becoming a scientist because it needs too much education.	0.97 (1.07)	0.84 (1.07)
A career in science would be dull and boring.	0.65 (89)	0.63 (.86)
I dislike listening to other people’s opinions.	0.51 (.79)	0.75 (1.04)
Finding out about new things is unimportant.	0.46 (1.06)	0.58 (1.28)
Composite	2.79 (1.64)	2.80 (1.73)

A final section included a new scale developed by the research team to measure how respondents recalled the hybrid nature of the program. On a ten-point scale, respondents were asked to compare three aspects of the program, the physical space, social environment and staff members, to their experiences at home, at school and with their friends – categories chosen to represent the three elements that teens pull from when creating hybrid spaces (Table 3). The survey also had other items used as evaluation measures to help inform future program design, but they were omitted from this analysis.

Table 3. Alumni responses to hybridity scale (1 = Less, 10 = More).

Prompt	Options (10-point Likert scale)	Means
		Female	Male
The Science Minors and Achievers program physical space felt…	like being at home	5.07 (.25)	5.17 (.33)
	like being at school	5.78 (.22)	5.76 (.32)
	like being with my friends	6.98 (.25)	7.51 (.32)
The Science Minors and Achievers program staff were…	like my parents or guardians at home	5.20 (.29)	4.40 (.31)
	like my teachers at school	6.25 (.24)	6.36 (.28)
	like my friends	5.50 (.26)*	6.35 (.33)*
The Science Minors and Achievers program social environment was…	like my social life at home	4.96 (.28)	4.92 (.36)
	like my social life at school	6.52 (.22)	6.48 (.32)
	like my social life with my friends	6.66 (.24)	7.13 (.31)

^*p < .05.

Semi-structured interviews

Two rounds of interviews were conducted in support of this study. The first was a semi-structured interview designed to look for deeper meaning behind the answers to the survey questions (Appendix A). Interviews were conducted via phone by 5 different interviewers, and took about 30 minutes to complete. They were done 1-3 months after participants returned the surveys. The second round of interviews were conducted with new alumni who graduated from the program one year after the first round of interviews. The goal of these new, more open-ended interviews was to probe feelings and memories closer to their graduation from the program. Additional topics explored in this round of interviews include childhood/family background experiences, relationships with key adult figures (e.g., teachers, parents, and SMA program staff), social groups, and future aspirations (Appendix B). Each of these interviews took 1-2 hours to complete and they were conducted in-person. All interviews were audio-recorded and transcribed. All names used in this paper are pseudonyms.

Analysis & results

Surveys

Survey data were analyzed using techniques from the General Linear Model to look for differences among groups. The TOSRA Likert data were first converted into ordinal data (Strongly Disagree = 0 to Strongly Agree = 5) with reverse items coded accordingly. Next, they were run through the Rasch Rating Scale Model using Winsteps 3.92.1 to convert into an interval format (Linacre, 2002; B. D. Wright & Masters, 1982). A mean TOSRA score, now in the form of logits, was calculated for each respondent. Statistical significance for our tests was set at the p = .05 level. Overall TOSRA mean scores were high (median raw score was 4.2). We found no statistical difference between male and female respondents (Table 4). We also found no relationships between the TOSRA mean scores and age, SES, race/ethnicity, or final career choice.

Table 4. Analysis of variance (ANOVA) of attitude toward science scores between groups.

	df	F	η	p
Age	9	1.21	.200	.312
SES	5	.425	.050	.829
Race/Ethnicity	9	.449	.084	.900
Final Career Choice	1	2.74	.036	.102

Note. N = 159.

The hybridity scale data was coded depending on which number the respondent circled. Responses to the hybridity scale were clustered around the middle. In general, the physical space, staff and social environment were often rated as being more similar to experiences they had with friends than experiences at school or home. There was a statistically significant difference between genders on one hybridity item - male respondents rated the staff as being more like friends than female respondents, F(163) = 3.95, p < .05.

Changes in career interest were compared using descriptives and frequency analysis. A little more than half of respondents recalled already being interested in a STEM career when they entered the program (Figure 2). Compared with male respondents (9%), female respondents (18%) showed greater increases with regard to interest in STEM careers while in the program. After leaving the program and entering college, both reported declines in interest but the decline of female respondents was sharpest, which is consistent with other studies of college STEM student trajectories (Hill, Corbett, & Rose, 2010).

Figure 2. STEM career interest by gender.

Interviews

Interviews were analyzed by looking for themes related to results we found in the survey data. Specifically, we focused on themes related to how respondents described their relationships with staff and how identities and agency were perceived. References to staff in the interview transcripts were coded by a single researcher according to whether they referred to staff as mentors, teachers, friends or parents - categories chosen from a first pass of emergent coding (Table 5). We found a difference between genders in that male respondents tended to refer to staff as mentors and friends and female respondents referred to staff more often as teachers and parents. Examples of female respondent references to staff follow:

Table 5. Characterization of participant-staff relationships.

Code	Male alumni (%)	Female alumni (%)
Mentor	75	10
Teacher	25	75
Friend	50	20
Parent	0	20

Aurora:

At the time I would have definitely looked at them as teachers since they were teaching us material.

Yvette:

…personality wise less like school teachers but they taught us everything, like the school teachers.

Therese:

They were like the best school teachers … it was also like a friendly relationship like we could laugh and joke and I could share my science jokes and I put stickers on their back and stuff. But I definitely looked to them as like a knowledge base if I had any questions about life, or school, or the curriculum…

Mila:

[They were] like teachers in the sense that they actually taught us what we knew.

Interestingly, while female respondents discourse concerning staff as “more like teachers” emphasizes the manner in which they [the staff] serve as knowledge brokers, it also highlights a certain degree of comfort and familiarity (e.g., Therese). Indeed, female respondents often adopted fictive kinship terminology in order to describe the role of and their relationship to staff (Jarrett, Jefferson & Kelly, 2010; Loyd & Williams, 2017). As one young woman reflected, “I mostly just remember them being really supportive and almost acting like a family.” Brandy, another former program participant, described her experience being involved in the program for such a long period of time (14–18 years old).

[T]o be in the program during those specific years, and to have people around you growing up. For the adults to see you grow up and changing. And seeing you every Saturday. I think it definitely does create a family feeling. Whether or not, some people who don’t show up that often, for maybe other reasons may not feel as close. But for those of us who consistently show up and see [refers to two staff] every Saturday, it’s definitely like a feeling of, “These are my people.” I know I could count on them if I had a problem, if I have a question.

As a matter of fact, for some young people it was precisely the semblance of family that drew them to the program in the first place and kept them coming. Many describe desiring a sense of belonging and wanting to create a community outside of their home and school life. One woman remarked, “Basically [it’s] like a family outside of your house … like a big family.”

Further, female respondents often described staff as occupying specific familial roles. When asked to describe the staff, one young woman conveyed, “I thought they were pretty nice. I like that Mr. Smith. He was like the grandfather from the program. We really liked him.” When the interviewer probed further, asking why the respondents liked Mr. Smith, she responded, “Because Mr. Smith was basically there since like day one. He started out in the minors program and if we had any problems or ever needed to talk to him he was there for us. He got things done … .” (Mila). She added, regarding a different staff member, “Miss Rivera, it’s like she looks, just like my mother, like fitting in just like my mom … ” (Mila). Another former female program participant explained, “Miss Rivera is like the mom.”

On the other hand, males were more likely to describe a mentoring relationship and also rarely mentioned the role of staff in serving as sources of new information. However, they did also include familial references.

Arnold:

[The staff] you know, they referred to them as by their first name so it wasn’t as, it wasn’t like a teacher student relationship at all … It was more like a buddy-buddy relationship … But you still look up to them as mentors you know like your big brother.

Jessie:

It was just the people. You know, honestly it was people, who, who I was around. You know. So, Mr. Jones, who’s an amazing, amazing guy, like maybe even a mentor to me, I honestly. He’s like a grandfather to me.

Beyond increased interest in the program, participants also described increased science interest and career pursuits due to staff encouragement and support. Tiana, who emphasized that she never wanted to go into teaching precisely because the majority of her family members are teachers, now wants to teach science; Manny, who obtained a scholarship to attend an ivy league university, has begun to consider engineering as a career due to his involvement in the program. Similarly, Cam didn’t know anything about engineering until his involvement in the SMA program and now he has decided to pursue it as a career. Ray was emphatic that were it not for the program, he would not be doing engineering. Andy remarked,

[B]eing a Science Achiever allowed me to see what kind of science I was really interested in, and what type of science I wanted to do [ … ] I became more interested in astronomy, I became more interested in computer science, to an extent where I want to make a career out of it. I became more interested in innovation, and wanting to design things, to not just, you know, benefit myself but to benefit society.

For this young man and others like him, the program either instilled in him the desire and/or offered the opportunity to construe his involvement as, in a sense, an act of citizenship building or civic engagement.

Herein we come to the concept of “thriving,” or the intention and ability “to integratively serve self and society” (Lerner et al., 2003, p. 176), as an embodied experience for SMA program participants. Scale and his colleagues elaborate that “[e]mpirically-supported dimensions of thriving orientation include prosocial behavior such as civic engagement, positive emotionality, openness to challenge and discovery, and a sense of hopeful purpose, all of which are reflected in the dimensions of successful young adult development we have put forward” (Scales et al., 2016, p. 166; see also Benson, 2003; Benson & Scales, 2009; Lerner et al., 2002; Scales & Benson, 2004). In fact, according to Andy, “[T]hat was something that was really important about Science Achievers, that you gain an outlook where you can not only make an impact on yourself by going out and getting a career in science, but also making an impact in society and in the world by discovering something.” He connected this desire to his positive experience with the program staff:

I think it was, a lot of it going back to the internship² last year when I, when I was [playing a Texan in a stage performance] everyone would be like imitating my accent cause they thought it was really good … Ms. Rivera just loved it, and she [was] like, “You’re gonna motivate a lot of kids like this.” And at that point it really mattered that I was a big part of this show, and I was gonna amaze all these kids, and, like, science was really cool. […]So that’s why volunteering at the Museum of Science and Industry, and motivating a kid to become a scientist and doctor, to find the answer to society’s problems, is very important.

Time and again, former SMA program participants express a form of altruism and a desire to “make a difference.” Frequently, these desires to “give back” or “make a difference” entail young people envisioning themselves in leadership positions. For example, Brandy wants to establish a Chicago-based non-profit organization that addresses the lack of girls and young people of color in STEM fields: “I do want to target underprivileged communities […] the obstacles they face. Adults need to be held accountable for making sure they succeed. Specifically in STEM, but also just in general.” She directly connects her desire to create a STEM-based non-profit organization that serves “underprivileged communities” to her involvement in the SMA program, specifically her own observations of staff:

I think it was when I started [the Program] … and I saw [a staff member] and her computer and I saw [another staff member] in his office. I was kind of looking around and started realizing, this is a business. We’re obviously coming to the museum, but these people are actually running an organization. And I was like, “I can do that, too.” And I felt this confidence, that I have been going through this program for so long and accumulated so much knowledge about how this was going to work, and I have the passion for it, that I can definitely recreate this experience for people in my own way.

Brandy’s statement reveals not only her desire to apply her STEM/science skills in order to enhance opportunities for underrepresented minority youth, but it also demonstrates a keen sense of confidence and self-awareness as an active agent, capable of generating change and making a positive impact in the world in which she lives.

Interview narratives also speak to the developmental concept of thriving in that program staff make a concerted attempt to build on young people’s pre-existing interests and passions and, more generally make them feel important and that they (and their opinions, interests, etc.) matter. These relationships, however, and more specifically staff encouragement and support, affected female participants in distinct ways—namely, their self-awareness around what it means to be a girl who “does” and is “good” at or “into” science. Pia described the staff,

They are mostly just guiding, and encouraging […] So I think maybe with different people, it … you wouldn’t get as many, you know, as many teens that interested, or excited, or willing to go out and talk to guests or anything like that.

She then provided an account of her own experience, noting that she “hated learning science in school” and found it “boring.” But, then, through her involvement in the program,

[N]ow I feel capable enough to, um, explain things to people. Um, even just having conversations with my friends sometimes now like I’m the science girl, ask her a question if you have any, you know. That’s kind of where I am now. I just feel like I’m more capable now as a teacher, too, as well.

Another respondent echoed similar sentiments, noting that she learned from the program “I can do this whatever it is, so science, engineering … ” She continues,

I remember in youth development³ we talked about women in STEM and I didn’t really feel like that was a big deal, but when I looked around and when Ms. Rivera was talking about it, it was, like, girls are given instantly Barbie dolls. And so now you see that the whole world is shifting to giving girls who are engineers Barbie dolls that are engineers. [And] there’s all these new Nickelodeon shows that my sister watches that is like about girls who are scientists. So basically the program helped me see that I can actually do this [italics added for emphasis].

Interview data also support how program participation transformed young girls’ relationship to science and, subsequently, career development. Izzy, a 22-year old female, talked about how her involvement in the SMA program changed science as a hobby for her to science as a viable career pursuit. She explained, “For a long time, science for me was more like the fun thing—that, like, fun hobby. And not what I was thinking career-wise. And so definitely having that [program] background made me feel more confident in pursuing it as a career.” We interviewed her just as she was wrapping up her undergraduate degree in forest science. While Izzy had a pre-existing interest in science that was transformed by the program, for others the program altered their disdain or indifference of science. Ali who identified as mixed-race reflected, “I didn’t really care about science when I first started [the program] … Because at first I thought science was kinda hard and just—I couldn’t understand it but, like, being at the Museum, it definitely helped me gain a better understanding of science topics.” At the time of the interview, Ali was completing her undergraduate degree in Math and Elementary Education, expressing a desire to teach math and science to elementary school children after graduation.

Indeed, examples abound of the impact that the program has had on its female participants. Britney, a 20-year old female paramedic, shared her views on how the program affected her, “I feel like it kind of pushed me towards what I’m doing now.” And, on the verge of graduating with a degree in neurobiology, 24-year old Tammy of mixed descent stated matter-of-factly, “This program started my love of science […] I think it was my real foundation for a love of science.” For Lena, a Latina female in her first year in college, her involvement in the SMA program completely shifted her interest in and relationship with science, particularly her ability to conceive of herself as a “science and math person.” In her own words,

[C]oming to the museum definitely opened up my love of science and math more than I thought it would. And it’s helped me realize that I don’t have to be a singer just ‘cause I went to a school for singing for four years, and that I can go with minimal science class [experience] … I didn’t even take Physics. And I can still go to a school [college] for chemical engineering and still get into it because it’s something I have a passion for. And if I’m willing to learn and teach about it, then I can do anything when it comes to what I like. And what I like is science. And what I like is math. And the museum kind of just helped me become comfortable with the fact that I’m a science and math person, and I enjoy it.

By virtue of her involvement in the SMA program, Lena’s relationship to science and math is altered, where she is has come to embrace the fact that she enjoys science and math or, as she puts, “become comfortable with the fact that I’m a science and math person.” She also expresses the belief that as long as someone has a desire to learn something, then they can and will learn it. Her statement reflects a shift in confidence around her own capabilities as well as the expansion of opportunity and possibility vis-à-vis her involvement in the SMA program.

Discussion

The most striking finding of the study is the difference in STEM career interests between female and male alumni while they were active in the SMA program. The SMA program model is not designed with any gender specific programming in mind and staffing was consistently gender balanced. This increase in female alumni interest is at odds with studies showing that, in general, STEM career interest of female high school students in the United States tends to drop more rapidly than male students during their high school career (Sadler, Sonnert, Hazari & Tai, 2012; Young, Ortiz, & Young, 2017).

So what could be the reasons for the increased STEM career interest by female participants? As measured by the TOSRA scores, we found no difference in reported attitudes⁴ towards science careers between the male and female respondents. Smist, Archambault, and Owen (1994) found gender differences among high school students using the same survey instrument, but that was with all survey items (compared to our focused subset) and in a different STEM educational climate. More recent studies using the same instrument with middle and high school students have not found gender differences (Ha, Cha, Kim, & Lee, 2007; Tegtmeier, 2009). Also, any underlying difference in attitudes between genders would more likely affect the absolute levels of STEM career interest, not the relative change in interest that we see. In the hybridity scores, we found no gender differences in how participants rated the physical space, social atmosphere or any relationships with current attitudes toward science.

Where we did find a difference was in how participants perceived staff. On the hybridity scales, female participants rated staff as being less like friends and more like teachers than male participants. This is strongly supported in the interview data where female participants were much more likely to compare staff to teachers and family. This supports results from Maltese and Tai (2010) where female graduate students in science were more likely to attribute their interest in science careers due to school-related experiences than male students. Also, recall that male students were more likely to describe staff as mentors. This is contrary to some studies suggesting that boys are less open to mentoring than girls (Liang, Bogat & Duffy, 2014) and that male youth tended to have fewer or less close mentoring relationships with informal mentors—those that naturally arise due to social connections (Liao & Sánchez, 2016). Mentorship in this program is more formal, so our results could suggest that male youth were more open to mentor type relationships in this setting. Overall, our results add mixed-method data to support Gupta and Negron’s (2017) recent phenomenological-based findings that emphasized the critical importance of caring and trusting staff-youth relationships in OST programs.

Implications for programming

Our data show the role of staff is quite complex and goes far beyond their responsibilities as facilitators of a program or even as educators. When serving as supportive adults invested in the participants’ lives they take on characteristics of the best teachers and mentors. Our findings show that female participants, in particular, responded strongly to that level of personal relationship building. This suggests that a robust OST program with a goal to increase STEM participation of females should invest in a model that supports strong and personal staff relationships. Staffing in OST programs tends to be dominated by younger staff with high levels of turnover (Vandell, Simzar, O’Cadiz, & Hall, 2016). This is significant because frequent staff turnover is a common challenge in OST programs (Bevan & Michalchik, 2013) and can subvert program continuity as well as long-term adult–youth relationships (Adams, Gupta, & Cotumaccio, 2014). Meanwhile, at the time of data collection SMA program staff all had college degrees with an average of 5 years working in the program. More generally, at least half of the staff had been with the Museum for ten years and all have been working full-time directly with youth for over a decade. Also, historically, the program has been staffed by people of color, reflecting the racial/ethnic backgrounds of many program participants that are frequently underrepresented in STEM/science fields.

The PYD raison d’etre that young people are resources to be developed (Benson & Scales, 2009; Lerner et al., 2002) is reflected in SMA youth participants’ accounts of their first-hand experiences in the program. Participants demonstrated a high degree of self-reflection that connected the four PARC elements—programs, activities, relationships, culture—of the program to their increased confidence and self-awareness as young adults. For the purposes of this paper, however, we focused on the “relationships” element, given its relevance to our findings on the impact of staff-participant relationships on positive youth development.

STEM OST programs may want to consider whether their program, like ours, is first and foremost an adolescent development perspective as opposed to a science education program. The SMA program takes place in the Museum of Science and Industry, Chicago, and scientific content is embedded in the program model. But the goal of the program is not to solely recruit teens into STEM careers; it is to develop young people. The staff sometimes tell the teens, “We are here to grow you.” The level at which this focus on personal growth writ large was reflected in the study findings was surprising. Despite being aimed at measuring various science outcomes, the impacts we found were more related to positive youth development and becoming better prepared for life after high school.

However, science was not absent—it was in the background providing a soundtrack to the program. Attitudes toward science among alumni was high and interest in STEM careers was consistently much higher than the national average, even accounting for the variability between genders. More importantly, the alumni data and conversations that led to the development of this framework were almost always within the context of science. When they talked about staff as teachers, female participants talked about how they were sources of scientific content knowledge. When they talked about the importance of the Museum as a physical space, they talked about specific scientific totems, such as the tesla coil or the 90 ft vortex. When they talked about the importance of like-minded friends, they referred to the fact that it was a safe space to be a science nerd. In this way, science is the medium by which teens navigated the program. Others have also found that science can be a mediating resource through which children learn more about themselves (Oztok & Arvaja, 2016).

Much of the best practices of science education are not mutually exclusive with the best practices of the positive youth development field. Our findings suggest that STEM-based OST programs that incorporate elements of PYD can have a significant impact in altering URM youth participants’ relationship to science. We acknowledge, at the same time, that resources in community-based and/or youth programs tend to be limited and we suggest allowing adequate investment in PYD strategies⁵. For example, a study of 300 science focused OST programs found only 59% offered professional development opportunities for their staff (Dahlgren et al., 2008), despite the fact that it has been identified as a key factor in supporting consistent program effectiveness (Feder & Jolly, 2017). Gender differences in science interest begins to emerge at the start of middle school and continues through high school (Tai, Liu, Maltese & Fan, 2006). This study shows that an OST program with PYD characteristics can attenuate or even reverse that trend. Science themed OST programs have shown to have an impact on scientific content knowledge and enthusiasm (Newell, Zientek, Tharp, Vogt, & Moreno, 2015). This study also shows its impact on career interest, even when science is not at the forefront of the program.

Implications for research

There have been calls for more research on specific components that make some OST programs more successful than others (Durlak, Mahoney, Bohnert, & Parente, 2010; J. N. Jones & Deutsch, 2011; NRC, 2015). Based on our findings, we argue that one such component that requires further research is the role of staff in OST programs (Theokas et al., 2005). Such research would contribute to studies on “differential effectiveness” or the notion that some youth may benefit from participation in a program more so than others (Greenberg & Lippold, 2013). Moreover, such research may also help address issues of race/ethnicity in PYD programs (Williams & Deutsch, 2016). In fact, Scales et al. (2016) argues:

“ … that developmental relationships with teachers and other adults have the potential to provide authentic empowerment of youth of color, working-class, and lower-income youth, by increasing their access to those kinds of relational influences that go beyond caring, to helping those young people stretch, expand, and become more savvy and powerful in the workings of the world. That is, such developmental relationships, useful for all youth, may be especially relevant for increasing the social capital that helps low-income students, students of color, and other historically marginalized young people have more options for dealing with these systemic limitations on their opportunities and making a successful transition to young adulthood.” (p. 156).

This is a retrospective study, but there is strong demand for future looking longitudinal studies of OST program outcomes. The fact that many programs are not multi-year frequently poses a challenge to researchers seeking to examine long-term impacts (Adams et al., 2014, p. 14). Moreover, multi-year programs provide important context and opportunity to examine over time the interactions between youth and their environment (Williams & Deutsch, 2016). Few researchers study the adult-youth relationship in OST programs empirically, and almost none longitudinally (J. L. Roth & Brooks-Gunn, 2016). Pertaining more specifically to the concept of thriving, Lerner et al. (2002) emphasizes, “Because thriving is a process concept, longitudinal analysis is needed to adequately appraise whether there is evidence that patterns of covariation exist over time in a manner reflecting the growth of a person-context relations promoting individual health and civil society” (p. 25). Larson (2000) echoes a similar sentiment, maintaining that “[t]he claim that structured youth activities promote positive youth development outcomes is more adequately tested by longitudinal studies” (p. 175). Also, he as well as many others call for the use of control groups where feasible. J. L. Roth and Brooks-Gunn (2016) underscore that “findings about program effectiveness from non-experimental designs are much less likely to be accepted as valid by researchers in other field or by policy makers […] Demonstrating effectiveness to these stakeholders requires more than descriptive data, no matter how compelling such data may be” (p. 198).

Limitations and future directions

This study’s main limitation lies in its retrospective design. First, those with positive memories are more likely to participate in surveys and interviews. To mitigate the effect, we offered substantial incentives and specifically requested help from those with negative memories, but it is still likely those with poor experiences probably did not participate at the same rate as others. Second, with a lack of a comparison group we cannot attribute the overall findings to the program itself. For example, the high science attitude scores are likely related to already high science interests alumni had when they entered the program. This is a common limitation of studies involving OST programs (Young et al., 2017). With these limitations in mind, the Museum has partnered with the University of Virginia Curry School Of Education to begin a study of current and future participants of the program. This new project addresses the field’s charge to integrate longitudinal research design, ethnographic data collection methods, and an experimental design. Referred to as The Developing YOUth! Project,⁶ it is following three cohorts of program participants for at least five years after they have left the program.

Conclusion

This study showed that an OST program that pays special attention to staffing can have an impact on career interest of their participants. This is especially true of the impact on adolescent girls, who report a different type of relationship with staff than the adolescent boy participants. Perhaps as a result of this relationship, they show a striking increase in STEM career interest compared to their male peers. Such OST programs, aligned with the best practices of PYD, could hold a clue to equalizing STEM opportunities for all youth.

Additional information

Funding

Dr. Price’s and Dr. Kares’ work was supported in part by National Science Foundation award DRL#1514593. Dr. Loyd’s work on this manuscript was supported in part by Grant Number K12HD055892 from the National Institute of Child Health and Human Development (NICHD) and the National Institutes of Health Office of Research on Women’s Health (ORWH). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Child Health and Human Development or the National Institutes of Health.

Notes

¹We recognize that gender is not always recognized as a binary, thus we asked the question using an open-ended prompt (“What is your gender?”). In this study, all alumni responded with a male or female identification.

²The Farrell Fellows Summer Internship is a paid, full-time summer internship at the Museum led by the SMA program staff. While not formally a part of this study, the majority of participants in the program are current or former SMA participants. Andy is referring to a time when the interns performed an original theatrical show on the science of bubbles.

³A typical program day is broken up into “modules” around specific themes, one of which is youth development.

⁴This is commonly conflated with personal interest, but in this study we consider those separate constructs.

⁵Summaries of best practices of the PYD field can be found in Catalano et al. (2004) and J. L. Roth and Brooks-Gunn (2003).

⁶https://www.msichicago.org/education/research-and-evaluation/developing-youth/.