Do outreach activities in secondary STEM education motivate students and improve their attitudes towards STEM?

ABSTRACT

The present study investigated outreach activities, developed by STEM-based companies or universities in co-creation with secondary education with the aim to inform students about and motivate them for a career in STEM by connecting the work-context with school-science. Although many of such activities are being offered, little is known about their effects. We investigated students’ perceptions with the outreach learning environment, perceived need-fulfilment, self-reported motivation and attitudes towards STEM. Data were gathered from 729 high-school students engaged in 12 activities in the USA and the Netherlands. The students completed a questionnaire, which contained questions about four elements of our theoretical frame based on the Self-Determination-Theory (SDT). Perceived needs-fulfilment and motivation were measured using the basic-psychological-needs-scale and the self-regulation-questionnaire. Attitudes were measured using the test-of-science-related-attitudes. Learning environment perceptions were measured in a previous study using subscales of what-is-happing-in-this-classroom (WIHIC), constructivist-learning-environments-scale (CLES) and classroom-environment-scale (CES) and typified by activity characteristics. Multilevel analyses of variance were conducted for the two motivation scales (controlled and autonomous-motivation) and the two attitude scales (social-implication and career-interest). Activity characteristics explained almost all variance in these variables between activities. Specific characteristics of outreach activities that statistically significantly related to autonomous motivation and positive general attitudes towards STEM were: workshop-format, understanding science, an out-of-school component. The attitude towards a possible STEM-career was positively associated with autonomous-motivation and negatively associated with controlled-motivation. Thus, outreach learning environments indeed created opportunities to increase students’ motivation in STEM and attitude towards STEM, but the impact varied according to particular characteristics of the activities.

KEYWORDS:

• Outreach
• STEM
• motivation
• subject-related attitudes
• secondary education

Introduction

Today’s society is based on technology and information. As technological knowledge and expertise is getting more specialised and economically more important, more and more jobs in the STEM domain are needed and this demand is expected to further increase in the upcoming years (OECD, 2008). In addition, STEM-based companies emphasise that as a result of the increased complexity of technology, employees need on the one hand new knowledge and on the other hand twenty-first century skills, such as flexibility, multidisciplinary problem-solving, communication, punctuality, critical thinking and creativity (Binckley, Erstadt, Herman, Raizen, Ripley, Miller-Ricci, & Rumble, 2012; Voogt & Roblin, 2010). As a response to these demands, secondary education is adapting STEM education (Ormond & Zandvliet, 2016). First, projects and courses are (re)designed with an emphasis on these twenty-first century skills. Second, STEM courses are linked to experiences and examples outside school to promote STEM to provide a context to apply these skills. One of the main objectives of these efforts is to increase the number of students in STEM related courses and to keep these students motivated for a future career in STEM to meet the demands for the workforce. To increase the number of students in STEM, and attract a wider variety of students for STEM, secondary education and STEM-based companies work together in outreach activities. In the present study we propose outreach as a possible means to achieve this.

Several contemporary STEM curricula intertwine real life contexts with the school context. During the last decade universities and colleges have added activities to the curriculum to smoothen the transition between secondary and higher education, for example by developing web-lectures, summer science programmes and student mentor programmes (Hayden, Ouyang, Scinski, Olszewski, & Bielefeldt, 2011; Jeffers, Safferman, & Safferman, 2004). Recently, STEM-based companies and universities or colleges have developed, together with secondary education, activities (both in-school and out-of-school) with the objective to inform students about or motivate them to pursue a career in STEM by connecting their STEM context with school science: outreach activities. These activities, so-called outreach, are defined as activities or processes of bringing information or services to people beyond usual limits (cf. Merriam-Webster dictionary). The activities developed with the industry or universities can thus be regarded as outreach activities. In the present study, the outreach activities or processes are developed in cooperation between secondary education and a STEM-based company (either public or private) or a university with information (or service) added that usually is not available for teachers and students (Vennix, den Brok, & Taconis, 2017). In these particular activities employees of STEM-based companies have an active role in supervising or guiding the activity.

Within this definition a diversity of activities is possible, such as sharing expertise and another perspective in guest-lessons, hands-on activities in workshops, showing workplaces and doing activities at the work environment, initiated and supervised real company based problems conducted at schools or at the company (Vennix et al., 2017). These activities are often additional to the regular curriculum. Overall, main characteristics of outreach activities are the active role of employees of a STEM-based company and the real life context. Although outreach activities often concern learning outside school, and may be seen as informal learning, learning is still structured with a desired outcome (Jarman, 2005). What most of the activities have in common is that a person out of school is involved (i.e. either a university student or an employee), and that high school students have the opportunity to connect to them. As the content is based on real life contexts, the students have the opportunity to explore their interests and abilities. As there is often no pressure to perform for a grade, students have some choice to attend or to choose a subject to work on. Therefore, outreach activities create a specific learning environment for students, with both formal and informal learning that may possibly increase students’ motivation for getting involved in STEM activities (Suter, 2016).

In an earlier study we were able to map the students’ perceptions of this particular learning environment with a validated questionnaire (Vennix et al., 2017). In that study, we collected data about different activities to be able to link specific activity characteristics to the most positively perceived activities. Students had an overall very positive perception of the outreach learning environment. Hands-on activities, activities with the goal of understanding science and obtaining new views of science and scientists were characteristics that contributed to a positive perception of the learning environment. Even though outreach activities apparently were perceived very positively, the question remained whether they actually motivated students for STEM and whether they increased the attitude towards a future science career. Positive perceptions of the learning environment are indicators of a potentially positive attitude towards STEM, and of high motivation to engage in STEM learning (Fraser, 2007). Adding outreach activities to the regular curriculum would potentially increase students’ intrinsic motivation to learn. As suggested by Deci and Ryan (2000), assuming that more positive perceptions of learning environment aspects would lead to higher satisfaction of the three basic psychological needs of relatedness, autonomy and competence (Black & Deci, 2000; Jang, Reeve, & Halusic, 2016; Lavigne, Vallerand, & Miquelon, 2007), one would expect high intrinsic motivation. From qualitative research about outreach programmes of universities and colleges, successful outreach activities were hands-one activities, free of choice and mentoring programmes (Cooper, Dann, & Harrison, 2010; MacLeish, Akinyede, Goswami, & Thomson, 2012). However, little if any empirical research has been conducted to actually investigate the link between outreach activities, motivation and attitudes towards STEM. Therefore our main research question in the present study is: What are the associations between students’ perceptions of the outreach learning environment, their needs fulfilment, and their motivation and attitudes towards STEM?

Theoretical framework

As the outreach activities in the present study typically have the objective to motivate students, our main focus in this study will be on motivation. In our overall conceptual framework four different concepts are assumed to relate to each other: the outreach learning environment, psychological basic need fulfilment, motivation and attitude towards STEM (Figure 1). In the next sections, the four aspects in our conceptual frame will be described in more detail and related to one another based on the self-determination theory by Deci and Ryan (2000).

Figure 1. Conceptual framework concepts.

Learning environment

Outreach activities relate the context to the classroom learning environment by showing applications in practice and such activities typically have a number of characteristics (Vennix et al., 2017). The first important characteristic is the context of these applications: this can be scientific, or more or less connected with either a community or a company context. The second characteristic is the location: for some activities students visit companies where they will sometimes meet up with students from other schools. As this is not always possible due to safety or quantity, some STEM-professionals will visit the students at school and for longer projects a combination of locations is also possible. The third characteristic is the objective of the outreach activity. The objective can vary from just trying to enhance interest in science to showing new views or understanding of science. The fourth characteristic is the main teaching method: this can vary between lecturing to mentoring students in project work. The fifth characteristic is selection: for some activities students need to be selected, due to company capacity and safety reasons, but for others there is unlimited access or participation.

In learning environments research the educational context is described and relationships between content, teaching practices and environmental variables are studied (Fraser, 2007). Several studies have provided evidence that students’ perceptions of the learning environment are linked to outcomes such as interest and attitudes (Baeten, Dochy, & Struyven, 2013; Kingir, Tas, Gok, & Vural, 2013; Ogbuehi & Fraser, 2007).

In several learning environment studies researchers have used the dimensions of Moos (1980) to characterise and map the learning environment: personal growth, relationship and system maintenance and change. As argued by Vennix et al. (2017), there is a relationship between Moos’ learning environment dimensions and the dimensions (needs) conceptualised in motivation research. These basic motivational needs are relatedness, competence and autonomy (Deci & Ryan, 1985). The need for autonomy refers to the extent to which students feel volitional in their decision to engage in academic activities (Deci & Ryan, 1985; Ratelle & Duchesne, 2014; Vansteenkiste, Sierens, Soenens, Luyckx, & Lens, 2009). The need for competence refers to the extent to which students understand and have the relevant skills to succeed (Deci & Ryan, 2000). The need for relatedness refers to the need to establish significant and satisfying relationships with others (Baumeister & Leary, 1995). The need for relatedness has conceptual overlap with Moos’ dimension of relationship, the need for competence has overlap both with relationship and personal growth, and the need for autonomy overlaps with bot personal growth and system maintenance and change (Vennix et al., 2017). All dimensions in both constructs should be equally present in the learning environment in order to facilitate both social and academic outcomes.

In the learning environment, teacher behaviour, activities and educational material may support the satisfaction of the basic needs (relatedness, competence and autonomy). Several dimensions in the learning environment may contribute to needs fulfilment, and learning environments research has provided a variety of concepts that describe these dimensions. To motivate students an important objective is that students’ are challenged to ask themselves: ‘Is this what I want to do in the future? Is this topic worth to study in high school to reach my future goal?’ Thus, outreach may give in potential students’ self-direction (Jang et al., 2016). Whenever the purpose of an outreach activity becomes meaningful to a student, it will contribute to the autonomy of that student (Assor, Kaplan, & Roth, 2002; Hardre & Reeve, 2003; Reeve & Jang, 2006; Vallerand, Fortier, & Guay, 1997). To support the autonomy of a student, (guest) teachers can suggest different options to do an activity. As outreach activities are sometimes short in time (i.e. a guest lesson) choices might be restrained. The autonomy support can still be met both by structuring the activity, giving students a rationale for this and trying to empathise with the students’ perspective (Sierens, Vansteenkiste, Goossens, Soenens, & Dochy, 2009). Students’ will then show more involvement during an activity. When (guest) teachers explain new ideas about science to students, the topics of school science will be extended and the students will be exposed to another way of working in class as outside school experts have a major role in these activities (innovation).

Feelings of competence can be stimulated through setting challenging and suitable goals, while providing informative feedback. This will make students feel empowered and will give them the confidence to explore alternative possibilities (i.e. personal growth) (Boggiano & Ruble, 1979; Deci, 1971). Outreach activities are characterised by adding subjects from real-world science that are connected to the students’ everyday out-of-school experience. This will challenge students and enhance the personal relevance of STEM for students. As real-world science has more uncontrollable variables than school science, students will have the opportunity to experience that scientific knowledge is evolving (i.e. uncertainty).

Commitment will strengthen even further when the teacher or task installs a sense of respect in the student, by for example expressing admiration or giving examples that the student identify with. The teacher then serves as a role model (Sjaastad, 2012): the relatedness need will then be fulfilled. As in outreach students get the opportunity to have short interactions with STEM-professionals by asking questions, the students will be able to establish links between themselves and professionals and narrow the gap between perceived and actual images of these professionals (Woods-Townsend et al., 2016). The extent to which the STEM-professionals or teachers support the students and the supportive interrelationships between students will sustain relatedness.

As shown in the study by Vennix et al. (2017) students’ perceptions of the outreach learning environment in terms of the aforementioned dimensions were more positive for outreach activities than for regular classrooms.

The expectation is that this positive perception of the learning environment is positively correlated with the perceived need fulfilment and this in turn will support motivation to pursue in STEM.

Needs fulfilment

In our study we will use the perceived satisfaction of each need (competence, relatedness and autonomy) as an indicator of the latent variable of the overall need satisfaction (Standage, Duda, & Ntoumanis, 2005). In our conceptual framework we propose that this outreach learning environment supports the fulfilment of the basic needs, and this in turn the motivation of students as argued in the Self-Determination Theory (SDT; Deci & Ryan, 1985, 2000).

Motivation

Motivation is a key function of outreach activities. Motivation refers to the reasons why someone acts or behaves in a particular way. A diversity of motivation theories distinguish between two types of motivation: extrinsic and intrinsic motivation. The reasons for particular action, behaviour or feelings of a person determines if the type of motivation is either extrinsic or intrinsic. Extrinsic motivation is regulated by reasons outside the person such as external rewards (fame or grades) and intrinsic motivation is internally regulated for reasons of inherent satisfaction of a person (Deci & Ryan, 2000; Eccles, 1983).

The Self-Determination Theory (SDT) refines this model of motivation, by posing that students can go through a process of internalisation. When the basic needs, competence, relatedness and autonomy, are met and balanced, this might lead towards more intrinsic motivation (Deci & Ryan, 1985). The different stages of internalisation can be summarised using the concepts of controlled and autonomous motivation. When the students’ motivation is controlled, this means they are so because they need to pass a mandatory test or they need to be present. When a student experiences shame for a low grade or absence, or takes pride in higher grades or participation, it means that they are a little more intrinsically motivated: the introjected level. In both cases the overall work ethic will predominantly be decided by external motivators and is referred to as controlled motivation (Vansteenkiste, Lens, De Witte, De Witte, & Deci, 2004). When the students’ motivation is autonomous, a student attends a class or activity because it serves a future purpose or goal that the student values and because interest of the student is consistent with their task. These last two indicators of motivation allow freedom of choice for the students; they support the autonomy of learners and are referred to as autonomous motivation (Vansteenkiste et al., 2004). Thus, within the motivation levels the model can differentiate between controlled levels and autonomy levels (Deci & Ryan, 2000; Vansteenkiste et al., 2004), which we will use in our study. Both controlled and autonomous motivation can be simultaneously present.

Students are expected to show low levels of self-reported controlled motivation and high levels of self-reported autonomous motivation, while attending outreach activities. First, in outreach activities certain external factors, such as testing, are absent or realised differently compared to regular class work. Secondly, the common objective of outreach is to connect the curriculum to real world problems and future competences and jobs in STEM that are needed. Therefore, the purpose of the activity will be clearer to students than that of regular class work. Overall students are expected to report more self-reported autonomous motivation during outreach activities than during regular class activities.

Outcomes of motivation

As has been demonstrated in previous research, motivation is related to attitudinal outcomes such as future intentions and persistence (Deci & Ryan, 2000; Vallerand & Ratelle, 2002). The more internally regulated motivated students are, the more positive their persistence, future intentions and cognitive engagement (Vallerand, Fortier & Guay, 1997). Autonomously motivated students have a positive attitude towards STEM courses and choosing STEM in the future (Eccles, 1983; Meece, 1990). The main objective for companies to develop outreach activities is to contribute to a positive attitude towards choosing STEM in the future.

In outreach activities companies provide information about the applications of science and they show students a glimpse of their work environment. As such, they are expected to contribute to the attitude towards the social implication of STEM. Second, outreach activities are expected to enhance students’ awareness of possibilities for STEM careers (Barmby, Kind, & Jones, 2008). Research in attitudes towards science distinguishes several attitudes (Fraser, 1981). As outreach activities have the specific objective to motivate students for STEM, the focus will be on the student perception of social implications of science as this refers to the connection to real life and on their interest in following a STEM career as this refers to the specific dimension of the outreach learning environment.

In the literature, several studies have found relations between (regular) perceived learning environments on the one hand and motivational and attitudinal outcomes on the other. Effective classroom management, such as adding structure, has been positively related to development of subject-related interest (i.e. math) (Kunter, Baumert, & Köller, 2007). In other studies, learning environment perceptions of regular classrooms, such as teacher behaviour, were related to motivation (Kunter et al., 2007; Opdenakker, Maulana, & den Brok, 2012; Deci & Ryan, 2000). These studies showed that teachers’ involvement was found to increase students’ feelings of relatedness, which in turn had a substantial impact on motivation and academic engagement (Opdenakker et al., 2012; Pavey, Greitemeyer, & Sparks, 2011; Stroet, Opdenakker, & Minnaert, 2013). Further, needs fulfilment (relatedness, autonomy and competence) was found to be positively related to intrinsic motivation (Kunter et al., 2007).

The above reported relationships between learning environment perceptions and motivational and attitudinal outcomes were obtained in regular classrooms. No empirical research is known by the authors about these relationships in outreach learning environments. In our present study the purpose was to investigate to what extent attitudes toward STEM and motivation during outreach activities are determined by need satisfaction and the learning environment.

Research questions

This study investigates the associations between students’ perceptions of the outreach learning environment and need fulfilment on the one hand, and self-reported motivation for STEM and attitudes towards STEM and future STEM career intentions on the other hand. The following more specific research questions are investigated:

1. What are students’ self-reported controlled and autonomous motivation while performing different outreach activities?

2. What are students’ attitudes towards STEM and future STEM career intentions after having performed outreach activities?

3. To what degree are students’ attitudes and motivation different between different (types of) outreach activities?

4. What relationships exists between, on the one hand, self-reported autonomous and controlled motivation, attitudes towards the social implication of STEM and career interest in STEM, and, on the other hand, characteristics of outreach activities, perceptions of the outreach learning environment and perceived needs fulfilment?

Research method

Sample

As this study is part of a larger study, the 729 participants (grades 7–11) in our first study (Vennix et al., 2017) were also questioned about their motivation and attitude. Of this group, from 706 students (97%) answers were obtained for external regulated motivation, from 701 students (96%) for internal regulated motivation, from 699 students (97%) for the attitude towards a future STEM career, and from 696 students (95%) for the attitude towards the social implication of STEM. Therefore we used data of 696 students (56,6% male, 43,2% female and 0,1% unknown) for our analyses. The students came from 35 different high schools.

Activities

The activities chosen for this study had a wide variety in terms of teaching method and location. They included, for example, traditional lectures either in-school or on location, workshops that last from one hour to projects over multiple days or weeks, either in-school or out-of-school. Another distinct characteristic was the objective of the activity: some activities focused on enhancing interest for STEM, other activities focused on understanding the science or showed new perspectives and applications of STEM. A full overview of activities, including characteristics described in our theoretical section, can be found in our first paper (Vennix et al., 2017). Table 1 shows a shortened version of the included activities in our study.

Table 1. Overview of activities (Modified from: Vennix et al., 2017).

Activity	Description
guest-lesson: LCD	Students in six different groups were taught by a company employee about liquid crystals and the application in a liquid crystal display (LCD). Students made their own working (simple) LCD.
guest-lesson: relativity and chaos	Two groups of juniors and seniors were taught by university professors about relativity and chaos.
guest-lesson: medical imaging	A group of 10th graders visited a health-tech company. Both a factory tour and a lecture with some theoretical background about medical imaging were part of the visit.
Make the match	Four groups of 7th graders were working on an assignment given by two different companies in a period of six weeks. The challenge was to find a solution for elderly or disabled people to move around more easily.
MRI-project	Juniors were given a technical challenge to find a simple and cheap solution to integrate in the production line.
Health-project	A group of juniors had to make a video or brochure about the examination in an MRI for young people.
Technasium	Students were working on assignments given by companies. The students made designs or did research on real problems.
Research-day	A STEM-based company opened their doors and 9th graders participated in at least three different workshops about the latest technology and applications.
Keurig kiezen	A programme with workshops offered by a Technical University for 9th graders, to show different areas of their research topics.
Beta2you	A one day project, theme light. Students had solve a challenge.
Masterclass enjoy engineering	Juniors worked in groups during a total of six days on different projects. They had to present their results in a poster session and presentation.
High tech room	A group of students from different high schools were challenged to advice the local government on mobility. The students were trained via masterclasses with experts, to come up with a well thought-out advice.

Instrumentation

The students completed a questionnaire, which contained questions about the four major elements of our theoretical frame: learning environment perceptions, need fulfilment, motivation and attitudes. All items in the questionnaire used a five-point Likert scale ranging from: (1) strongly disagree to (5) strongly agree.

To map perceptions of the Outreach learning environment we used the questionnaire constructed for our first study (Vennix et al., 2017), which was based on parts of existing learning environment questionnaires, such as the What is Happening in this Classroom (WIHIC, Fraser, Fisher, & McRobbie, 1996) and the Constructivist Learning Environments Survey (CLES, Taylor, Dawson, & Fraser, 1995). This questionnaire maps perceptions on the following dimensions or scales: cohesiveness, teacher support, involvement, personal relevance, uncertainty and autonomy support. To map needs satisfaction of the students, we used two items for the need competence, two items for the need autonomy and three items for the need relatedness based on the basic psychological needs scales (Deci & Ryan, 2000). From these scale means a total need satisfaction score was calculated (Quested et al., 2011). To map motivation, students completed a 16 item questionnaire based on the Academic Self-Regulation Questionnaire (SRQ-A, Deci & Ryan) consisting of four subscales. Each subscale contained four items associated with the different levels of motivation: extrinsic motivation, introjected motivation, identified motivation and intrinsic motivation. We calculated a controlled motivation score by averaging extrinsic and introjected motivation scores and an autonomous motivation score by averaging identified and intrinsic motivation. The questions on attitudes were based on the Test Of Science Related Attitudes (TOSRA) (Fraser, 1981). Both the subscales career interest and social implication were selected from this instrument and had 7 items each. Thus, for each of the four subsections existing (parts of) instruments were used, which have elaborately been tested for reliability and validity by their original creators, for example via exploratory and confirmatory factor analyses, internal consistency and other validity indicators. For all scales we refer to scores between 2.8 and 3.2 as neutral, scores below 2.8 as negative and scores above 3.2 as positive. Table 2 shows some example items for each scale used, and scale means.¹

Table 2. Means, standard deviations, Cronbach’s alpha and sample items.

		Number of items	M student level	SD student level	Cronbach’s alpha	Sample items
Learning environment	Cohesiveness (WIHIC)	8	4.07	.60	.79	I worked well with other students.
	Teacher support (WIHIC)	7	3.60	.80	.90	The (guest) teachers’ questions helped me to understand.
	Involvement (WIHIC)	8	3.62	.74	.86	I explained my ideas to others.
	Innovation (CES)	4	3.70	.72	.67	New ideas were being tried out.
	Personal relevance (CLES)	4	3.60	.85	.79	I learned interesting things about the world inside and outside of school.
	Uncertainty (CLES)	4	3.52	.82	.78	I learned that science cannot always provide (perfect) answers to problems.
	Autonomy (LCQ)	6	3.48	.88	.91	I felt that the (guest)-teacher provided me choices and options.
Needs	Need fulfilment	7	3.78	.63	.71	While working during the activity I felt very competent.
Motivation	Controlled motivation	8	2.40	1.02	.85	I am motivated for STEM, because, that is what others (parents, friends) expect me to do.
	Autonomous motivation	8	3.66	.89	.93	I am motivated for STEM because I enjoy doing it.
Attitude	Social implication	7	3.63	.68	.77	Science and technology can help to make the world a better place in the future.
	Career interest	7	3.36	.86	.83	Working in a science laboratory of the high tech industry would be an interesting way for me to earn a living.

To investigate the quality of the scales for the present study several analyses were undertaken. The Cronbach’s alpha coefficient, indicating scale reliability, ranged between 0.67 (for innovation) and 0.93 (for autonomous motivation) (Table 2). Thus, the scales displayed satisfactory internal consistency. To see whether the subscales measured distinctively aspects of the learning environment, motivations and attitudes, correlations between subscales within all of the four subsections were computed. The correlations were adequately low to conclude that the scales seemed to measure distinct aspects (the average correlation for the seven scales of the learning environment was .46, the correlation between the two attitudes .59 and the correlation between both types of motivation .06). Therefore, the data of the instrument were used for conducting further analyses.

Analysis

To answer the research questions a number of analyses were performed. To answer the first two research questions, the means and standard deviations (e.g. descriptive) for all the scales were computed for the entire sample of students. To answer research question two and three an ANOVA was used to examine the percentages of variance at the level of the activities.

In order to answer research question four, multilevel analyses of variance were conducted with SPSS for the two motivation scales (controlled and autonomous motivation) and the two attitude scales (social implication and career interest). The models consisted of two levels – activity and student – and were tested in three steps. First, an empty model without independent variables was tested, with an output of raw percentages of variance in the motivation and attitude scores at the levels of the student and the activity. Second, a model with student variables was tested. Statistically non-significant variables were deleted and the remaining model contained only significant student variables. Third, activity characteristics (Table 3) were added to the final model in a similar way as in the second model. For each category one of the characteristics was chosen as baseline. In the models for attitudes, also the motivation variables (autonomous and controlled) were added and tested for statistical significance. Regression coefficients for variables and their standard errors were determined as well as the effect size. The effect size was obtained with the formula: standardised coefficient = coefficient * (standard deviation independent variable/standard deviation dependent variable) (e.g. Snijders & Bosker, 1999). These effect sizes were subsequently interpreted as either small (range 0 to .2), medium (range .2 to .5) or large (range .5 to 1) (e.g. Cohen, 1988).

Table 3. Activity characteristics as used in statistical analyses (Adapted from: Vennix et al., 2017).

Factor (activity level)	categories
location	• in-school • out-of-school, with own school • out-of-school with multiple schools • combination, both in and out-of-school
context	• scientific • community • company • student personal life
selection	• no selection of students • selection
teaching method	• long project • short workshop • short project • lecture-based
objective	• enhance interest and engagement in science • understanding of science and scientific concepts • new views of science and scientists

The −2*loglikelihood (−2LL) was computed, and indicates the distance between model and data (Embretson & Reise, 2000). If the decrease in −2LL was significant between two models, the next model was computed. Finally, the percentages of variance at the different levels and the percentages of explained variance by variables were computed for each model.

Results

Students’ motivation during outreach activities and students’ attitudes towards the social implications of STEM and future STEM career intentions

To answer the first two research questions, scale means and standard deviations for all variables were calculated and are presented in Table 2. No neutral scores were found. As can be seen, students’ perceptions of the learning environment were overall very positive, ranging from 3.48 (autonomy) to 4.07 (cohesiveness) on a five-point scale. Perceived needs fulfilment was also very positive (M = 3.78). Controlled motivation was negative (M = 2.40), whereas autonomous motivation was positive (M = 3.66). On average, students attending outreach activities reported more autonomous motivation than controlled motivation.

Students’ attitude towards a future STEM career was slightly positive. The attitude towards the social implication of science (M = 3.63) was even more positive than the attitude towards a future career in STEM.

Differences in students’ attitudes and reported motivation between different outreach activities

Differences between activities were clear from the analysis of variance (ANOVA) as shown in Table 4. Differences in students’ career interest and attitude towards the social implication of STEM were for 19% and 21% attributable to differences between activities. Differences in students’ reported autonomous and controlled motivation for STEM were for 11% and 15% related to differences between activities.

Table 4. Motivation and attitudes, variance between activities.

	F (11,694)	p	η^2*
Controlled motivation	9.96	.00	.15
Autonomous motivation	6.74	.00	.11
Social implication	12.69	.00	.19
Career interest	14.71	.00	.21

Relationships between self-reported motivation and attitudes, characteristics of outreach activities, perceptions of the outreach learning environment and perceived needs fulfilment

Controlled motivation

Table 5 shows the results of the multilevel analyses for reported controlled motivation. Only statistically significant regression coefficients are reported. Outcomes of the empty model showed on average a relative low score for controlled motivation. In our sample, 17% of the variance was located at the activity level. By adding activity characteristics almost 16% of the total variance was explained, equating to about two-third of the variance at the activity level.

Table 5. Controlled motivation, significant (p = .05), regression coefficients, variance components and effect sizes for all outreach activities.

Controlled motivation		Empty model (0)	Individual level (1)	Activity level (2)	Effect sizes in activity model
Student level	Constant	2.32 (.13)	2.35 (.11)	2.89 (.19)
	Personal relevance		−.11 (.04)	−.12 (.04)	−.12
	Autonomy support		.28 (.04)	.29 (.04)	.28
Activity level, context	Scientific			−.88 (.24)	−.33
	Society/community			−.62 (.24)	−.28
	Company			ns	ns
variance	Student	82.6	79.6	79.6
	activity	17.4	12.0	4.6
	explained	–	8.4	15.8
−2*log(like)		1934.0	1896.7	1887.9
Difference log (df)			37.3 (2)	8.8 (2)

Baseline: context = student life, ns = non-significant.

The results suggest that the amount of reported controlled motivation will increase when students perceive more autonomy support and that it will decrease when students perceive more personal relevance. The effect size of the personal relevance perception was small, all other effect sizes were higher, although they still could be considered as small effects (Cohen, 1988). Perceived needs fulfilment was not statistically significantly related to controlled motivation. At the activity level only the context was partly related to controlled motivation. The level of reported controlled motivation was lower for activities with a society and scientific context compared to activities with a context related to students’ lives.

Autonomous motivation

Table 6 shows the results of the multilevel analyses for self-reported autonomous motivation. In our sample almost 9% of the variance was located at the activity level. By adding activity characteristics 43% of the total variance was explained, amounting to almost all variance at the activity level.

Table 6. Autonomous motivation, significant (p = .05), regression coefficients, variance components and effect sizes for all outreach activities.

Autonomous motivation		Empty model (0)	Individual level (1)	Activity level (2)	Effect sizes in activity model
Student level	Constant	3.75 (.09)	3.59 (.10)	4.06
	Gender		.26 (.05)	.27 (.05)	.15
	Personal relevance		.22 (.03)	.20 (.03)	.22
	Need fulfilment		.35 (.03)	.36 (.03)	.40
Activity level location	Both in-school and out of school			.89 (.22)	.48
	Out-of-school with classmates			.24 (.12)	.11
	Out-of-school with students from other schools			ns	ns
objective	New views of science and scientists			.39 (.08)	.22
	Understanding of science and scientific concepts			1.05 (.15)	.42
teaching method	Lecture			.78 (.19)	.34
	Workshop			.76 (.21)	.40
	Short project			.68 (.26)	.13
selection	No			−.52 (.13)	−.24
variance	Student	91.4	57.5	55.7
	activity	8.6	10.3	1.1
	explained	–	32.2	43.2
−2*log(like)		1784.6	1510.9	1471.1
Difference log (df)			273.7 (3)	39.8 (9)

Baselines: gender = female; location = in school; objective = enhancing interest; teaching method = long project; selection = yes. ns = non-significant.

Autonomous motivation was positively related to perceived personal relevance and perceived needs fulfilment. If students perceived the activity as personally relevant and their needs (relatedness, competence and autonomy) were perceived to be more fulfilled, students reported higher autonomous motivation. All the activity characteristics had a statistically significant relationship with autonomous motivation, except for context. Lectures, short projects and especially workshops resulted in higher reported autonomous motivation compared to long projects. With respect to goals of the activity, understanding science and new views of science and scientists resulted in higher autonomous motivation compared to activities with the goal to enhance interest in science. Last, an activity that was located both in school and out of school resulted in higher reported autonomous motivation compared to an in-school activity. The strongest effects (i.e. medium effect size) were found for workshops, an activity conducted both in and out-of-school, the objective of understanding science and needs fulfilment. Male students reported slightly more autonomous motivation compared to female students.

Attitude towards the social implication of STEM

Table 7 shows the results of the multilevel analyses for the attitude towards the social implication of STEM. In our sample almost 22% is of the variance was located at the activity level. By adding activity characteristics 65% of the total variance was explained, and nearly all variance at the activity level.

Table 7. Attitude towards social implication of STEM, significant (p = .05), regression coefficients, variance components and effect sizes for all outreach activities.

Attitude towards social implication of STEM		Empty model (0)	Individual level (1)	Activity level (2)	Effect sizes in activity model
Student level	Constant	3.77 (.09)	3.65 (.06)	4.50 (.17)
	Gender		.10 (.02)	.12 (.04)	.09
	Teacher support		.07 (.02)	.08 (.02)	.12
	Innovation		.05 (.02)	.05 (.02)	.07
	Autonomous motivation		.22 (.02)	.20 (.02)	.29
	Controlled motivation		−.29 (.02)	−.28 (.02)	−.41
Activity level location	In-school.			ns	ns
	Both in-school and out-of-school			ns	ns
	Out-of-school with classmates			.33 (.07)	.17
objective	Understanding science			.51 (.07)	.29
	New views of science and scientists			.21 (.06)	.15
variance	Student	78.4	38,0	34.7
	activity	21.6	4.2	.1
	explained	–	57.7	65.3
−2*log(like)		1341.5	1069.1	1031.9
Difference log (df)			272.4 (5)	37.2 (8)

Baselines: gender = female; location = out of school with students from other schools; objective = enhancing interest, ns = non-significant.

The attitude towards the social implication of STEM was positively related to autonomous motivation and strongly negatively related to controlled motivation. Perceived teacher support as well as activities perceived as innovative were positively associated with attitudes towards the social implication of STEM. Out-of-school activities with classmates resulted in more positive social implications attitudes compared to activities out-of-school with students from other schools. Activities with the main objective to understand science resulted in higher attitudes compared to activities with the objective to enhance interest.

Attitude towards career interest in STEM

Table 8 shows the results of the multilevel analyses for the attitude towards career interest in STEM. In our sample 14% is of the variance was located at the activity level. By adding activity characteristics 64% of the total variance was explained and all variance at the activity level.

Table 8. Attitude towards career interest in STEM, significant (p = .05), regression coefficients, variance components and effect sizes for all outreach activities.

Attitude towards career interest in STEM		Empty model (0)	Individual level (1)	Activity level (2)	Effect sizes in activity model
Student level	Constant	3.48 (.10)	3.22 (.05)	3.98 (.15)
	Gender		.31 (.04)	.31 (.04)	.18
	autonomous motivation		.55 (.02)	.55 (.02)	.64
	controlled motivation		−.17 (.02)	−.16 (.02)	−.19
Activity level location	In-school.			ns	ns
	Out-of-school with classmates			.30 (.07)	.14
	Out-of-school with students from other schools			.28 (.08)	.12
objective	Understanding science			.22 (.08)	.10
	New views of science and scientists			.14 (.06)	.08
variance	Student	85.7	35.2	35.6
	activity	14.2	2.2	0
	explained	–	62.6	64.4
−2*log(like)		1690.4	1188.9	1171.4
Difference log (df)			501.5 (3)	36.3 (9)

Baselines: gender = female; location = combination of both in-school and out-of-school; objective = enhancing interest. ns = non-significant.

The attitude towards career interest in STEM was strongly positively related to autonomous motivation and strongly negatively related to controlled motivation. Male students reported more interest in a future career in STEM compared to female students. Students attending out-of-school activities (both with classmates and students from other schools) had a more positive attitude towards a career interest in STEM compared to activities with a combination of locations. If the main objective was to understand science students’ attitude towards a career interest was more positive compared to an activity to enhance interest.

Discussion and conclusion

Conclusion and interpretation

In this study, we explored the associations between students’ perceptions of the outreach learning environment, motivation and attitudes towards STEM and future STEM career intentions. In our view, the findings have shed light on some interesting insights that might be of general value to enhance the number of students choosing for STEM and keep them motivated to pursue a possible career in STEM.

First, students’ perceptions of the outreach learning environment were overall very positive, as also reported in an earlier study (Vennix et al., 2017). Adding outreach activities to the curriculum would thus potentially add value for students’ reported intrinsic motivation for getting more involved in STEM activities, assuming that more positive perceptions of the learning environment elements would also lead to higher perceived satisfaction of the three needs, relatedness, autonomy and competence (Black & Deci, 2000; Jang et al., 2016; Lavigne et al., 2007). The results of our present study showed very positive results for students’ perceived autonomous motivation. In addition students’ controlled motivation was rated below average as an indication that this type of motivation was relatively absent. Students’ attitude towards the social implication of STEM was very positive as well, as an indication of a successful real life connection and embedding of STEM in society applications. Although, students’ attitudes towards a possible STEM career were positive, their scores were not as positive as those for their attitudes towards the social implication of STEM.

Second, there was quite some variance in students’ reported motivation and attitudes. The variance was for 15–20% located at the activity level. These percentages were in line with other learning environments studies, typically between 7 and 15% of the variance in student outcomes is related to differences at the class level (see for example Den Brok, Brekelmans, & Wubbels, 2004; Frenzel, Pekrun, & Goetz, 2007). For this reason we used multi-level analyses to explain differences in motivation and attitudes between activities. With these analyses we showed that activities systematically differ from one another in their levels of motivation and attitude, and that almost all the variance can be explained by activity characteristics.

Third, the most important factor of the outreach learning environment related to both the autonomous and controlled motivation was students’ perception of the personal relevance of the activity. The perceived personal relevance was negatively related to controlled motivation and positively to autonomous motivation. This is in line with former studies that showed that promoting general applications of STEM, for example health applications, positively influenced students’ interest in the subject (Jones & Kirk, 1990). In outreach activities, need fulfilment appeared to be positively related to autonomous motivation. This is in line with prior findings of the positive association between needs fulfilment and autonomous motivation (Deci & Ryan, 2000; Stroet et al., 2013). Teacher support and innovation were significant learning environment aspects that influenced the attitude towards the social implication of STEM positively. Indirectly, these findings might be indicators that both relatedness and autonomy needs were fulfilled (Sierens et al., 2009; Sjaastad, 2012). To enhance an attitude towards a possible career interest, interestingly no associations were found for the learning environment dimensions.

Fourth, at the activity level the context of an activity was negatively associated with controlled motivation. This means that context was an important aspect to lower the reported controlled motivation. The strongest negative association was for activities with either a scientific or a community context. Although, context was not significant related to autonomous motivation, the association that context decreased controlled motivation indicated that context contributed to higher qualitative motivation (Vansteenkiste et al., 2009).

Fifth, reported autonomous motivation was highest for activities that were located both in and out-of-school. Also, out-of- school activities with own classmates scored higher for autonomous motivation than activities located in school or out-of-school with students from other schools. Both these characteristics might emphasise the relevance of the application in direct connection to the school subject. Also attitudes towards STEM were positively associated with out-of-school activities. Although the combination of both in and out-of-school was less pronounced in increasing the attitudes compared to autonomous motivation. Thus, location was an important factor influencing both autonomous motivation and attitudes towards STEM: an out-of-school location supported an authentic learning environment (Herrington, Reeves, & Oliver, 2014).

When the objective of an outreach activity was to explain a STEM application, this caused a higher score for autonomous motivation and attitudes towards social implication and a future STEM careers. Students wanted to understand what they saw and had to do. This supported also the connection between the application and the school subject, and thus the need for competence (Boggiano & Ruble, 1979).

Last, autonomous motivation was also positively related to the teaching method. Students preferred workshops over projects and lectures. As in this teaching method most hands-on activities were performed, students were actively involved (Holstermann, Grube, & Bögeholz, 2010). Attitudinal outcomes were not statistically significantly related to the teaching method. As expected, both attitudes were positively related to autonomous motivation and negatively related to controlled motivation (Deci & Ryan, 2000; Vallerand & Ratelle, 2002). These findings supported our line of reasoning, as the learning environment influences the autonomous motivation by supporting the basic needs as proposed by the SDT. Both the controlled and autonomous motivation influenced students’ attitudes towards STEM.

Limitations

Although we conducted a multiple case study with a variety of twelve activities and 729 students, we might have missed certain characteristics of outreach activities, or some activities might be influenced by other factors. As a consequence generalisation of the results for now remain within the current set of types of activities and the Dutch/American context.

Second, students were questioned just once about their perceptions, motivation and attitudes, after the attended outreach activities. Since outreach activities are increasingly popular and students might attend outreach activities more than once, questioning students after every activity could verify the same effects as we have found in our present study. Also, students might have entered outreach activities with an already high(er) attitude and motivation; our study did not control for initial attitudes or motivation.

Third, with our multilevel analysis we investigated associations between perceptions of the learning environment, reported motivation and attitudes. To determine causality, longitudinal designs or intervention studies are needed, as well as different types of analyses, such as path analysis.

Fourth, we found that on average students’ reported autonomous motivation was higher than students’ controlled motivation. To understand the different levels of motivation (controlled and autonomous) and the effects of the outreach characteristics and in particular the effects on the attitudes towards a future STEM career, motivational profiles cluster analysis could be done.

Fifth, to gain deeper insight in the processes that determine if a student becomes more motivated, a case study with qualitative data collection could be done to understand what processes take place during an outreach activity and why students get involved or not.

Implications

Several implications can be drawn from the results of our study. In general the outreach learning environment was perceived positively, as well as perceived satisfaction of the basic needs (competence, relatedness and autonomy). In addition, we were able to relate specific characteristics of outreach activities to more autonomous motivation and more positive attitudes towards STEM.

To enhance autonomous motivation, the outreach learning environment should be personally relevant for a student, with the main objective to understand the science. This can be achieved using real-life examples, for example in the Dutch context working on solutions for increased visibility for bikers in order to bike to school safely. Although the teaching method for a longer project is positively related to autonomous motivation, the shorter ones showed larger effect sizes. This implies that the external guide is more involved and in that perspective the activity provide a completely different learning experience compared to regular science courses. In addition, activities in which students worked with classmates showed higher reported autonomous motivation than activities in which students worked with other students they did not know. Thus, activities where students work together with external guides (i.e. experts) and classmates on a real-life problem, are more likely to enhance autonomous motivation. Also projects do not have to last for a long period, to enhance the autonomous motivation.

Although autonomous motivation is positively associated with both attitudes, the effect size for the attitude towards a future career is larger. This suggests that the out-of-school component is important to get a more complete picture of the implications of STEM and of possible career opportunities.

Based on these findings, it is suggested that teachers and schools should give more attention to adding outreach activities to the curriculum, in particular activities with an out-of-school element, activities connecting to the personal relevance of the students, connecting the activity to curriculum-aspects and focusing activities on the understanding of science.

Acknowledgement

The first part of this study was published in learning environments research, therefore we had to refer to this paper several times.

This paper is part of a larger study about motivational aspects of STEM-based outreach activities in secondary education. The authors want to thank all the participants (students and schools), that were willing to participate in the second part of this study. This made it possible not only to get more understanding about the perceived outreach learning environment, but gave us also the opportunity to learn more about the relationship between this particular outreach learning environment, student motivation for STEM and attitudes towards a possible STEM career.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Johanna Vennix http://orcid.org/0000-0001-6295-7471

Notes

1 The complete questionnaire can be obtained from the first author.