1,945
Views
1
CrossRef citations to date
0
Altmetric
Research Article

A research-practice cooperation to support elementary school teachers’ diagnostic competencies based on a working theory of talent development in STEM

, , , , &

ABSTRACT

We present a teacher professionalization project for promoting elementary school teachers’ diagnostic competencies in recognizing their students’ potential in STEM subjects. Teacher professionalization is fostered through the teachers’ personal involvement in the development of subject-specific diagnostic materials as well as through continuing training on the fundamental knowledge needed to detect and support gifted students. On the basis of the Talent Development in Achievement Domains (TAD) framework, we reached a common working theory of talent and its development in mathematics and science for elementary school-aged students based on the available scientific evidence and approved by teachers. We share a multidimensional, dynamic view of talent development that includes abilities, personality traits, and skills. We describe how our working theory and diagnostic materials can foster teachers’ diagnostic competencies as well as talent discovery and promotion in order to foster students’ development of creative productivity in STEM fields.

Introduction

Diagnosing students’ abilities and learning needs is a crucial task of teachers both in general (Pellegrino, Citation2014) and for the identification of talent in science, technology, engineering, and mathematics (STEM) specifically (Siegle & Powell, Citation2004). For one, assessments by teachers inform their teaching methods and educational designs. Talented students in particular have been found to require substantially differentiated learning environments with curriculum and teaching that provide appropriate pace, depth, and breadth (Knopfelmacher & Kronborg, Citation2003; Kronborg & Plunkett, Citation2013; Maker & Schiever, Citation2010; Pierce & Adams, Citation2004). In addition, assessments by teachers influence students’ development because they can lead to certain expectations, which in turn influence further development (Jussim & Harber, Citation2005; Wang et al., Citation2018). Teachers’ diagnostic competencies are therefore of high practical relevance. The current paper describes a cooperative project between elementary school teachers and educational researchers that aims to address some of the challenges in diagnosing STEM talent in students. The aim of this project is to promote the diagnostic competencies of elementary school teachers in STEM subjects (i.e., mathematics and science).

In the following, we first provide background information on STEM talent and its development as well as on teachers’ diagnostic competencies. We then present the project, which comprises the development of a common working theory of talent development in STEM, the development of diagnostic material based on this working theory, and its formative evaluation in the school context. Due to the focus of the special issue on eminence, we describe the construct of creativity and its relation to achievement in more detail, as creativity is critical to eminence (Rena F. Subotnik et al., Citation2011). Finally, we discuss how the promotion of teachers’ diagnostic competencies supports the development of student talent in STEM. Here, we again focus on the promotion of creativity and its contribution to the development of talent. Overall, the present paper presents an example of a research-practice partnership (Yurkofsky et al., Citation2020) that aims to improve talent development in STEM in elementary school students by supporting teachers’ diagnostic competencies in diagnosing STEM talent in students.

Talent development and STEM talent

Talented students are those students whose performance is above average in one or more achievement domains. These students successfully transformed their potential into above-average performance levels within a complex developmental process with numerous influences, including learning opportunities (Rena F. Subotnik et al., Citation2011). Most generally, talent development can be defined as the development of a person’s potential for achievement into actual achievement. According to the Talent Development in Achievement Domains (TAD) framework (Preckel et al., Citation2020), there are four successive levels, with the first two being most relevant for talent development in STEM in elementary school. (1) Aptitude describes the constellation of factors that are predictive of an individual’s positive development of achievement or future performance. (2) Competence refers to a cluster of related and systemically developed abilities, knowledge, and skills that enable a person to act effectively in a situation with multiple options. (3) Expertise refers to a high level of consistently superior achievement. (4) Transformational achievement refers to levels of accomplishment that go beyond expertise by generating creative responses or interpretations that require breaking domain boundaries, combining different domains, or creating new questions.

The TAD framework conceptualizes talent development as dependent on multiple factors with a focus on person-related variables (i.e., abilities, personality traits, and acquired skills). These person-related variables (i.e., predictors of talent development) partly vary by domain. We therefore outline the empirical evidence for domain-specific predictors of talent development in STEM. summarizes empirical evidence, mainly taken from meta-analyses and literature reviews, for abilities, personality traits, and acquired skills that correlate positively with the development of achievement and creative productivity in STEM fields.

Table 1. Predictors and correlates of STEM achievement.

Due to space limitations, we cannot discuss each variable listed in in detail. Rather, we point readers to the original sources and focus on creativity as one example. The role of creativity in talent development has mainly been investigated in cross-sectional studies. A recent meta-analysis by Gajda et al. (Citation2017) included cross-sectional studies published since the 1960s and revealed an average correlation of r = .22 with academic achievement. Gajda et al. (Citation2017) found that this rather weak relation between creativity and academic achievement was moderated by several factors. First, verbal creativity tests showed higher relations with academic achievement than non-verbal tests. Second, the relation was higher for middle school students than for primary school students. Moreover, the relation was weaker for high school and tertiary students compared to middle school students. This latter finding might be explained by the increasing domain-specificity in higher education, which was not considered by most of the studies included in the meta-analysis (Gajda et al., Citation2017). Third, the relation between creativity and achievement was higher when using standardized achievement tests rather than grade point averages (GPAs). Freund and Holling (Citation2008) found that the relation between creativity and GPA varied strongly between different classrooms. This could be because some teachers appreciate creative thinking and acting in their classrooms, whereas others do not. In general, the relevant literature indicates that teachers prefer to teach highly intelligent rather than highly creative students, as their unusual questions and answers or their liveliness may disrupt the class (Krampen, Citation2019). With respect to longitudinal studies, there is no evidence yet that creativity is a causal factor in predicting later academic achievement. Nevertheless, some findings document that people identified as creative during their school years tend to become creative and productive adults (Runco et al., Citation2010; Torrance, Citation1972).

These illustrative findings for creativity help to demonstrate some important aspects with regard to talent development. Even if relations between a construct and achievement are weak in the early stages of talent development, this construct might increase in its importance later on. That is, low correlations at one level of talent development do not necessarily indicate that the construct is not important. In addition, the importance of a construct also depends on the context, e.g., how much one’s teacher values and supports it. Therefore, it is of utmost importance that teachers know what comprises potential in a certain domain to be able to adequately support relevant student traits.

In sum, the TAD framework (Preckel et al., Citation2020) and related research has shown that talent discovery and development is a long process. Talent development leading to outstanding performance and sometimes transformational achievement depends on interactions between individuals and the environment (R. F. Subotnik et al., Citation2018; Ziegler, Citation2005). The foundation of talent trajectories includes abilities, personality traits, and skills that are modifiable by education. In addition to one’s potential in a particular domain, talent development also requires specific types of environments (e.g., knowledgeable teachers) and specific types of responses to environmental pressures (e.g., persistence, engaging in deliberate practice). Well-established abilities, personality traits, and skills enable individuals to persevere on demanding talent development paths despite setbacks, anxiety, and lack of support (Olszewski-Kubilius et al., Citation2016). Furthermore, the movement from aptitude to competence requires increasing commitment to intentional learning, study, and practice in areas of interest and talent (Olszewski-Kubilius et al., Citation2016). To pave the way for young people to specialize in their field, increased interest and talent in a specific domain should always be valued and supported by teachers. Without an accumulation of all of these interacting factors, talent development is not likely. Hence, this is where teachers and their diagnostic competencies come into play.

Teachers’ diagnostic competencies

Beginning in the 1980’s with Schrader and Helmke’s (Citation1987) assertion that accurately judging students’ abilities is a vital factor in differentiated instruction, diagnostic competencies are now regarded as a core aspect of teachers’ expertise (e.g., Baumert & Kunter, Citation2006; Van Ophuysen, Citation2010; Weinert et al., Citation1990). At first, diagnostic competence was defined as the ability to judge students’ performance level correctly. Later on, Spinath (Citation2005) showed that the accuracy of teachers’ judgments is not determined by a single ability and therefore suggested avoiding the term “diagnostic competence” as a single competence related only to accurately judging students’ characteristics. Judging the requirements of learning materials is also essential for initiating successful learning processes in the classroom, and therefore the construct of diagnostic competence also needs to include the accurate estimation of the difficulty of tasks and materials (McElvany et al., Citation2012). Hence, teachers with high diagnostic competencies are able to judge the demands of learning tasks as well as students’ potential, knowledge, performance, and other school-related behavior, and to use this information to adapt their teaching accordingly. Meta-analyses document the size of relations between teachers’ assessments and the actual characteristics of their students, including their cognitive abilities (Machts et al., Citation2016) and their academic achievements (Südkamp et al., Citation2012). Individual studies on performance assessments of students by their teachers show that both in primary and secondary school, high-performing students tend to be underestimated whereas lower-performing students tend to be overestimated (Bates & Nettelbeck, Citation2001; Westphal, Citation2016). Potentially high-performing students, such as highly intelligent children with only average school performance, are rarely recognized by their teachers as talented (Hanses & Rost, Citation1998). The same applies to students whose potential lies in “unusual” areas, such as girls who are talented in STEM (Dicke et al., Citation2012). Moreover, diagnostic competencies of teachers should be conceptualized as domain-specific (Shulman & Sherin, Citation2004). They are related to the didactic knowledge of teachers in a specific subject, to knowledge about the diagnostic potential of tasks and their cognitive and implicit knowledge requirements, and to teachers’ knowledge about students’ learning processes (Baumert & Kunter, Citation2013).

A cooperative project to support teachers’ diagnostic competencies in STEM

In our project LUPE (acronym for Leistung unterstützen, Potenziale erkennen “supporting achievement, assessing potential”), a research team cooperates with 23 elementary schools in Germany over a five-year period. Currently, we are in the third year of the project. The project is characterized by a research-practice partnership (RPP; Yurkofsky et al., Citation2020), in which the teachers and researchers cooperate on an equal footing as experts in their field. In educational research, several approaches have already considered this RPP. Such approaches are referred to as “research for practice” instead of “research on practice” (Schrader et al., Citation2020) or joint work as “improvers” instead of distinguishing between “knowers” and “doers” (Bryk, Citation2015). The involvement of practitioners and researchers differs highly by approach. Hence, it is useful to position the RPP on two dimensions to illustrate the degree of involvement of practitioners and researchers in the research process.

In project LUPE, the involvement of practitioners and researchers differs by stage (i.e., 1. development of the working theory, 2. development of diagnostic material, and 3. formative evaluation; see ). The active participation of practitioners results in different advantages depending on the stage of the process (Burns et al., Citation2011). Initially developing a common working theory of practitioners and researchers supports understanding and acceptance. The development of diagnostic material with extensive involvement of researchers ensures an evidence-based approach. Extensive involvement of both practitioners and researchers in the practical testing and formative evaluation of the material leads to educational outcomes that are appropriate for the teachers’ needs and the school context. That is, the RPP approach ensures a strong link between research and practice, safeguarding that the theoretically well-founded material has also been assessed by the users as successful in practice (Schrader et al., Citation2020).

Figure 1. Research-practice partnership at different stages of the project LUPE.

Figure 1. Research-practice partnership at different stages of the project LUPE.

shows the process of the RPP in LUPE. In Stage 1, to derive a common working theory we discussed the assumptions of the TAD model for talent development in STEM and the predictors and correlates of STEM achievement (see ) with the teachers of the 23 schools in workshops. We assessed teachers’ knowledge about talent development and student-related variables that indicate potential for STEM, compared this knowledge with scientific evidence in a contrasting juxtaposition, and agreed on a working theory conceptualizing potential in STEM as multidimensional and dynamic (for a detailed description of this process and its results see Mack et al., Citation2021).

Figure 2. Developmental process of the diagnostic material.

Figure 2. Developmental process of the diagnostic material.

In Stage 2, based on the developed working theory, we engaged in the development of diagnostic materials. Together with the teachers, we decided to develop full assessment units that can be carried out with a whole class and allow teachers to base their assessments not only on task solutions but also on systematic observations and interviews with the students. Each assessment unit focuses on one or two predictors of talent development in STEM that were included in the working theory (e.g., creativity; see ). This narrow focus promotes a better observation of the respective predictor in the students. The development of an assessment unit is comprised of three parts: 1. selection of the predictor(s), 2. literature search for task materials, 3. creation of a first draft. In Stage 3, the first draft of the unit then enters a formative evaluation phase (see ).

Assessment unit on creativity

In the following, the individual steps of this process are described in more detail for the example of creativity. We developed an assessment unit for creativity because of its positive relation with STEM achievement (Gajda et al., Citation2017) and its prominent role in talent development (Rena F. Subotnik et al., Citation2011). In an extensive literature search, we identified task material related to creativity. Together with didactics experts, we adapted tasks and developed them into a unit. We chose open-ended tasks that are characterized by “the existence of more than one (preferable many more than one) possible pathways, responses, approaches, or lines of reasoning” (p. 3; Sullivan et al., Citation2000). gives an example: “Dividing Squares” is an open-ended task on figural divergent thinking in this unit. Here, children see the empty square () and are instructed to “divide the square into 3 equal parts; think of as many different or unusual and new ways to do this as possible.” The children have 20 minutes to complete the task. Teachers can take notes in their evaluation protocol both during and after the task. The information on the evaluation protocol for teachers provides clues on what characterizes creativity and how this can become evident in student behavior (e.g., an idea that no other student had). The evaluation protocol includes information on: the number of ideas, the categories assigned to the ideas, the variety of ideas, and the originality of ideas, evaluated on 5-point Likert scale ranging from very weak to very strong. High creativity is characterized by children producing many, different, and original solutions. show possible solutions that are different and differ in originality. The solutions differ in the kind of lines used (straight and even, ; straight and diagonal as well as uneven, ; straight and curved, ). The originality of the solutions, indicated by the infrequency of their occurrence, increases from ).

Figure 3. Possible solutions for “Dividing Squares” task used to observe creativity.

Figure 3. Possible solutions for “Dividing Squares” task used to observe creativity.

Creativity is typically assessed by open-ended tasks, which have certain advantages for their use in the assessment units. Open-ended tasks allow students to choose different approaches for task solutions, to activate their individual knowledge and resources, and to apply solutions that they themselves consider important for the process. Students get the opportunity to show what they can do and to produce new and original ideas that no other student has had before. Teachers, on the other hand, get the opportunity to systematically observe individual indicators of divergent thinking ability such as the quantity, variety, and originality of ideas. Additionally, teachers learn how to utilize the creativity demands embedded in the task material and how to use this information for identification purposes.

Open-ended tasks have proven to be effective for the identification and promotion of STEM potential, for example, related to mathematical creativity (e.g., R. Leikin & Lev, Citation2007). With the help of open-ended tasks, it is possible to distinguish between students with different ability levels (R. Leikin & Lev, Citation2007) because they offer students the opportunity to engage in accordance with their prior knowledge and abilities (Sullivan, Citation2011). Students produce results that reflect their level of competencies, for instance, with regard to the complexity and refinement of their answers (Hertzog, Citation1998). Low-performing students benefit from the lower entry threshold of open-ended tasks. In contrast, high-performing students might enrich their answers or go into more depth (Sullivan, Citation2011). Therefore, especially in heterogeneous classes, open-ended tasks provide useful information for identifying and fostering talent (Hertzog, Citation1998).

The complete material for the creativity unit comprises a definition of the goals of the unit, the classification of the predictor in this unit in our working theory, the worksheets for the students, the observation and evaluation protocol for teachers, and explanations on the scientific background of the predictor creativity and the content of the unit. In addition, the material contains student examples as well as tips for differentiating the material for students of different ability levels.

The assessment unit on creativity is currently undergoing the formative evaluation process, where teachers provide continuous feedback on the material, before and after its use in practice. The research team then revises the material, and the teachers test the material in practice again (see ). Throughout the practical testing phase, the project teachers use the material during their regular lessons, collect student responses, and document their experiences. The feedback from the teachers forms the basis for the optimization and further development of the material.

Discussion: Fostering talent development by enhancing teachers’ diagnostic competencies

The understanding of talent development in a specific domain (in project LUPE, the focus is on STEM) is important for teachers at a practical level to identify STEM talent in their students and to offer appropriate educational services to them. In this regard, our common working theory can be used as an orientation for teachers regarding which abilities, personality traits, and skills are important for the promotion of talent at different stages of elementary school. The associated assessment units support teachers in identifying these abilities, personality traits, and skills in their students. In our project, the teachers are personally involved in the development and formative evaluation of the materials. In addition, they receive continuing training on the fundamental knowledge needed to identify and support talented students as well as on how to use the provided materials in their classroom. Both contribute to the teachers’ professionalization for identifying and supporting talented students. The cooperation with teachers in a partnership of equals in a RPP ensures that the materials are tailored to the needs of the schools (The Design-Based Research Collective, Citation2003). As active members of the development process, the teachers ensure that the materials can easily be integrated into everyday school life. The assessment units are complete lessons in which all students are able to work on open-ended tasks at their specific skill level. The units facilitate the systematic observation of different predictors of talent and achievement in STEM (see , in which we give an overview of all constructs included). In contrast to checklists or closed-ended talent assessments, teachers are guided to focus on particular attributes and observable behaviors, which enhances the quality of the assessment. Furthermore, teachers learn that abilities, personality traits, and skills are expected to change over time. To do justice to this fact, our assessment units can be used several times over the course of the first four years in elementary school, making it possible to keep a development portfolio for each student (see ).

Figure 4. LUPE assessment units for year 1–4 of elementary school.

Figure 4. LUPE assessment units for year 1–4 of elementary school.

Supporting teachers in assessing and fostering students’ creativity

Creativity plays a central role in the development of talent (Krutetskii, Citation1976; Renzulli, Citation1978; Rena F. Subotnik et al., Citation2011). It is both a predictor and outcome of talent development. The interaction between personality traits and environmental factors determine, to a great extent, its development (Roza Leikin & Pitta-Pantazi, Citation2013). Teachers can use the awareness that comes from understanding the importance of creativity in the talent development process to further improve their teaching. Many studies have emphasized situational factors affecting creativity, such as educational practices and freedom to pursue problems (Beghetto, Citation2016; Schacter et al., Citation2006). The first step is for teachers to realize that nurturing creativity is an essential piece of the instructional process. For students to be willing to express their creativity, they must feel that their ideas, especially those that are unconventional, are welcome in the classroom. Nickerson (Citation1999) argued that a balanced environment (i.e., an environment that is both demanding and supportive) is necessary for creativity to flourish. For an extensive report on creativity and critical thinking as important learning goals, the OECD (Vincent-Lancrin et al., Citation2019) worked with networks of schools and teachers in eleven countries to develop a set of pedagogical resources that exemplify what it means to teach, learn, and make progress in creativity and critical thinking in primary and secondary school. They pursued a similar procedure as we did, whereby they focused on promotion instead of diagnostics. Through examples of lesson plans, teachers in the field gave feedback, implemented the proposed teaching strategies, and documented their work. They agreed that the promotion of abilities like creativity can be accomplished through tasks with high demands on these abilities.

Teachers also need to be supported with classroom activities that foster creativity. Hadzigeorgiou et al. (Citation2012) argue that science is based on creativity and present several methods that teachers can use in science education to foster their students creativity: Creative problem solving (e.g., measuring the height of a building using a barometer or tennis ball), creative writing (e.g., a day without gravity), creative science inquiry (e.g., investigating possible factors that might have an effect on the illumination of a room), creating analogies to understand phenomena and ideas (e.g., the phenomenon of resonance), challenging students to find connections among apparently unrelated facts and ideas, and mystery solving (e.g., detective work in order to explain the disappearance of something). Open-ended tasks or project-based learning (Krajcik & Blumenfeld, Citation2006), which are included in our assessment units, allow students to activate their resources and surprise with new and original ideas. In this regard, we also encourage teachers in our project to transfer the concepts introduced in our units to develop new materials. For instance, existing school material could be examined for diagnostic and support potential related to creativity and the other constructs within our working theory.

In sum, the cooperative approach of our project and the materials provided in our assessment units are not only meant to contribute to the discovery of talent, but also to its promotion. Our interventions are designed to create new beneficial routines, such as long-term individual monitoring and support and a focus on student resources and potential. Even if not all students can achieve exceptional achievements in a domain, the goal should be to help students to make the best possible use of their talents.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Mireille Krischler

Mireille Krischler, Dr., is a postdoctoral researcher at the University of Luxembourg in the Department of Social Sciences. From 2018 to 2020 she worked at the University of Trier in the Department of Psychology, Giftedness Research and Education. She worked as a postdoctoral researcher in the project LUPE whose main objective is to support the academic development of (potentially) high-performing students by promoting their teachers’ diagnostic competencies. Her main areas of research encompass attitudes and competencies of teachers required to work with heterogeneous student populations as well as diagnostics and promotion of students with special educational needs with a focus on inclusion.

Elena Mack

Elena Mack is a Ph.D. student at the University of Trier in the Department of Psychology, Giftedness Research and Education. She also works as a research associate in the project LUPE whose main objective is to support the academic development of (potentially) high-performing students by promoting their teachers’ diagnostic competencies. Her professional research interests are talent development, psychological diagnostics, and diagnostic competencies of teachers.

Jessica Gnas

Jessica Gnas is a Ph.D. student at the University of Trier in the Department of Psychology, Giftedness Research and Education. She also works as a research associate in the project LUPE whose main objective is to support the academic development of (potentially) high-performing students by promoting their teachers’ diagnostic competencies. Her professional research interests are diagnostic skills of teachers, psychological diagnostics and stereotyping of gifted pupils.

Moritz Breit

Moritz Breit is a Ph.D. student at the University of Trier in the Department of Psychology, Giftedness Research and Education. His research interests center on the nature and development of cognitive ability. His current research focuses on ability differentiation, particularly Spearman’s law of diminishing returns. Moritz obtained his BA and MA in Psychology at the University of Trier.

Julia Matthes

Julia Matthes is a Ph.D. student at the University of Trier in the Department of Psychology, Giftedness Research and Education. She also works as a research associate in the project LUPE whose main objective is to support the academic development of (potentially) high-performing students by promoting their teachers’ diagnostic competencies. Her research interests include need for cognition, academic achievement and development, and diagnostic competencies of teachers.

Franzis Preckel

Franzis Preckel is full professor for giftedness research and education at the Department of Psychology of the University of Trier, Germany. She received her doctor’s degree from the University of Muenster, Germany, in 2002. Her areas of expertise include intelligence and giftedness, talent development, personality factors related to achievement, and psychological assessment including test construction. In 2017, Franzis received the path breaker award of the AERA special interest group on Research on Giftedness, Creativity, and Talent. She has served as co-editor of Gifted and Talented International and Diagnostica. Franzis has published her research in top-ranked journals including Psychological Bulletin or Perspectives on Psychological Science.

References