604
Views
0
CrossRef citations to date
0
Altmetric
TEACHER EDUCATION & DEVELOPMENT

The design process of a questionnaire measuring teachers’ innovative behavior

ORCID Icon, ORCID Icon & ORCID Icon
Article: 2283641 | Received 30 Aug 2022, Accepted 10 Nov 2023, Published online: 13 Dec 2023

Abstract

Teachers’ Innovative Behavior (TIB) is increasingly important for today’s education. However, available instruments do not meet the complexity of TIB as an individual, collective, planned and unplanned construct. The aim of the present study was to construct a new instrument that meets the requirements of this complexity. We started with a compilation of the most suitable items from 12 existing questionnaires, selected by experts. After determining the most appropriate items, face-validity was investigated by interviewing primary and secondary school teachers concerning essential terminology. Lastly, with a sample of 178 primary and 159 secondary school teachers, we preliminary assessed the dimensionality of the questionnaire by means of an explorative factor analysis (EFA). The results show that the new questionnaire appears face-valid and structurally solid, but still needs to be tested in practice among large groups of teachers, in order to confirm dimensionality and other forms of validity.

1. Introduction

In recent years, there has been a growing interest in teachers’ innovative behavior (TIB) in the field of educational research. This growing interest is particularly evident considering the increasing number of publications on the subject as the years progress (Thurlings et al., Citation2015). The growing popularity of TIB as a research field can be explained by a change in society, in which 21st century skills, of which TIB is one, are increasingly important (Voogt & Pareja Roblin, Citation2010). This focus on 21st century skills requires that teachers continually learn, develop and are innovative together with their colleagues, in order to provide students with an education that fits the current era (OECD, Citation2020). The teaching profession entails the constant involvement in various reciprocal interactions within a school context, in which the teacher has influence on pupils, colleagues and superiors, and in which he/she is influenced by these different stakeholders as well (Castelijns et al., Citation2009; Lin & Sanders, Citation2017; Vermeulen, Citation2016). TIB cannot solely be approached from the perspective of the individual teacher, as is the case for human behavior and development. TIB, like individual and group behavior, is the result of interaction with the context (Vermeulen, Citation2016). Despite these elements interacting within an organization, the concept of TIB has rarely been examined in a comprehensive configuration, including this interactive perspective, in which all the elements within an organization are interrelated. In addition to the distinction between individual and collective TIB, a recent review study (Author, Citationsubmitted) showed that the difference between planned and unplanned TIB is present in the literature as well. In professional development, the relevance of distinguishing planned, formal learning processes from informal, unplanned learning processes has been recognized for decennia (Marsick & Watkins, Citation1990) and potentially leads to different outcomes (Vermeulen, Citation2016). In the field of Human Resource Development (HRD), a typology was built based on a review study which distinguishes between human development that occurs individually versus collectively and planned versus unplanned (Garavan et al., Citation2015). As TIB is a form of development (Runhaar, Citation2008; Vermeulen, Citation2016), it can also be organized in this typology (Author, Citationsubmitted; Garavan et al., Citation2015). Although several instruments measuring TIB have been developed and validated, these instruments are mostly limited to individual, unplanned TIB (Author, Citationsubmitted). Consequently, instruments measuring other types of TIB, for example planned or collective types of TIB, are still lacking, as is the research on interaction between the different types of TIB. In order to fill this knowledge gap, an instrument covering a more extensive range of TIB, from individual to collective and planned to unplanned TIB, is needed. The aim of the present study is therefore to develop a more comprehensive instrument that measures the multiple facets of TIB.

1.1. Innovative behavior

TIB is often described as an individual process that begins with the recognition of a problem and the generation of ideas and solutions to this problem (Scott & Bruce, Citation1994). The individual then seeks support for the ideas within the organization and applies the ideas into practice (Scott & Bruce, Citation1994). TIB involves processes and products that seem to be intertwined in the current literature (Messmann & Mulder, Citation2012). The definition for TIB most widely used in educational research (Thurlings et al., Citation2015) is that of Janssen (Citation2000, p. 288), who describes innovative behavior as: “the intentional creation, introduction and application of new ideas within a work role, group or organization, in order to benefit role performance, the group, or the organization”. When this definition is dissected, it covers three phases, idea generation, idea promotion and idea realization, in which a teacher can be involved individually or in any social configuration and at any time (Janssen, Citation2000; Scott & Bruce, Citation1994). The phases do not have to follow each other in a sequence (Messmann & Mulder, Citation2012), which means TIB can be present if only one of the three phases is involved (Thurlings et al., Citation2015). Idea generation involves coming up with new ideas, usually related to problems or new trends at work. During the next phase of TIB, the idea is promoted amongst colleagues. This phase is necessary to get support within the organization to eventually implement the idea. The final phase is idea realization, where the idea is put into practice. Because practice can give rise to new ideas through new problems or trends, TIB is considered a cyclical process in which new ideas are continuously generated, which is why some authors also included the phase of reflection in regard to TIB (Lambriex-Schmitz et al., Citation2020; Messmann & Mulder, Citation2012). Given the cyclical character of TIB, reflection can be considered the start of a new cycle, where it is incorporated into idea generation, hence it is not necessary to perceive reflection as a separate dimension. Moreover, reflection can lead to small adjustments, which means that TIB in the context of reflection is sometimes not much more than a routine that leads to anchoring of behavior. The same authors (Lambriex-Schmitz et al., Citation2020; Messmann & Mulder, Citation2012) who use the phase of reflection also include the phase of opportunity exploration. This phase is described as: “the recognition and comprehension of problems and needs in one’s work context that creates an opportunity for change and improvement” (Messmann & Mulder, Citation2012, p.44). Because opportunity exploration with this definition is only an instigator of idea generation (Janssen, Citation2000), it is not used by several authors in the operationalization of TIB (De Jong & Den Hartog, Citation2010; Janssen, Citation2000; Scott & Bruce, Citation1994). Recently, Lambriex-Schmitz et al. (Citation2020) added the dimension of sustainability to the operationalization of Messmann and Mulder (Citation2012). However, this added dimension breaks away from the cyclical character of TIB because, if an innovative idea is permanent, it is incorporated into the organization and at some point, it is no longer considered innovative. Janssen’s (Citation2000) original definition of TIB, (idea generation, promotion and realization) remains the most appropriate for this study, given the following three considerations: reflection can be seen as part of idea generation, opportunity exploration can be considered as just an instigator and lastly, sustainability interferes with the cyclical character of TIB.

1.2. Typology of TIB

In order to structure the concept of TIB, the typology on professional development, derived from HRD (Garavan et al., Citation2015) could serve as a blueprint, as this typology focuses on developing the human potential. This typology is based on the work of Lee (Citation2001) who, based on available HRD literature, distinguishes four strongly interrelated forms of development. Based on these forms of development, two axes emerge along which development takes place: known versus unknown endpoint of development and unitary versus co-regulated identity. A subsequent study (Garavan et al., Citation2015) empirically confirms Lee’s (Citation2001) model, where the known versus unknown endpoint axis is transformed into planned versus unplanned, leaving the definition of the known versus unknown endpoint unchanged. The unitary versus co-regulated identity is transformed into individual versus collective development (Garavan et al., Citation2015). The difference between the approaches of Lee (Citation2001) and Garavan et al. (Citation2015) regarding this axis is that Lee (Citation2001) assumes co-regulation in which the individual is seen in interaction with the collective, while Garavan et al. (Citation2015) explicitly assumes interdependence, which indicates that collective is more indicative of a joint collective activity. This corresponds to the views of Kelchtermans (Citation2021, July), who describes this interdependence as duality (not to be confused with dualism). Duality in TIB means that the types of TIB are always present together, interact with each other and thereby reinforce each other.

TIB can be considered a specific form of development (Runhaar, Citation2008; Vermeulen, Citation2016), consequently Garavan et al. (Citation2015) typology can be used as a foundation to develop a questionnaire for TIB. The position on the axes (planned-unplanned and individual-collective) should preferably be based on process and product, which are often mentioned together when defining TIB in current literature (Messmann & Mulder, Citation2012). The generation of new ideas can be seen as a process, while the actual idea can be considered a product. This distinction between process and product is useful to differentiate between individual and collective TIB (Gerlak & Heikkila, Citation2011; Simons et al., Citation2001). Individual TIB refers to an individual teacher generating, promoting or realizing new ideas. An example of individual TIB is an individual teacher who thinks of new ways to teach pupils how to multiply. TIB is considered individual if the process, the product or both are individual (Castelijns et al., Citation2013; Simons & Ruijters; Simons et al., Citation2001). TIB is considered collective if a group or team of teachers are collectively generating, promoting or realizing new ideas. An example of collective TIB is a conversation between a group of teachers in which they all recognize a problem and brainstorm for new solutions, as a collective group endeavor. The planned vs unplanned dimensions are distinguished in a similar way, based on process and product. Unplanned TIB involves the generation, promotion and realization of ideas as a spontaneous, unplanned process and unplanned product. For example, a teacher might meet a colleague from another school on a train and strike up a conversation. Through this interaction, they come up with new insights for their class. TIB is considered planned if either the process or the product is planned. An example of planned TIB is a group of teachers who are not satisfied with the progression of pupils through the different grades. These teachers consciously plan a trajectory together to generate ideas for solving this problem. In conclusion, unplanned TIB includes both an unplanned process and an unplanned product while in planned TIB, the process, the product or both can be planned (Gerlak & Heikkila, Citation2011; Simons et al., Citation2001).

When the axes planned versus unplanned and individual vs collective from the typology for TIB are combined, four types of TIB emerge (Table , Garavan et al., Citation2015): Autonomous (individual and unplanned), acquisitive (individual and planned), dialogical (collective and unplanned) and networked (collective and planned). These four types of TIB are not mutually exclusive but have a complementary character (Lee, Citation2001). For example, in the staff room, a certain cooperative work form is discussed by two teachers. As the discussion progresses, more and more colleagues join in and become interested in the discussion (dialogical TIB). They are very enthusiastic about the method after trying it once, and it then becomes a fixed item on the agenda of the monthly teacher meeting. At this planned meeting, the teaching staff decides to systematically introduce this cooperative form of working (networked TIB). The introduction of the new, cooperative form then leads to new ideas for individual teachers (autonomous or acquisitive TIB), as well as for new didactics used by whole subject areas (dialogical or networked TIB). This example illustrates how the four types of TIB can influence and blend into each other.

Table 1. Subtypes of TIB, based on Garavan et al. (Citation2015)

1.3. Existing instruments

Since research into innovative behavior has increased significantly in recent years (Thurlings et al., Citation2015), one can expect that the necessary instruments to measure this behavior have also been developed. However, an earlier review study (Author, Citationsubmitted, Citationsubmitted) revealed that quantitative studies based on the definition of TIB (Janssen, Citation2000) made use of existing operationalizations of the autonomous type of TIB (Author, Citationsubmitted, Citationsubmitted); this contrasts with the other three types of TIB where qualitative methods were used in a limited number of studies on these types of TIB. The focus on this individual, unplanned perspective of TIB has no obvious explanation, apart from the sharing of tools within certain research communities.

Over the past twenty years, autonomous TIB has been operationalized with eleven different questionnaires in 26 different publications (Author, Citationsubmitted, Citationsubmitted). A number of these frequently used questionnaires originated outside the educational context (De Jong & Den Hartog, Citation2010, Citation2005; Scott & Bruce, Citation1994) or were developed to measure TIB in a specific branch of education, such as vocational education (Messmann & Mulder, Citation2012) or higher nursing education (Lambriex-Schmitz et al., Citation2020). Because all quantitative instruments were limited to autonomous TIB, the interaction between the different types of TIB could not be investigated, nor the affecting variables of collective TIB, by means of quantitative measurement.

1.4. Design requirements for a new quantitative instrument

To develop an instrument which can encompass the distinguished facets of TIB, it seems appropriate to combine the definition of TIB (Janssen, Citation2000) with the types of TIB (Garavan et al., Citation2015). Given the presence of the word intentional in the definition (Janssen, Citation2000), it can be argued that this definition falls into the acquisitive developmental type (individual and planned), characterized by Garavan et al. (Citation2015), by words such as deliberate and purposeful. Since TIB, as stated earlier, is part of development (Runhaar, Citation2008; Vermeulen, Citation2016), it seems logical to extrapolate Janssen’s (Citation2000) definition, consisting of the three phases (idea generation, promotion and realization), combined with Garavan’s et al. (Citation2015) typology of development. By factoring typical words that fit the types of TIB into the phases of TIB, definitions per type were created that served as a point of reference for the new questionnaire. The typical words were: “I” for individual, “we as a team” for collective, “conscious” for planned and “spontaneous” for unplanned.

1.5. Research question

The blind spots in previous research on TIB that were revealed by the constructed typology (Author, Citationsubmitted, Citationsubmitted), hinder quantitative research on reciprocal relations and specific affecting variables, which lead to the following research question: What dimensions should be considered in developing a comprehensive questionnaire for TIB to be used in practice?

2. Materials and methods

2.1. The design process of a new quantitative instrument

In the current literature, 12 different, individually unplanned TIB questionnaires were used in 26 quantitative studies. This large quantity of questionnaires can act as an anchoring point to develop a comprehensive instrument, which allows measuring all types of TIB. A total of 129 separate questions or statements were extracted from these questionnaires. In a preliminary effort to discern the most pertinent questions, a group of experts, consisting of fellow researchers and practitioners with an interest in the topic of TIB, participated in a round table session during the 2019 April conference. There were eight female participants in the roundtable session, ranging in age from 24 to 63, who originated from European countries. The participants were interested in the subject, as was described in the conference proceedings, were employed at a university or university of applied sciences and were familiar with the use of questionnaires and TIB. After a brief introduction to the types of TIB, the participants were asked to assess which questions from existing lists were best suitable for measuring (individual unplanned) TIB, and to which dimension (idea generation, promotion or realization), as described by Janssen (Citation2000), they belonged. After this first step, the number of usable items was reduced to 28, whereby very similar items from different questionnaires were merged into one. Next, the items were translated into Dutch using the method of back-translation (Cha et al., Citation2007). When the translation was finished, the items were critically examined and altered in relation to the typology of TIB. This approach led to a balanced number of three questions per stage of TIB, which resulted in a total of nine questions per type of TIB. In particular, questions were developed in relation to the types of TIB that remained underexposed in the existing questionnaires. These questions were based on the definitions and the corresponding characteristics derived from the typology of development (Garavan et al., Citation2015). The individual component was operationalized with the word “I”, while collective TIB was operationalized by “we as a team…”. For the planned type of TIB, the word “conscious” was used, while unplanned TIB was operationalized by “spontaneous”.

2.2. Face validity

To investigate how teachers would understand and interpret the reformulated and translated questions, the face validity of the items was investigated. This was done by interviewing three primary school teachers and four secondary school teachers in the Netherlands after they had completed the questionnaire. The interviewees included three women between the ages of 27 and 56 teaching in primary education and two men and two women between the ages of 32 and 45 teaching in secondary education.

A small interview guide was created, which focused on the underlying dimensions of the questions and explored how teachers interpreted the newly formulated questions. The teachers were asked how they interpreted terms as “my team”, “difficult problems”, “job as a teacher”, “the organization”, “spontaneous” and “conscious”. The interview was conducted the same day the interviewees had completed the questionnaire. Each interview lasted approximately 20 minutes and the audio was recorded for analysis.

After the interviews, the recordings were played back, with the research team paying attention to how the underlying terms and latent dimensions of the questionnaire were interpreted. The information about the interpretation of terms was relevant, as these terms formed the core of the questionnaire (Table ). “Spontaneous” was the operationalization for unplanned TIB, in which teachers randomly generalized, promoted or realized ideas. The opposite of spontaneous was “conscious”, where teachers generated, promoted or realized ideas at planned moments, with a preconceived plan. “Team” referred to a collective context in the school with which a teacher identified. This could be a cohort team or a subject team. The terms “work as a teacher”, “the organization” and “difficult problems” were intended to be interpreted broadly in the questionnaire, referring to all the things that a teacher encounters during the job.

Table 2. Summary of interpretation of the concepts (rows) in the questionnaire by five interviewed teachers (columns)

Apart from the terms “spontaneous” and “team”, it appeared that the terms used in the questionnaire and its dimensions were understood by the interviewees as intended by the researchers. There were minor differences observed between the primary and secondary school teachers. The meaning of the word “spontaneous” was not clear for two interviewees, who indicated that an example of spontaneous behavior would be helpful (Table ). “Conscious” was interpreted as intended by all interviewees, namely, planning or thinking of new ideas in advance (Table ). The definition of “team” was ambiguous for two of the interviewees. For the secondary school teachers, it was not clear whether team referred to the team following a cohort of students or the subject team, teaching the same subject, as both could be the case at the same time. As primary schools are often smaller compared to secondary schools, primary school teachers referred to “team” as all their colleagues in the school (Table ). All interviewees interpreted “work as a teacher” broadly, as intended, namely as everything that is related to being a teacher. This also applied to “organization”, which all interviewees understood to refer to organizational or managerial matters that took place outside the classroom. “Difficult problems” were also broadly interpreted, namely as all problems that can occur in a lesson or group. Several examples of spontaneous and conscious behavior were included in the final questionnaire.

2.3. Statistical validation

2.3.1. Design and sample

After establishing its face validity, the questionnaire was distributed among a wide selection of 57 primary and 11 secondary schools in the Netherlands. This resulted in a sample of 178 (53%) primary school teachers and 159 (47%) secondary school teachers, who filled out the complete questionnaire. This roughly represents the population in which the number of primary school teachers (61%) is also overrepresented compared to secondary school teachers (39%) (Ministry of Education, Culture and Science, Citation2021). Regarding gender in primary education, 23 (13%) males and 150 (84%) females completed the questionnaire, with 5 (3%) respondents not indicating their gender. This seems consistent with the population of primary education teachers consisting of 87% women and 13% men (Ministry of Education, Culture and Science, Citation2021). In secondary education, the questionnaire was completed by 76 (48%) men and 81 (51%) women and the gender of 2 (1%) respondents was unknown. This also matches reasonably well with the population of secondary education teachers, in which 45% are men and 55% are women (Ministry of Education, Culture and Science, Citation2021).

The questionnaire was designed utilizing a five-point scale, encompassing response categories ranging from “1. totally disagree” to “5. totally agree.” The inclusion of clear numbering for intermediate options followed the recommendation by Casper et al. (Citation2020). This approach ensured that the collected data were at the interval level, which is particularly advantageous for computing total or average scores in latent constructs, as commonly applied in practical contexts (Norman, Citation2010; Robitzsch, Citation2020). Respondents filled out the questionnaire via a hyperlink that was sent by email and a newsletter of the schools. Completion of the questionnaire was voluntary, and each teacher had the opportunity to not give information about their institution or gender. Teachers were also asked about their number of years of educational experience, rather than their age, as this was more relevant for the present study and ensured anonymity to a greater extent. In total, the questionnaire was completed by 337 respondents.

2.3.2. Procedure

Questionnaire distribution started after the boards of the institutions gave written permission to approach teachers. After the ethical review committee of the University had granted permission, teachers were approached to participate in the present study. The questionnaires were administered using an online data collection tool, which allowed the questionnaires to be accessed via a hyperlink, which was distributed via email or newsletter. Two reminders were sent out for teachers to participate. Once participants opened the questionnaire by clicking on the hyperlink, they first had to grant permission for the data to be processed for scientific research purposes before the questionnaire would launch. It took respondents approximately 15 minutes to complete the questionnaire. After the questionnaires were completed, the data was anonymized and stored in the protected environment of the University.

2.3.3. Analyses

To identify the factors within the newly developed questionnaire, exploratory factor analysis (EFA) was conducted. EFA was preferred over principal component analysis (PCA), as the objective was to identify latent constructs in the present questionnaire (Watkins, Citation2018). We assessed the clustering of the data using the intraclass correlation coefficient (ICC). A minimal ICC value (<.05; Heck & Thomas, Citation2009) indicates negligible variation between groups, indicating the absence of clustered data. Next it was checked whether the data was suitable for EFA, using Bartlett’s test for sphericity and the Kaiser-Meyer-Olkin test. The factors were extracted based on Principal Axis Factoring (PAF), which is a commonly utilized method in EFA (Watkins, Citation2018). PAF was preferred over maximum likelihood (ML)-based extraction because it does not make assumptions about normality distribution and is considered more accurate in retrieving all major factors (Watkins, Citation2018). The number of factors was determined based on parallel analysis (Horn, Citation1965; Watkins, Citation2018) and minimum average partials (MAP; Velicer, Citation1976; Watkins, Citation2018). Parallel analysis especially was preferred over the Kaiser criterion (Eigenvalue >1), as it is more precise in determining the number of factors based on a comparison with a Monte-Carlo simulated dataset (Goretzko et al., Citation2021; Watkins, Citation2018). Next, the factor solution was rotated in an oblique way (Oblimin), as this eases interpretation when factors are expected to be correlated (Field, 2018). After interpreting the EFA, the reliability statistics for the components were determined using McDonald’s Omega. This reliability measure does not assume tau-equivalence and is therefore preferred over Cronbach’s alpha (Hayes & Coutts, Citation2020). Omega also works for a two-item scale, as it leads to the same reliability measure as the Spearman Brown reliability, which is assumed to be the best measure when dealing with a two-item scale (Eisinga et al., Citation2013). Next, the scale scores were compiled based on the regression method to calculate Pearson’s correlation coefficients between the scales. The Pearson’s correlation coefficients were interpreted based on Cohen (Citation1992), who classifies correlations < .30 as small, between .30 and .50 as intermediate and > .50 as strong. All analyses took place using the “psych” package (Revelle, Citation2019) in Jamovi (the Jamovi project, Citation2021).

3. Results

The 337 returned questionnaires contained no missing data, so all respondents could be included in the following analyses. Given the diverse distribution of respondents across a total of 68 schools, we used the Intraclass Correlation Coefficient (ICC) to examine the potential clustering of the data (Table ). Our analysis revealed that only items 35 (“As a team, we spontaneously apply new ideas for solving difficult problems”) and 36 (“As a team, we consciously create new ideas for solving difficult problems”) had variances (6% and 7%), greater than 5%, which could be attributed to the schools in which the teachers were employed. For a few items, the ICC was found to be negative, signifying that the differences within groups were notably smaller than the differences between groups (Kenny et al., Citation2002; Shieh, Citation2016). Descriptive statistics including mean, standard deviation, skewness, and kurtosis were then assessed at the item level (Table ). Since item 2 (“I consciously create new ideas to carry out my work as a teacher”) and item 26 (“I consciously apply new ideas to carry out my work as a teacher”) did not meet all the requirements, we used principal axis factoring (PAF) for factor extraction. PAF was chosen for its robustness in dealing with non-normally distributed items. Bartlett’s test for spherety showed that the correlation matrix (Table ) was not random, X2 (561) = 8401, p < .001and the KMO statistic was .88, both making the data suited for EFA (Watkins, Citation2018).

Table 3. Mean (m), standard deviation (sd), skewness, kurtosis and intraclass correlation coefficient (ICC) for al questionnaire items (n = 337)

Table 4. Pearson correlations (p < .05) between all questionnaire items (n = 337)

Based on parallel analysis (Horn, Citation1965) and MAP, the EFA conducted on all 36 items revealed the presence of seven factors (Table ; Figure ), which together explain 65.5% of the total variance (Table ). The items 1 (“I spontaneously get new ideas to carry out my work as a teacher”), 2 (“I consciously create new ideas to carry out my work as a teacher”) and 6 (“I consciously create new ideas that are good for the organization”) were removed from the analysis, as they showed weak factor loadings on the dimensions they belonged to (factor loading < .40; Stevens, Citation1992). Moreover, item 6 loaded negatively on the opposite, collective unplanned dimension. Because the factor solution was not sufficient at this point the items 25 (“I spontaneously apply new ideas to carry out my work as a teacher”) and 26 (“I consciously apply new ideas to carry out my work as a teacher”) were removed because they showed the lowest communalities (item 25: h2 = .462; item 26: h2 = .427). As the factor solution clearly discriminates between individual and collective TIB, we chose to report the collective and individual scales separately.

Figure 1. Representation of statistically validated factors in the framework of Garavan et al. (Citation2015).

Figure 1. Representation of statistically validated factors in the framework of Garavan et al. (Citation2015).

Table 5. Questionnaire items and their rotated factor loadings (pattern matrix) in relation to the seven factors: (1) collective idea promotion, (2) individual idea promotion, (3) collective planned idea generation and realization, (4) collective unplanned idea generation, (5) individual generation and realization to solve problems, (6) individual planned organizational idea realization and (7) collective unplanned idea realization (n = 337)

Table 6. Discovered factors of TIB, divided into individual and collective factors, the number of items, reliability and explained variance

3.1. Collective

The first collective factor “collective idea promotion (IP)” included all items regarding collective IP, not distinguishing between planned and unplanned IP (Table ). The second collective factor was “collective planned idea generation and realization”. This factor contained all collectively planned, formulated items concerning idea generation and idea realization (Table ). The third collective factor, “unplanned idea generation”, was formed by three unplanned idea generation items and an individual idea generation item (item 5) that was aimed at the organization: “I spontaneously get new ideas that are good for the organization” (Table ). The fourth collective factor, “collective unplanned idea realization”, was formed by all collective, unplanned idea generation items (Table ).

3.2. Individual

The first individual factor, “individual IP” was formed by all individually formulated IP items, also not distinguishing between planned and unplanned IP (Table ). The second individual factor was called “individual generation and realizations to solve problems”. This unexpected factor clustered the four individual items together that emphasized the solving of difficult problems (Table ). The third individual factor was called “planned organizational idea realization”. This factor was formed by two items concerning generating new ideas for the organization (Table ).

3.3. Factor correlations

After determining the latent dimensions, scale scores were calculated based on the regression method used for the correlation matrix (Table ). The correlation matrix showed all positive correlations of which most significant correlations were small (<.30; Cohen, Citation1992). The individual factors however, correlated with each other in a moderate way (r > .30 - < .50; Cohen, Citation1992). The only other moderate correlation with collective factors were found between the individual and collective idea promotion (r = .343, p < .05) and between collective idea promotion and unplanned idea generation (r = .399, p < .05) (Table ).

Table 7. Pearson’s correlation coefficients (p < .05) between the dimensions of TIB (n = 337)

4. Discussion

The aim of the present study was to construct a questionnaire that encompasses the various characteristics of TIB. To achieve this goal, the following research question needed to be answered: What dimensions should be considered in developing a comprehensive questionnaire for TIB to be used in practice? This question was answered by first selecting the most suitable questions in relation to our TIB concept, based on existing questionnaires and the opinion of experts. Next, face validity was investigated by interviewing teachers in the field, this led to some reformulated items. Then, the newly developed questionnaire was administered to a sample of primary and secondary school teachers and EFA was conducted, partially confirming the assumption that the stages of TIB (Janssen, Citation2000) occur in combination with the types of TIB (Garavan et al., Citation2015).

Considering the typology of Garavan et al. (Citation2015), the individual and collective dimensions can be distinguished (Figure ). The difference between planned and unplanned TIB is only clear within the collective dimension, where the scale that combines planned idea generation and idea realization was distinguished from unplanned idea generation and unplanned idea realization (Figure ). Regarding the stages of TIB, formulated by Janssen (Citation2000) and which hypothetically occur within the types of development (Garavan et al., Citation2015), idea promotion could be clearly distinguished in the individual and collective dimensions, without discriminating between planned and unplanned TIB. In the dialogic (collective unplanned) part, idea generation and realization could be distinguished clearly, while these dimensions formed a single construct in the networked (collective planned) part of the quadrant. At the individual side of the framework, apart from idea promotion, the stages of Janssen (Citation2000) could not be clearly distinguished. Instead, items clustered around the concepts of “solving difficult problems” and “ideas for the organization”.

A total of five items, all concerning individual TIB, were removed from the questionnaire because of different reasons. First, items 1, 2 and 6 were removed as they showed too small factor loadings on the dimensions to which they theoretically belonged. Next, items 25 and 26 were removed as they showed low communalities.

The items 1, 2, 25 and 26 were related to the questions concerning “generating or applying new ideas while carrying out the work as a teacher”. The removal of these four items suggests that by “generating or applying new ideas while working as a teacher”, TIB is not well operationalized in the present sample. It is likely that generating and applying new ideas is already inherently part of a teacher’s work to such an extent that it is no longer perceived as new and innovative and therefore does not fit with the other individual questions. This corresponds with the fact that teachers in general often improvise (Sorensen, Citation2017) and thus implicitly generate new ideas on the job and put them into practice. Another explanation for the removal of four individual items comes from Kanter (Citation1988), who argues that complex forms of TIB transcend the individual and take place between teachers, thus complex new ideas are more often generated or applied in collective configurations.

Item 6 concerned the conscious creation of new ideas for the organization, which logically cross loaded negatively with the opposite, collective unplanned dimension. Besides this cross-loading, item 6 had too small of a factor loading on the scale to which it should have belonged.

The planned and unplanned dimensions could not be distinguished on the individual side of the framework. Given that this distinction was identified within the collective side of the framework, one might tentatively argue that it does not solely arise from the choice of words used for operationalization (conscious vs. spontaneous). In other words, at the individual level, autonomous and acquisitive TIB appear to be closely interconnected. This suggests that teachers either do not discriminate between planned and unplanned TIB at the individual level, or that the theoretically highly correlated dimensions of the framework could not be discriminated by de EFA.

The results of the present study imply that a distinction between individual and collective TIB should be made in the future. Although individual and collective TIB seem significantly related, the present study showed that their distinction is also evident. Furthermore, research should investigate the interaction between the quadrants and how they possibly mutually reinforce each other.

4.1. Limitations

By adding a collective dimension, the present exploratory study suggests a completely new approach to the concept of TIB. Because this exploratory study is the first quantitative step in validating this questionnaire, it has several limitations.

The first limitation relates to the fact that the sample was not randomly drawn. Even though the questionnaire was offered to teachers at a wide variety of primary and secondary schools resembling the population (Ministry of Education, Culture and Science, Citation2021), the external validity or generalizability of the results may be limited. In addition, two of the included items had ICC values slightly above the 5% criterion (Heck & Thomas, Citation2009). Since this only occurred for two items and the percentages are still considered low (Bland, Citation2000; Thompson et al., Citation2012), these values are not expected to increase the likelihood of a type 1 error, the chance of finding a nonexistent result. Some ICC values were negative which could be caused by about ten primary schools only containing a single teacher. However, these values were very low and are not likely to cause sampling error. Despite these shortcomings the current sample can still be considered as a reasonable starting point for a first exploratory study, and for further development of the current questionnaire.

The second limitation concerns the method of analysis. The EFA chosen to examine the dimensionality of the questionnaire is appropriate for this exploratory study. However, based on the theoretical model and the clear distinction of individual and collective factors, a second order dimension could be expected, clustering the individual and collective factors from the current research. Such a hierarchical structure needs to be investigated further by confirmatory factor analysis (CFA) (Byrne, Citation2001). In addition, the invariance should be investigated, because the different groups, primary and secondary teachers, may show different factor structures.

The third and most important limitation of the current questionnaire development is the lack of research on construct validity. Construct validity is traditionally investigated based on convergent and discriminant validity (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, Citation2014). Convergent validity involves a positive correlation between the construct measured and an existing instrument, while divergent validity involves a small correlation to theoretically non-related constructs (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, Citation2014). For the individual part of the current questionnaire, it is impossible to determine convergent validity according to the current philosophy of science, as the current questionnaire was based on questions of existing questionnaires. Therefore, correlating the current questionnaire to existing TIB questionnaires would present us with a Münchhausen trilemma (Kirwin, Citation2018), confirming our results with what we originally used as a template. Because convergent validity could not be established, it was also not possible to establish divergent validity, as the correlations of divergent constructs must be less than the convergent constructs (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, Citation2014). Collective TIB could theoretically be correlated to collective learning. But, as far as we know, there are no instruments available that measure collective learning in the specific group we investigated, namely teachers. Besides, TIB involves only partial aspects of collective learning. In summary, it was not possible to investigate construct validity because the current questionnaire is based on existing questionnaires and is too specific in terms of target group and concept, to relate to collective learning.

4.2. Future research

Based on the previously mentioned limitations, recommendations for follow-up research can be made. The first recommendation lies with further statistical validation of the questionnaire. Based on the preliminary findings in the present study, a larger sample could be tested in a confirmative factor analysis (CFA) with a hierarchical structure (Byrne, Citation2001) to provide more insights on the entire construct. For instance, based on the present study’s results, it might be expected that individual TIB and collective TIB form latent constructs in the questionnaire. Besides this statistical recommendation, future research should also further explore TIB as a concept. The present study is the first to approach TIB from a collective perspective, using team as a specific operationalization. Perhaps the concept of TIB reaches even further and is not limited to individual teachers or the team but extends to the level of the organization, as is the case with learning (Lin & Sanders, Citation2017).

Future research should go beyond the individual teacher and consist of various facets that interact in line with the present study. For example, interviews could be conducted to get a more nuanced picture of TIB, while carrying out the work as a teacher in relation to collective TIB. In this way, more insight can be gained into why the items around individually generating and applying ideas to perform the work as a teacher do not fit the dimensions of the present study. Insights of such an interview study would allow the present instrument and its possible future developments to provide a foundation for a better understanding of TIB processes in practice, which is important for conducting research on how TIB can be facilitated and stimulated in 21st century education.

4.3. Implications for practice

The present questionnaire is the first TIB measurement instrument that recognizes that TIB takes place in collective forms, making it more suitable for application within the structure of a school where teachers are always part of a team. Since it can be argued that a collective of teachers can achieve more than the same individual teachers can achieve on their own, the present study research paves the way to further examine relevant factors in relation to collective forms of TIB. Understanding how to foster TIB in teacher teams can be a sustainable development for schools working toward a learning organization.

Supplemental material

Author_Bios_minors.docx

Download MS Word (13 KB)

Acknowledgments

We thank Dr. Eric Robbers, Maartje van der Lek (MSc), Maud Goedemans (MSc) and Antonie Rozema (MSc) for their assistance with the data collection.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/2331186X.2023.2283641.

Additional information

Notes on contributors

Stefan Robbers

Stefan Robbers, MSc, is a Phd student at the faculty of educational sciences at the open university (OU) of the Netherlands. After several years of working as a teacher in secondary education, he started a PhD project on innovative behavior of teachers. During the years of his PhD, he developed an interest in collective learning, social network analysis, and review studies. In addition, he was involved as a teacher in the Master’s in Educational Sciences at the Open University of the Netherlands.

Arnoud Evers

Arnoud Evers, PhD, is assistant professor at the Faculty of Educational Sciences at the Open University (OU) of the Netherlands. He is visiting scholar at the Teaching Systems Lab (TSL) group of MIT (Cambridge, Massachusetts). He has also previously been Board Member of the European Association for Practitioner Research on Improving Learning (EAPRIL). His main research areas are: teachers’ learning at work, innovative schools and behavior and HRD. He published, among others, in Review of Educational Research, Human Resource Development Review, European Journal of Training and Development, Studies in Continuing Education, and Vocations and Learning.

Marjan Vermeulen

Marjan Vermeulen, Prof, is professor at the faculty of Educational science of the Open University of the Netherlands. Her professorship is funded by teacher education institute ‘De Kempel’ and education advice organization ‘Onderwijs maak je samen’ (Education is made together). Her expertise lies in supporting educational practice as well as educational research in the field of professional development and innovative behavior of teachers, the school as organization of work for teachers and school leadership, educational development and innovation.

References

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014) . Standards for educational and psychological testing. American Educational Research Association.
  • Author, [submitted]. Framing teachers’ innovative behavior: A systematic review.
  • Bland, J. M. (2000). Sample size in guidelines trials. Family Practice, 17(1), 17–23. https://doi.org/10.1093/fampra/17.suppl_1.s17
  • Byrne, B. M. (2001). Structural equation modeling with AMOS basic concepts, applications, and programming. Lawrence Erlbaum Associates.
  • Casper, W. C., Edwards, B. D., Wallace, J. C., Landis, R. S., & Fife, D. A. (2020). Selecting response anchors with equal intervals for summated rating scales. Journal of Applied Psychology, 105(4), 390–409. https://doi.org/10.1037/apl0000444
  • Castelijns, J., Koster, B., & Vermeulen, M. (2009). Vitaliteit in Processen Van Collectief Leren: Samen Kennis Creëren in Basisscholen en Lerarenopleidingen [vitality in processes of collective learning: Creating knowledge together in primary schools and teacher training colleges]. Garant.
  • Castelijns, J., Vermeulen, M., & Kools, Q. (2013). Collective learning in primary schools and teacher education institutes. Journal of Educational Change, 14(3), 373–402. https://doi.org/10.1007/s10833-013-9209-6
  • Cha, E. S., Kim, K. H., & Erlen, J. A. (2007). Translation of scales in cross‐cultural research:issues and techniques. Journal of Advanced Nursing, 58(4), 386–395. https://doi.org/10.1111/j.1365-2648.2007.04242.x
  • Cohen, J. (1992). Statistical power analysis. Current Directions in Psychological Science, 1(3), 98–101. https://doi.org/10.1111/1467-8721.ep10768783
  • De Jong, J., & Den Hartog, D. N. (2010). Measuring innovative work behavior creativity and innovative Management. Journal of Management, 19(1), 23–36. https://doi.org/10.1111/j.1467-8691.2010.00547.x
  • De Jong, J. P. J., & Den Hartog, D. N. (2005). Determinanten Van Innovatief Gedrag: Een Onderzoek Onder Kenniswerkers in Het MKB. [Determinants of Innovative Behavior: A Survey of Knowledge Workers in SMEs.]. Gedrag & Organisatie, 18(5), 235–259. https://doi.org/10.5117/2005.018.005.001
  • Eisinga R, Grotenhuis M te and Pelzer B. (2013). The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown?. Int J Public Health, 58(4), 637–642. 10.1007/s00038-012-0416-3
  • Garavan, T. N., McGuire, D., & Lee, M. (2015). Reclaiming the “D” in HRD: A typology of development conceptualizations, antecedents, and outcomes. Human Resource Development Review, 14(4), 359–388. https://doi.org/10.1177/2F1534484315607053
  • Gerlak, A. K., & Heikkila, T. (2011). Building a theory of learning in collaboratives: Evidence from the Everglades Restoration Program. Journal of Public Administration Research and Theory, 21(4), 619–644. https://doi.org/10.1093/jopart/muq089
  • Goretzko, D., Pham, T. T. H., & Bühner, M. (2021). Exploratory factor analysis: Current use, methodological developments and recommendations for good practice. Current Psychology, 40(7), 3510–3521. https://doi.org/10.1007/s12144-019-00300-2
  • Hayes, A. F., & Coutts, J. J. (2020). Use omega rather than Cronbach’s alpha for estimating reliability. But …. Communication Methods and Measures, 14(1), 1–24. https://doi.org/10.1080/19312458.2020.1718629
  • Heck, R. H., & Thomas, S. L. (2009). An introduction to multilevel modeling techniques. Lawrence Erlbaum Associates Publishers.
  • Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. https://doi.org/10.1007/BF02289447
  • The jamovi project. (2021). Jamovi. (Version 2.0) [Computer Software]. Retrieved from https://www.jamovi.org.
  • Janssen, O. (2000). Job demands, perceptions of effort‐reward fairness and innovative workbehaviour. Journal of Occupational and Organizational Psychology, 73(3), 287–302. https://doi.org/10.1348/096317900167038
  • Kanter, R. M. (1988). When a thousand flowers bloom: Structural, collective and social Conditions for innovation in organization. Research in Organizational Behavior, 10, 169–211. https://doi.org/10.1016/B978-0-7506-9749-1.50010-7
  • Kelchtermans, G. (2021, July). Onderwijsvernieuwing en schoolontwikkeling: terugblik en vooruitblik op onderzoek en theorievorming [Educational innovation and school development: looking back and ahead to research and theory building] [invited symposium]. Proceedings of the Onderwijs Research Dagen (ORD) 2021 conference, Utrecht, the Netherlands.
  • Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. A. (2002). The statistical analysis of data from small groups. Journal of Personality and Social Psychology, 83(1), 126–137. https://doi.org/10.1037/0022-3514.83.1.126
  • Kirwin, C. (2018). Pulling oneself up by the hair: Understanding Nietzsche on freedom. Inquiry, 61(1), 82–99. https://doi.org/10.1080/0020174X.2017.1371829
  • Lambriex-Schmitz, P., Van der Klink, M. R., Beausaert, S., Bijker, M., & Segers, M. (2020). Towards successful innovations in education: Development and validation of a multi-dimensional innovative work behavior instrument. Vocations & Learning, 13(2), 313–340. https://doi.org/10.1007/s12186-020-09242-4
  • Lee, M. (2001). A refusal to define HRD. Human Resource Development International, 4(3), 327–341. https://doi.org/10.1080/13678860110059348
  • Lin, C. H., & Sanders, K. (2017). HRM and innovation: A multi‐level organizational learningperspective. Human Resource Management Journal, 27(2), 300–317. https://doi.org/10.1111/1748-8583.12127
  • Marsick, V. J., & Watkins, K. E. (1990). Informal and incidental learning in the workplace. Routledge.
  • Messmann, G., & Mulder, R. H. (2012). Development of a measurement instrument for innovative work behavior as a dynamic and context-bound construct. Human Resource Development International, 15(1), 43–59. https://doi.org/10.1080/13678868.2011.646894
  • Ministry of Education, Culture and Science. (2021). Trend report labor market teachers primary education, secondary education and vocational education 2021 [Trendrapportage arbeidsmarkt leraren PO, VO en mbo]. Rijksoverheid.
  • Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education, 15(5), 625–632. https://doi.org/10.1007/s10459-010-9222-y
  • OECD. (2020). Professional collaboration as a key support for teachers working in challenging environments. Teaching in Focus, 34. https://doi.org/10.1787/c699389b-en
  • Revelle, W. (2019). Psych: Procedures for Psychological, Psychometric, and Personality Research. [R Package]. Retrieved from https://cran.r-project.org/package=psych.
  • Robitzsch, A. (2020). Why ordinal variables can (almost) always be treated as continuous variables: Clarifying assumptions of robust continuous and ordinal factor analysis estimation Methods. Frontiers in Education, 5, 589965. https://doi.org/10.3389/feduc.2020.589965
  • Runhaar, P. (2008). Promoting Teachers’ Professional Development. Dissertation, University of Twente.
  • Scott, S. G., & Bruce, R. A. (1994). Determinants of innovative behavior: A path model of individual innovation in the workplace. Academy of Management Journal, 37(3), 580–607. https://doi.org/10.5465/256701
  • Shieh, G. (2016). Choosing the best index for the average score intraclass correlation coefficient. Behavior Research Methods, 48(3), 994–1003. https://doi.org/10.3758/s13428-015-0623-y
  • Simons, P. R. J., Ruijters, M. C. P., & P, M. C. (2001). Work related learning: Elaborate, expand, externalize. In L. Nieuwenhuis (Ed.), Dynamics and stability in VET and HRD (p. 2001). Twente University Press.
  • Sorensen, N. (2017). Improvisation and teacher expertise: Implications for the professional development of outstanding teachers. Professional Development in Education, 43(1), 6–22. https://doi.org/10.1080/19415257.2015.1127854
  • Stevens, J. P. (1992). Applied multivariate statistics for the social sciences (2nd ed.). Erlbaum.
  • Thompson, D. M., Fernald, D. H., & Mold, J. W. (2012). Intraclass correlation coefficients typical of cluster-randomized studies: Estimates from the Robert Wood Johnson Prescription for Health projects. The Annals of Family Medicine, 10(3), 235–240. https://doi.org/10.1370/afm.1347
  • Thurlings, M., Evers, A. T., & Vermeulen, M. (2015). Toward a model of explaining teachers’ innovative behavior. Review of Educational Research, 85(3), 430–471. https://doi.org/10.3102/0034654314557949
  • Velicer W F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41(3), 321–327. 10.1007/BF02293557
  • Vermeulen, M. (2016). Leren Organiseren. Een Rijke Leeromgeving Voor Leraren En Scholen [learning to organise a rich learning environment for teachers and schools]. Open University.
  • Voogt, J., & Pareja Roblin, N. (2010). 21st Century skills. University of Twente.
  • Watkins, M. W. (2018). Exploratory factor analysis: A guide to best practice. Journal of Black Psychology, 44(3), 219–246. https://doi.org/10.1177/0095798418771807