Measuring twenty-first century skills: development and validation of a scale for in-service and pre-service teachers

Abstract

A self-report scale that measures teachers’ confidence in teaching students about twenty-first century skills was developed and validated with pre-service and in-service teachers. First, 16 items were created to measure teaching confidence in six areas: information literacy, collaboration, communication, innovation and creativity, problem solving, and responsible citizenship. Then, the items were tested in two groups of pre-service teachers and two groups of in-service teachers respectively. Exploratory factor analyses and confirmatory factor analyses were conducted to examine the scale’s validity and reliability. The analyses identified a three-construct scale including innovation and problem solving, collaboration, and utility of technology for pre-service teachers and a one-dimension scale, cross-functional skills, for in-service teachers. This study suggests that different scales should be used to measure in-service teachers’ and pre-service teachers’ confidence in teaching twenty-first century skills.

Keywords:

• twenty-first century skills
• instrument development
• factor analysis
• survey research
• teacher effectiveness

Notes

1. Conventionally, SRMR smaller than .05, RMSEA smaller than .06, CFI larger than .96, and TLI larger than .95 are considered to be a good fit (Hu and Bentler 1999). Whereas SRMR smaller than .08, RMSEA smaller than .1, CFI larger than .90, and TLI larger than .90 are considered to be an acceptable fit (Brown and Cudeck 1993).

2. Loading size larger than .60 was considered to be high and larger than .45 was considered acceptable. Loading size larger than .35 in over one factor was considered as cross-loading (Pett, Lackey, and Sullivan 2003; Silvera, Martinussen, and Dahl 2001; Tabachnick and Fidell 2001).

3. Four factors had an eigenvalue larger than 1, indicating a four-factor solution. The scree plot also showed that from the fourth factor on, the fractions of the total variance explained by each successive factor were minimal. Model fit indices for each of the four factor models were shown in Table 2.

4. According to Cohen (1988), a correlation of .30 to .50 was moderate.

5. The factor loadings from .75 to .81 for Utility of Technology, .55 to .82 for Collaboration, and .64 to .81 for Innovation and Problem Solving.

6. Three factors had an eigenvalue larger than 1, indicating a three-factor solution. The scree plot showed that, after the third factor, the fraction of the total variance explained by each successive factor dramatically dropped. Model fit indices for each of the four factor models were shown in Table 4.

Additional information

Funding

This work was supported by the Research and Evaluation Program at the Nellie Mae Educational Foundation.