Teachers’ exposure to professional development and the quality of their instructional technology use: The mediating role of teachers’ value and ability beliefs

Abstract

One way to help teachers use technology effectively is through professional development (PD). However, understanding of how exposure to PD relates to teachers’ personal characteristics is limited. The purpose of this research was to investigate the relationships among PD exposure, teachers’ abilities and values, and teachers’ quality of technology integration according to Bloom’s taxonomy. By surveying 724 middle and high school teachers, and using structural equation modeling, this study showed that values mediate the influence of PD exposure on technology integration. Results suggest that PD may be most effective when targeting improving teachers’ values in addition to enhancing technology-related skills.

Keywords:

• Teacher professional development
• value belief
• classroom technology integration
• barriers

The need for teachers to use technology, to improve both teaching practices and student learning, has become more and more evident. One of the most important ways to help teachers use technology more effectively is the provision of professional development (PD). PD are programs offered to in-service teachers in order to enhance teachers’ knowledge, strategies, as well as other related teacher characteristics that may influence their teaching (Barrett et al., 2012). Empirical evidence has shown that many PD programs are effective at improving how teachers integrate technology in the classroom. Teachers who are exposed to quality professional development are more likely to see an increase in their skills and abilities (Liu et al., 2015; Xie et al., 2017). Given recent evidence of the importance of value beliefs in how teachers ultimately integrate technology, scholars have called for PD programs to also address teachers’ value beliefs toward instructional technology use (Cheng et al., 2020; Cheng & Xie, 2018; Er & Kim, 2017; Kim et al., 2017; Liu et al., 2015). However, few studies have thus far answered the call to examine the relationship between PD exposure and teachers’ technology-related value beliefs. Improved understanding on this topic would bolster efforts to target value beliefs as an outcome of PD. Furthermore, a more integrated examination of the relative influence of PD exposure on both teachers’ perceived ability and value beliefs would offer a more holistic picture of how PD programs can help teachers use technology in more consistent and purposeful manners. In order to design effective and targeted PD programs, more evidence is needed to understand the arc of possible teacher outcomes.

Given that institutions are investing in PD programs, more research is needed to understand different factors that may influence the efficacy of these efforts. The purpose of this research is to investigate how teachers’ exposure to PD programs are associated with their perceived ability and value beliefs, and how these variables in turn influence their technology integration practices.

Literature review

Barriers to technology integration model

The ways in which teachers use technology in the classroom are influenced by multiple factors. The Barrier to Technology Integration model (Ertmer, 1999; Hew & Brush, 2007) described these factors as ‘barriers’ that hinder how and how much teachers integrate technology. According to Ertmer’s model (1999) these barriers may be external to teachers, also called first-order barriers, or they may be internal to teachers, also called second-order barriers. It is important to understand these barriers in order to translate technology integration policies into meaningful changes in how technology is used in the classroom (Ertmer et al., 2012). For several decades, research has been conducted to define these barriers and develop strategies to overcome them.

First-order barriers include such factors as institutional culture and vision, access to technology, and PD opportunities (Ertmer, 1999; Hew & Brush, 2007). The culture and vision of the institution may include the expectations conveyed, or lack thereof, and support provided by school leadership (Al-Bataineh et al., 2008; Ertmer, 1999; Fu, 2013; Hew & Brush, 2007). Access to technology may include both the availability of devices for students to use, and support in using and maintaining the devices. Merely having devices does not necessarily mean all teachers and students have access to them (Hew & Brush, 2007). For example, available computers may be arranged in labs, organized on a mobile cart for checkout, or divided among classrooms with a few devices in each (Ryan & Bagley, 2015). Additionally, schools may or may not have dedicated technology support personnel. The third external barrier, PD opportunities, may include a lack of easily accessible training for teachers to learn strategies for implementing technology in their classrooms (Yildirim, 2007). First-order barriers had been shown in studies to be significant obstacles to achieving technology integration (Ertmer, 1999; Ertmer et al., 1999; O’Mahony, 2003).

Second-order barriers include value beliefs and ability. Value beliefs encompass attitude – the like or dislike of something – and beliefs about technology’s usefulness for the classroom (Hew & Brush, 2007). Ability beliefs may include a lack of perceived knowledge or skills for integrating technology into the classroom. Hew and Brush (2007) further explain that technology integration ability relates to the use of specific technology, pedagogical practices supported by technology, and classroom management practices. Both second-order barriers rely on teachers’ internal perceptions of the ways in which technology may or may not fit into their teaching practice. They are crucial to translating available technology and related resources into actual usage in the classroom (Ertmer et al., 1999; O’Mahony, 2003; Vongkulluksn et al., 2018).

In the last decade, many research and policy programs have focused on overcoming first order barriers. For example, the Every Student Succeeds Act passed in 2015 provided $1.65 billion dollars in grants toward “improv[ing] the use of technology in order to improve the academic achievement and digital literacy of all students” (ESSA, n.d.). Much of the money went into purchasing devices, building the necessary infrastructure in schools, and providing professional learning opportunities for teachers related to technology (Reynolds et al., 2016). Many researchers have focused on the overall impacts of technology availability to learning (e.g. Bebell & O’Dwyer, 2010; Zheng et al., 2016), as well as how arrangements of technological resources may influence their usage in practice (e.g. Jenson & Rose, 2006; Tondeur et al., 2015). Through collective efforts, first-order barriers have, to a large extent, been minimized. Researchers have thus recently turned their attention to mitigating second-order barriers.

Studies have shown that teacher beliefs have direct relationships with how teachers use technology in the classroom, thus translating technology availability into actual practice (Cheng et al., 2020; Ertmer et al., 2012; Vongkulluksn et al., 2018; Wozney et al., 2006; Xie et al., 2019). For example, Ertmer et al. (2012) conducted interviews with twelve teachers who had previously won awards based on their classroom technology integration practice. In these interviews, teachers indicated that internal factors were key to their classroom practices. Given the noted importance of value beliefs, multiple scholars have called for more in-depth examinations of the ways in which second-order barriers can be improved (Er & Kim, 2017; Kim et al., 2017; Liu et al., 2015).

Professional development for improving teachers’ ability and value beliefs

Because both ability and value beliefs are important factors for technology integration, educational technology research should focus on finding ways to foster these positive teacher characteristics through PD programs. PD programs may be compulsory, targeted to teachers with certain characteristics, or voluntary (Barrett et al., 2012). Most PD programs have focused on improving teachers’ ability beliefs for integrating technology. Some examples are Evaluating Digital Contents for Teaching and Instructional Excellence (Kim et al., 2017; Xie et al., 2017) and Collaborative Professional Development for technology integration (Liu et al., 2015). Kim et al. (2017) conducted a year-long comprehensive PD course in which participants attended face-to-face sessions and completed online modules. These focused on the knowledge and processes involved in evaluating digital learning content and technology integration training geared at improving teachers’ technological, pedagogical and content knowledge. The participants were 171 teachers from five school districts across central Ohio. Through this, teachers became more familiar with and more confident in evaluating digital content for classroom use. Liu et al. (2015) conducted a PD program with three pairs of mentor teachers and pre-service teachers. The PD was enacted in four cycles, each of which included a teaching demonstration by the mentor on how to integrate technology and a lesson taught by each pre-service teacher. The program had a significant influence on teachers’ perceived ability toward integrating technology with their content-specific curriculum. These and many other PD programs focused on improving teachers’ knowledge of and confidence toward technology integration.

Despite its prominence in technology integration models such as the Barrier model, only a few PD programs focused on improving teachers’ value beliefs for technology. One exception is a study by Er and Kim (2017), which described a PD program targeting teachers’ value beliefs using the Episode-Centered Belief Change (ECBC) model. This model utilized teachers’ memories of classroom observation to discover what beliefs they hold that may impede effective technology integration. These memories can be used in professional development to change or restructure teachers’ value beliefs. The ECBC program is only one of a few specifically geared toward improving technology-related value beliefs. More work is thus needed to understand how technology-related PD programs may influence teachers’ value beliefs. Indeed, several researchers have identified a greater need to focus on value beliefs for future PD programs and related studies. For example, Cheng and Xie (2018) noted that PD programs should address value beliefs as one of the target outcomes. Kanaya et al. (2005) also identified that quality technology PD programs should include addressing teachers’ current needs and interests, rather than presenting technology separately from the curricular needs. By doing so, a quality PD program has the opportunity to shed light on the utility of technology, addressing teachers’ value beliefs about technology integration practices.

While the literature on technology PD programs is extensive, there are several notable gaps. First, many studies of teacher PD programs in general and technology PD programs in particular have focused on groups of teachers who self-selected to participate (Barrett et al., 2012; Teitel, 2000; Vallett et al., 2018). Thus, results may not reflect trends for teachers with varying degrees of ability and value beliefs (Barrett et al., 2012; Vallett et al., 2018). Second, as previously mentioned, most technology PD programs target improving teachers’ ability and perceived ability for technology integration, while only a few focused on improving their value beliefs. Third, current studies have primarily examined the relationship between PD exposure and these targeted outcomes in isolation. However, other studies in educational technology have pointed to the close relationship between perceived ability and value beliefs (Hsu et al., 2017; Vongkulluksn et al., 2018). Teachers who have high perceived ability are also more likely to have high value beliefs toward technology integration. Also, factors that influence one variable have also been shown to influence another. For example, levels of technology knowledge were shown to influence both perceived ability and value beliefs (Inan & Lowther, 2010). Given the close relationship between perceived ability and value beliefs, teachers who are exposed to PD programs would likely improve on both of these factors at the same time. Understanding the interplay between perceived ability and value beliefs, as well as how PD programs may influence both teacher factors at the same time is useful for tailoring future PD programs to intentionally target such malleable factors.

How professional development improves technology integration

The aim of technology PD programs is to help teachers use technology in a more consistent and purposeful manner. However, scholars have noted that the ways in which PD exposure influence teachers’ behaviors are often indirect, mediated through variables such as perceived ability and value beliefs (Schrader & Lawless, 2004; Vongkulluksn et al., 2018). Effective technology integration begins with teacher change (Ertmer & Ottenbreit-Leftwich, 2010; Harris, 2005). Therefore, PD programs aiming to influence change in how technology is used in the classroom should target changing teacher-related factors (Ertmer & Ottenbreit-Leftwich, 2010). This not only includes improving teachers’ strategic and instructional knowledge, but also their perceived ability and attitudes. Empirical research also supported the mediating role of perceived ability and value beliefs. For example, in their comprehensive study of 1,382 in-service teachers, Inan and Lowther (2010) found that an increase in teachers’ proficiency with technology is associated with gains in teachers’ perceived ability and value beliefs. These two variables then have a direct influence on how much teachers integrate technology in the classroom. Inferring from results of their study, ability and value beliefs may also mediate the relationship between technology proficiency gained through professional development and teachers’ technology integration practice. While some evidence exists for the mediating role of ability and value beliefs, few have specifically tested how PD exposure may work to improve teachers’ technology use through these variables. Understanding how perceived ability and value beliefs play a role in translating PD exposure to teacher behavior outcomes will clarify how PD works to improve integration effectiveness, as well as to encourage including these crucial teacher variables as markers of PD effectiveness.

The purpose of technology PD programs, and indeed technology integration efforts overall, is to leverage available technology tools for improved student learning. The International Society for Technology in Education (ISTE) has standards for educators that provide guidelines to encourage and enable learners. These include, for example, setting personal goals to learn about and apply new technology-based pedagogical approached and reflecting on them, modeling for colleagues the evaluations and adoption of digital resources, collaborating and planning with other educators to create authentic learning experiences with technology, and fostering student learning and curiosity using digital resources that maximize students’ deep learning (ISTE, 2020). However, PD programs often measure their ultimate effectiveness in terms of the quantity of technology used by participating teachers, the types of technology used (Al-Bataineh et al., 2008; Kanaya et al., 2005; O’Mahony, 2003; Yildirim, 2007), and broad modes of use (e.g. a research tool, a productivity tool, a communication tool, etc.; Barron et al., 2003). They tend to talk about what tools teachers use and how much teachers use them. Because the ultimate goal of teacher technology professional development is to help teachers gain proficiency in technology integration in order to improve students’ outcomes (ISTE, 2020), we need to assess PD’s influence in a way that is more connected to student learning (Lawless & Pellegrino, 2007), and how well the technology integration facilitates the learning process (Davies, 2011). It is imperative that teachers develop quality technology-enhanced learning experiences that allow students to develop their cognitive skills in both lower and higher cognitive domains.

A review of the extant literature revealed that few studies have examined how students used technology to engage in cognitive learning tasks. Some studies have used qualitative data, such as student interviews (Dietrich & Balli, 2014) and observations (Dugdale et al., 1998; Heflin et al., 2017), while others surveyed students’ learning engagement in digital environments, but utilized measures of general cognitive engagement not specific to technology use (Gašević et al., 2017; Pellas, 2014; Sun & Rueda, 2012). Thus, it is useful to examine a framework from which a measurement tool might be developed.

One way to measure the quality of technology integration practices is to analyze teachers’ technological instructional strategies within the cognitive domain, i.e. the use of technology to facilitate lower- and higher-order critical thinking tasks. Bloom’s Digital Taxonomy (Churches, 2010), based on the original Bloom’s Taxonomy (Bloom, 1956) and the revised taxonomy (Anderson & Krathwohl, 2001) depicts how teachers can use technology and digital tools to facilitate student learning experiences and outcomes. It illustrates a hierarchy of digital learning activities progressing from lower-order learning tasks to higher-order learning tasks. Lower-order cognitive tasks refer to those used for remembering, understanding, and applying, while higher-order tasks are those used for analyzing, evaluating, and creating (Churches, 2010; Krathwohl, 2002). Students can engage in tasks at any level, at any time. At the lower-order level, students may use digital flash cards to memorize words or make bookmarks to webpages related to a specific project, while higher-order thinking may involve posting discussion comments and replying to others.

Achieving higher-order cognitive skills is an important life-skill. The ability to think critically about information stays with the student throughout their lives (Churches, 2010). Technology can be used in many ways to help students achieve higher-order cognitive tasks and obtain life-long skills, such as organizing information in a way that students can critically analyze and make judgements about their sources. As stated by Churches (2010) and others, the goal of technology integration is not about the technology itself, but about how the technology is used to achieve greater learning outcomes. When teachers see the value of technology and effectively integrate technology into instructional practice, technology tools can extend learning in different ways. To that end, we characterized teacher practices with technology in this study in terms of their use of technology to facilitate lower- and higher-order critical thinking tasks (Churches, 2010; Krathwohl, 2002).

The current study

Consistent with the gap in research related to relationships among exposure to professional development, ability and value beliefs, and teachers’ technology integration practices as they relate to integration of technology for lower and higher cognitive tasks, the present study aims to answer the following research questions:

1. Does PD exposure significantly predict the quality of technology use (e.g. for lower and higher cognitive tasks)?

2. Do second-order barriers (ability and value beliefs) significantly predict the use of technology for lower and higher cognitive tasks?

3. Is the relationship between PD exposure and instructional practices with technology mediated by ability beliefs?

4. Is the relationship between PD exposure and instructional practices with technology mediated by value beliefs?

As shown in the theoretical model (Figure 1), the hypotheses are:

Figure 1. Theoretical Model.

H1-3: PD exposure, value, and ability beliefs will significantly predict greater use of technology for lower- and higher-order tasks.

H4: Ability beliefs will significantly explain teachers’ translation of PD exposure into actual classroom practices.

H5: Value beliefs will significantly explain teachers’ translation of PD exposure into actual classroom practices.

Materials and methods

Participants and procedure

The sample for this study were 724 sixth- to twelfth-grade math, English language arts, science, and social studies teachers in 17 schools across a midwestern state in the United States. Schools were sampled from those with existing partnerships with the authors’ research team. Participating schools range in having 46.9 to 82.7 percent of teachers with at least a master’s degree. The average class size for participating teachers is 15 students. Ninety-one percent of teachers in the sample taught grades 9^th to 12^th, 4 percent of teachers exclusively taught grades 6^th to 8^th, and the remaining 5 percent taught both middle and high school grades.

In the spring of 2017, participating teachers were asked to respond to an online survey using Qualtrics about how they use technology for instruction and their perceptions of barriers to technology integration, which included: access to technology, institutional vision and culture, and exposure to professional development. The survey also included items measuring teachers’ beliefs about the value of technology and their perceived abilities with technology.

Measures

Perceptions of first-order barriers

A Likert-scale survey from Kopcha (2012) was used to measure teachers’ perceptions of first-order barriers, namely institutional (4 items, Cronbach’s α = .793) and access (4 items, Cronbach’s α = .807), and PD exposure (3 items, Cronbach’s α = .824). These Cronbach’s α are in acceptable range, which has been estimated from best practices in current literature as between 0.70 and 0.95 (Bonett & Wright, 2015; Tavakol & Dennick, 2011). The items were constructed such that higher responses on the 5-point scale represented facilitative conditions toward technology integration. A sample item of PD exposure is ‘I feel adequately trained on the skills needed to use technology and digital contents.’ As noted, these are perceptions of access, for example, not actual access (number of computers per student, etc.), therefore these are latent variable constructs specific to each teacher.

Perceptions of second-order barriers

Value beliefs about technology use were examined through the theoretical lens of the expectancy-value theory, as has been supported in prior research (Cheng et al., 2020). Value belief was measured using a 7-point Likert-scale survey from Wigfield and Eccles (2000; 6 items, Cronbach’s α = .941), with endpoints indicated. Consistent with Expectancy-Value Theory of Achievement Motivation, value was measured using the latent factors of intrinsic value (2 items), utility (2 items), and attainment (2 items). These constructs together make up the value variable. A sample item of intrinsic value is ‘In general, I find spending time on integrating technology into my lessons (very boring – very interesting).’

A Likert-scale survey from Kopcha (2012) was used to measure teachers’ ability beliefs (5 items, Cronbach’s α = .86), which is a widely used and validated instrument to measure teachers’ perceived ability toward technology integration. The items were constructed such that higher responses on the 5-point scale represented higher levels of ability belief. A sample item is ‘I have the skills necessary to use technology in the classroom.’

Technology integration practices: lower and higher order usage

The outcome variables were from a survey based on Bloom’s Digital Taxonomy, previously created by the authors (Vongkulluksn et al., 2019). Nine items targeted technology use for lower-order tasks (remembering (3 items), understanding (3 items), and applying (3 items); Cronbach’s α = .899) and nine items targeted higher-order tasks (analyzing (3 items), evaluating (3 items), and creating (3 items); Cronbach’s α = .936). Teachers reported how often they ask students to do specific activities using technology (0 = never to 5 = very often). A sample item for evaluating is ‘How often do you ask students to use technology to compare multiple sources of information?’ Based on Bloom’s taxonomy, these tasks indicate levels of complexity of student learning. As such, technology use for higher-order tasks indicates a higher quality of usage.

Statistical analysis

We calculated the intra-class correlations (ICCs) analysis to check if it is necessary to conduct multi-level modeling for the present study. The observed ICCs range from 0.018 to 0.045 over the measured variables, which indicates low school-level variability. So, it was not necessary to conduct a multi-level analysis (Woltman et al., 2012). Structural equation modeling (SEM) was used to examine the associations of each barrier on teachers’ technology integration practices, including for usage for lower-order and higher-order tasks.

Preliminary analysis includes the reliability coefficients, descriptive analysis and bivariate correlation. Data were analyzed by using Mplus 7.4. The missing values were coded as 9999 and automatically handled by maximum likelihood estimation approach in Mplus. Eleven separate measurement models were tested to examine the measurement for each latent variable. To assess the fit of the model, we used five criteria: the chi-square, root mean square error of approximation (RMSEA), the comparative fit index (CFI), Tucker Lewis Index (TLI) and standardized root mean square residual (SRMR). For the chi-square test, a p-value greater than .05 shows a good overall model fit, failing to reject the null hypothesis that the specified model and observed data are similar. For both RMSEA and SRMR, values closer to 0 represent a good fit, with the cutoff for adequate fit being .08. CFI values greater than or equal to .90 represent a good model fit, while TLI values should be close to .95 to represent a good model fit as compared to the null model (Kline, 2016).

Results

To test and compare the associations of each barrier on teachers’ technology integration practices, structural equation modeling (SEM) was used. A hypothetical model was drawn and tested.

Preliminary analysis

Table 1 displays the descriptive statistics of the variables and the missing rate. We found that the study was able to capture teachers with wide variabilities in perceived ability and value, perceived external barriers to technology integration, and technology integration practices. Specifically, teachers on average had moderate perceptions of their exposure to technology professional development, scoring slightly higher than the median score on this variable with 3.55 out of 5. The minimum scores on these factors range from the lowest possible score of 1 for most indices (except perceived ability) to the maximum score of the highest possible score of 5 or 7. These results show that some teachers in the sample had very negative beliefs regarding barriers to technology integration and reported rare usage of technology in the classroom. The missing rate ranges from 0% to 2.8%. We ran Little’s MCAR test to examine the missing data. The result suggests that the data are completely missing at random χ²(302)=333.072, p=.106, therefore, it allows us to analyze the data with maximum likelihood method.

Table 1. Descriptive Statistics.

Variable	Mean	SD	Min	Max	Missing
Institution	4.10	0.654	1	5	0%
Access	3.95	0.758	1	5	0%
PD	3.55	0.851	1	5	0.6%
Ability	3.89	0.666	2	5	1.1%
Value	5.09	1.137	1	7	2.6%
Low-order	3.29	0.871	1	5	2.6%
High-order	3.03	0.988	1	5	2.8%

Pearson correlation analyses were performed on all the variables in this study (see Table 2). From the correlation matrix, we can see that all variables are highly correlated with each other. PD exposure has a significant relationship with both value beliefs and ability. In addition, PD, value beliefs, and ability all show significant positive correlations with lower-order technology use and high-order technology use.

Table 2. Correlations.

	1.	2.	3.	4.	5.	6.	7.
1. Institution	1
2. Access	.759**	1
3. PD	.605**	.620**	1
4. Value	.456**	.498**	.566**	1
5. Ability	.139**	.163**	.303**	.275**	1
6. Low-order	.079*	.106**	.254**	.256**	.829**	1
7. High-order	.224**	.287**	.423**	.529**	.522**	.445**	1

** p < 0.01;

* p < 0.05.

Measurement model

SEM comprises the confirmatory factor analysis (CFA) and path analysis (PA). We used a two-step strategy (Kline, 2016) to analyze the structural regression model CFA to test how well the measured variables represent the latent constructs according to the theory. In the first step, we first conducted the factor analysis for each latent variable with no error covariance. The results of the latent variables for ability beliefs, as well as lower-order and higher-order technology usage suggest the re-specification of the measurement models based on the modification indices (>3.84).

In the original measurement model for lower-order technology use, all measurement items are loaded to the lower-order. We recategorized all items with Remember, Understand and Apply as it is in the Bloom’s Taxonomy during the re-specification procedures. The result shows that the model still needs to be improved, χ²(23) = 171.741, p<.001, SRMR = 0.037, RMSEA = 0.100, CFI = 0.954, TLI = 0.928. We modified the model by allowing the correlation between the residuals of apply1 and apply3. Apply1 and apply3 refer to asking students to connect what they learn to real life situations and share their work with other students or with the outside community, which are highly related. Students who share their work outside of the class spheres would also likely connect their learning to the real world. The item understand3 is also reloaded to both Apply and Understand. This item, worded as, ‘Demonstrate their understanding of class content,’ may be viewed as both Understand and Apply. The verb ‘demonstrate’ falls under both cognitive categories. The result shows that the data fits the model well (see Table 3).

Table 3. Measurement Models.

	χ^2	df	χ^2 p-value	CFI	TLI	RMSEA	RMSEA 90%CI	SRMR
Institution	12.407	2	0.002	0.992	0.975	0.088	0.046–0.138	0.015
Access	9.437	2	0.009	0.993	0.979	0.082	0.035–0.138	0.015
PD	0.000	0	0.000	1.000	1.000	0.000	0.000–0.000	0.000
Ability	7.444	3	0.059	0.998	0.993	0.047	0.000–0.092	0.009
Value	13.282	6	0.039	0.998	0.995	0.043	0.009–0.075	0.008
High-order	153.660	22	0.000	0.971	0.952	0.096	0.082–0.111	0.027
Low-order	95.595	21	0.000	0.977	0.980	0.074	0.059–0.089	0.025

In terms of the higher-order use of technology, adding three categories (Analyze, Evaluate, Create) to the measurement models did not yield a significant improvement on the model fit, χ²(24) = 307.6, p<.001, SRMR = 0.053, RMSEA = 0.135, CFI = 0.937, TLI = 0.906. To improve the measurement model, residuals covariance were added between analyze2 and analyze3. Analyze2 measured if teachers asked students to use technology to ‘conduct research that requires multiple sources of information’ and analyze3 measured if they asked students to ‘compare multiple sources of information.’ These two actions are similar and may have tapped into similar instructive behaviors. Residual covariance was also added between create1 and create3. These items use the verbs ‘create’ and ‘produce,’ which may be viewed as closely related. These modifications led to a better model fit (see Table 3).

As for the ability variable, we added the error correlation between ability1 and ability4, both referring to teachers feeling comfortable teaching with technology and spending class time resolving technology issues. These two items are related since not having to resolve technical issues might be related to teachers’ comfort in using instructional technology. We also added residual correlation between ability2 and ability5, each referring to having skills necessary to use technology in the classroom. The model has a better fit with added residuals correlations, χ²(3) = 7.444, p = 0.059, SRMR = 0.009, RMSEA = 0.047, CFI = 0.998, TLI = 0.998.

Next, we explored the measurement model with all seven latent variables (institution, access, PD, value, ability, higher-order and lower-order). The goodness-of-fit indices were satisfactory: χ²(705) = 193.634, p< .001, SRMR = 0.044, RMSEA = 0.053, CFI = 0.940, TLI = 0.933.

Structural model

Based on the measurement model, we performed structural regression analysis to examine the relationships among the latent variables. The first-order barrier variable (PD exposure) is considered as an exogenous variable. The second-order barrier variables that include value and ability have direct paths as well indirect paths, taken as mediators between PD exposure and the endogenous variables (lower-order technology use and higher-order technology use). Institutional culture and technology access were also specified as control variables, having direct relationships with the endogenous technology use variables. According to the result, the structure model fits the data well: χ²(709) = 1959.904, p<.001, RMSEA = 0.051, CFI = 0.939, TLI = 0.933, SRMR = 0.047. Figure 2 is the final structural model. All significant pathways are depicted with path coefficients.

Figure 2. Structural Model with Significant Paths.

Results showed that some relationships were statistically significant, whereas other were not. When controlling for institutional culture and access to technology, the results reveal that PD exposure was a significant predictor of both value and ability beliefs (p < 0.001). We use the alpha level of 0.05 for all statistical tests. When p < 0.05, it can be inferred that there is a significant chance (more than 95%) of the null hypothesis being rejected. In structural equation models, the null hypothesis is that there is no association between the two variables as specified. While value beliefs had a significant influence on teachers’ use of technology for both types of cognitive tasks, ability beliefs did not show significant direct relationships with either type of technology integration practices (see Table 4). To detect direct and indirect effects, a bootstrap analysis based on 10,000 resampling iterations was conducted, with bias-corrected bootstrapping confidence intervals (BCBCI) examined for statistical significance (Preacher & Hayes, 2008). Table 4 presents the direct and indirect effects for the structural model. The result shows that PD exposure has significant indirect effects on both types of technology integration practices. In the model, PD exposure represented a positive direct effect of 0.284 (p<.001, 95% BCBCI [0.098, 0.479]) and a significant indirect effect through value beliefs of 0.187 (p<.001,95% BCBCI [0.077, 0.289]) on technology integration in higher-order learning tasks. Additionally, PD exposure also has a positive direct effect on teachers’ technology use in lower-order tasks (0.285, p<.001, 95% BCBCI [0.095, 0.481]); the indirect effect is also found to be significant on low-order tasks, with an effect of 0.170 (p<.001, 95% BCBCI [0.049, 0.272]).

Table 4. Effects Decomposition Analysis.

	Value	Ability	Low-order			High-order
	Dir.	Dir.	Dir.	Ind.	Tot.	Dir.	Ind.	Tot.
PD	0.5***	0.749***	0.285*	0.170***	0.455***	0.284*	0.187***	0.47***
Value	–	–	0.609***	–	–	0.474***	–	–
Ability	–	–	−0.180	–	–	−0.067	–	–

* p < .05;

** p < .01;

*** p < .001.

Discussion

The purpose of this study was to explore specific factors that help explain the relationship between professional development exposure and teachers’ uses of technology in the classroom. The results showed that technology-related professional development has a direct relationship with in-service teachers’ use of technology for both lower- and higher-order tasks in the classroom. This finding was similar to those from previous empirical research, such as in Kim and associates’ (2017) study in which participation in training sessions aimed at helping teachers analyze and critique digital tools were found to improve the ways in which teacher used digital tools in the classroom . While exposure to professional development opportunities is important, the research points to several teacher factors that play an important role in translating teachers’ professional development experiences into actual classroom practices. Because the study collected data from teachers who did not self-select to participate in a particular professional development program, these findings add special insights into factors of technology integration that matter for teachers with a wide range of characteristics.

First, the results showed that teachers’ ability beliefs toward integrating technology into regular classroom use are directly affected by professional development exposure. This is not surprising as it has been previously shown that most professional development programs aim to improve teachers’ technology integration abilities. Liu et al. (2015), for example, found that their mentorship program had a significant influence on teachers’ ability beliefs for integrating technology into their curriculum. Much of the previous research analyzed the direct effects of PD exposure on teachers’ ability beliefs. The current research extends previous results by analyzing the direct effect of PD exposure on ability in addition to its relationship with teachers’ integration practices. The results showed that PD exposure has a large effect on ability, confirming that PD programs primarily designed to improve teachers’ abilities are working.

Second, teachers’ technology-related value beliefs were also shown to be an important factor for how they used technology in the classroom, confirming prior research (Cheng et al., 2020). Although the relationship was not as strong as that with ability, the analysis points to the close associative relationship between PD exposure and teachers’ value beliefs. As previously stated, not many professional development programs focus directly on teachers’ values. In one of the few studies, Er and Kim (2017) found that explicitly focusing on teachers’ beliefs about using technology through their memories was effective in changing their beliefs. Perhaps if more programs are designed to convey to teachers the value that technology has for student learning, the influence of professional development on value beliefs could be enhanced. Arming teachers with both the ability to integrate technology in meaningful ways as well as the belief that technology can make a difference for student learning is a way to implement impactful technology integration initiatives (Vongkulluksn et al., 2018).

In addition to showing a strong relationship between PD exposure and value, the current study establishes that value is a mediator between PD exposure and teachers’ use of technology for both lower- and higher-order classroom tasks. In fact, the indirect effects of PD exposure on technology use for both lower- and higher-order tasks through value are stronger than the corresponding direct effects of PD exposure on these teacher practice variables. This finding is new and not previously analyzed or stated in the extant literature. Er and Kim (2017), for example, analyzed the direct effect of a PD program on teachers’ value for using technology, but did not examine teachers’ classroom practices. The current research extends previous findings by showing that, while PD exposure is important for increasing teachers’ ability beliefs, it may be important to structure future professional development opportunities to directly and explicitly address teachers’ value for technology integration. Teachers’ value beliefs may hold untapped potentials in the way that they translate professional development into actual behaviors in the classroom.

Finally, we examined the potential mediation effect of ability between PD exposure and technology use for lower- and higher-order tasks. The findings indicate that ability is not a significant mediator on the relationship. In the current study, as well as others, PD exposure has a strong direct relationship with teachers’ classroom practices (e.g. Kim et al., 2017; Liu et al., 2015). Because PD often aims to improve ability, the ability perception itself perhaps does not add to additional variance explained in the prediction model. This does not mean that we should stop focusing on improving ability in PD. In fact, the results confirmed previous research and showed that PD is working to improve teachers’ ability beliefs. However, the results also show that perhaps PD programs can be improved by also targeting other factors like value beliefs.

In summary, these findings confirm previous research showing that when access to technology is improved, there are still other barriers to technology integration (Ertmer et al., 2012; Vongkulluksn et al., 2018; Wozney et al., 2006). Providing an institutional culture and vision for technology use in the classrooms is important, as is providing teachers and students with access to technology. However, once these needs are met, teachers are still in need of quality professional development programs that focus on not only improving their abilities, but also helping teachers to see the value of using technology in the classroom. Teachers’ abilities to integrate technology into lessons, as well as their value beliefs about technology, have significant effects on how teachers translate their exposure to professional development opportunities into actual classroom practices. Professional development, as shown by the current research, is and continues to be important for teachers to use technology in high quality ways that benefit student learning. While most professional development focuses on skills and abilities, it has become clear that a focus on teachers’ value beliefs is also critical to the relationship.

Practical implications

These findings confirm past research showing that PD exposure is important for teachers’ technology integration practices. Investment in professional development focusing on training teachers to use and integrate technology into their classrooms should continue. While a focus on knowledge and skills or abilities is important, this research demonstrates the importance of teachers’ values as well. Because value has strong influence on technology integration practices and also significantly mediates the relationship between PD exposure and classroom practices, it is important for teachers to be given opportunities to participate in PD programs that focus on improving their value beliefs. Thus, PD providers may consider integrating value-enhancing activities alongside those targeting teachers’ technology integration ability. For example, past research has shown that asking teachers to reflect on how technology is relevant for their teaching practice (Kale & Akcaoglu, 2018) and allowing teachers to design their own technology-integrated lessons (Hixon & Buckenmeyer, 2009; Scott & Mouza, 2007) may improve teachers’ value beliefs related to technology integration. Further, teachers may be led to recall memories about their own successful experience with using technology in the classroom, which have been shown to increase teachers’ value beliefs about technology (Er & Kim, 2017). These practices may also be adopted in training programs for pre-service teachers in order to help develop positive value beliefs toward technology integration for teachers entering the profession. By offering PD programs along with a value beliefs intervention, targeting at once two of the most influential first- and second-order barriers to technology integration, the effects of these programs on teacher practices maybe multiplied compared to a solely skills-focused program.

Additionally, these results suggest that PD programs may be most influential when they intentionally recruit teachers with low value beliefs and endeavor to help these teachers shift toward a more positive value belief for technology integration. This study identified many teachers with low perceived ability and value for technology use, highlighting that there may still be an ability and value gap when it comes to teachers’ outlook of technology integration. This research is unique in that it assessed general professional development exposure and did not focus on pools of teachers who self-selected to participate in a particular professional development program. As such, we were able to find that, for these teachers with varying characteristics related to technology integration, both perceived ability and value were instrumental for how they used technology in the classroom. Moreover, participation in PD programs have the potential to change these crucial factors for more positive outcomes. Therefore, reaching out to teachers with initially low ability and/or value beliefs may be an as yet under-addressed way to make technology integration efforts translate to practical effects for more students.

These results also suggest that more work needs to be done to identify ways to reach teachers with initially low value beliefs. Professional development programs that target this population may need to be structured in specific ways that are distinct from traditional programs already present. For example, asking for teacher volunteers to participate in technology professional development programs may not attract teachers who believe that technology does not matter much for student learning. At the same time, compulsory attendance may not yield positive teacher motivation toward program participation or application of learned skills. Furthermore, additional strategies for improving technology-related value beliefs are needed, especially those that aim to improve the perceptions of teachers who may have past negative experiences with technology. Given the lack of research in this area, additional work is still needed to address both how best to recruit teachers with initially low value beliefs and how to structure professional development programs that work to improve their perceptions of technology’s value for student learning.

Limitations and future direction

There were several limitations of the present study. First, the measures of extrinsic and intrinsic barriers to technology integration utilized in this study represented teachers’ individual perceptions. As such, all barrier measures were self-reported. Additionally, we did not have the opportunity to collect observational data on teachers’ technology integration practices, and these too were collected via self-report measures. Due to the large sample size and number of classrooms, collecting observational data on each teacher was outside the capabilities of the researchers. There is a known social desirability bias in self-report measures. People do not always report the truth, with biases stemming from both limitations in their conscious self-knowledge as well as the tendency to align reported behaviors and perspectives with what they viewed to be socially desirable (Barker et al., 2002). To offset this validity concern, some scholars recommend using a combination of data collection methods in addition to self-report (Greene, 2015; Pintrich, 2004). However, this was not logistically possible in the current study. Instead, we endeavored to use well-validated scales for constructs with existing measurement and ran a validity examination via confirmatory factor analysis of the newly created scale to measure teachers’ facilitation of cognitive tasks according to Bloom’s Digital Taxonomy. With these concerns, additional research using more objective and observer-reported data to examine relationships among PD exposure, teacher beliefs, and technology integration practices are needed to corroborate the trends found here.

Another limitation is that the sample for this study is from one state in midwestern United States. This study also did not include more objective school-level variables, such as number of devices present or available technical support. However, the characteristics of school demographics, teachers, specialists, and administrators are diverse. We have confidence that these results could be replicated in other schools and regions across the US. Additionally, we did not probe types of technology professional development programs teachers engaged in and whether they were compulsory, targeted to teachers with specific characteristics, or voluntary. Additional research is needed to understand how teachers in different types of PD programs may exhibit different patterns among PD exposure, ability and value beliefs, and technology integration practices. Finally, the analysis did not include longitudinal trends. The data collection was completed at one time, and thus sequential effects could not be examined. An analysis of these variables over time is suggested for future research.

Despite its limitations, the present study adds to current technology integration research by highlighting both ability and value beliefs as salient factors that should be the focus of future professional development programs. While teachers continue to face barriers to technology integration in their classrooms, past and present research has shown that teachers’ own value beliefs about technology are at least as salient to their practice as their ability to integrate technology. To acknowledge and target teachers’ value beliefs could prove the most successful way to increase the quality of technology use. Professional development has immense potential in terms of improving teachers’ ability and value beliefs, ultimately resulting in enhanced student learning experiences.

Disclosure statement

The study reported in this paper is based upon work in the EDCITE: Evaluating Digital Content for Instructional and Teaching Excellence project and the College Ready Ohio project supported by the Straight A fund from the Ohio Department of Education. The funding source had no involvement in the study design, data collection, data analysis, writing, and decision to submit the article for publication. The conclusions and recommendations expressed in this article do not necessarily reflect the views of the Ohio Department of Education.

Additional information

Notes on contributors

Margaret A. Bowman

Margaret Bowman earned her Bachelor of Science in Education from Ashland University with a teaching license in Middle Grades Education, and her Master of Education from Tiffin University. She was a middle school Math and Language Arts teacher for 6 years, before joining the Academic Design Department at McGraw-Hill Education in Columbus, Ohio. She currently works as an Academic Designer in the Middle School Mathematics Department, writing and designing digital curriculum. Margaret is also a research associate in the Research Laboratory for Digital Learning at The Ohio State University.

Vanessa W. Vongkulluksn

Dr. Vanessa Vongkulluksn is an Assistant Professor of Educational Assessment, Evaluation, and Research in the Department of Educational Psychology and Higher Education at University of Nevada, Las Vegas. She earned her Ph.D. in Education with a concentration in Educational Psychology and Quantitative Methods from Rossier School of Education, University of Southern California. She has focused her research on examining factors that impact learning and motivation in technology-integrated contexts, particularly the influence of digital and information literacy skills. She has over eight years of experience in school-based research and statistical analyses of data related to learning, cognition, and motivation.

Zilu Jiang

Zilu Jiang received her Master’s degree in Teaching Chinese to Speakers of Other Language from Lanzhou University, China. She worked as an online Chinese teacher and course designer in Chinese Teaching Center at the Open University of China 2012–2017 and also served as an online Chinese instructor for American high school students at Confucius Institute at Michigan State University in 2013–2016. She has designed and developed online courses for multiple levels and also involved in the online curriculum design and online flipped classroom research. She was selected as an online Chinese teacher at the STARTALK program at the University of Virginia in 2016. She also provides workshops and teacher training on distance teaching and technologies integration in classrooms.

Kui Xie

Dr. Kui Xie is Ted and Lois Cyphert Distinguished Professor and Professor of Learning Technologies in Department of Educational Studies at The Ohio State University (OSU). He is the director of The Research Laboratory for Digital Learning. He earned his Ph.D. in Instructional Psychology and Technology from University of Oklahoma. His scholarship focuses on K-12 technology integration and teacher professional development, motivation and engagement in digital learning, technology intervention and learning environment, learning analytics and research methods.