Teacher collaboration and students’ digital competence - evidence from the SELFIE tool

ABSTRACT

This paper explores the relationship between students’ digital competence acquisition, teaching practices, and teacher professional learning activities. We analysed insights provided by 59,452 teachers through SELFIE, an online self-reflection tool for schools’ digital capacity. Using ordinary least squares regressions with school fixed effects, we focus on students’ digital competence and find that the use of digital technologies in cross-curricular projects is the teaching practice most related to the acquisition of students’ digital competence. On the other hand, we also find that teachers’ participation in teacher networks is highly correlated with the implementation of cross-curricular projects with digital technologies. The results further suggest that the use of digital technologies for teacher collaboration (in professional learning activities and in implementing cross-curricular projects) can have great potential and importance in the digital age, both for teachers and learners.

KEYWORDS:

• Computer literacy
• educational cooperation
• teacher development
• training needs
• self evaluation (individuals)

Introduction

European Commission (2018) defines digital competence as ‘the confident, critical and responsible use of, and engagement with, digital technologies for learning, at work, and for participation in society’. More specifically, according to the Digital Competence Framework for Citizens (Carretero, Vuorikari, and Punie 2017; Vuorikari et al. 2016), it comprises five components: information and data literacy; communication and collaboration; digital content creation (including programming and intellectual property related questions); safety (including digital well-being and competences related to cybersecurity); and problem-solving with digital tools.

Digital competence and digitalisation will be fundamental to future economic and social prosperity in Europe (Gruber 2019; Staeritz, Stephanblome, and Von Oettingen 2018). For that reason, the European Council has included digital competence as one of the eight key competences needed for personal fulfilment, a healthy and sustainable lifestyle, employability, active citizenship and social inclusion (European Commission 2018). In the future, some estimates say that 90% of jobs will require basic digital competence (European Commission 2016). However, in 2019, only 56% of EU-27 individuals had basic or above basic overall digital skills (Eurostat 2020).

Consequently, the early development of children’s and teenagers’ digital competence is considered an essential need for European countries. Research has shown that the intensive but unguided use of digital technologies by primary and secondary students does not lead automatically to high levels of digital competence (Van Dijk and Van Deursen 2014; Fraillon et al. 2019). That is why educational systems should play an important role in developing digital competence of citizens (e.g., European Commission/EACEA/Eurydice 2019). To do so, teachers need to know how to use digital technologies purposefully and efficiently. However, even though many teachers have participated in training in this area in the last years (European Commission 2019b), they still do not feel very competent using digital technologies and teaching students how to use them (European Commission 2019a). Consequently, teachers in the European Union (EU) continue considering training on the pedagogical use of digital technologies a priority (OECD 2019b).

The above situation reveals a need to have better information and practices to guide teachers, schools, providers of continuous professional learning,¹ and policymakers in at least two aspects. First, it is crucial to have a better understanding of the most effective uses of digital technologies in the classroom for developing students’ digital competence. In this line, one of the objectives of our paper is to respond to the following research question (RQ):

• RQ1: What kind of teaching practices using digital technologies are linked to students’ digital competence acquisition?

Secondly, to optimise teachers’ time in participating in professional learning activities on the use of digital technologies for teaching practices, it is also important which training mode is selected. In this paper, by answering to RQ1, we are in a good position to contribute to this need and take a step further by shedding light on the relationship between professional learning methods and teaching practices. Therefore, we formulate our second RQ as follows:

• RQ2: Which teacher professional learning activities are related to the teaching practices that have been identified as more closely correlated to students’ digital competence acquisition?

Figure 1 below shows schematically the proposed correlations chain investigated in this paper by putting together both RQ.

Figure 1. Analytical model of the research questions.

To meet our goals, we use aggregated and anonymised data from SELFIE, an online tool for collective reflection on schools’ digital capacity² (more information about the tool can be found in materialsro and methods section). SELFIE includes a set of questions providing information that allows answering the research questions of this paper. In total, more than 400,000 participants took part in SELFIE in the school year 2018–2019, including almost 60,000 teachers, who are the focus of this paper. Thus, using this data, we can estimate a range of ordinary least squares (OLS) regressions on the relationships presented in Figure 1.

To the best of our knowledge, our paper is unique because it exploits a novel dataset, which allows using multivariate models, controlling for school and educational level characteristics, to explore the complete correlational chain between students’ digital competence acquisition; teaching practices using digital technologies; and teacher professional learning activities. The paper also provides concrete comparative results between different teaching practices and professional learning activities that can be useful to inform stakeholders’ decisions.

The remaining of the paper is structured as follows: first, we present a literature review covering the areas of interest that justify the analysis performed in the paper. Then, we present the tool where the data come from, the variables used in the paper, and its descriptive statistics. Next, we outline the methods of our modelling strategy before presenting and discussing the results of our regressions. Finally, in the conclusions section, we sum up the outcomes of the paper and propose ideas for future research.

Literature review

Students’ digital competence

Young people are very often considered ‘digital natives’ who are inherently competent and confident with digital technologies due to the intense use they make of them. However, research evidence also shows that there is no such thing as a ‘digital native’ (Kirschner and De Bruyckere 2017). Research has found a link between the use of digital technologies and the development of students’ digital competence (Escueta et al. 2017; Graafland 2018; Matzat and Sadowski 2012). However, research also shows that unguided use of digital technologies is not enough neither for promoting productive and beneficial uses nor for acquiring high digital competence levels. The recent International Computer and Information Literacy Study – ICILS (Fraillon et al. 2019), shows that grade 8 students make more extensive use of technologies outside the school than inside, using them more for leisure than for other purposes. The study also reveals that young people do not develop sophisticated digital skills through exposure to and use of digital devices alone. Other studies go in the same line, indicating that unguided exposure to and use of digital devices alone is not sufficient for young people to learn how to use technologies critically and responsibly, not only as consumers but also as producers of information (e.g., Pérez Escoda, Castro Zubizarreta, and Fandos Igado 2016; Van Dijk and Van Deursen 2014).

International surveys confirm that European students need to increase their digital competence levels. The 2nd Survey of Schools: ICT in Education (European Commission 2019a) found that European students (ISCED³ levels 2 and 3) do not feel confident in all the areas of digital competence as defined in the Digital Competence Framework for Citizens (Carretero, Vuorikari, and Punie 2017; Vuorikari et al. 2016). They feel most confident about their digital competence in communication and collaboration as well as with activities linked with information and data literacy, whereas they felt least confident in areas of digital content creation and problem-solving using digital technologies. Also, the evidence from ICILS study (Fraillon et al. 2019) shows that there are (advanced) skills gaps even in those areas where students think they perform better. For example, only 2% of grade 8 students demonstrated the ability to assess information found online critically. Consequently, educational systems and teachers need to adapt to be ready to foster early development of children’s and teenagers’ digital competence.

The education systems approach to digital competence

Due to the importance of digital competence, education systems are gradually including it in their academic programmes and curricula. A 2019 study shows that recently, the majority of EU countries have reformed their school curriculum to include the development of students’ digital competence into it directly or indirectly (European Commission/EACEA/Eurydice 2019). In many countries, the development of students’ digital competence in schools has relied primarily on classes on Information and Communication Technologies (ICT) or computer science (Bocconi et al. 2016). This approach has narrowed its scope and excluded some sub-competences and skills, such as critical use of online information, from the curriculum (e.g., OECD 2019a). In Europe, this is especially true in secondary education. According to a recent report (European Commission/EACEA/Eurydice 2019) more than half of European systems consider digital competence as a transversal theme in primary education, but this percentage decreases secondary or tertiary education. However, some countries, in Europe and beyond, are innovating in the way of integrating digital competence in the curriculum. The Australian Curriculum, Assessment and Reporting Authority (OECD 2019a), the new Finnish curricula ICT competence and multiliteracy (Tulivuori 2019), and the Estonian ProgeTiger programme (Conrads et al. 2017) are good examples of initiatives that aim the development of digital competence at school level and using a transversal approach that encompasses different subjects. In addition, an essential feature of the ProgeTiger programme was that teachers benefited from the support of local teacher networks related to the programme (Conrads et al. 2017).

Teachers’ confidence and use of digital technologies in education

Teachers play a central role in fostering students’ digital competence, and hence they need to be knowledgeable and confident when using and guiding on how to use digital technologies. Evidence from the ICILS study (Fraillon et al. 2019) shows that teachers are more likely to promote computer and informational literacy and computational thinking in their teaching if they are confident users of ICT; if they have positive views towards ICT; and if they feel their school has a collaborative approach to the use of ICT in teaching. However, according to TALIS 2018, in the EU-22, 48% of teachers say that they ‘frequently’ or ‘always’ let students use ICT for projects or classwork (OECD 2019b). Moreover, cross-country differences are significant, varying from 90% of teachers using ICT in Denmark to merely 29% in Belgium. Such differences are only partially related to the level of digital infrastructure in schools, as only around one in four school principals across OECD countries who responded to the same survey report a shortage or inadequacy of digital technology for instruction (OECD 2019b). Teacher digital skills and confidence seem to play a more important role in the use of digital technologies.

At the moment, European teachers do not feel very confident in properly using digital technologies. A possible explanation can be the lack of importance of teachers’ digital competence in pre-service teachers education curriculum (Instefjord and Munthe 2016; Gudmundsdottir and Hatlevik H 2018). A survey on ICT use in schools points out that only half of the European teachers feel confident in their abilities. In essence, 45% of primary education teachers and 52% of the ones in lower and upper secondary general education describe themselves as not very digitally active or confident about their ability to integrate digital technologies in their pedagogical practice (European Commission 2019a). Along the same lines, only 39% of teachers in EU-22 feel ‘well prepared’ or ‘very well prepared’ for the use of technology for teaching. However, in the same study, 56% of EU-22 respondents said that the topic had been included in their recent professional learning activities (European Commission 2019b).

Teachers’ professional learning on the use of digital technologies

The data above shows that the emphasis on policies focusing on the development of teachers’ competence for the integration of digital technologies in their professional practices can play a crucial role in enhancing the quality of education. Despite the high participation level in training on the use of digital technologies for teaching, this is still considered a need by teachers and is ranked high in Europe (OECD 2019b, annexe C, table I.5.21). Thus, understanding the effective design of professional learning activities in the context of digital technologies is of key importance.

New insights about teacher professional learning show that there are some features linked to higher effectiveness. According to literature, activities should focus on the content of the subject taught incorporating active learning and offering opportunities for teacher collaboration. Additionally, the provision of support, in terms of coaching or experts, and opportunities to reflect and give feedback, are other crucial elements. (Darling-Hammond, Hyler, and Gardner 2017; Cordingley et al. 2015).

Such features are starting to be seen in the design and provision of professional learning activities, and even some structured teacher professional development courses start incorporating them (Vuorikari 2019). In parallel to this trend, the nature of professional learning is changing over the last ten years. Teachers tend to participate more and more in professional learning activities that are less structured, some of these activities are even informal in their nature, and their advantages are that they can be carried out in a timelier manner and be more strongly connected to teachers’ individual needs (Vincent-Lancrin et al. 2019).

Teachers’ networks and collaboration

Participation in teacher networks or in-school teacher collaboration are examples of less-structured professional learning activities. In Europe, one in three lower secondary teachers participates in a network of teachers specifically formed for their professional learning (OECD 2019b, annexe C, table I.5.7).⁴ In the last five years, there has been an average growth of 5% in this participation. The number of teachers participating in such communities and networks varies across countries within Europe. In Estonia, the figure is highest at 59%, in Lithuania, Croatia and Sweden the figure is around half of the lower secondary teachers, whereas it is around 20–25% in Spain, Czech Republic, Slovak Republic, Austria and Cyprus (OECD 2019b, annexe C, table I.5.7). This variety might be due to the availability of such communities, their status as part of the official offering of continuing professional learning, and how teachers are incentivised to participate.⁵ Time is also an essential factor – teachers participate more in professional learning activities in their own free time than during working hours (European Commission 2019a).

International organisations (European Commission 2013; OECD 2014) consider that peer collaboration and networks are essential to gain sufficient skills to cope with the changing learning environments, and especially for those changes derived from the incorporation of digital technologies.

Research also shows that teachers working in schools that support the use of technology through a planned and collaborative approach are more likely to promote the development of student digital competence (Fraillon et al. 2019).

Teaching practices using digital technologies, learning outcomes, and students’ digital competence

The benefits of using digital technologies in compulsory education to develop student digital competence and other learning outcomes may largely depend on schools’ and teachers’ ability to create conditions for adequate use and to adapt teaching methods to be conducive to student learning (Spiezia 2010). Several studies have tried to look into the link between the use of digital technologies in the classroom and its impact on students’ achievement, for example in the cognitive domain such as enhanced attainment in a given subject. Research has shown contradictory results (Escueta et al. 2017; Gubbels, Nicole, and Groen 2020; Hanushek and Woessmann 2015; OECD 2015; Rosén and Gustafsson 2014). However, such conflicting associations might be partially explained by the fact that studies do not always focus on the same type of activity done with digital technologies (Rosén and Gustafsson 2014).

The implementation of different activities and pedagogies facilitated by digital technologies can have heterogeneous impacts in different competences (e.g. collaboration through digital technologies vs learning language). Moreover, research shows that they can affect students’ attainment in the same subject differently. Therefore, it is important to identify what works and for which purpose. The literature shows that there are some uses of digital technologies that tend to be more associated with positive results than others (Tamim et al. 2011). This is the case for the use of technology for interactive learning (Bernard et al. 2009), behavioural interventions, and differentiated computer-assisted learning (Escueta et al. 2017), especially when the latter is used for mathematics teaching (Escueta et al. 2017; Cheung and Slavin 2013).

In this line, a recent report from OECD suggests that teachers should apply innovative pedagogies that ‘correspond to learners’ needs, prior competencies and digital literacy, following clear instructional designs and put forward new ways of collaboration and learning’ (OECD 2019a, 191). Also, Lakkala, Ilomäki, and Kantosalo (2011) indicate that a rich and integrated use of various technological tools in a meaningful context can help at developing digital competence. The authors show that in general, activities that are based on complex and challenging tasks, promoting collaboration, project work and students’ knowledge creation are commonly identified by the literature as the adequate ones for the development of students’ digital competence.

Material and methods

Instrument

Our study uses data from SELFIE,⁶ a free online tool developed by the European Commission to help schools to reflect on incorporating digital technologies into teaching and learning. SELFIE supports schools to plan their digital strategies, by highlighting what is working well, where improvement is needed and what the priorities should be. The tool gathers – anonymously – the views of students, teachers and school leaders on how technology is used in their school, as part of an exercise of collective reflection at school level. This reflection is done through three questionnaires (for school leaders, teachers and students) that comprise short statements and questions using a simple 1–5 agreement scale.⁷ The statements cover areas such as leadership, infrastructure, teaching practices, professional learning, assessment practices and students’ digital competence.⁸ Upon completing SELFIE, each school receives an interactive school report that provides aggregated data with insights on the strengths and weaknesses of their use of digital technologies for teaching and learning. Participants can receive personalised certificates of participation, and the schools can get open badges.

SELFIE’s main objective is to provide a snapshot of the use of digital technologies to each participating school. However, by aggregating information from different participating schools, the tool also provides information that can be used for research purposes. SELFIE data are well suited for measuring the relationship between the use of digital technologies for teaching and learning, the students’ acquisition of digital competence and teachers’ participation in professional learning as the questionnaires include a range of detailed questions on these topics and a large number of individuals answered the questionnaire.

Sample

In this paper, we use data collected from SELFIE during the first year of its use, i.e. from the launch of the tool on 25 October 2018 to 31 July 2019⁹. At the time of writing, the total SELFIE database comprises 410,238 participants (11,610 school leaders, 59,452 teachers and 339,176 students) from 4,228 schools¹⁰ in 38 different countries. In the first year, the tool was available in the 24 official languages of the European Union as well as in some other languages (e.g., Albanian, Georgian, Russian, Serbian and Turkish). The tool was available for primary (ISCED 1), lower secondary (ISCED 2), upper-secondary general (ISCED 3-GEN), upper-secondary vocational (ISCED 3 – VET) and post-secondary non-tertiary education levels (ISCED 4 – PSNTE).

The schools decide on whether to participate or not in SELFIE. Hence, schools with a higher interest in reflecting about digital technologies may be more likely to participate, posing a slight bias in the data. However, although the data is not representative, it is a unique dataset that covers a large sample of schools in different countries and provides evidence on how they are using digital technologies. Moreover, it allows us to learn more about the practices of the schools who are actively dealing with the process of incorporating digital technologies.

In this paper, we will focus on the responses of teachers, as they constitute information about participation in professional learning and what happens in the classrooms in terms of teaching practices and students’ digital competence acquisition. Thus, we will analyse the responses of almost 60,000 teachers, most of them coming from across Europe.

Variables and descriptive statistics

Table 1 shows the main descriptive statistics of the variables used for our analysis (see appendix 1, table 2 for a more complete definition of the variabes used in the paper). Part I in the table includes a set of variables measuring the perceived usefulness of participation in different types of professional learning activities on the use of digital technologies. This part only includes responses from those teachers who declared to have participated in each of the proposed activities. It is interesting to observe that in-school collaboration is highly rated by teachers as an integral part of professional learning, whereas it is often considered as an outside school activity (e.g. course or workshop to be attended). From our data, we can observe that teachers highly appreciate training by peers: professional learning activities that allow learning from other teachers within the same school and through online or offline collaboration are considered, on average, the most useful ones (mean = 3.80). They are closely followed by ‘other in-house training’ activities organised by the school (mean = 3.77) and face-to-face (F2F) professional learning (mean = 3.76). All other activities follow with a certain margin.

Table 1. Descriptive statistics.

I- Usefulness of professional learning activities on the use of digital technologies	Mean	S.D.	n	II- Teaching practices using digital technologies	Mean	S.D.	n	III-Student digital competence	Mean	S.D.	n
Within school collaboration	3.80	1.01	46,854	Tailoring to needs	3.71	1.00	57,726	Responsible behaviour	3.79	1.00	57,180
Other in-house training	3.77	1.06	45,797	Fostering creativity	3.73	1.04	57,151	Safe behaviour	3.75	1.01	56,947
F2F professional learning	3.76	1.10	41,543	Engaging students	3.70	1.02	57,191	Learning to communicate	3.72	1.00	56,223
Online professional learning	3.62	1.13	39,694	Student collaboration	3.49	1.12	56,066	Checking information quality	3.64	1.00	56,740
In-house mentoring/coaching	3.58	1.10	41,115	Cross-curricular projects	3.37	1.17	54,489	Creating digital content	3.59	1.04	56,223
Accredited programmes	3.57	1.16	38,695					Giving credit to other’s work	3.51	1.04	55,990
Professional networks	3.55	1.15	39,296
Study visits	3.40	1.25	36,532

In contrast, study visits (for instance to other schools, businesses or organisations) have the lowest perceived usefulness (mean = 3.4) and the highest standard deviation (S.D.). These results are in line with previous research that point to the usefulness of collaborative professional learning activities that involve peer learning (Darling-Hammond, Hyler, and Gardner 2017; Cordingley et al. 2015). In most of the cases, the in-school training is probably also timelier and better aligned with the needs of the teachers of each school. The results also point to the importance of considering the school as a learning organisation for teachers and students alike.

Part II in the table shows a set of items that aims at measuring the level of use of digital technologies to support various teaching practices. In SELFIE data, teachers have the highest means in the used of digital technologies for fostering creativity (mean = 3.73), followed by the use of digital technologies for tailoring teaching to students’ needs (mean = 3.71) and for engaging students (mean = 3.70). Interestingly, teachers perceive the use of digital technologies for fostering student collaboration and for the development of cross-curricular projects as the less common practices (means are 3.49 and 3.37, respectively).

Finally, part III offers a closer look at the variables measuring students’ digital competence acquisition as perceived by teachers. According to the data, teachers state that, on average, acquiring competences to behave responsibly online (mean = 3.79), behave safely online (mean = 3.75) and learn to communicate (mean = 3.72) are a priority in schools’ approach to digital competence. On the other hand, knowing how to give credit to others’ work found online (mean = 3.51), creating digital content (mean = 3.59) appear to be practices which are less taught.

Methods

In this paper, we run two different Ordinary least squares (OLS) regressions to give response to the research questions. This is a common and appropriate statistical procedure used to estimate the relationship between one or more independent variables and a dependent variable (Hutcheson and Sofroniou 1999; Wooldridge 2016). In particular, regression analysis describes how the typical value of the dependent variable changes when any one of the independent variables increases or decreases, while holding the other independent variables constant .¹¹ The typical change in the dependent variable related to a variation of one unit in an independent variable is known as coefficient.

In a first step, we estimate the following straightforward general model using OLS to analyse the relationship between digital teaching practices and digital competence acquisition of students, as perceived by teachers:

(1) $Digcomp=\alpha +\beta \ast practices+\gamma \ast schoo{l}_{\hspace{0.17em}}+{\epsilon }_i$(1)

where: Digcomp is an index computed as the average value of the variables measuring teachers reporting on students’ digital competence acquisition in their schools (see Appendix 1, table 2)¹² and corresponds to the dependent variable that is being predicted or explained; α is the constant; practices is a vector of variables measuring teachers’ digital teaching practices (see Appendix 1); β is the vector of coefficients associated to practices and school is a proxy of school/ISCED level fixed effects.¹³ This last variable aims at controlling for different unobserved school and educational level characteristics (e.g. school strategy for digital education, implementation of digital education by peers, digital equipment of the school, support to professional learning activities, school socioeconomic status, school educational level, etc.) that can be correlated with the obsereved independent variable.¹⁴ γ is the vector of coefficients associated with school. Finally, ε_i is the error term of the model.

In a second step, we aim to shed light on which professional learning activities are better correlated with the implementation of cross-curricular projects using digital technology, the practice that has been previously identified as the most closely linked to the development of students’ digital competence. In this analysis, we limit our sample to those teachers who declare to have participated in all types of proposed professional learning activities on the use of digital technologies during the last year. This limitation is due to the design of the question, and to guarantee comparability between all professional learning activities. As a result, the analysis is based on 30,584 individuals. Then, we run the following OLS model:

(2) $CC=\alpha +\beta \ast PLactivities+\gamma \ast schoo{l}_{\hspace{0.17em}}+{\epsilon }_i$(2)

where: CC is the teacher’s declared level of implementation of cross-curricular projects using digital technologies, and professional learning activities (PLactivities) is a vector of variables measuring the self-reported usefulness of each professional learning activity on the pedagogical use of digital technologies (see Appendix 1). β is the vector of coefficients associated to PLactivities. We use a measure of teachers’ perceived usefulness (instead of participation) for two reasons. First, because we want to compare views from teachers who have participated in all the proposed activities, and second, because we want to find out which professional learning practices are considered more useful for the implementation of cross-curricular projects using digital technologies, independently of their quality and perceived usefulness for other topics. The school part of the equation was included for the same reason explained above for equation (1)(1) $Digcomp=\alpha +\beta \ast practices+\gamma \ast schoo{l}_{\hspace{0.17em}}+{\epsilon }_i$(1) . γ is the vector of coefficients associated with school and ε_i is the error term of the model.

Finally, since in each school level there is more than one grade, and due to privacy reasons our data does not allow us to account for it, in both OLS models, we estimated cluster-corrected Huber-White estimators in which teachers are considered to be clustered by grade within each different school and level. In both models, we checked multicollinearity, and the tests were acceptable to proceed with the analysis.¹⁵

Results

Figure 2 presents the results for RQ1 from the OLS model (1) regressing the variables related to teaching practices with digital technologies on our indicator of students’ digital competence acquisition.¹⁶ The results show that all teaching practices included in the regression (see Table 1) have a significant and positive effect on teachers’ perception of students digital competence acquisition. A closer look to the OLS coefficients allows us to see that the teaching practice that is the most closely related to the acquisition of students’ digital competence is engaging students in the use of digital technologies in cross-curricular projects, followed by the use of digital technologies to tailor teaching to students’ individual needs. According to teachers’ responses, the effect of using digital technologies to foster students’ creativity and collaboration are practices less associated with students’ digital competence acquisition.¹⁷

Figure 2. Relationship between digital learning practices and digital competence of students. n = 52,520. R square: 0.50. Notes: Dots indicate the beta values of the OLS regression, while horizontal lines show the 95% confidence interval using robust SE.

Once we have identified the most effective teaching practice in terms of students’ digital competence acquisition, we go a step further with RQ2 and aim at understanding better which professional learning activities are plausible to be determinants of the implementation of cross-curricular projects using digital technologies. Figure 3 presents the results from the OLS model (2), where the dependent variable is the level of implementation of cross-curricular projects using digital technologies in the classroom. The independent variables, on the other hand, correspond to the level of usefulness that teachers attributed to different modes of professional learning activities on the pedagogical use of digital technologies,¹⁸ controlling also school/ISCED level characteristics using fixed effects and using robust standard errors. Results indicate that the use of digital technologies for cross-curricular project work is significantly linked in particular to the teachers’ perceived usefulness of professional networks.¹⁹ However, it is also clear that all the analysed professional learning activities on the pedagogical use of technology positively correlate with the use of digital technologies for cross-curricular projects.

Figure 3. Relationship between the usefulness of professional learning activities and cross-curricular projects. n = 30,584 (only those with experience in all the above professional learning activities during the last year). R square:0.44. Notes: F2F stands for face-to-face. Dots indicate the beta values of the OLS regression, while horizontal lines show the 95% confidence interval using robust SE.

Discussion

Digital competence is considered as key for living and working in societies in Europe and elsewhere. Therefore, its acquisition is becoming more and more an educational priority. This paper contributes to a better understanding of the adequate uses of digital technologies in the classroom for developing students’ digital competence. Using two related research questions, we analysed the relationship between teachers’ professional learning activities on the pedagogical use of digital technologies, teaching practices using digital technologies in the classroom, and students’ digital competence acquisition. Figure 4 depicts schematically the main results that emerge from our analysis.

Figure 4. Main results of the analysis.

Using self-reported data from 59,452 teachers that participated in the SELFIE tool in the school year 2018–2019 and controlling for schools’ characteristics, we used first OLS regression to consider the relationship between teaching practices using digital technologies and students’ digital competence acquisition (RQ1). We found that all the proposed teaching practices using digital technologies (see Figure 2) are positively and statistically correlated with students’ digital competence acquisition as perceived by teachers, and this happens even after controlling for the other practices. This confirms previous results highlighting the importance of allowing, and enabling, students to use a variety of technologies, and for different purposes, for a more complete development of their digital competence (Lakkala, Ilomäki, and Kantosalo 2011).

More importantly, the results also show that the use of digital technology in implementing cross-curricular projects is the teaching practice with the highest correlation. Therefore, the results are in line with previous research in the field suggesting that the pedagogical settings and methods that better support the development of digital competence include solving multidisciplinary problems (Pruulmann-Vengerfeldt, Kalmus, and Runnel 2008; Tierney, Bond, and Bresler 2006) and project work (Erstad 2010; Lakkala, Ilomäki, and Kantosalo 2011).

Interestingly, the descriptive statistics of this paper (see Table 1) suggest that teachers perceive the use of digital technologies in cross-curricular projects as the least common teaching practice among the analysed ones. This result can indicate that it may be an effective practice, however, still under-used one and/or that teachers do not have enough knowledge on how to develop such cross-curricular projects in their classrooms. There is an increasing interest in teacher professional learning as an important strategy for supporting students’ complex skills that they need for further education and work. Showcasing and modelling good teaching practices, which could eventually impact various students’ learning outcomes, can provide teachers with a clear vision of what best practices look like (Darling-Hammond, Hyler, and Gardner 2017). Step-by-step implementation guides and example of pedagogical cross-curricular projects using digital technologies are made readily available for European teachers by eTwinning, for example.²⁰ Importantly, incentivising teachers to take up such practices is also important and something that policy makers and school authorities should focus on.

Focusing on the different coefficients of the proposed teching practices, we see that the ‘implementation of cross-curricular projects’ and ‘tailoring teaching to the needs of students’ are practices more correlated with the development of students’ digital competence than ‘activities fostering students’ creativity’ or ‘promoting group or team work through digital tools’. A possible explanation is that participation in creative and collaborative activities is normaly more autonomous and less structured than ‘participation in cross-curricular projects or in activities tailored by the teacher to specific needs’, which usually require more teacher engagement. Thus, the results presented here are consistent with the literature showing that the development of digital competence is better achieved with external guidance (e.g. Van Dijk and Van Deursen 2014: Fraillon et al. 2019).

The RQ2 focused on the relationship between the use of digital technology in cross-curricular projects and teachers’ participation in professional learning activities. This analysis can help to identify ways to prepare teachers for activities oriented to the early development of children’s and teenagers’ digital competence in the school context. The following modes of professional development activities were considered: face-to-face professional learning; online professional learning; (within school) collaboration; professional networks; in-house mentoring/coaching; other in-house training; study visits; accredited programmes. The results indicate that all these modes are positively associated with teachers’ implementation of cross-curricular projects in the classroom. In general, these results show the importance of teachers’ participating in professional learning activities that focus on the pedagogical use of digital technologies. This finding is in line with recent studies showing that in Europe, teachers’ need for training on the use of digital technologies for teaching is ranked high (OECD 2019b).

Our findings also reveal that there is a clear link between the perceived usefulness of teachers participating in professional networks and the implementation of cross-curricular projects in the classroom. The positive development across OECD countries shows that teachers’ participation in unstructured professional learning activities, such as teacher networks, is growing (OECD 2019b). The European Commission also supports the importance given to teacher peer collaboration for coping with changing environments and acquiring knowledge on the use of digital technologies (European Commission 2013; OECD 2014). In order for schools to reap the best benefits in terms of pedagogical practices that also improve students’ various leanring outcomes, schools should align their policies and practices so that the conditions within the school do not inhibit the effectiveness of professional learning. The SELFIE tool with its self-reflection nature, especially in areas of leadership, teaching practices and professional learning, can help schools in setting up such policies and follow them up on annual bases.

Last, our findings suggest that even if teachers find structured accredited programmes on the use of digital technologies somehow useful for their general skills, they are not necessarily the best way to learn how to use technologies in teaching in a way that it promotes students’ digital competence acqusition. Consequently, public authorities could invest in improving the offer of accredited programmes, but more importantly, also recognise the value of other modes of professional learning activities – especially teacher networks that promote teacher collaboration have proven useful in our study.

The evidence presented in this paper contributes to the knowledge in the field in many ways. Contrary to most of the literature on the impact of digital technologies on student learning, our study provides empirical insights about specific types of teaching practices and professional learning activities that involve the use of digital technologies and that focus on alternative learning outcomes such as students’ digital competence acquisition. Most of the information on professional learning activities from large-scale international surveys is obtained from univariate statistics where the teachers report about it for a specific ISCED level or age group. However, this paper contributes to the literature by using robust multivariate models taking into consideration the relationship between teaching practices, students’ digital competence acquisition and professional learning activities, controlling for school and ISCED level characteristics.

Secondly, the evidence presented in this paper gives large-scale empirical support to existing conceptions and case studies about the development of students’ digital competence and effectiveness of teachers’ professional learning activities on the pedagogical use of digital technologies. Moreover, our two-step analysis is a novelty in the field: it allows to obtain information on teaching practices that can improve students’ digital competence in addition to what can be done to improve the relationship between professional learning activities and these specific practices. This paper also gives unique comparisons between specific alternatives for professional learning activities and teachers’ use of technology in the classroom. Finally, our results are also a good example of how aggregated data from self-reflection tools, such as SELFIE, can be analysed for delivering useful and informative results for teachers, school leaders, professional learning providers and policymakers.

Yet, our research also has limitations. First, the variables used in the study may have some measurement error as they are self-reported by teachers and not directly observed by the researchers, but this is a standard proxy used in surveys and the literature. It implies that the measurement of students’ digital competence acquisition is a proxy based on teachers’ view rather than on a direct measurement. Secondly, our data comes from different countries, some of them are overrepresented (Spain, Bulgaria, Serbia and Turkey), therefore it has to be taken into account for generalisation purposes.²¹ Finally, it is important to highlight that, although we control for a good set of factors, including unobserved school and educational level ones using fixed effects, our results do not imply causality. In the two OLS models presented, reverse causality and confounding variables can still play a role in explaining the correlation. One might argue that a tendency towards innovation can be a crucial confounding variable influencing the results in our two models. However, as a robustness check, we run the models controlling for teachers’ digital competence, which can be linked to a propensity to innovation, and the results maintained stable. Moreover, there is not an apparent reason to think that the implementation of cross-curricular projects is the most innovative teaching practice among the analysed ones, and teaching practices are central components in our two models. However, other unobserved variables linked to the teachers could still play a role.

Conclusions and future research

This paper explored the relationship between students’ digital competence acquisition, teaching practices, and teacher professional learning activities by analysing insights from close to 60,000 teachers throughout Europe. Using ordinary least squares regressions with school fixed effects, it was shown that the use of digital technologies in cross-curricular projects is the teaching practice most related to the acquisition of students’ digital competence. On the other hand, it was also shown that teachers’ participation in teacher networks is highly correlated with the implementation of cross-curricular projects using digital technologies. The main conclusion from these results is that collaboration among teachers should be promoted for a more efficient development of students’ digital competence. Both, across-school and within-school teacher networks can be useful for teacher professional learning and for the development of cross-curricular projects that facilitate an early-acquisition of students’ digital competence.

Moreover, some pedagogical implications for an effective digital competence acquisition can be derived from our findings. First, the importance of guidance in teaching activities oriented to the development of students’ digital competence. Secondly, the importance of being exposed to the use of technologies for a variety of educational goals, teaching practices and subjects.

The results presented also provide a sound basis for future counterfactual analysis looking for the causality of professional learning activities and specific teaching practices. For future research, it would also be useful to have a look at the combination of different teaching practices and how they complement each other in the acquisition of students’ digital competence.

Disclosure of potential conflicts of interest

The authors declare that they have no competing interests.

Acknowledgments

The European Commission has funded this research. The authors would like to thank the colleagues from the European Commission´s Joint Research centre who provided feedback on early versions of the paper: Nikoleta Giannoutsou, Romina Cachia, Lilian Weikert, and Yves Punie. Thanks also go to the members of the SELFIE community who contributed to the development, testing and implementation of the tool. Finally, special thanks go to the thousands of school leaders, teachers and students who have used the tool so far to reflect on the digital capacity of their schools.

Additional information

Funding

This work was supported by the Directorate-General for Education and Culture [JRC Ref. 35346-2018 // EAC 2018-0747].

Notes on contributors

Jonatan Castaño Muñoz

Jonatan Castaño Muñoz, PhDis a researcher at the Joint Research Center of the European Commission from where he gives scientific evidence to inform European educational policies. Jonatan is part of the Human Capital and Employment Unit and his research is focused on education and skilling, especially on the use of digital technologies in school, higher, and adult education. His research is not only interested in the pedagogical dimension of the use of digital technologies in education but also in the socio-economic implications that this use entails. Jonatan has a degree in political sciences, a master in applied social research techniques, and an interdisciplinary PhD in ‘Information and Knowledge Society’. He also contributes regularly to conferences and journals in his research fields.

ORCID: https://orcid.org/0000-0002-6112-2728

Twitter: https://twitter.com/jcastanom

Linkedin: https://www.linkedin.com/in/jonatan-castaño-muñoz-3a264043/

Researchgate: https://www.researchgate.net/profile/Jonatan_Castano-Munoz

Riina Vuorikari

Riina Vuorikari, PhD joined the Joint Research Centre (JRC) of the European Commission in July 2013. She contributes to research and policy support in the field of digitalisation of education and training, and is interested in innovation in education. Her work focuses on Digital Competence for Citizens and Teacher Professional Development, especially through teacher networks. Her new themes cover areas such as Educational Makerspaces and AI in Education, but also impact evaluations.

Riina has degrees in education (M.Ed in 1998 in Finland) and hypermedia (DEA in 1999 in France). Her PhD, which was completed in 2009, is from the Dutch research school for Information and Knowledge Systems. Since 1999, she has worked in the field of digital education. Her main interest is dealing with issues related to the adoption of new technologies and innovation in education. Before joining the JRC in Sevilla, she worked as a Project Manager and Research Analyst. Many of her clients were based in Brussels, including educational organisations such as European Schoolnet, and national and regional education authorities (e.g. Ministry of Education).

Patrícia Costa

Patrícia Costa, PhD is a researcher at the Center for Mathematics Applied to Economic Forecasting and Decision (CEMAPRE). She worked as a scientific officer at European Commission´s Joint Research Centre (JRC) from 2011 to 2020 where she contributed to research and policy support in the areas of Educational Assessment, Applied Statistics, Comparative Education and Psychometrics. She has been a data analyst and psychometrician in the Human Capital and Employed Unit. At the JRC her research was focused on secondary data analysis of international large-scale surveys and on the validation of instruments. She has also been involved in supporting the Commission by providing technical expertise in statistical analyzes of student and adult skills. In addition, she regularly organizes workshops and serves as a scientific committee member.

Patrícia holds a Ph. D in the area of Psychometrics/Statistics and received a master and bachelor degrees in Mathematics. Before joining the JRC, Patrícia was a researcher at the Laboratory of Applied and Computational Statistics in Portugal. There she worked on a School Effectiveness project in the area of Mathematics and in this context she developed new Item Response Theory methods. She also worked as a Mathematics and Statistics university professor.

ORCID: https://orcid.org/0000-0002-2943-4270

LinkedIn: https://www.linkedin.com/in/patricia-costa-531517bb/

Ralph Hippe

Ralph Hippe, PhD is an expert at Cedefop. Previously he was a scientific officer in the European Commission’s Joint Research Centre (JRC), leading the SELFIE tool in the area of vocational education and training. Moreover, he has worked for the London School of Economics, Sciences Po Strasbourg and University of Tübingen, and has been a consultant for the World Bank and a visiting researcher at IDDRI/Sciences Po Paris. Hippe holds a PhD in economics from University of Strasbourg, BETA/CNRS and University of Tübingen (summa cum laude). He was awarded four dissertation awards for his PhD thesis on human capital and economic growth. He also holds Master’s degrees in business administration and management from Ecole de Management Strasbourg and University of Tübingen.

Panagiotis Kampylis

Panagiotis Kampylis, PhD is a Senior Research Fellow at the Italian National Research Council – Institute for Educational Technology. He worked as a scientific officer and team lead at European Commission´s Joint Research Centre (JRC) from January 2012 to August 2020. He has led the design, development, testing and implementation of SELFIE tool for schools´ digital capacity since its inception until August 2020. He graduated from Jyvaskyla University (Finland) in 2010 with his PhD in Cognitive Science (thesis on fostering creative thinking in education settings). His research interests include creativity and innovation in education and training, technology-enhanced learning, arts-education, computational thinking and digital and entrepreneurship competences. Before joining the JRC in 2012, Panagiotis worked many years as a primary school teacher, music teacher, head teacher, teacher trainer and creativity researcher. He has been an invited speaker at various events in the field of education, training, and lifelong learning. His scientific publications have been cited widely (see at https://bit.ly/3brr04y).

ORCID: https://orcid.org/0000-0003-1760-0347

Website: www.pankampylis.eu

Twitter: https://twitter.com/pankampylis

Linkedin: https://www.linkedin.com/in/pankampylis/

Notes

1. We define teachers’ professional learning as an overarching term that includes intentional learning activities aiming to improve teaching practices and learning outcomes. Professional learning activities vary in their degree of structure. They range from a well-structured format (e.g. courses or workshops) to less structured formats (e.g. participation in networks or peer learning). Usually, structured activities are also referred as continuous professional development by the literature (see: Darling-Hammond, Hyler, and Gardner 2017).

2. Based on the concepts of organisational capability (Killen, Beetham, and Knight 2017) and schools digital maturity (Balaban, Redjep, and Calopa 2018) we define digital capacity as the extent to which culture, policies, infrastructure as well as digital competence of students and staff support the effective integration of technology in teaching and learning practices (Costa, Castano-Munoz, and Kampylis 2021).

3. ISCED refers to the International Standard Classification of Education. ISCED 2 refers to lower secondary and ISCED 3 to upper-secondary.

4. The OECD-31 average is 40%.

5. For example, in Iceland teachers can now get official professional learning credit for their participation in online communities and reportedly, school heads are happy with it (Vuorikari 2019).

6. https://ec.europa.eu/education/schools-go-digital_en

7. In the scale, 1- indicates strong disagreement and 5- strong agreement. A sixth option was offered to allow indicating that a question is not applicable in respondent’s context. This last option was marked as missing in our analysis.

8. Based on the theoretical framework entitled European Framework of Digitally-Competent Educational Organisations (DigCompOrg).

9. The European Commission’s Joint Research Centre collects aggregated data with all responses of participants that is used for research purposes. The authors were given access to the data and the analyses performed for this paper fully comply with the data protection and privacy policy of SELFIE (available at https://ec.europa.eu/education/tools/selfie/privacy_en) that foresees the use of aggregated and anonymised data collected through the tool for publications.

10. In SELFIE, a school comprises one ISCED level.

11. This method estimates the relationship by minimising the sum of the squares in the difference between the observed and predicted values of the dependent variable shaped as a straight line and is adequate to the kind of variables used in this study. More information can be found at Wooldridge (2016).

12. The six items composing the index have been validated as an area using Confirmatory Factor Analysis in (Costa, Castano-Munoz, and Kampylis 2021).

13. For instance, in the case of a school that offers ISCED levels 1, 2 and 3 three dummy variables were included: school * ISCED1, school * ISCED2, and school * ISCED 3.

14. More information can be found at Taylor and Von Fintel (2016).

15. Results are available on request from the authors.

16. For robustness check we also run this model adding control variables referring to teachers’ digital competence in order to check if the competence level could influence the results (see Table 3). The results obtained are in line with the ones from model (1).

17. We present results for all teachers, but results are robust if we run the regression only including those teachers who have participated in all the proposed professional learning activities in the last year. Therefore, results remain stable if we limit our sample to teachers included in equation (2).

18. Results from the same model with variables referring to professional learning participation are consistent and show that participation in professional networks is the activity that is most related to the implementation of cross-curricular activities using ICT. Results are available upon request to the authors.

19. For all analyses presented in this section robustness checks were performed in order to check if teachers’ age group and experience could influence the results. The findings were very similar to the ones from this section and no significant heterogeneous effects were found.

20. See https://www.etwinning.net/en/pub/get-inspired/projects.cfm and https://www.etwinning.net/en/pub/get-inspired/kits.cfm for an example of a detailed guide on pedagogical design of cross-curricular projects.

21. More details can be provided by the authors upon request.

Appendices

Appendix

Table 2. Variables included in the models.

Description	Variables – Reported by teachers
Students digital competence index (DigComp)	Index was computed as the average of the six variables below (values from 1–5):
	Students learn how to behave safely online
	Students learn how to behave responsibly when they are online
	Students learn how to check that the information they find online is reliable and accurate
	Students learn how to give credit to others work they have found online
	Students learn how to create digital content
	Students learn how to communicate using digital technologies
Digital teaching practices (practices)	Variables (values from 1–5):
	Tailoring to needs
	Fostering creativity
	Engaging students
	Student collaboration
	Cross-curricular projects
Level of implementation of cross curricular projects (CC)	Students engagement in using digital technologies in cross-curricular projects (values from 1–5):)
Usefulness of professional learning activities (PLactivities)	Variables (values from 1–5):
	Face-to-face professional learning
	Online professional learning
	Within school collaboration
	In-house mentoring/coaching
	Other in-house training
	Study visits
	Accredited programmes

Table 3. Variables included in the robustness check (See footnote 16).

Description	Variables – Reported by teachers
Teachers’ digital competence proxies	Variables (values from 1–5):
	Use of online educational resources
	Creation of digital resources
	Use of virtual learning environment
	Use of digital technologies for school-related communication