Home broadband and student engagement during COVID-19 emergency remote teaching

ABSTRACT

During the academic year 2019–2020, school buildings worldwide closed in response to the COVID-19 pandemic, necessitating a rapid shift to distance education. This study assessed the influence of high-speed broadband availability on student engagement with distance learning during this period in Ireland. Employing data from a representative sample of 206 secondary schools, student engagement as perceived by school principals was estimated to have been more adversely affected among schools located in areas with lower coverage of high-speed broadband. This may be partly explained by a lower probability of poorer student engagement among schools that deployed live online video teaching. While the costs and benefits must be considered, these findings may support the case for government intervention to provide greater equity in access to high-speed broadband. Where distance learning is required in future, secondary teachers should be supported in the use of live online teaching to better foster student engagement.

KEYWORDS:

• distance education
• online learning
• secondary education
• broadband
• COVID-19

Introduction

In response to the COVID-19 pandemic, governments across the world closed school buildings to reduce the spread of the virus, forcing an abrupt transition to distance learning. There were nationwide school closures in 188 countries in April 2020, with 1.5 billion children and young people worldwide not physically attending a school premises (UNESCO, 2020). Distance learning has been defined as an educational setting “in which the expectation is that the student and instructor will not be physically co-present in the same location” (Allen et al., 2004, p. 403). This study investigated the impact of the shutdown of school buildings, and consequent move to distance learning in response to COVID-19, on secondary school student engagement in the Republic of Ireland.

Our aim is to provide an understanding of how engagement in learning on a remote basis was affected by the availability of high-speed broadband and access to information and communications technology (ICT) devices in students’ homes. Research has shown that “if engagement is to be socially just then all students should not only have equality and equity of access to activities, learning at similar levels, but also have similar opportunities beyond school related to these activities” (Zyngier, 2008, p. 1769). The contribution of opportunities beyond school to student learning is likely to have increased during the pandemic.

Studying factors that affect student engagement is important since engagement is positively associated with academic achievement (Lei et al., 2018), and evidence suggests there is scope for interventions targeting engagement to improve academic achievement (Fredricks et al., 2016). Longitudinal evidence has also linked student engagement with post-school and occupational outcomes (Abbott-Chapman et al., 2014). Greater insight and evidence on whether student engagement is related to the availability of high-speed broadband at home can inform policymaking and education planning during this period, as well as into the future as transitions to digital societies accelerate globally. The European Commission (EC, 2020) has argued that emergency situations such as COVID-19 demonstrate the necessity for universal access to high-speed broadband, with online education cited as one motivation. Previously, the EC (2016) also identified access to online learning as a motivating factor in its vision for a broadband speed of at least 100 megabits per second available to all European households.

This article presents the findings of a quantitative analysis of evidence gathered from school leaders during the shutdown of school buildings in Ireland.

Impact of COVID-19 school closures

Studies examining the impact of COVID-19 school closures on primary and secondary students, typically using survey data, are rapidly emerging from a range of international jurisdictions. In the Netherlands, primary and secondary students from more advantaged backgrounds were found to benefit from higher levels of parental support and better resources such as owning a computer, indicating a likely effect on inequality in educational opportunities (Bol, 2020). Similar issues were evident in the United Kingdom, where school-aged students from middle-class backgrounds were found to be twice as likely as students from working-class homes to participate in live and recorded lessons online every day, and higher levels of concern were apparent among teachers in deprived areas in relation to students having access to electronic devices and adequate Internet access (Cullinane & Montacute, 2020).

School-aged children from better-off families in the United Kingdom were also found to spend 30% more time on home learning and to have better access to learning resources (Andrew et al., 2020; Eyles et al., 2020). A study of households in the United States of America with students in K-12 (primary and secondary) schools found that less educated parents spent the same amount of time helping their children as parents with more education, but that they faced more problems in relation to resources, such as computer and Internet access (Bansak & Starr, 2021). A lack of infrastructure, such as electricity and Internet access, was also found to give rise to education inequalities amid COVID-19 school building closures in parts of Kenya (Mabeya, 2020). Other emerging research on the experiences of secondary schools during COVID-19 include studies based in Germany (Züchner & Jäkel, 2021) and Serbia (Kovács Cerović et al., 2021), with the use of, and access to, technology and digital media highlighted as important factors in shaping the learning experience.

Drawing on literature that examined the impact of missing school due to absenteeism, regular breaks, or school closures, Kuhfeld et al. (2020) projected that primary and secondary students in the US would only experience 63%–68% of usual learning gains in reading, and 37%–50% in mathematics, over the academic year due to COVID-19 closures. Another consequence of school building shutdowns has been the outright cancellation of in-person examinations. Andersen and Nielsen (2020) contended that secondary school exam cancellations may result in poorer future exam performance. As well as reduced academic learning, the socio-emotional learning of children of all ages may also be impacted (Outhwaite, 2020), which has also been found to affect academic attainment (Panayiotou et al., 2019). Some studies have also suggested COVID-19 school closures may reduce future earnings of current students (Azevedo et al., 2020; Psacharopoulos et al., 2020). In addition to impacting educational inequality and learning, significant challenges relating to the health and mental well-being of students as a result of school closures have been documented (Lee, 2020; Liang et al., 2020; Rundle et al., 2020; Workman, 2020).

Distance learning

Although COVID-19 has dramatically increased the use of some elements of distance learning, the practice of distance learning has been the subject of a substantial body of research. It has been pointed out in the literature that while schools worldwide switched to a version of distance learning in response to COVID-19, the short-term nature of these emergency arrangements meant that they did not amount to robust forms of distance learning, which would entail additional instructional design and planning (Doukakis & Alexopoulos, 2020). The term emergency remote teaching has been proposed as a more accurate description of the continued education while school buildings were closed during this period (Bozkurt et al., 2020; Crompton et al., 2021; Doukakis & Alexopoulos, 2020).

Several studies have compared the performance of primary and secondary students in distance education with students in traditional classroom-based education, generally finding no significant difference in outcomes (Cavanaugh, 2001; Cavanaugh et al., 2004; Means et al., 2013). A meta-analysis found no significant difference when aggregating data of all 51 included studies across secondary and higher education, but considerable variation in results between studies was noted (Zhao et al., 2005). Much of the research in this area, however, has focused on adult education (Allen et al., 2004; Rice, 2006). A recent systematic review of literature on the use of online distance learning noted that the field has moved away from an emphasis on theoretical articles to more empirical studies (Arnesen et al., 2019). This review built on previous reviews of this area (Barbour & Reeves, 2009; Cavanaugh et al., 2009; Hasler-Waters et al., 2014; Rice, 2006). Another recent systematic review (Crompton et al., 2021) considered empirical research from 2010 to 2020 on the use of emergency remote teaching in primary and secondary education in response to emergencies that cause school building closures. The review noted that the vast majority of this research has been conducted since the start of the COVID-19 pandemic, indicating the growing research interest in this area.

Lesson delivery factors, such as an appropriate balance of human resources and technology, have been identified as factors in successful distance learning (Zhao et al., 2005). A distinction may be drawn between asynchronous formats of communication for distance learning such as pre-recorded video and synchronous formats such as live, interactive video. Allen et al. (2004) found no significant reduction in student performance from utilizing asynchronous instead of synchronous learning formats, although Zhao et al. (2005) highlighted the importance of live human instruction for successful distance learning.

Secondary school distance learning is also shaped by student characteristics such as self-regulation skills and responsibility (Roblyer & Marshall, 2002). For successful distance learning, Roblyer and Marshall asserted that students also require access to appropriate technology. Teachers in Finland and India identified that not all students were in an equal position to make a transition to remote education in response to COVID-19, with barriers including existing technology, practices, and skills affecting students to varying degrees (Iivari et al., 2020). In Northern Ireland, almost a quarter of households engaged in primary and secondary homeschooling due to COVID-19 did not have access to a printer, and only half of students had their own device for accessing online content (Walsh et al., 2020).

Studies have also pointed out that the skills required of teachers are different for online distance learning than for traditional learning. One study reviewed literature on the competencies required of primary and secondary teachers for online learning (Pulham & Graham, 2018), while another systematic review found wide variation in the content of programs designed to prepare primary and secondary teachers for online distance learning (Moore-Adams et al., 2016).

Student engagement

There is broad agreement in the literature that student engagement encompasses three dimensions: behavioral, incorporating participation, effort, persistence and positive conduct; emotional, covering interactions with teachers and classmates and a sense of belonging; and cognitive, including self-regulated learning (Fredricks et al., 2016). The presence of strong relationships between students and teachers has emerged as a crucial factor in promoting student engagement (Fredricks et al., 2016; Nguyen et al., 2018; Quin, 2017; Taylor & Parsons, 2011; Yang et al., 2018). High levels of support from teachers (Havik & Westergård, 2020; Lam et al., 2016; Strati et al., 2017; Virtanen et al., 2018) and the provision of regular student feedback (Fredricks et al., 2016; Taylor & Parsons, 2011) have been found to enhance student engagement. Higher engagement has also been attributed to a positive school climate, characterized by high expectations in academic and discipline domains, coupled with a strong supportive ethos between teachers and students (Konold et al., 2018; Yang et al., 2018).

Classroom technology has been described as a “tool for engaged learning” (Taylor & Parsons, 2011, p. 14). Bond and Bedenlier (2019) proposed a framework for conceptualizing how classroom technology can influence student engagement, informed by literature examining educational technology and student engagement. Benefits of the use of ICT in the classroom were found to include enhanced participation, greater collaboration and the development of skills (McCoy et al., 2016). It should be noted, however, that the use of ICT in a classroom is different to using ICT for distance learning, and the benefits of classroom ICT integration may not necessarily extend to a distance learning setting.

There is also evidence to show that student engagement can be shaped by school structural characteristics, as well as the composition of the student body. Student engagement was found to be higher for smaller schools (Fredricks et al., 2004) and for more socioeconomically advantaged school settings (McCoy, Smyth, et al., 2014). A number of studies also highlighted higher engagement levels among single-sex girls’ schools and for female students more generally (Frawley et al., 2014; Havik & Westergård, 2020; Lam et al., 2016).

It is also worth noting research suggesting that increased student engagement has not always necessarily been positive. Student burnout among highly engaged students was identified among students from the United States of America and Finland in one study (Salmela-Aro et al., 2016). In the Irish context, high stress levels in advance of the final secondary school state examination have repeatedly been highlighted, with evidence showing that this persists into the post-school period, particularly for high-achieving female students (McCoy, Smyth, et al., 2014). Greater reliance on individual ICT devices for learning can also have unintended consequences. Technology-related distraction can adversely impact on learning or academic performance (Dempsey et al., 2019; Zureick et al., 2018), and cyberbullying has been found to impact engagement (Yang et al., 2020).

One study of the impact of COVID-19 school closures in Ireland indicated that barriers to engagement for secondary students include a lack of interest, a lack of support, and a lack of access to devices, all of which were more prevalent in schools with greater numbers of students from disadvantaged backgrounds (Devitt et al., 2020). The research also suggested that the mode of content delivery was associated with student engagement, with better levels of engagement associated with more interactive and collaborative approaches such as the use of assessment with feedback.

Contribution of this research

The quantitative analysis presented in this article adds to a rapidly growing evidence base that charts the experience of schools in transitioning to distance learning during COVID-19. We aim to broaden the literature on the effectiveness of distance learning relative to more traditional, classroom-based methods. This study posed the following research question: Is student access to high-speed broadband and to ICT devices at home associated with the impact of the shift to distance learning on student engagement? Given established links between student engagement and academic achievement (Lei et al., 2018), understanding the conditions shaping student engagement with distance learning is important, particularly in the context of COVID-19 school shutdowns. The study examined this question among secondary schools in Ireland and employed novel objective indicators that identified schools located in areas characterized by a lower availability of high-speed broadband.

Secondary education in Ireland

In the academic year 2019–2020, there were 723 secondary-level schools in Ireland (Department of Education and Skills [DES], 2020). Schools are publicly funded, although there is a subset of 51 fee-charging secondary-level schools operating. A total of 198 secondary-level schools are included in the Delivering Equality of Opportunity in Schools (DEIS) program, which aims to address educational inequality by providing additional resources to schools identified as having a high proportion of students from disadvantaged backgrounds (DES, 2017; McCoy, Quail et al., 2014). The majority of schools are English-language-medium schools, and there are 49 Irish-language-medium schools, known as a Gaelscoil. Further details on the secondary education system in Ireland are outlined in Appendix A.

Materials and methods

Data collection

An online survey of secondary school leaders (typically school principals or deputy principals) in the Republic of Ireland was fielded 2 months after the closure of school buildings (Mohan et al., 2020). School leaders are well placed to provide a rich insight on the lived experiences of their students and matters on the ground as they are key decision-makers in responding to the challenges of providing continuity of education. Moreover, during this unprecedented period, school leaders were found to have had more frequent contact with teachers, students, and parents (Mohan et al., 2020).

The sampling frame encompassed all 723 secondary schools. The survey was emailed to school principals on 13 May 2020, followed up with several reminders before the survey closed on 29 May 2020. A total of 234 responses were submitted, securing a 32.4% response rate. The sample of responses received was representative of schools nationally as recorded in the secondary school census for 2019–2020 (DES, 2020) in terms of geographic location (see Figure B-1 in Appendix B), other key school characteristics (see Figure B-2 in Appendix B), and broadband availability. We found that high-speed broadband was available to less than 90% of residences in the imputed catchment areas of 54% of schools in our sample. The corresponding figure for all 723 secondary schools in Ireland was 46%. The response rate varied by question, and for this study the sample was reduced to 206 schools that had complete data for the outcome variables of interest.

Descriptive statistics for key school characteristics of the sample are documented in Tables 1 and 2. The quantitative analysis related to the level of the school, providing a picture of the overall experience of each school from the perspective of school leaders, rather than the individual experiences of teachers or students within schools.

Table 1. Descriptive statistics: Categorical variables

Categorical variable	N	%
Outcome
Worse student engagement	144	69.9
Worse student engagement (3rd year)	133	64.6
Worse student engagement (6th year)	114	55.3
Worse delivery of lesson content	95	46.1
Worse student attendance	158	76.7
Catchment area characteristics
High-speed broadband available for less than 90% of residences	112	54.4
Distance teaching methods used in most or all classes
Live video (e.g., Zoom, Google Hangouts):
Yes	121	58.7
No	84	40.8
Missing	1	0.5
Prerecorded video:
Yes	63	30.6
No	141	68.5
Missing	2	1.0
Kahoot or similar platform:
Yes	73	35.4
No	130	63.1
Missing	3	1.5
Google Classrooms or Microsoft Teams:
Yes	178	86.4
No	27	13.1
Missing	1	0.5
Shared presentation (e.g., PowerPoint, Prezi):
Yes	108	52.4
No	94	45.6
Missing	4	1.9
Paper-based (e.g., textbooks, worksheets):
Yes	69	33.5
No	131	63.6
Missing	6	2.9
School structural characteristics
Gender:
Mixed	145	70.4
Girls only	32	15.5
Boys only	29	14.1
School size:
Small (24–350 students)	67	32.5
Medium (358–610 students)	72	35.0
Large (616–1,538 students)	67	32.5
Fee-paying	19	9.2
In DEIS program	61	29.6
Gaelscoil (Irish-language-medium school)	14	6.8
Individual ICT devices provided to students	138	67.0
Technology-driven:
Yes	88	42.7
No	91	44.2
Missing	27	13.1
Total	206	100

Note. N denotes number of valid responses (schools). The original responses of the categorical outcome variables are provided in Table B-1 in Appendix B.

Table 2. Descriptive statistics: Continuous variable

Continuous variable	N	Mean	SD	Min.	Max.
Catchment area characteristics
Average household income (€)	206	46669.8	8606.7	26888.58	69071.8

Note. N denotes number of valid responses (schools). SD denotes standard deviation.

Variables of interest

Outcomes

School leaders were asked to rate the effect of the transition on student engagement with learning, student attendance in classes, and delivery of lesson content, each on a 5-point Likert scale from much better to much worse. School leader perceptions of the effect of the shutdown on student engagement among 3rd and 6th year students due to sit Junior and Leaving Certificate examinations in June 2020 were also reported on a 5-point scale from very positive to very negative.

The response categories for the perceived engagement variables were aggregated to create binary indicators, framed as a negative outcome. For example, the binary outcome worse student engagement was assigned a value of 1 for a much worse or worse rating, while similar, better or much better responses were given the value of 0, forming the reference category. Details on the wording of questions, responses, and the construction of variables are provided in Appendix C, with the specific response categories chosen by respondents detailed in Table B-1 in Appendix B.

This analysis examined five outcomes, descriptive statistics for which are provided in Table 1. The primary outcome of interest was the level of engagement by all students as a result of school closures, as perceived by school principals. This measure of student engagement is likely to capture behavioral engagement, including participation and effort, rather than emotional or cognitive engagement. A conceptual framework of factors that may influence the impact of school closures on student engagement, informed by the literature reviewed in the Introduction section, is illustrated in Figure 1. Secondary outcomes included engagement among 3rd year (Junior Certificate) students, engagement among 6th year (Leaving Certificate) students, delivery of lesson content and overall student attendance.

Figure 1. Conceptual framework of factors of impact of COVID-19 school closures on student engagement

Main exposure variables of interest

The capacity of students to engage in distance learning may have been affected by access to high-speed broadband and access to ICT equipment at home. To investigate this, we generated aggregate variables of broadband availability and median household income (as an indicator of economic resources to purchase ICT devices) within the areas surrounding schools. A priori, we anticipated that a lack of access to high-speed broadband or economic resources was likely to result in reduced student engagement during the COVID-19 shutdown.

We created an indicator of the availability of high-speed broadband for every residence in Ireland by spatially linking each residence with the 2019 National Broadband Plan map (Department of Communications, Climate Action and Environment, 2019). This map, illustrated in Figure 2, distinguishes between three areas: (a) where high-speed broadband is currently available, (b) where there is no high-speed broadband available but where a commercial entity currently plans to roll out fiber broadband in the future, and (c) where there is no high-speed broadband available and where state intervention is required to provide broadband through the National Broadband Plan. This defines high-speed broadband as having a download speed of 30 megabits or higher per second.

Figure 2. National broadband plan map for Republic of Ireland, Quarter 3, 2019, based on data from Department of Communications, Climate Action and Environment (2019)

Using data from the 2016 census, we assigned each residence the household median gross annual income of the electoral division they were located in (Central Statistics Office, 2020).

For the purposes of this study, we drew a circular buffer around each school representing the catchment area, with each school’s buffer distance determined by the extent to which the school’s surrounding area was urban. We assigned a distance of 8 km to schools in highly urban areas, and 24 km to schools in highly rural areas, with the buffer distances based on data from both the National Household Travel Survey 2017 (Mooney et al., 2018) and the Growing Up in Ireland survey (McNamara et al., 2020). We then measured the proportion of residences for which high-speed broadband was available, and the average household income, within each school’s imputed catchment area. For our analysis, we created an indicator variable for each school concerning whether high-speed broadband was available to over 90% of residences in the imputed catchment area, on the basis that high-speed broadband would need to be available to the vast majority of students for it to be practicable for teachers to utilize applications requiring high-speed connectivity. Descriptive statistics of the high-speed broadband availability variable and average household income in imputed school catchment areas are provided in Tables 1 and 2.

Other variables of interest

Additional variables of interest included the use of various distance teaching methods and the prior use of technology in the classroom. Respondents were asked about the use of live video (e.g., via Zoom or Google Hangouts), shared presentations (e.g., via PowerPoint or Prezi) and prerecorded video. Responses were provided on a 4-point scale consisting of all, most, some and no classes, which were aggregated to a binary indicator of whether each method was used in either most or all classes.

School leaders were asked about the extent to which individual ICT devices, such as laptops, tablets, or mobile phones, were used by students for educational purposes in classrooms prior to the shutdown. An indicator of whether individual devices were used by over half of students in the school before the shutdown was created, which may signal whether schools had already embraced these technologies for teaching. Whether ICT equipment, such as school laptops or tablets, had been provided to students prior to, or as a result of, the shutdown was also specified. A chi-square test of independence between these two indicator variables on ICT devices confirmed that they were independent of each other (chi-square statistic = 1.0, p value = 0.31).

Statistical analysis

As shown in Table 1, 69.9% of schools in our sample reported a negative impact on student engagement as a result of the shutdown. A logistic regression model of this outcome variable may be expressed as follows (Equation 1):

$P(worseengagemen{t}_i=1,0|broadband,X)=\mathrm{\Lambda }\left(\alpha +{\beta }_{0}broadban{d}_i+\sum {\beta }_k{X}_{ki}+{\epsilon }_i\right)$

In Equation 1, Λ represents the cumulative distribution of the logistic function and ${\epsilon }_i$ represents an error term. The outcome $worseengagemen{t}_i$ is equal to 1 for School $i$ if that school reports a negative impact on student engagement, and 0 otherwise. The variable $broadban{d}_i$ represents whether high-speed broadband was available for over 90% of School $i$’s catchment area.

$X_{ki}$ represents a vector of all other independent variables, including the natural logarithm of catchment area average household income, along with the previous provision of devices to students and the previous use of individual devices in the classroom (technology-driven). Our models also adjusted for key school structural characteristics, as depicted in Figure 1 and outlined in Table 1, which may be expected to affect the outcome variable. For example, DEIS status schools are characterized by populations of students from lower income backgrounds, and student engagement with distance learning may have been lower among these schools (Devitt et al., 2020). We combined mutually exclusive fee-paying schools (Fee-paying in Table 1) and Irish-medium-language schools (Gaelscoil in Table 1) into one category due to small numbers in each group, to distinguish these types of schools, which are likely to have more socio-economically advantaged student populations (O’Connell et al., 2006), from all other schools. It is worth noting that while our models adjusted for key school structural characteristics, our survey did not capture information on other school-level factors that may be important, such as school climate or ethos (Konold et al., 2018; Yang et al., 2018).

We hypothesized that broadband availability may have affected how distance teaching was delivered, and it is this method of delivery that may have impacted student engagement. To explore this hypothesis, logistic regression models of overall student engagement were run where the broadband exposure variable was replaced by an indicator of a particular method of distance learning. For example, Equation 2 describes a model which includes the use of live video in most or all classes by School $i$:

$P(worseengagemen{t}_i=1,0|livevideo,X)=\mathrm{\Lambda }\left(\alpha +{\beta }_{0}livevide{o}_i+\sum {\beta }_k{X}_{ki}\right)+{\epsilon }_i$

Finally, we carried out robustness checks of the results from the main models using the original ordered categories of the student engagement outcome in an ordered logistic regression model. Descriptive statistics for these categories are presented in Table B-1 in Appendix B. Given small cell sizes for the much better and better categories, these were combined with the similar to give 3 categories: similar or better, worse and much worse.

All results are presented in the form of odds ratios. A statistically significant odds ratio greater than 1 indicates that the respective independent variable is associated with a higher likelihood of the outcome occurring, while an odds ratio lower than 1 indicates a lower probability.

Results

Table 3 presents results of the main model described in Equation 1, assessing factors in the impact of school closures on student engagement. The outcome in Model A is student engagement among the overall school population, while Models B and C focus specifically on Junior (3rd year) and Leaving Certificate (6th year) students. The results for Model A indicate that where the availability of high-speed broadband in the catchment area of schools was lower, it was almost three times more likely that the principal reported a negative impact on student engagement from the shutdown. In Model A, schools in areas with higher average income were also significantly more likely to report a negative impact on student engagement, and schools characterized as having previously been technology-driven were less likely to report a negative impact.

Table 3. Odds ratios for negative impact of shutdown on student engagement, delivery of content, and student attendance

	Shutdown has had negative impact on:
	Overall student engagement (A)	Junior Certificate (3rd year) student engagement (B)	Leaving Certificate (6th year) student engagement (C)	Delivery of lesson content (D)	Overall student attendance (E)
Catchment area characteristics
High-speed broadband availability:
More than 90% of residences	[ref.]	[ref.]	[ref.]	[ref.]	[ref.]
Fewer than 90% of residences	2.95***	1.13	1.58	1.36	2.21*
	(1.17)	(0.43)	(0.60)	(0.50)	(1.00)
Log of catchment area average income	13.22**	1.21	6.83*	24.60***	11.55*
	(15.44)	(1.34)	(6.96)	(25.63)	(14.44)
School structural characteristics
Gender:
Mixed	[ref.]	[ref.]	[ref.]	[ref.]	[ref.]
Girls only	0.42**	0.35**	0.70	0.47*	0.23***
	(0.19)	(0.15)	(0.30)	(0.21)	(0.10)
Boys only	0.54	0.50	1.40	0.72	0.61
	(0.26)	(0.22)	(0.62)	(0.31)	(0.32)
Size:
Small (24–350 students)	[ref.]	[ref.]	[ref.]	[ref.]	[ref.]
Medium (358–610 students)	1.02	1.49	1.13	1.41	1.05
	(0.43)	(0.59)	(0.43)	(0.52)	(0.47)
Large (616–1,538 students)	0.95	1.27	1.57	0.97	1.32
	(0.45)	(0.51)	(0.63)	(0.38)	(0.71)
In DEIS program	2.21*	1.93*	3.82***	1.23	2.21
	(1.02)	(0.75)	(1.53)	(0.43)	(1.17)
Fee-paying or Gaelscoil	0.36**	0.45*	0.91	0.27***	0.30***
	(0.15)	(0.20)	(0.30)	(0.13)	(0.13)
Individual ICT devices provided to students	1.38	0.69	1.13	1.17	1.09
	(0.51)	(0.23)	(0.37)	(0.37)	(0.44)
Technology-driven	0.47**	0.88	0.62	0.40***	0.66
	(0.17)	(0.30)	(0.20)	(0.13)	(0.27)
N	206	206	206	206	206

Note: *** denotes significance at 1% level, ** at 5% level, * at 10% level. Robust standard errors in parentheses.

N denotes number of valid responses (schools).

Table 3 also includes odds ratios of similar logistic regression models, looking at delivery of lesson content (Model D) and overall student attendance (Model E). As with student engagement in Model A, a negative impact on the delivery of content in Model D was more likely in areas of higher average income, while technology-driven schools were less likely to report poorer content delivery. Model E was reflective of Model A, suggesting relatively robust results, with a negative impact on student attendance more likely among schools in areas with lower broadband availability.

Figure 3 compares the proportion of schools which employed various teaching methods in most or all classes between schools in areas with high broadband availability and those in areas with lower coverage. The favored methods of distance teaching differed by availability of high-speed broadband, notably with live video utilized in a higher proportion of schools in areas with high broadband availability.

Figure 3. Distance teaching methods used in most or all classes by availability of high-speed broadband

Note. High broadband availability indicates schools where high-speed broadband is available for more than 90% of catchment area residences.

Summary results for six different models based on Equation 2, with student engagement as the outcome and distance teaching methods employed in some or all classes as the main exposure of interest, are presented in Table 4.

Table 4. Summary of odds ratios for negative impact of shutdown on student engagement

	Shutdown has had negative impact on:
	Overall student engagement
(F) Live video	0.41**
	(0.16)
(G) Prerecorded video	1.02
	(0.35)
(H) Kahoot or similar online platform	1.47
	(0.58)
(I) Google Classroom or Microsoft Teams	0.65
	(0.37)
(J) Shared presentation	0.96
	(0.32)
(K) Paper-based	1.12
	(0.40)
N	206

Note: *** denotes significance at 1% level, ** at 5% level, * at 10% level. Robust standard errors in parentheses. These results each correspond to fully adjusted models.

N denotes number of valid responses (schools).

As shown in Table 4, Model F, in which the method of distance teaching included was the use of live video, was the only model where the method of distance teaching was estimated to have a statistically significant odds ratio. In schools that used live video in some or all classes, the odds that the principal reported a negative impact on student engagement was less than half that of schools that did not report using these applications, independent of other school characteristics.

Table B-2 in Appendix B presents results for the ordered logistic regression model as a robustness check on our main results. As in Model A (Table 3), a negative impact on student engagement was more likely among schools in areas of lower broadband availability.

Discussion

Econometric modelling revealed that student engagement during COVID-19 school building closures in Ireland was perceived by secondary school principals to be more negatively affected among schools located in areas that had a less than universal availability of high speed-broadband. The level of student engagement was also associated with the method of distance learning used, with lower odds of worse student engagement in schools that employed live video for teaching.

The signals found in Model A in relation to broadband availability and economic resources were not reflected in models focusing on students due to sit state examinations in June 2020, that is, Models B and C. It is possible that the impact on engagement with learning for these groups was instead mainly affected by the cancellation of state examinations, and that this cohort-specific factor diminished any potential influence of broadband availability or economic resources.

In the primary model of student engagement, Model A, higher average household income in a school’s catchment area, which may affect access to ICT devices as well as other aspects of socioeconomic advantage, was significantly associated with a higher likelihood of worse engagement. This appears to be a counterintuitive result, suggesting that student educational engagement was more adversely affected in areas of higher-than-average income, independent of other school characteristics and of broadband availability. One possible explanation for this is that the outcomes represented a perception of change, for example, student engagement being rated worse than normal, and that baseline student engagement may have been systematically higher in areas characterized by a higher average income prior to the shutdown of schools. It may also be that parents residing in more affluent areas may have had greater expectations of schools and school leadership to maintain student engagement during the period, relative to those of more deprived areas. However, we note that this result did not persist in the ordered logistic regression analysis, and therefore this may be considered inconclusive.

Based on this evidence, the availability of high-speed broadband appears to have been important for engaging with distance learning using ICT. From a public policy perspective, this suggests that educational equality may have been affected by a lack of availability of high-speed broadband in certain regions, at least in a context of distance learning. It is likely that the availability of broadband affected how distance learning was delivered, which in turn could have impacted student engagement. The use of live video in most or all classes, a method that could be considered particularly reliant on high-speed broadband relative to others, appeared to be the only distance teaching method that significantly influenced our measure of student engagement.

This result concurs with the findings of Devitt et al. (2020), which revealed that student engagement was higher during the COVID-19 shutdown in Ireland when distance teaching methods were more interactive and collaborative. No such association for prerecorded videos or shared presentations was found, which may corroborate research highlighting the importance of live human instruction in distance learning (Zhao et al., 2005).

As 69.9% of schools reported a perceived negative impact of closures on student engagement (see Table 1), this could be interpreted as suggesting that distance learning was less effective than classroom-based learning among secondary students, contrary to some research that found no significant difference in effectiveness between distance learning and traditional learning methods (Allen et al., 2004; Cavanaugh, 2001; Cavanaugh et al., 2004; Means et al., 2013; Zhao et al., 2005). However, it may be that this was a short-term effect as teachers and students adjusted to an unexpected and unprecedented change in teaching and learning in the context of a pandemic.

Some schools in Ireland present themselves as technology-driven (Marcus-Quinn et al., 2019; McCoy et al., 2016). It is also worth noting that results presented in this paper suggest that a negative impact on student engagement and on lesson delivery was less likely among schools that previously utilized individual student devices in the classroom more intensively. These schools may have been better equipped to rapidly shift to distance learning. Moreover, it suggests that students’ experience of distance learning varied by the extent to which ICT had previously been integrated into the classroom, raising important questions over educational inequality.

Strengths and limitations

This quantitative analysis employed data from a nationally representative sample of secondary schools in Ireland, capturing one third of the country’s secondary schools. Models controlled for key structural school characteristics such as gender, size, and disadvantaged status, which helped to account for preexisting differences between schools. Geographic data to characterize school catchment areas where students were likely based while school buildings were closed, including objective data on the availability of high-speed broadband, complemented the survey. This allowed us to gain an understanding of the ability of students to engage in distance learning from home.

A number of limitations of this study must also be acknowledged. The survey was targeted at school principals and deputy principals, rather than teachers or students, and responses thus reflected the subjective perceptions of school leaders who may not necessarily have been directly involved in distance teaching and learning. However, we contend that school leaders, as key decision-makers within schools, were well placed to provide a rich insight on the experiences of students.

In addition, school catchment areas were imputed rather than directly observed. The cross-sectional nature means causality cannot be inferred and an omitted variable bias may exist where other characteristics such as school climate, which may be important for student engagement, could not be accounted for.

The study focused on distance learning in secondary education. Although our findings are relevant to researchers, policymakers, teachers, students, and other stakeholders in secondary education, further research is required to determine if any findings can be extended to primary or tertiary education.

Policy implications and directions for future research

While the costs, risks, and benefits of government intervention to provide high-speed broadband to remote areas must be fully evaluated, the influence of broadband availability on student engagement with distance learning could be regarded as one argument in favor of intervention from an educational equality perspective. Universal access to high-speed broadband for online learning has been championed by the EC (2016, 2020). The finding in relation to the use of live online video suggests that teachers could be better encouraged and supported in using these mediums for distance learning. In light of our findings in relation to the relatively better experience of technology-driven schools, enhanced integration of ICT in the classroom more generally could be advantageous in terms of increasing flexibility in learning, thus affording greater preparedness for unforeseen events.

Future research could consider the long-term impacts of the rapid transition to distance learning and the cancellation of state examinations, for example, investigating effects on future test scores, with particular regard for vulnerable groups such as students from lower income backgrounds or those with additional needs.

Conclusions

Secondary school principals perceived a general decline in student engagement during COVID-19 school building closures in Ireland, and statistical modeling revealed that a perceived reduction in engagement was more likely among schools located in areas characterized by lower coverage of high-speed broadband. This may be partly explained by a lower probability of reduced student engagement found among schools that employed live online video applications for teaching in most classes.

This study adds to the emerging literature assessing the impact of COVID-19 on teaching and learning. Studies of the rapid transition to distance learning due to COVID-19 help to document this unparalleled set of events and can develop understanding in this area.

Acknowledgments

The authors would like to acknowledge the contribution of the survey participants who gave their time to inform this work and the support of the National Association of Principals and Deputy Principals in Ireland.

Disclosure statement

The authors have no declarations or conflicts of interest to declare.

Data availability statement

The data that support the findings of this study are available from the corresponding author upon request.

Additional information

Funding

This research is supported by the Economic and Social Research Institute’s Electronic Communications Programme, jointly funded by the Department of Communications, Climate Action and the Environment and the Commission for Communications Regulation in Ireland.

Notes on contributors

Ciarán Mac Domhnaill

Ciarán Mac Domhnaill is a PhD student in the School of Economics and Finance, University of St Andrews. Environment, health, education, and energy economics are among his research interests.

Gretta Mohan

Gretta Mohan is a research officer at the Economic and Social Research Institute in Ireland and assistant professor at Trinity College Dublin. Her research interests include the economic and social impacts of electronic communications and digital technologies.

Selina McCoy

Selina McCoy is an associate research professor at the Economic and Social Research Institute in Ireland and professor at Trinity College Dublin. Her research focuses on educational inequality, inclusion, and digital technologies in teaching and learning.

Appendix A

Secondary education in Ireland

The Irish education system comprises three key stages: (a) primary, (b) secondary, and (c) further and tertiary education. Education is compulsory up to the age of 16, and secondary school caters for students between 12 and 18 years. In comparative terms, the Irish educational system can be characterized as general, rather than vocationally specific, in nature (Smyth & McCoy, 2011). While the established Junior Certificate program at lower secondary level (1st to 3rd years, including a Junior Certificate state examination in 3rd year) was previously criticized as being overly reliant on rote learning and memorization techniques, the Department of Education and Skills’ (DES) recently introduced Framework for Junior Cycle 2015 (DES, 2015) focuses on the integration of innovative approaches to teaching and assessment with the chief aim of enhancing students’ overall experiences within the typical school environment (Marcus-Quinn et al., 2019). However, upper secondary (4th to 6th years) qualifications continue to be assessed on the basis of student performance in external, standardized examinations. This is a high-stakes system, where access to tertiary education is based on grades achieved in the upper secondary state examination taken in 6th year, the Leaving Certificate (Banks & Smyth, 2015).

In the academic year 2019–2020, there were 723 secondary schools in Ireland (DES, 2020). Education is mostly publicly funded, with a minor private component of 51 fee-paying secondary schools (Newman, 2014). In total, 198 secondary schools are included in the Delivering Equality of Opportunity in Schools (DEIS) program, which aims to address educational inequality by providing additional resources to schools identified as having a high proportion of students from disadvantaged backgrounds (DES, 2017; McCoy et al., 2014). The majority of schools are English-language-medium schools, and there are 49 Irish-language-medium schools, known as a Gaelscoil.

Although there is a growing appreciation of the potential for digital technologies in supporting learning, infrastructural barriers and constraints in terms of teacher skills and competencies have limited the pedagogical integration of technologies in Irish schools (McCoy et al., 2016; Smyth & McCoy, 2021). In particular, Ireland has lagged behind many developed countries in terms of integrating digital technologies in education (McCoy et al., 2016; Smyth & McCoy, 2021). Further, despite stated policy objectives, experts have observed an absence of clear policy on technology use in schools, with high school autonomy meaning decisions on technology use are made at school level (Marcus-Quinn et al., 2019). All schools in Ireland have developed ICT plans, though some distinguish themselves as technology-driven, while others favor a more blended or traditional approach; therefore, students’ experience of technology in school varies widely (Marcus-Quinn et al., 2019; McCoy et al., 2016). Secondary school teachers in Ireland were found to have positive attitudes towards ICT but were more likely to use technology in preparing classes rather than in the classroom itself (Coyne et al., 2015). Within and across schools, teachers were found to vary in confidence in using ICT (McCoy et al., 2016).

To reduce the spread of COVID-19, the government of Ireland mandated the closure of all education settings on 12 March 2020. There was little warning of the shutdown, so schools had minimal time to prepare for the overnight move to distance learning. In April 2020, the DES announced the cancellation of Junior Certificate (3rd year) state examinations. Following a period of considerable uncertainty, the cancellation of the Leaving Certificate (6th year) state examinations was announced on 8 May 2020 and final assessments were switched to a system of calculated grades. Additional resources totaling €10 million were allocated across all primary and secondary schools to support students experiencing difficulties in engaging with distance learning (Mohan et al., 2020).

References

• Banks, J., & Smyth, E. (2015). ‘Your whole life depends on it’: Academic stress and high-stakes testing in Ireland. Journal of Youth Studies, 18(5), 598–616. https://doi.org/https://doi.org/10.1080/13676261.2014.992317
  <!--${if: isGetFTREnabled}-->

  <!--${/if:}--> <!--${ifNot: isGetFTREnabled}-->Taylor & Francis Online<!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--> Web of Science ®<!--${googleScholarLinkReplacer: 10.1080%2F13676261.2014.992317 journal %E2%80%98Your+whole+life+depends+on+it%E2%80%99%3A+Academic+stress+and+high-stakes+testing+in+Ireland author%3DJ.+Banks%26author%3DE.+Smyth 2015 Banks%2C+J.%2C+%26+Smyth%2C+E.+%282015%29.+%E2%80%98Your+whole+life+depends+on+it%E2%80%99%3A+Academic+stress+and+high-stakes+testing+in+Ireland.+Journal+of+Youth+Studies%2C+18%285%29%2C+598%E2%80%93616.+ 598-616 Journal+of+Youth+Studies 18 5 https%3A%2F%2Fdoi.org%2F10.1080%2F13676261.2014.992317 %00empty%00 %00null%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_2_1 10.1080%252F13676261.2014.992317 url_ver%3DZ39.88-2004%26rft.genre%3Darticle%26rft_id%3Dinfo%3Adoi%2F10.1080%252F13676261.2014.992317%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aulast%3DBanks%26rft.aufirst%3DJ.%26rft.date%3D2015%26rft.atitle%3D%25E2%2580%2598Your%2520whole%2520life%2520depends%2520on%2520it%25E2%2580%2599%253A%2520Academic%2520stress%2520and%2520high-stakes%2520testing%2520in%2520Ireland%26rft.jtitle%3DJournal%2520of%2520Youth%2520Studies%26rft.volume%3D18%26rft.issue%3D5%26rft.spage%3D598%26rft.epage%3D616 %00empty%00}-->
• Coyne, B., Devitt, N., Lyons, S., & McCoy, S. (2015). Perceived benefits and barriers to the use of high-speed broadband in Ireland’s second-level schools. Irish Educational Studies, 34(4),355–378. https://doi.org/https://doi.org/10.1080/03323315.2015.1128348
  <!--${if: isGetFTREnabled}-->

  <!--${/if:}--> <!--${ifNot: isGetFTREnabled}-->Taylor & Francis Online<!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--> Web of Science ®<!--${googleScholarLinkReplacer: 10.1080%2F03323315.2015.1128348 journal Perceived+benefits+and+barriers+to+the+use+of+high-speed+broadband+in+Ireland%E2%80%99s+second-level+schools author%3DB.+Coyne%26author%3DN.+Devitt%26author%3DS.+Lyons%26author%3DS.+McCoy 2015 Coyne%2C+B.%2C+Devitt%2C+N.%2C+Lyons%2C+S.%2C+%26+McCoy%2C+S.+%282015%29.+Perceived+benefits+and+barriers+to+the+use+of+high-speed+broadband+in+Ireland%E2%80%99s+second-level+schools.+Irish+Educational+Studies%2C+34%284%29%2C355%E2%80%93378.+ 355-378 Irish+Educational+Studies 34 4 https%3A%2F%2Fdoi.org%2F10.1080%2F03323315.2015.1128348 %00empty%00 %00null%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_3_1 10.1080%252F03323315.2015.1128348 url_ver%3DZ39.88-2004%26rft.genre%3Darticle%26rft_id%3Dinfo%3Adoi%2F10.1080%252F03323315.2015.1128348%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aulast%3DCoyne%26rft.aufirst%3DB.%26rft.date%3D2015%26rft.atitle%3DPerceived%2520benefits%2520and%2520barriers%2520to%2520the%2520use%2520of%2520high-speed%2520broadband%2520in%2520Ireland%25E2%2580%2599s%2520second-level%2520schools%26rft.jtitle%3DIrish%2520Educational%2520Studies%26rft.volume%3D34%26rft.issue%3D4%26rft.spage%3D355%26rft.epage%3D378 %00empty%00}-->
• Department of Education and Skills. (2015). Framework for Junior Cycle 2015. https://www.education.ie/en/Publications/Policy-Reports/Framework-for-Junior-Cycle-2015.pdf
  <!--${if: isGetFTREnabled}--><!--${/if:}--> <!--${ifNot: isGetFTREnabled}--><!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--><!--${googleScholarLinkReplacer: %00empty%00 web Framework+for+Junior+Cycle+2015 %00empty%00 2015 Department+of+Education+and+Skills.+%282015%29.+Framework+for+Junior+Cycle+2015.+ %00empty%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_4_1 %00empty%00 url_ver%3DZ39.88-2004%26rft.genre%3Dweb%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aucorp%3DDepartment%2520of%2520Education%2520and%2520Skills%26rft.date%3D2015%26rft.atitle%3DFramework%2520for%2520Junior%2520Cycle%25202015 %00empty%00}-->
• Department of Education and Skills. (2017). DEIS plan 2017: Delivering equality of opportunity in schools. https://www.education.ie/en/Publications/Policy-Reports/DEIS-Plan-2017.pdf
  <!--${if: isGetFTREnabled}--><!--${/if:}--> <!--${ifNot: isGetFTREnabled}--><!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--><!--${googleScholarLinkReplacer: %00empty%00 web DEIS+plan+2017%3A+Delivering+equality+of+opportunity+in+schools %00empty%00 2017 Department+of+Education+and+Skills.+%282017%29.+DEIS+plan+2017%3A+Delivering+equality+of+opportunity+in+schools.+ %00empty%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_5_1 %00empty%00 url_ver%3DZ39.88-2004%26rft.genre%3Dweb%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aucorp%3DDepartment%2520of%2520Education%2520and%2520Skills%26rft.date%3D2017%26rft.atitle%3DDEIS%2520plan%25202017%253A%2520Delivering%2520equality%2520of%2520opportunity%2520in%2520schools %00empty%00}-->
• Department of Education and Skills. (2020). Data on individual schools. https://www.education.ie/en/Publications/Statistics/Data-on-Individual-Schools/
  <!--${if: isGetFTREnabled}--><!--${/if:}--> <!--${ifNot: isGetFTREnabled}--><!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--><!--${googleScholarLinkReplacer: %00empty%00 web Data+on+individual+schools %00empty%00 2020 Department+of+Education+and+Skills.+%282020%29.+Data+on+individual+schools.+ %00empty%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_6_1 %00empty%00 url_ver%3DZ39.88-2004%26rft.genre%3Dweb%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aucorp%3DDepartment%2520of%2520Education%2520and%2520Skills%26rft.date%3D2020%26rft.atitle%3DData%2520on%2520individual%2520schools %00empty%00}-->
• Marcus-Quinn, A., Hourigan, T., & McCoy, S. (2019). The digital learning movement: How should Irish schools respond? The Economic and Social Review, 50(4), 767–783. https://www.esr.ie/article/view/1332
  <!--${if: isGetFTREnabled}--><!--${/if:}--> <!--${ifNot: isGetFTREnabled}--><!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--><!--${googleScholarLinkReplacer: %00empty%00 journal The+digital+learning+movement%3A+How+should+Irish+schools+respond%3F+The+Economic+and+Social+Review%2C+50%284%29%2C+767%E2%80%93783 author%3DA.+Marcus-Quinn%26author%3DT.+Hourigan%26author%3DS.+McCoy 2019 Marcus-Quinn%2C+A.%2C+Hourigan%2C+T.%2C+%26+McCoy%2C+S.+%282019%29.+The+digital+learning+movement%3A+How+should+Irish+schools+respond%3F+The+Economic+and+Social+Review%2C+50%284%29%2C+767%E2%80%93783.+ %00empty%00 %00empty%00 %00empty%00 %00null%00 %00empty%00 %00empty%00 %00null%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_7_1 %00empty%00 url_ver%3DZ39.88-2004%26rft.genre%3Darticle%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aulast%3DMarcus-Quinn%26rft.aufirst%3DA.%26rft.date%3D2019%26rft.atitle%3DThe%2520digital%2520learning%2520movement%253A%2520How%2520should%2520Irish%2520schools%2520respond%253F%2520The%2520Economic%2520and%2520Social%2520Review%252C%252050%25284%2529%252C%2520767%25E2%2580%2593783 %00empty%00}-->
• McCoy, S., Lyons, S., Coyne, B., & Darmody, M. (2016). Teaching and learning in second-level schools at the advent of high-speed broadband (Paper No. 51; ESRI Research Series). Economic and Social Research Institute. https://www.esri.ie/system/files/media/file-uploads/2016-04/RS51.pdf
  <!--${if: isGetFTREnabled}--><!--${/if:}--> <!--${ifNot: isGetFTREnabled}--><!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--><!--${googleScholarLinkReplacer: %00empty%00 journal Teaching+and+learning+in+second-level+schools+at+the+advent+of+high-speed+broadband+%28Paper+No.+51%3B+ESRI+Research+Series%29.+Economic+and+Social+Research+Institute author%3DS.+McCoy%26author%3DS.+Lyons%26author%3DB.+Coyne%26author%3DM.+Darmody 2016 McCoy%2C+S.%2C+Lyons%2C+S.%2C+Coyne%2C+B.%2C+%26+Darmody%2C+M.+%282016%29.+Teaching+and+learning+in+second-level+schools+at+the+advent+of+high-speed+broadband+%28Paper+No.+51%3B+ESRI+Research+Series%29.+Economic+and+Social+Research+Institute.+ %00empty%00 %00empty%00 %00empty%00 %00null%00 %00empty%00 %00empty%00 %00null%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_8_1 %00empty%00 url_ver%3DZ39.88-2004%26rft.genre%3Darticle%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aulast%3DMcCoy%26rft.aufirst%3DS.%26rft.date%3D2016%26rft.atitle%3DTeaching%2520and%2520learning%2520in%2520second-level%2520schools%2520at%2520the%2520advent%2520of%2520high-speed%2520broadband%2520%2528Paper%2520No.%252051%253B%2520ESRI%2520Research%2520Series%2529.%2520Economic%2520and%2520Social%2520Research%2520Institute %00empty%00}-->
• McCoy, S., Quail, A., & Smyth, E. (2014). The effects of school social mix: Unpacking the differences. Irish Educational Studies, 33(3), 307–330. https://doi.org/https://doi.org/10.1080/03323315.2014.955746
  <!--${if: isGetFTREnabled}-->

  <!--${/if:}--> <!--${ifNot: isGetFTREnabled}-->Taylor & Francis Online<!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--> Web of Science ®<!--${googleScholarLinkReplacer: 10.1080%2F03323315.2014.955746 journal The+effects+of+school+social+mix%3A+Unpacking+the+differences author%3DS.+McCoy%26author%3DA.+Quail%26author%3DE.+Smyth 2014 McCoy%2C+S.%2C+Quail%2C+A.%2C+%26+Smyth%2C+E.+%282014%29.+The+effects+of+school+social+mix%3A+Unpacking+the+differences.+Irish+Educational+Studies%2C+33%283%29%2C+307%E2%80%93330.+ 307-330 Irish+Educational+Studies 33 3 https%3A%2F%2Fdoi.org%2F10.1080%2F03323315.2014.955746 %00empty%00 %00null%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_9_1 10.1080%252F03323315.2014.955746 url_ver%3DZ39.88-2004%26rft.genre%3Darticle%26rft_id%3Dinfo%3Adoi%2F10.1080%252F03323315.2014.955746%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aulast%3DMcCoy%26rft.aufirst%3DS.%26rft.date%3D2014%26rft.atitle%3DThe%2520effects%2520of%2520school%2520social%2520mix%253A%2520Unpacking%2520the%2520differences%26rft.jtitle%3DIrish%2520Educational%2520Studies%26rft.volume%3D33%26rft.issue%3D3%26rft.spage%3D307%26rft.epage%3D330 %00empty%00}-->
• Mohan, G., McCoy, S., Carroll, E., Mihut, G., Lyons, S., & Mac Domhnaill, C. (2020). Learning for all? Second-level education in Ireland during COVID-19 (ESRI Survey and Statistical Report Series No. 92). Economic and Social Research Institute). https://www.esri.ie/system/files/publications/SUSTAT92_3.pdf
  <!--${if: isGetFTREnabled}--><!--${/if:}--> <!--${ifNot: isGetFTREnabled}--><!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--><!--${googleScholarLinkReplacer: %00empty%00 journal Learning+for+all%3F+Second-level+education+in+Ireland+during+COVID-19+%28ESRI+Survey+and+Statistical+Report+Series+No.+92%29.+Economic+and+Social+Research+Institute%29 author%3DG.+Mohan%26author%3DS.+McCoy%26author%3DE.+Carroll%26author%3DG.+Mihut%26author%3DS.+Lyons%26author%3DC.+Mac+Domhnaill 2020 Mohan%2C+G.%2C+McCoy%2C+S.%2C+Carroll%2C+E.%2C+Mihut%2C+G.%2C+Lyons%2C+S.%2C+%26+Mac+Domhnaill%2C+C.+%282020%29.+Learning+for+all%3F+Second-level+education+in+Ireland+during+COVID-19+%28ESRI+Survey+and+Statistical+Report+Series+No.+92%29.+Economic+and+Social+Research+Institute%29.+ %00empty%00 %00empty%00 %00empty%00 %00null%00 %00empty%00 %00empty%00 %00null%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_10_1 %00empty%00 url_ver%3DZ39.88-2004%26rft.genre%3Darticle%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aulast%3DMohan%26rft.aufirst%3DG.%26rft.date%3D2020%26rft.atitle%3DLearning%2520for%2520all%253F%2520Second-level%2520education%2520in%2520Ireland%2520during%2520COVID-19%2520%2528ESRI%2520Survey%2520and%2520Statistical%2520Report%2520Series%2520No.%252092%2529.%2520Economic%2520and%2520Social%2520Research%2520Institute%2529 %00empty%00}-->
• Newman, C. (2014). Education: Market failure and government interventions. In J. O’Hagan (Ed.), The economy of Ireland: National and sectoral policy issues (12th ed., pp. 338–361). Gill and Macmillan.
  <!--${if: isGetFTREnabled}--><!--${/if:}--> <!--${ifNot: isGetFTREnabled}--><!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--><!--${googleScholarLinkReplacer: %00empty%00 book Education%3A+Market+failure+and+government+interventions.+In+J.+O%E2%80%99Hagan+%28Ed.%29%2C+The+economy+of+Ireland%3A+National+and+sectoral+policy+issues+%2812th+ed.%2C+pp.+338%E2%80%93361%29 author%3DC.+Newman 2014 Newman%2C+C.+%282014%29.+Education%3A+Market+failure+and+government+interventions.+In+J.+O%E2%80%99Hagan+%28Ed.%29%2C+The+economy+of+Ireland%3A+National+and+sectoral+policy+issues+%2812th+ed.%2C+pp.+338%E2%80%93361%29.+Gill+and+Macmillan. %00empty%00 %00null%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_11_1 %00empty%00 url_ver%3DZ39.88-2004%26rft.genre%3Dbook%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aulast%3DNewman%26rft.aufirst%3DC.%26rft.date%3D2014%26rft.btitle%3DEducation%253A%2520Market%2520failure%2520and%2520government%2520interventions.%2520In%2520J.%2520O%25E2%2580%2599Hagan%2520%2528Ed.%2529%252C%2520The%2520economy%2520of%2520Ireland%253A%2520National%2520and%2520sectoral%2520policy%2520issues%2520%252812th%2520ed.%252C%2520pp.%2520338%25E2%2580%2593361%2529%26rft.pub%3DGill%2520and%2520Macmillan %00empty%00}-->
• Smyth, E., & McCoy, S. (2011). The dynamics of credentialism: Ireland from bust to boom (and back again). Research in Social Stratification and Mobility, 29(1), 91–106. https://doi.org/https://doi.org/10.1016/j.rssm.2011.01.002
  <!--${if: isGetFTREnabled}-->

  <!--${/if:}--> <!--${ifNot: isGetFTREnabled}-->Crossref<!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--> Web of Science ®<!--${googleScholarLinkReplacer: 10.1016%2Fj.rssm.2011.01.002 journal The+dynamics+of+credentialism%3A+Ireland+from+bust+to+boom+%28and+back+again%29 author%3DE.+Smyth%26author%3DS.+McCoy 2011 Smyth%2C+E.%2C+%26+McCoy%2C+S.+%282011%29.+The+dynamics+of+credentialism%3A+Ireland+from+bust+to+boom+%28and+back+again%29.+Research+in+Social+Stratification+and+Mobility%2C+29%281%29%2C+91%E2%80%93106.+ 91-106 Research+in+Social+Stratification+and+Mobility 29 1 https%3A%2F%2Fdoi.org%2F10.1016%2Fj.rssm.2011.01.002 %00empty%00 %00null%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_12_1 10.1016%252Fj.rssm.2011.01.002 url_ver%3DZ39.88-2004%26rft.genre%3Darticle%26rft_id%3Dinfo%3Adoi%2F10.1016%252Fj.rssm.2011.01.002%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aulast%3DSmyth%26rft.aufirst%3DE.%26rft.date%3D2011%26rft.atitle%3DThe%2520dynamics%2520of%2520credentialism%253A%2520Ireland%2520from%2520bust%2520to%2520boom%2520%2528and%2520back%2520again%2529%26rft.jtitle%3DResearch%2520in%2520Social%2520Stratification%2520and%2520Mobility%26rft.volume%3D29%26rft.issue%3D1%26rft.spage%3D91%26rft.epage%3D106 %00empty%00}-->
• Smyth, E., & McCoy, S. (2021). School experiences and post-school pathways in the Republic of Ireland. In E. A. Marshall & J. E. Symonds (Eds.), Young adult development at the school-to-work transition: International pathways and processes (pp. 205–221). Oxford University Press.
  <!--${if: isGetFTREnabled}-->

  <!--${/if:}--> <!--${ifNot: isGetFTREnabled}-->Crossref<!--${/ifNot:}--><!--${if: isGetFTREnabled}--><!--${/if:}--><!--${googleScholarLinkReplacer: 10.1093%2Foso%2F9780190941512.003.0009 book School+experiences+and+post-school+pathways+in+the+Republic+of+Ireland author%3DE.+Smyth%26author%3DS.+McCoy%26 2021 Smyth%2C+E.%2C+%26+McCoy%2C+S.+%282021%29.+School+experiences+and+post-school+pathways+in+the+Republic+of+Ireland.+In+E.+A.+Marshall+%26+J.+E.+Symonds+%28Eds.%29%2C+Young+adult+development+at+the+school-to-work+transition%3A+International+pathways+and+processes+%28pp.+205%E2%80%93221%29.+Oxford+University+Press. 205-221 %00null%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 %00empty%00 getFTREnabled FULL_TEXT %00empty%00}--><!--${sfxLinkReplacer: e_1_3_5_1_2_6_13_1 10.1093%252Foso%252F9780190941512.003.0009 url_ver%3DZ39.88-2004%26rft.genre%3Dbook%26rft_id%3Dinfo%3Adoi%2F10.1093%252Foso%252F9780190941512.003.0009%26rfr_id%3Dinfo%3Asid%2Fliteratum%253Atandf%26rft.aulast%3DSmyth%26rft.aufirst%3DE.%26rft.date%3D2021%26rft.atitle%3DSchool%2520experiences%2520and%2520post-school%2520pathways%2520in%2520the%2520Republic%2520of%2520Ireland%26rft.aulast%3DMarshall%26rft.aufirst%3DE.%2520A.%26rft.btitle%3DYoung%2520adult%2520development%2520at%2520the%2520school-to-work%2520transition%253A%2520International%2520pathways%2520and%2520processes%26rft.spage%3D205%26rft.epage%3D221%26rft.pub%3DOxford%2520University%2520Press %00empty%00}-->

Appendix B:

Additional figures and tables

Figure B-1: Regional representativeness of survey sample

Figure B-2: Representativeness of survey sample, key school characteristics

Table B-1. Outcome variables: original response categories

Variable	N	%
Student engagement:
Much better	0	0
Better	8	3.9
Similar	54	26.2
Worse	107	51.9
Much worse	37	18.0
Do not know	0	0
Impact on student engagement (3rd year):
Very positive	3	1.5
Positive	23	11.2
Neither negative nor positive	46	22.3
Negative	100	48.5
Very negative	33	16.0
Do not know/not applicable	1	0.5
Impact on student engagement (6th year):
Very positive	1	0.5
Positive	30	14.6
Neither negative nor positive	50	24.3
Negative	80	38.8
Very negative	34	16.5
Do not know/not applicable	11	5.3
Delivery of lesson content
Much better	1	0.5
Better	13	6.3
Similar	96	46.6
Worse	85	41.3
Much worse	10	4.9
Do not know	1	0.5
Student attendance in lessons
Much better	1	0.5
Better	3	1.5
Similar	44	21.4
Worse	111	53.9
Much worse	47	22.8
Do not know	0	0
N	206	100

Note: N denotes number of valid responses (schools).

Table B-2. Ordered logistic regression odds ratios for negative impact of shutdown on student engagement

	Shutdown has had negative impact on:
	Overall student engagement
Catchment area characteristics
High-speed broadband availability:
More than 90% of residences	[ref.]
Fewer than 90% of residences	2.20**
	(0.76)
Log of catchment area average income	5.65*
	(5.52)
School characteristics
Gender:
Mixed	[ref.]
Girls only	0.44*
	(0.19)
Boys only	0.48*
	(0.19)
Size:
Small (24–350 students)	[ref.]
Medium (358–610 students)	0.81
	(0.28)
Large (616–1,538 students)	1.00
	(0.39)
In DEIS program	2.30**
	(0.80)
Fee-paying or Gaelscoil	0.49
	(0.22)
Individual ICT devices provided to students	1.26
	(0.40)
Technology-driven	0.42***
	(0.13)
N	206

Note: *** denotes significance at 1% level, ** at 5% level, * at 10% level. Robust standard errors in parentheses. N denotes number of valid responses (schools).

Appendix C:

Additional details on survey questions

Overall student engagement

Overall, compared to normal in-school learning, how would you rate teaching and learning since the school closures?

Student engagement with learning responses:

• Much better

• Better

• Similar

• Worse

• Much worse

• I don’t know

Indicator created for negative impact:

• 0 if much better, better, similar, don’t know.

• 1 if worse, much worse.

Junior Certificate (3rd year) student engagement

What effect is the COVID-19 shutdown having on the following in your school?

Junior Certificate student engagement responses:

• Very positive

• Positive

• Neither negative nor positive

• Negative

• Very negative

• Don’t know or not applicable

Indicator variable created for negative impact:

• 0 if very positive, positive, neither negative nor positive, don’t know or not applicable.

• 1 if negative, very negative.

Leaving Certificate (6th year) student engagement

What effect is the COVID-19 shutdown having on the following in your school?

Leaving Certificate student engagement responses:

• Very positive

• Positive

• Neither negative nor positive

• Negative

• Very negative

• Don’t know or not applicable

Indicator variable created for negative impact:

• 0 if very positive, positive, neither negative nor positive, don’t know or not applicable.

• 1 if negative, very negative.

Delivery of lesson content

Overall, compared to normal in-school learning, how would you rate teaching and learning since the school closures?

Delivery of content responses:

• Much better

• Better

• Similar

• Worse

• Much worse

• I don’t know

Indicator created for negative impact:

• 0 if much better, better, similar, don’t know.

• 1 if worse, much worse.

Overall student attendance

Overall, compared to normal in-school learning, how would you rate teaching and learning since the school closures?

Student attendance and participation in lessons responses:

• Much better

• Better

• Similar

• Worse

• Much worse

• I don’t know

Indicator created for negative impact:

• 0 if much better, better, similar, don’t know.

• 1 if worse, much worse.

Other variables of interest

Distance teaching methods

Across all teachers in school what approaches to distance teaching and learning have been employed during the closure period? Respondents select response from following grid:

Table

	All classes	Most classes	Some classes	No classes
Paper-based learning e.g., use of textbooks, worksheets, exercise booklets
Virtual learning via live video (e.g., Zoom, Google Hangouts, etc.)
Pre-recorded video
Shared presentations (PowerPoint, Prezi, etc.)
Google Classroom or Microsoft Teams
Other online learning platform (e.g., Kahoots, StudyClix)

Indicator variables created for use of each method in most or all classes:

• 0 if some classes, no classes.

• 1 if all classes, most classes.

Individual ICT devices provided to students

Has your school provided ICT equipment to students?

Responses:

• Yes – prior to school closures

• Yes – as a result of school closures

• We are in the process of sourcing or acquiring ICT to provide

• No

Indicator variable created of ICT equipment provided to students:

• 0 if we are in the process of sourcing or acquiring ICT to provide, no.

• 1 if yes – prior to school closures, or yes – as a result of school closures.

Technology driven

Prior to the shutdown, what proportion of students used the following individual device(s) for educational purposes in any class? Respondents select response from following grid:

Table

	Most or all	More than half	About half	Less than half	None or almost none
Laptops, netbooks, mini notebooks
Tablets
Mobile phones or smartphones

Indicator variable created of use of ICT devices in classroom by over half of students prior to shut down:

• 0 if about half, less than half, none or almost all for all three ICT device types.

• 1 if most or all, more than half for any ICT device type.