E-learning engagement gap during school closures: differences by academic performance

ABSTRACT

We study the impact of COVID-19 school closures on differences in online learning usage by regional academic performance. Using data from Google Trends in Italy, we find that during the first lockdown, regions with a previously lower academic performance increased their searches for e-learning tools more than higher-performing regions. Analysing school administrative and survey data before the pandemic, we find that both teachers and students in lower performing regions were using no less e-learning tools than higher performing ones. These two findings suggest that the COVID-19 shock widened the e-learning usage gap between academically lower and higher-performing regions. Exploiting the regional variation in school closure mandates during the 2020–2021 academic year, we report that the patterns detected after the first lockdown were no longer present. Regions with different previous academic performance had the same response in terms of online learning usage when faced with stricter school closures.

KEYWORDS:

• E-learning
• COVID-19
• Education
• Inequality
• School closures

JEL CLASSIFICATION:

• C31
• C81
• I24
• H75

Acknowledgments

We thank Thomas Crossley, Andrea Ichino, Anna Lavizzari, Eleonora Erittuand Pedro Freitas, for very helpful comments and suggestions. We also thank the very usefuldiscussions at the 2nd Joint IZA and Jacobs Centre Workshop, 37th AIEL National Conference,and at the internal seminars of the European University Institute and Universidad Carlos III deMadrid. Finally, we gratefully acknowledge the financial support from the European University Institute COVID-19 Knowledge Hub and the support provided by the institution in grantingaccess to safely secured data. This article was completed when Amer-Mestre took service at theEuropean Commission, Joint Research Centre. The contents of this article do not necessarilyreflect the position or opinion of the European Commission. Declarations of interest: none.

Disclosure statement

This article was completed when Amer-Mestre took service at the European Commission, Joint Research Centre. The contents of this article do not necessarily reflect the position or opinion of the European Commission. Declarations of interest: none.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/00036846.2023.2167924

Notes

¹ The following papers have found none to negative average effects of e-learning tools on academic performance: Brown and Liedholm (2002), Fairlie and Robinson (2013), Figlio, Rush, and Yin (2013), Joyce et al. (2015), Beuermann et al. (2015), Bando et al. (2017), Cristia et al. (2017), and Lu and Song (2020).

² Established in 1997, among other tasks, INVALSI is entrusted with administering periodic tests to evaluate students’ academic achievement at different levels of education.

³ Andrew et al. (2020) show that the gap in the time used for learning between primary school students from high and low socioeconomic status increased in England.

⁴ For example, Carlana and La Ferrara (2021) find that an intervention giving free, individual, online tutoring to disadvantaged students in Italy substantially increased students’ academic performance. Angrist, Bergman, and Matsheng (2020) show that SMS and phone calls to parents minimize learning loss when school close.

⁵ Five days later, on March 9th, the president declared a national lockdown. On March 11th, all commercial activity except for supermarkets and pharmacies were closed, and on March 21st, the Italian Government closed all non-essential businesses and industries and restricted the movement of people.

⁶ Under each category, the Government implemented different measures to contain the spread of COVID-19. These measures mostly regulated social gatherings and events, and the ability to move across cities and regions. Thresholds in the value of specific epidemiological indicators measured at the regional level, such as relative COVID-19 active cases and the share of occupied beds in intensive care units, determined the changes across colour zones.

⁷ In January 2021, a new lower colour category was introduced, ``white”, where most of the measures present in the yellow category would not be in place. For schooling activity, however, this new white zone imposed the same measures as those present in its subsequent higher category, yellow.

⁸ Based on the data collected by Carlana and La Ferrara (2021) on 427 teachers in 76 middle schools all over Italy, by the month of June 2021 more than 96% of the teachers were providing synchronous online classes, and around 85% of the teachers provided some asynchronous videos additionally$-$usually no more than one hour per week. Right after the launch of the website, on March 26th, the Italian Ministry of Education passed the Ministerial Decree n.187, which allocated resources as follows: 1) 70 million euro to buy IT devices, such as tablets or computers, to lend temporarily to students in need, as well as to help these students improve their internet connection; 2) 10 million to allow schools to equip themselves with platforms and digital tools useful for distance learning and; c) 5 million euro to train teachers on methodologies and techniques for distance teaching. Due to bureaucracy delays, however, the help did not arrive to all in need.

⁹ The website didattica a distanza, created by the Italian Ministry of Education as a way to help teachers and students to have a smoother transition into e-learning promoted three different platforms: G Suite, provided by Google (which includes Google Classroom and Google Meetings), Microsoft Office 365 (which includes Word, PowerPoint, Excel, Outlook and Teams), provided by Microsoft, and WeSchool, provided by the Italian main communication company.

¹⁰ In fact, Google Classroom and WeSchool feature as the third and fourth most searched of all words in the list of ten trending words of Italy during the year 2020, only after Coronavirus and Elezione U.S.A (U.S.A elections) keywords, which take the first and second places respectively.

¹¹ INVALSI grades are reported according to the WLE (Weighted likelihood estimates) of individual parameters of the Rasch model (Rasch 1993) where 200 matches the national average.

¹² Table A2 in the Appendix A reports the same type of analysis for the event study.

¹³ As explained in Section III, the value of the index for a given term in each of the series – corresponding to each of the regions – is a value relative to each series’ own peak i.e if Lombardy takes the value of 70 and Campania takes the value of 50 on the index on a given date for a given term, it means that in that particular date, that term was searched 70% as much as in its most searched day in Lombardy and 50% as much as in its most searched day in Campania. We still do not know whether in that day and for that term, Lombardy had a higher search intensity than Campania or the opposite was true.

¹⁴ PISA 2015 provides data for Bolzano, Campania, Lombardy and Trento, while PISA 2018 provides data for Bolzano, Toscana, Sardegna and Trento. Note that both Bolzano and Trento (which form Trentino-Alto Adige) have a considerably lower share of publicly managed schools and therefore might be using e-learning differently than schools managed by the State. Excluding these two regions, PISA 2018 does not include any other region from the ‘above median performance’ group we consider in our main analysis. Therefore, PISA 2015 is best suited for our analysis.

¹⁵ Despite Campania being a much poorer region than Lombardy, one could wonder if results are driven by higher access to ICT, by students in Campania. We use the ICT Familiarity Questionnaire, which asks about device availability at home, and find that this is not the case.

Additional information

Funding

The work was supported by the European University Institute [COVID-19 Knowledge Hub]