More is not always better: evidence from a randomised experiment of computer-assisted learning in rural minority schools in Qinghai

ABSTRACT

The education of poor and disadvantaged populations has been a long-standing challenge for education systems in both developed and developing countries. Drawing on data from two randomised controlled trials involving two cohorts of grade 3 students in poor rural minority schools in China’s Qinghai province, this paper explores the effects of computer-assisted learning (CAL) on student academic and non-academic outcomes for underserved student populations, and how interactions between the CAL programme and existing classroom resources affect the programme effectiveness. Results show that CAL could have significant beneficial effects on both student academic and non-academic outcomes. However, when the scope of the programme expanded to include a second subject (in this case, math – which was added on top of the Mandarin subject matter that was the focus of the first phase of the programme), some schools had to use regular school hours for CAL sessions. As a result, the phase II programme did not generate any (statistically) significant improvement over the first phase.

KEYWORDS:

• Education
• development
• computer-assisted learning
• random assignment
• test scores
• China
• rural minorities

Acknowledgements

We acknowledge financial support from the National Natural Science Foundation of China (grant numbers 71473240 and 71110107028), the 111 Project (Grant number B16031). We would also like to acknowledge Bowei Lee and his family, Acer Computers, DELL, Ford Foundation, Guo Foundation and Narada Foundation for their generous support for REAP’s Technology and Human Capital theme area. We are also grateful to the hard work of dozens of volunteers from the Chinese Academy of Sciences, Northwest University of Xi’an, Qinghai Minorities University made this paper possible.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. There are 592 national designated poverty counties among the more than 2000 county-level jurisdictions in China. The Leading Group of the Alleviation of Poverty gave counties the designation in the 1990s based on the severity of the level of poverty in the county.

2. The number of 57 was determined by our budget for field survey and data collection.

3. Please refer to Lai et al. (2015) for detailed description of the implementation of the phase I programme. The main purpose of this paper is to examine the interactions of classroom inputs and the CAL programme by comparing the phase I and phase II programmes, and thus descriptions of the phase I programme implementation is much simplified.

4. In assessing the results, one may ask if part of the CAL effect was an ‘extra teacher effect.’ In fact, we initially considered adding an additional treatment arm – as was included in Banerjee et al. (2007). The additional treatment would have provided remedial tutoring in the form of traditional teacher-based after school tutoring sessions. However, when we tried to implement this, we were not allowed to do so by the school district. Many Chinese school districts have strict regulations against the running of traditional teacher-based after school tutoring sessions. In addition, as discussed in the body of the paper, we did not allow teachers to teach Mandarin or interact with students about Mandarin-related questions during the CAL sessions (except that the teachers could answer questions about computer operations from the students). In fact, the scope for interaction was not great since none of the CAL supervisors were Mandarin teachers. There was little ability and no incentive for them to disregard the rules of the programme and teach children additional Mandarin.

5. He, Linden, and MacLeod (2008) also found that an Indian English Education Curriculum implemented through teacher training improved student math and English scores rather than just their English scores. They conjectured that this was due to the fact that when teachers implemented the English language CAL sessions, they were able to teach English more efficiently so that during the normal class period, teachers could devote more time to teaching other topics, such as math. While interesting for the study sites in South Asia, this is unlikely to be the case in our study. In our case, teachers were unlikely to spend less time on Mandarin during school and more time on math as the in-school hours for each subject are already delineated according to a strictly followed class schedule. In addition, it is unlikely that this schedule would change as in almost all Chinese schools Mandarin teachers teach Mandarin and math teachers teach math. Therefore, even if the CAL programme was helping improve the student performance in Mandarin, there would be no incentive for Mandarin teachers to allow math teachers to tutor children during their class period. Because of this, the mechanism that is at work in the He et al. case, almost certainly does not apply.

6. We will further explore the possibility of this explanation in the next section.

7. In our sample 86 per cent of the students are rural minority students whose native language is not Mandarin. Therefore, in theory, all (of most) of these minority students might be more or less affected by the language barrier. In particular, as the Salar, Tu and Tibetan minorities have their own spoken languages, it might be more difficult for students of these ethnic groups to understand the content of math programme in Mandarin (52% of our sample are students of these three ethnic minority groups).

8. We will further explore the possibility of this explanation in the next section.

Additional information

Notes on contributors

Fang Lai

Dr Fang Lai is a research associate at the IMPAQ International. She formerly worked as a research associate at Stanford University and assistant professor at the School of Education in New York University. She obtained her PhD in Agricultural and Resource Economics from University of California at Berkeley. Her work mainly focuses on evaluation of education programs, assessment of teacher and school effectiveness, school choice and educational reform, equity of education, social interaction and peer influence. She has authored and co-authored multiple papers in peer-reviewed academic journal in these fields.

Linxiu Zhang

Dr Linxiu Zhang is a professor and deputy director at the Center for Chinese Agricultural Policy (CCAP), Chinese Academy of Sciences (CAS). She is the co-director of a collaborative program called Rural Education Action Program (REAP). She obtained her PhD from Reading University. Her research concentrates on policy relevant studies on rural development in China, particularly on poverty alleviation, labor market development, public investments, and the economics of rural education and healthcare. She has published more than 200 papers in peer-reviewed journals and also received numerous awards. The most recent ones include “Ten Most Outstanding Women in Science” award from CAS (2013), TWAS “Celso-Furtado Prize” in Social Sciences (2013), “Fudan Management Excellence Award” (2014) and a TWAS fellow (2014).

Yu Bai

Dr Yu Bai is an assistant professor at the Center for Experimental Economics in Education (CEEE), Shaanxi Normal University in China. He obtained his PhD from the Center for Chinese Agricultural Policy (CCAP), Chinese Academy of Sciences (CAS). His research focuses on rural education in China, particularly on migration, technology and human capital, as well as healthcare.

Chengfang Liu

Dr Chengfang Liu is an associate professor with tenure at the China Center for Agricultural Policy (CCAP), School of Advanced Agricultural Science (SAAS), Peking University (PKU). She obtained her PhD in agricultural and resource economics from the University of California (Davis). For more than 15 years, her research has been in the field of agricultural and rural development, with focus on the provision of rural infrastructure, human capital, and migration. Her work is highly empirical, drawing on a large number of data sets that she and her coauthoring teams have collected in rural China. She has published 80 papers both domestically and internationally, of which 40 in SCI/SSCI indexed journals. Chengfang is currently working on how to carry out rigorous impact assessments in agricultural and rural development policies. She is the principal investigator of Rural Education Action Program (REAP).

Yaojiang Shi

Dr Yaojiang Shi is a professor and the founding director of the Center for Experimental Economics in Education (CEEE), Shaanxi Normal University, China. His research focuses on the emergence and development of economic enterprises in rural China and China's education reforms. He uses empirical research to identify important leverage points for education policy that addressed the needs of the rural poor. He has published more than 80 papers in peer-reviewed journals. He and his team has produced 25 policy reports and sent directly to national and provincial government leaders for review.

Fang Chang

Dr Fang Chang is an assistant professor at the Center for Experimental Economics in Education (CEEE), Shaanxi Normal University in China. Her research focuses on rural development as well as rural education. She obtained her PhD in public economics from China’s Northwestern University. She has published more than 10 papers in peer-reviewed journals. Recently, she has been working on teaching methods and teacher performance payment at the compulsory level in rural China.

Scott Rozelle

Dr Scott Rozelle is the Helen Farnsworth Senior Fellow in the Food Security and the Environment Program of Stanford University’s Freeman Spogli Institute for International Studies. He is the co-director of Rural Education Action Program (REAP). Rozelle’s research focuses almost exclusively on China and is concerned with three general themes; a) agricultural policy, including the supply, demand, and trade in agricultural commodities, b) the rural environment; and c) issues of poverty alleviation with a focus on rural education. He is the chair of the Board of Academic Advisors of the Center for Chinese Agricultural Policy. He has won numerous awards including International Collaboration Award of Chinese Academy of Science, China International Science and Technology Cooperation Award, Friendship Award of the People’s Republic of China.