Computer assisted learning as extracurricular tutor? Evidence from a randomised experiment in rural boarding schools in Shaanxi

Abstract

This paper uses a clustered randomised field experiment to explore the effects of a computer assisted learning (CAL) programme on student academic and non-academic outcomes in poor, rural public schools in China. Our results show that a remedial, game-based CAL programme in math held outside of regular school hours with boarding students in poor rural public schools improved standardised math scores by 0.12 standard deviations. Students from poorer families tended to benefit more from the programme. However, CAL did not have any significant impact on either Chinese language standardised test scores or non-academic outcomes.

Keywords:

• education
• development
• computer assisted learning
• random assignment
• test scores
• China
• rural schools

Acknowledgement

We would like to acknowledge Dell Inc. for their generous support for REAP's Technology and Human Capital theme area. The hard work of dozens of volunteers from the Chinese Academy of Sciences, Northwest University of Xi'an and Qinghai Minorities University made this paper possible.

Notes

1. Self-efficacy of studying is a psychological concept that measures one's belief in one's ability to succeed in learning and problem solving in a certain subject. One's sense of efficacy of studying can play a major role in how one approaches goals, tasks and challenges related in the study of a subject. Individuals with higher levels of self-efficacy in studying typically take control over their own learning experience and are more likely to participate in class and prefer hands-on learning experiences.

2. There are 592 national designated poverty counties among the more than 2000 county-level jurisdictions in China. The Leading Group of the Alleviation of Poverty gave counties the designation in the 1990s based on the severity of the level of poverty in the county.

3. We only had 240 computers for the 36 treatment schools, which was not enough to accommodate all students of the same grade (or even the same class) simultaneously. Therefore, the students of the same grade split into several groups and took turns in using the computer for CAL sessions. Moreover, the CAL protocol required each student to have had two 40-minute sessions per week and existing school schedules could not accommodate all students that much computer time. As a result, it was infeasible for the CAL programme to accommodate boarding students of all of the four grades (third to sixth).

4. Metacognition is defined as ‘cognition about cognition’ or ‘knowing about knowing.’ It refers to a level of thinking that involves active control over the process of thinking that is used in learning situations. Planning the way to approach a learning task, monitoring comprehension and evaluating the progress towards the completion of a task: these are the skills that are metacognitive in nature. Similarly, maintaining motivation to see a task to completion is also a metacognitive skill. The ability to become aware of distracting stimuli – both internal and external – and sustain effort over time also involves metacognitive or executive functions. Metacognition helps people to perform many cognitive tasks more effectively (Metcalfe and Shimamura 1994).

5. To measure the self-efficacy of math studying, a professor in psychometrics and measurement in Beijing Normal University helped us choose among the 12 indicators of math attitudes used in TIMSS 2003 and developed a seven-item scale of self-efficacy of math studying that is appropriate to use under the context of elementary schools in China.

6. For example, the CAL programme might have improved the student's general learning ability and thus the student Chinese test score might also increase. The CAL programme might have also taken up so much of the student's time and energy in learning math that the student had less time and energy to spend on Chinese. In this case, the CAL programme in math might negatively affect the student academic performance in Chinese.

7. The test scores are normalised to standardised scores with mean equal to zero and standard deviation equal to one.

8. The only exception is that compared to the control group, grade 3 students whose mother had a high school degree benefited more from the CAL intervention than those whose mother had no high school degrees (row 2, column 2). This result is difficult to explain. However, one explanation is that since this heterogeneous treatment effect is only weakly significant at the 10 per cent level, it may be appearing by chance. In addition, we find no significant evidence of CAL intervention heterogeneous programme effects for other student demographic and family characteristics (that is, female, age and family off-farm), or for the student baseline math test scores or for the students’ access to computers before the programme started (results not included in the table for simplicity).