Evaluating the effectiveness of a family literacy programme on the attainment of children with English as an additional language – a cluster randomised controlled trial

ABSTRACT

Background: A cluster randomised controlled trial was conducted to test whether the offer of Family Skills, a programme targeted at the parents of reception year (4–5 year-old) pupils with English as an Additional Language (EAL), raised attainment.

Purpose: There is little existing evidence of whether family literacy programmes delivered in school settings are effective in raising attainment among pupils with EAL in the English context. This study seeks to address this gap.

Programme description: Eleven two and a half hour sessions were delivered during the school day, by trained trainers, to the parents of pupils with EAL. Sessions aimed to enhance parents’ knowledge of effective literacy strategies.

Sample: In total, 115 primary schools in England were recruited to the study. Each school identified pupils in reception year that had EAL. The parents of these children were invited to take up the programme.

Design and methods: A two-arm parallel cluster randomised control trial was conducted, with schools randomised to intervention and control conditions. The primary outcome measure was literacy attainment. The evaluation also included a mixed methods process evaluation.

Results: The estimated effect size for the primary outcome based on adjusted intention-to-treat analysis, with a full set of covariates, was 0.03 (95% CI: −0.14 to 0.21). Not all parents invited to take up the intervention did so and it proved difficult to obtain a reliable measure of take-up.

Conclusions: Estimated effect sizes ranged from 0.13 in an unadjusted analysis to 0.03 in the full-adjusted analysis. Our results do not reach statistical significance at the 95% level. We discuss ways the intervention might be improved and address the issue of the low take-up of Family Skills.

KEYWORDS:

• Family literacy programme
• English as an additional language (EAL)
• literacy attainment
• primary education
• cluster randomised controlled trial (RCT)

Acknowledgments

We are grateful for the comments of anonymous referees on earlier drafts of this paper.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. It is important to note that the relative improvement in performance of pupils with EAL who have higher levels of language proficiency does not appear to be related to their EAL status per se, but rather other socio-demographic and economic factors correlated with both proficiency and higher attainment, specifically socio-economic factors (Demie and Hau 2013; Strand and Demie 2005).

2. For further details, see https://www.cem.org/reception-baseline-assessment (accessed, 23 May 2018).

3. See Hattie (2008) for a discussion of effect sizes in education, in which he suggests an effect size of 0.20 is considered small or modest.

4. Neither test corrects for the clustered nature of the data and therefore results represent conservative tests for baseline imbalances between intervention and control samples.

5. The raw score is the score obtained in the CEM Base assessment which is unstandardized and not age-adjusted. Simple mean outcomes at follow-up on the CEM Base assessment were 137.5 (SD = 25.02) and 133.4 (SD = 24.67) for intervention and control groups respectively.

6. The effect sizes reported in Table 2 are derived from results obtained from fitting a series of hierarchical linear regression models using maximum likelihood. Effect sizes were derived from the regression results according to an equation provided by Hedges (2011, equation 31, page 360). The standard derivation required for the derivation of effect sizes is calculated from the pooled within treatment group unconditional variance. The intra-class correlation coefficients required to adjust the pooled within treatment group variance were obtained from Model 1. The ratio between the effect size and regression coefficient for each of the four models in Table 2 is constant across the models (subject to rounding and a slightly smaller estimation sample for Model 4) as the same unconditional variances and intra-class correlation coefficients used in the calculations. The 95 percent confidence intervals were obtained from an equation for an approximation of the variance for the effect size provided by Hedges (2011, equation 33, page 361); this yields a conservative estimate of the variance.

7. The correlation between pre and post-test scores for the full sample was 0.73.

8. It is important to note that parents in control schools were generally unable to access Family Skills. Technically, therefore, one sided non-compliance is assumed . There was some evidence that one school in the control group did access Family Skills but that the number of children implicated in this was so low as to make this form of non-compliance in the control group effectively ignorable.

Additional information

Funding

The research described in the paper was funded by a consortium led by the Education Endowment Foundation with Unbound Philanthropy and The Bell Foundation.