Measuring and understanding patterns of change in intervention studies with children: Implications for evidence-based practice†

†We would like to thank ICAN, the health trusts involved and the two research officers, Kerry Williams and Belinda Seeff, who collected the data.

Abstract

Purpose: Comparisons across studies of the effects of intervention are problematic. Such analyses raise both methodological and statistical challenges. A single data set was examined to investigate whether different established approaches to measuring change in children with specific language impairments alter the conclusions that can be drawn regarding the efficacy of an intervention.

Methods: Measures of cognitive and language skills were collected at baseline and at six months following an intervention. Reliable and valid psychometric measures were used. Data from the intervention study were used to explore the patterns of results obtained using four different measures of change: change of diagnostic category, differential improvement across assessment measures, item specific changes and predictors of individual change.

Results: Associations between different tests purporting to measure similar constructs were modest. The measures identified different children as impaired both at baseline and follow-up. No effect of intervention was evident when a categorical analysis of impairment was used. Both treatment and comparison children changed significantly across time on the majority of measures, providing evidence of development, but specific effects of the intensive intervention were evident using ANCOVAs. Item analysis indicated that one of the standardized language tests adopted in the evaluation was insensitive to change over a six month period. Change in individual children's performance was predicted by language level on entry to the project.

Conclusion: The implications of the results are discussed in terms of the range of analytic approaches available to intervention researchers and the need to consider combinations of methods when analysing outcome data.

†We would like to thank ICAN, the health trusts involved and the two research officers, Kerry Williams and Belinda Seeff, who collected the data.

Keywords:

• Language
• measuring change
• intervention
• SLI

Acknowledgements

The authors wish to acknowledge the contribution of the Department of Health in the UK for funding this project through I CAN, Kerry Williams and Belinda Seeff for their contribution to the data collection, Jorg Prieler for his contribution to the interpretation of the item analysis and Dan Wright for his statistical advice.

Notes

†We would like to thank ICAN, the health trusts involved and the two research officers, Kerry Williams and Belinda Seeff, who collected the data.

NOTE

1.  ANCOVA is generally the preferred methods of analysis for interventions of this type. However, a series of repeated ANOVAS on the same data set provided the same results.