Economic evaluation of the Happy Talk pilot effectiveness trial: A targeted selective speech, language and communication intervention for children from areas of social disadvantage

Abstract

Purpose: This study presents a cost-effectiveness analysis of a targeted selective pre-school intervention programme, “Happy Talk”, which focuses on language development, by simultaneously enhancing parental interaction and the pre-school environment.

Method: Happy Talk (delivered to 77 children) is an add on intervention, and is compared to usual care, adopting a healthcare perspective. Cost-effectiveness analyses were carried out using the Pre-school Language Scale 5- Total (PLS-5) for baseline analysis and the Child Health Utility Instrument (CHU9D) in a secondary analysis.

Result: Baseline cost-effectiveness analysis showed Happy Talk was more effective (6.3 point change in total PLS-5 standard score – effect size 0.463SD and more expensive (€82.06) than usual care (cost-effectiveness ratio is €13.02 per unit change). Employing a proxy to estimate monetary net benefit, the benefits outweigh the costs, showing that it is cost-effective. However, results do not persist when health-related quality of life outcome measures are considered.

Conclusion: Findings suggest a targeted selective public health approach, could be considered value for money to reduce the societal burden of children with low levels of speech, language and communication. However, measurement of longer term outcomes and a larger trial are required, to definitively inform policy changes.

Keywords:

• economic evaluation
• language intervention
• children

Introduction

Effective speech, language and communication skills are central to an individual’s overall development. There are a number of negative associations between speech, language and communication needs (SLCN) and future outcomes, including poor academic success (Field, 2010), lower levels of literacy (Catts, Fey, Tomblin, & Zhang, 2002), social-emotional and behavioural difficulties (Beitchman et al., 2001; Qi & Kaiser, 2003) and poorer employment prospects (Law, Rush, Schoon, & Parsons, 2009). If difficulties are untreated, there are long-term health, educational and societal consequences (Law et al., 2009), which in turn place a significant economic burden on society. This burden is particularly evident in socially disadvantaged areas, where 40–50% of children entering pre-school, have significantly poorer speech and language skills than expected for their age (Locke, Ginsborg, & Peers, 2002). Interventions in the early years are one of the most effective ways of improving children’s long-term outcomes, particularly for those who are socially disadvantaged (Campbell et al., 2014; Heckman, Moon, Pinto, Savelyev, & Yavitz, 2010). The premise is that they bring greater returns for investment than those later in a child’s development, through hypothesised effects of developmental cascades wherein “skill begets skill” (Heckman et al., 2010).

In keeping with the United Kingdom Medical Research Council (MRC) guidelines for the development and evaluation of complex interventions (Craig et al., 2008), evaluations should progress from efficacy to effectiveness and cost-effectiveness, before scaled implementation. While efficacy trials evaluate whether interventions work under optimal, highly controlled conditions, effectiveness trials test whether interventions work when delivered by practitioners to typical clients in real world settings. However, although this is the recommended progression and commonly carried out in other disciplines, it is rare in speech-language pathology (SLP). In the field of SLP, most studies do not move beyond the efficacy stage of evaluation and for those that do, research on their cost-effectiveness is usually overlooked (Bercow, 2008). However, given the limited nature of resources available to commissioners and policy makers, the need for quality economic evaluations is more pressing than ever, to inform how funds are allocated and to ensure that public sector resources are used wisely. Economic evaluations employ standardised methodology to jointly examine costs and effects to determine if an intervention is worth doing compared to alternatives (Drummond, Sculpher, Claxton, Stoddart, & Torrance, 2015). Here we present an economic evaluation of a targeted selective community-based language intervention programme (Happy Talk), for which effectiveness has been established (Frizelle et al., 2021).

Economic evaluations, financial costs and benefits of SLCN interventions

Despite the long-term adverse effects of poor speech, language and communication skills, literature regarding the health, educational, and societal costs associated with SLCN is relatively scarce. In establishing indirect costs associated with work participation and wages in Australia, Cronin, Reeve, McCabe, Viney, and Goodall (2020) employed a human capital approach (where payback on an investment is measured in terms of increased production in the market place) to estimate the projected difference in lost wages between children of different levels of severity of language difficulties. They found that with SLP treatment an annual average of A$355 per person could be saved, equating to savings of A$5.37 billion across a lifetime (2020 Australian dollars). In 2010, using a similar approach, Marsh et al,. estimated UK savings in relation to effective SLP intervention, and found that based on an increase in adult earnings of £4325 per person, annual savings would be in the region of £878M (in 2010 British pounds).

A recent review by Le et al. (2020) considered service utilisation costs and health-related quality of life associated with low language and of the 11 papers that were included most focussed on healthcare and/or out-of-pocket (OOP) costs to families. Depending on the methodology there was considerable variation in costs between studies (ranging from A$430 to A$2560 (in 2017) for a two year period). Overall, healthcare and OOP costs were significantly higher for those with language difficulties than those with typical language (Cronin et al., 2017; Le et al., 2017).

As children with SLCN progress to school, they are at increased risk of falling behind their peers, and consequently are more likely to require additional supports (Stothard, Snowling, Bishop, Chipchase, & Kaplan, 1998). Despite this, no studies included in the Le et al. review investigated the associated educational costs for children with low language. In a report for the children’s charity I CAN, based on local authority spending, Hartshorne (2009) estimated special educational costs associated with persistent SLCN to equal £4.1 billion annually in the UK. Additionally, because of lower levels of literacy and numeracy, children with SLCN tend to complete fewer years of formal education and consequently are more likely to be either unemployed or in less skilled employment (Felsenfeld, Broen, & McGue, 1994; Law et al., 2009; Marmot, Allen, Boyce, Goldblatt, & Morrison, 2020), again placing extra financial burden on society. In sum, the evidence base outlining the specific costs of SLCN is no way comprehensive, particularly with respect to the education and employment sectors. However, it is clear that the financial burden is substantial (Hartshorne, 2009; Marmot et al., 2020). One way to reduce this burden has been through early investment in high quality pre-school programmes, many of which have undergone cost benefit analyses and report benefit cost ratios of up to 8:1 (Rolnick & Grunewald, 2003). However, it is not enough to examine the costs and financial implications of early intervention programmes. We need to pay close attention to changes in outcomes and assess if this change is cost effective.

The need for cost-effectiveness evaluations of SLCN interventions

Our review of the literature shows that previous studies have considered the economic burden of SLCN (Hartshorne, 2009), return on investment associated with speech and language programs (Rolnick & Grunewald, 2003), and conducted cost benefit analyses (Ludwig & Phillips, 2007; Schweinhart, 2005) or partial cost-effectiveness analyses (no direct comparison e.g. Gibbard, Coglan, & MacDonald, 2004). While some studies have looked at the cost-effectiveness of programmes with broader outcomes (e.g. Knight et al., 2019; Ludwig & Phillips, 2008), there is a dearth of studies examining cost effectiveness in speech and language interventions (Bercow, 2008). One study in which a cost-effectiveness analysis was completed was carried out by Boyle, McCartney, Forbes, and O’Hare, (2007) in relation to a randomised controlled trial of individual versus group models of SLP, for children with developmental language disorder. Based on 15 weeks of therapy, Boyle reported an average cost of £786 per child, with costs ranging from £493 to £1144. While a cost-effectiveness analysis was performed using changes in Clinical Evaluation of Language Fundamentals – 3^rd edition (CELF-3) total language scores, reliable estimates of cost-effectiveness could not be made. Dickson et al. (2009) followed this, employing longer-term evidence. While no significant post-intervention differences were reported between modes of delivery (equivalent effectiveness), indirect group therapy was considered the most cost effective. Gibbard, Coglan, and MacDonald (2004) also used expressive and receptive language assessment measures in their analysis (Reynell Developmental Language Scales, (Reynell, 1983) and PLS- 3 (Boucher & Lewis 1997)), but did not perform a full cost-effectiveness evaluation, as there was no direct comparator.

The lack of cost-effectiveness studies in SLP can be explained in part by the fact that many studies do not move beyond the efficacy stage of evaluation, however the majority of those that do, do not report sufficient cost information to allow the full cost of the interventions to be estimated. In the context of a financially constrained health care service, we have not only to consider clinical effectiveness but also the resource implications of any programme. Therefore, we are not only interested in whether an intervention programme will result in increased speech, language and/or communication outcomes, but we need to know the most efficient method of intervention to improve outcomes. Including cost-effectiveness evaluations (embedded in trials) allows policy-makers and commissioners to evaluate the real economic cost of an additional intervention programme or of changing existing practice. Importantly, because clinical outcomes are part of the intervention comparison in a cost-effectiveness analysis, the impact on children is central to the evaluation and therefore, decision makers cannot automatically conclude that a cheaper, but less effective intervention, is necessarily preferable.

The current study

In this study, an economic evaluation was conducted from the perspective of the healthcare provider, to determine if the Happy Talk targeted selective intervention programme was cost effective, relative to standard pre-school care. In this assessment of costs and outcomes, costs accruing to the publicly-funded Irish health system and health benefits accruing to individuals are included. Happy Talk is a manualised training and language support programme delivered by speech-language pathologists, to parents and early childhood educators simultaneously, in socially disadvantaged areas. While the overall programme supports children between 0 and 6 years the focus of this evaluation was on the pre-school component (children aged between 2;08 years and 5;06 years). With the exception of Knight et al. (2019) and Barnett, Escobar, and Ravsten (1988), most intervention studies in which parents and teachers were simultaneously involved, have not considered costs (e.g. Bierman, Heinrichs, Welsh, Nix, & Gest, 2017; Lonigan & Whitehurst, 1998). Other programmes, designed specifically to target the SLCN needs of socially disadvantaged children in Ireland, have not yet measured effectiveness relative to controls, and consequently have not explored whether these programmes are cost-effective. To the best of our knowledge, in the field of SLP, there have only been three paediatric studies into which any kind of cost effectiveness analyses have been integrated (Boyle et al., 2007, 2009; Dickson et al., 2009 and Gibbard et al., 2004). To keep pace with what funders and decision makers require, we need to integrate cost-effectiveness analyses into the research practice of our discipline. This study will therefore add to the limited information currently available nationally and internationally, on the cost-effectiveness of pre-school language intervention programmes in general and more specifically on targeted programmes, which simultaneously engage with parents and early childhood educators. The paper presents the within trial economic evaluation of the 11-week Happy Talk intervention, in line with national guidelines for conducting economic evaluations (HIQA, 2020). The following research questions were addressed:

1. What are the total costs and costs per child of delivering the Happy Talk language intervention programme, from a healthcare perspective?

2. Is Happy Talk a cost-effective programme (a) based on language outcomes (b) based on quality of life measures?

3. What is the uncertainty around the cost-effectiveness decision?

4. Is there value in collecting additional information (Value of Information Analysis)?

Method

Intervention

The Happy Talk programme is a targeted selective community-based language intervention, programme, which has been developed and refined by speech-language pathologists over a number of years. The programme, now manualised, offers training and support to parents and early childhood educators, in socially disadvantaged areas. The parent programme is made up of 12 one-hour sessions, with each session including group training and individual coaching. Core components of the programme include listening skills; phonological awareness; sharing books and stories and core interaction strategies (modelling, expanding, balancing questions and comments). The premise is that by changing levels of parental responsiveness, this will result in home environments that promote the development of language, which will result in positive changes in child language.

Additionally, 4 staff workshops are provided with a similar focus as that described for parents, but which also incorporate the sharing of resources and information on language development in young children. The first workshop takes place before the 12-week parent programme begins and includes a coaching session with staff in their respective preschools. The remaining three, workshops, follow each four-week parent intervention block. The aim of these workshops is to affect how classroom environments are structured as well as the quality and quantity of interactions with children,. Importantly, the programme is embedded in preschools to tackle issues accessing educational supports, which are problematic in socially disadvantaged groups. Due to illness experienced by the speech-language pathologist delivering the intervention during the trial, the programme was delivered over 11 rather than 12 weeks. The costs are therefore based on an 11-week programme.

Trial

A quasi-experimental single blind study design was used to compare Happy Talk (an add on intervention), to usual care across 4 preschools. Due to the small number of preschools supporting socially disadvantaged children in the area (n = 6), we did not use random assignment to the intervention and control arms. To control for this, we included the four preschools that were most closely aligned pre-intervention on the Communication Supporting Classroom Observation Tool (CSCOT), a tool designed to profile the oral language environment of the classroom. speech-language pathologists who completed the CSCOT were not involved with the programme and were blind to which preschools would receive the intervention. Preschools were allocated to the intervention and control arms based on size, as we could not guarantee that there would be sufficient funding to offer the control preschools the Happy Talk programme the following year. Ethically, it was therefore appropriate to assign the two larger preschools to the intervention arm, to ensure that a greater number of children would be offered the intervention. All children (and their parents) attending the four preschools were invited to take part in the Happy Talk evaluation. Non-English speaking children and children with queried intellectual disability were excluded from the evaluation. Bilingual children were included. Happy Talk training was open to all staff in the intervention preschools, with places prioritised for those working with children who were participating in the study. Children and parents were assessed pre- and post the intervention by a team of speech-language pathologists blind to study arm.

Participants

The intervention was delivered to 77 children and 68 parents (with limited involvement of 4 siblings). Due to a lack of parental consent, children absent on the day of assessment (n = 2), refusal to engage with the assessment process (n = 8) and exclusions (n = 7), pre and post outcome measures are available for 35 children. Figure 1 in the Supplementary material shows the flow of participants through the trial (also available in Frizelle et al., 2021). Note that 81 refers to the number who consented to be part of at least one aspect of the trial.

Figure 1. Incremental cost-effectiveness planes.

Assessment measures and outcomes

Details of all measures and outcomes are reported in a companion paper (Frizelle et al., 2021). The outcomes reported here are those pertinent to our economic evaluation, undertaken from a healthcare perspective i.e. the costs to the healthcare provider and the benefits accruing to the child.

Children’s language was the primary outcome and was measured using the Pre-school Language Scale – 5^th edition (PLS-5) (Zimmerman, Steiner, & Pond, 2014), a standardised measure which includes a receptive language score, an expressive language score and a composite (receptive and expressive) total score. The secondary child outcome measure was the Child Health Utility instrument (CHU9D) (Furber & Segal, 2015), a generic preference- based measure, using parent report scales to measure health related quality of life, in young children. The CHU9D (for children < 5years) consists of 11 questions and parents are asked to base their responses on how their child is feeling on the day of completion. It consists of a descriptive system and a set of preference weights, which give utility values for each health state described by the descriptive system, allowing the calculation of QALYs (Mpundu-Kaambwa et al., 2017).

The trial showed that compared to controls, children receiving Happy Talk improved by 0.6SD in relation to their receptive language (9.2 point change in PLS-5 comprehension score) and 0.46SD with regard to total language score (6.3 point change in PLS-5 total score), thereby indicating large and moderate intervention effect sizes respectively. No statistically significant effect was shown for CHU9D.

Economic evaluation

A trial-based economic evaluation was conducted comparing the “add on” intervention – Happy Talk, to usual care. The baseline evaluation was a cost effectiveness analysis (CEA), wherein effects were measured using PLS5-Total.

Also, in line with national guidelines a cost utility analysis (CUA) (wherein effects are measured as quality adjusted life years gained (QALYs) was undertaken, albeit with reservation owing to sensitivity concerns and issues with estimating QALYs over such a short period (8 months). Here utility estimates are measured using the CHU9D. Both analyses employed an 8-month time frame and were conducted using Microsoft Excel software.

Costs

Costs of the intervention were measured using standard national guidelines, whereby resources associated with delivery of the intervention and usual activity were identified, measured, and valued in 2020 Euros (Health Information and Quality Authority (HIQA), 2020). Only direct costs of the trial (excluding measurement) were included. Resources included were personnel costs and materials. The personnel costs included the cost of the speech-language pathologist's time, including tax and pension costs and accounting for overheads. Records of the speech-language pathologist's time given to the preparation and implementation of the programme, were logged throughout the trial. All materials and printing costs were estimated at current market rates. The trial included measurements at baseline and post intervention, as the cost of these measurements were common across the control and intervention, these costs were excluded from the cost-effectiveness analysis. The therapist’s travel costs from place of work to pre-schools was covered by the 25% overhead referred to above. It was assumed that in implementing usual activity no additional costs were incurred and the opportunity cost of pre-school staff’s time was not accounted. As the intervention costs were incurred within one year, no discounting was required.

Cost effectiveness

To examine cost effectiveness, the incremental costs are compared to the incremental benefit (measured in additional point change in PLS-5 total and QALYs). If the intervention is more effective i.e. generates more benefit, and is less costly than the comparator, it is considered cost effective and it dominates the comparator. Alternatively, if the intervention is less effective i.e. generates less benefit, and more expensive, the intervention is not considered cost effective and is dominated by the comparator. If the intervention is more costly and more effective or less costly and less effective, the Incremental Cost Effectiveness Ratio (ICER) is computed as the incremental costs divided by incremental effects. In the cost utility analysis this is compared to the nationally accepted cost effectiveness threshold (€45 000/QALY) (HIQA, 2020). If the ICER is less than the threshold it can be considered cost effective. In the absence of an estimate of what society would be willing to pay for an additional point change in total PLS-5 (i.e. a ceiling ratio) a proxy is employed to estimate monetary net benefit as follows. Marsh, Bertranou, Suominen, and Venkatachalam (2010) found that every £1 invested in speech and language in the UK generates £6.43 through increased lifetime earnings. This return on investment (1:6.43) is applied to the cost-effectiveness ratio to estimate the expected return in monetary terms. This is compared to the cost of the intervention, if the expected return is greater than the cost at this cost-effectiveness ratio, it would be reasonable that society would be willing to pay that ceiling ratio for an additional point change in total PLS-5.

Handling uncertainty

In every economic evaluation, and its model, uncertainty and heterogeneity occur. Uncertainties are costly and increase the risk of making the incorrect decision regarding the cost effectiveness of an intervention and its comparators. There are various types of uncertainties that can occur. Here we investigate parameter and decision uncertainty. Parameter uncertainty refers to the accuracy with which input parameters are calculated. Imprecision can arise from using limited sample evidence to estimate input parameters such as probabilities, costs, utilities, and treatment effects for populations. To investigate parameter uncertainty, a probabilistic sensitivity analysis was employed. This involved characterising uncertainty in the parameters, using normal distributions (effectiveness and normal parameters and a Gamma distribution (on utility parameters)). The uncertainty was then propagated through the model using a Monte Carlo simulation, which generates 10 000 simulated point estimates. For further information on this process and the distributions, see Briggs, Claxton, and Sculpher (2006).

The results are presented on Incremental Cost Effectiveness Planes. These are four quadrant diagrams which plot the incremental costs and effects of the intervention under evaluation compared to the alternative per child in a two-dimensional space. The incremental costs are on the vertical axis and effects are on the horizontal axis. If the intervention is more effective and less costly than the alternative (the impact falls in the South-East quadrant) it is said to have dominance compared to the alternative and is the recommended option. There is also dominance where the intervention under consideration is more costly and less effective than the comparator (the impact falls in the North-Western quadrant) and the comparator should be recommended. The decision is more ambiguous however when the intervention under consideration is more effective and more expensive (North-Eastern quadrant) or less effective and less expensive (South-Western quadrant). In these scenarios to compare between alternatives an external ratio (ceiling ratio) measure is used to analyse the difference in costs and effects between interventions. As discussed above this is readily available when effectiveness is measured in terms of QALYs (currently €45 000/QALY (HIQA, 2020).

The results from the Monte Carlo simulation are then used to consider the uncertainty surrounding the cost effectiveness decision. Here net benefit can be estimated employing different ceiling ratios and the probability of Happy Talk being cost effective at each ceiling ratio is estimated and presented on the cost-effectiveness acceptability curve. These summarise the uncertainty surrounding the cost-effectiveness decision for various ceiling ratios. Net benefit is estimated by multiplying incremental benefits by ceiling ratio and subtracting costs. If net benefit is positive the intervention can be considered cost effective (as benefits outweigh the costs) and if negative the comparator is considered cost effective. The values on the horizontal axis represent cost-effectiveness ceiling ratios, which represent the maximum society is willing to pay for an additional unit of health gain (measured as QALYS here) (Fenwick, O’Brien, & Briggs, 2004)

Scenario analysis

In the trial an experienced speech and language therapist at senior grade delivered the intervention, which is reflected in the costing. However a staff¹ grade speech and language therapist provided with the appropriate training could deliver the intervention at a reduced cost (€11.81 per hour). We estimate the impact of this change in a scenario analysis. (For the purposes of the analysis it is assumed that the cost of providing the training is absorbed into existing professional development training budgets).

Value of information analysis

The economic evaluation presented here is estimated using existing information and resulting decisions (to adopt and implement Happy Talk or not in this case), are based on expected net benefit which are subject to uncertainty. That is to say the decision is correct given current information, but should uncertainties diminish as new information becomes available, a different decision could be made. There costs associated with making the wrong decision and collecting addition information is expensive. Bayesian Value of Information analysis (VoI) was employed to investigate if there was potential value in collecting additional evidence. The Expected Value of Perfect Information (EVPI) investigates what society would be willing to pay to eliminate all the uncertainty surrounding the adoption decision. It is calculated as the difference in the net benefit of the decision made with perfect information and that made based on current information (Briggs et al., 2006). As information is non-exclusive, the overall EVPI for a population can be estimated and it puts an upper bound on the value of future research (Briggs et al., 2006; Claxton, 1999; Claxton & Posnett, 1996). Based on actual and predicted births from 2018 to 2022 (from the Central Statistics Office, 2020) the number of children for whom Happy Talk would be suitable was estimated at 50 500 over a 5-year period. This estimate indicates a predicted demand for pre-school services at 277 665 over the next 5 years, while applying the following predicted material deprivation rates for children: 23% in 2020; 19% in 2021 and 16% (assuming we return to 2018 levels of 16%) from 2022 post the Covid −19 pandemic (Regan & Maitre, 2020).

Result

Total costs/costs per child

The intervention was delivered to 77 children across two sites. Baseline demographic information is available for both intervention and control groups in the companion paper (Frizelle et al., 2021) and in Table I in the Supplementary material. While there appear to be between group differences in the education levels of the parents, more than half of the parents did not report their education level (n = 49 out of possible 81) which may have biased the result, particularly as the vast majority of the 49 were in the intervention group. In addition, the Pobal Index of deprivation² (2016) was calculated for each pre-school and showed no significant difference between the deprivation levels of the intervention versus the control preschools (p = 0.87). The total cost of the intervention was €6318, of which personnel costs were 83%. The cost per child was €82.06, giving an incremental cost between the intervention and control of €82.06 per child. The cost analysis is given in Table I.

Table I. Cost analysis: Happy talk.

	Hours/# units	€/unit	€ Baseline: Senior SLT	€ SA: Grade SLT
Speech & Language Therapist: Senior; grade	139^a	37.73; 25.56^b	5244.27	4372.88
Consumables
Refreshments^b			310.00	310.00
Materials:			100.00	100.00
Calendars	77	4.00 ^c	308.00	308.00
Books	77	4.00^c	308.00	308.00
Posters	6	8.00^c	48.00	48.00
Total			6318.27	5446.88
Average/School^d			3159.14	2723.44
Average/Child^e			82.06	70.74

^aRetrospective estimate based on therapists own records, includes sessions with parents (55 hours); staff (12 hours); preparation for sessions (1 hour of preparation per hour) and coaching (5 hours).

^bDepartment of Health Salary scales, adjusted for Pay Related Social Insurance (PRSI), Pensions and overhead as per HIQA (2020).

^cProject financial records.

^dTwo schools.

^eBased on 77 children.

Cost-effectiveness analysis and uncertainty

The effectiveness results demonstrate that Happy Talk delivers greater improvement in Total PLS-5 scores than the comparator (Frizelle et al., 2021). However, CHU9D decrease in both Happy Talk and the comparator between baseline and follow-up. (Table II in the Supplementary material).

Table II. Economic evaluation results.

		CEA		CUA		Scenario Analysis
	Cost (€)	PLS5	ICER (€/ unit change PLS5)	QALYS^a	ICER (€/QALY)	Cost (€)	CEA: ICER (€/ unit change PLS5)	CUA: ICER (€/QALY)
Happy Talk	82.06	3.10		–0.0153		70.74
Usual Care	0	–3.20		–0.0135		0
Incremental	82.06	6.30	13.02	–0.0018	Dominated^b	70.71	11.23	Dominated

CEA: Cost Effectiveness Analysis; CUA: Cost Utility Analysis; PLS-5: Pre-school language scales 5^th Edition (Zimmerman et al., 2014); ICER: Incremental Cost Effectiveness; QALYS: Quality Adjusted Life Years.

^aQALYs estimated using CHU9D.

^bDominated indicates Happy Talk is less effective and more expensive than usual care so cannot be considered cost effective.

The CEA demonstrates Happy Talk is more expensive (€82.06) and more effective (6.3 point change in total PLS-5 standard score – effect size 0.46SD³) than the control (Table II) and the cost effectiveness ratio is €13.02 per unit change.

As illustrated in Figure 1(a), there is no uncertainty around the existence of difference in costs, however there is considerable uncertainty around the existence and extent of differences in effects.

Figure 2. Cost-effectiveness acceptability curves.

This uncertainty translates to decision uncertainty as presented on the cost-effectiveness acceptability curve (Figure 2(a)). If society was willing to pay €28.25 (point where the probability that Happy Talk is cost effective is 0.5) or more for a one-point improvement in PLS-5 Total, then Happy Talk would be more cost effective than the control.

Using the return on investment ratio estimated from the literature (1: 6.43) (Marsh et al., 2010), the expected return of an investment of €13.02 is €83.75, which is greater than the cost of Happy Talk (€82.06). It would therefore be reasonable to infer that €13.02, for a one point improvement in PLS-5 Total, is value for money, and Happy Talk could be considered cost effective. The scenario analysis demonstrates, if a staff grade speech-language pathologist delivers the intervention, assuming effectiveness remains constant, the ICER decreases even further to €11.26 per unit change in PLS-5 Total.

When effects are measured in QALYs (using CHUD9) the positive effects of Happy Talk do not persist. Here Happy Talk is more expensive (€82.06) and marginally less effective (-0.002), so cannot be considered cost effective (see Figure 2(b)). This dominance remains in the scenario analysis wherein a staff grade speech-language pathologist delivers the intervention.

Expected value of perfect information

The Expected Value of Perfect Information (EVPI) associated with the decision between Happy Talk and usual practice at a ceiling ratio of €45 000/QALY was €64.1 million over 5 years for the predicted population (50 491) or €6347 per person (Figure 2 in Supplementary Materials). The EVPI provides a maximum value for the return on further research for this population group and thus provides an upper bound on the potential value for additional research in the Irish context.

Discussion

In this paper we present a within trial economic evaluation of Happy Talk – a targeted selective pre-school intervention programme aimed at improving the speech, language, and communication abilities of children from socially disadvantaged areas. The evaluation was conducted from the viewpoint of the healthcare provider, and the analysis conducted in line with national guidelines in Ireland (HIQA, 2020), which are similar to those favoured by many decision makers in the health care arena (ISPOR, 2014). From the perspective of the healthcare provider the total cost of delivering Happy Talk was €6318 and the cost per child was €82.06. Because of the dearth of economic evaluations in the field of SLP, and the disparate methods chosen in the few studies available, there is no valid comparison costed intervention, with which to compare these costs.

Based on the primary outcome measure, PLS-5 Total score, there was a statistically significant increase in language, in favour of the intervention group. Employing these results in a cost-effectiveness analysis (CEA) demonstrates Happy Talk is more effective and more expensive than the control. While a nationally accepted ceiling ratio is not available to indicate what society is willing to pay for an improvement in PLS-5 score, the proxy employed in its absence (from Marsh et al., 2010) suggests the benefits could be greater than the costs (€13.02/improvement in PLS-5 Total), so Happy Talk does appear to represent value for money.

The costs of Happy Talk presented here are based on a senior grade therapist (as per the trial), but costs could be further reduced by training staff grade therapists to implement the programme – while acknowledging the cost of this training. Although staff grade therapists have delivered the programme in the past, prior to delivery, they have shadowed senior therapists for a significant period, to develop skills in maintaining positive cross-disciplinary co-practice. Indeed, the happy talk model, when delivered outwith a trial context, has a strong focus on developing the social aspects of collaboration, which are central to the success of programmes aiming to engage with practitioners from different disciplines (McKean, Law, et al., 2017). These social aspects of collaboration, which are linked to trust, engagement, and alliance, would need to be explicitly incorporated into the Happy Talk training programme to facilitate delivery by less experienced staff.

In the field of education, an effect size of 0.25 is accepted as a sufficient improvement to warrant changing practice and policy (Ramey, 2018), however this needs to be considered within the context of cost. Given the reported Happy Talk effect size (.46 SD) along with the fact that the programme appears to represent value for money, our findings add to the evidence suggesting that, a targeted selective public health approach, could reduce the societal burden of children with low levels of speech, language, and communication. However, measurement of longer-term outcomes and a larger trial are required, to be more definitive with respect to policy changes.

The gold standard for economic evaluations are cost utility analyses, wherein outcomes are measured in terms of QALYs. This is the secondary analysis presented here and shows that when outcomes are measured using CHUD9, Happy Talk would not be considered cost effective, even if delivered by a staff grade speech-language pathologist. Underlying these results are the difference in effectiveness measures, challenging the sensitivity of health-related quality of life instruments such as the CHU9D and QALYs for this population cohort and intervention. In contrast to the significant increase in language score for the intervention group, the CHU9D utility score declined in both the control and the intervention group from baseline to follow-up. This lack of a positive effect (previously discussed in Frizelle et al., 2021) may be because of the immediacy of the post-intervention outcome measure administration, in that an increase in overall language ability had not yet translated into an increase in health-related quality of life. It is reported in the literature that children with low language skills experience health-related quality of life difficulties at twice the rate of their typically developing peers (McKean, Reilly, et al., 2017). Therefore, it may be that these intervention effects would be observed in the long-term, however, future research is needed to confirm this.

The lack of an effect in relation to health-related quality of life may also reflect the relatively small sample size. The effects shown for language were large and would therefore be detected in a relatively small sample, however, smaller effects which may be reflective of health-related quality of life, may not emerge in this sample.

The decline in CHU9D also causes us to question the sensitivity of generic measures to detect improvements captured in the language measure, in that they may not represent the quality of life constructs that would be expected to change with language improvements, particularly in the short term. This potential lack of sensitivity (relative to more condition specific instruments) has been acknowledged in the literature (Foster Page, Thomson, Marshman, & Stevens, 2014; Drummond et al., 2015). There are measures designed specifically for communicative domains, such as FOCUS (Focus on the Outcomes of Communication Under Six), which aim to measure participation rather than quality, and which have identified a clinically meaningful difference score, however, whether a suitable algorithm could be developed to map FOCUS to a utility measure like Euro-Qol-5D to facilitate QALY estimates warrants further research. The sensitivity of generic health-related quality of life instruments is an on-going challenge not confined to SLCN, and investigating how improvements in language score can be generalised into functional improvements, which might give QALY gain, is a priority area for further research. Similar recommendations are made for translating word finding ability into functional improvements, which might result in appreciable QALY gains when assessing interventions for adults with aphasia (Latimer et al., 2020). (Note: The original study (Frizelle et al., 2021) also included Paediatric Quality of Life Inventory (PedsQL) scores, which can be mapped onto CHU9D to estimate QALYs. These were excluded from this analysis as they showed no statistically significant difference in effect and therefore the cost effective conclusions would be similar to those reported using CHU9D.)

In the absence of reliable utility data to estimate QALYs, effectiveness measures in “natural units” can be employed, as demonstrated here. However, this approach has been taken in only a few other studies to date (e.g. Boyle et al., 2007; Dickson et al., 2009). By using the standard cost effectiveness analysis methodologies employed in other clinical fields, we are advancing the use of economic evaluations in the speech and language arena. Such standard methodology is applicable to all healthcare interventions for the Department of Health/Health Service Executive in line with national HIQA guidelines in Ireland or NICE (National Institute for Clinical Excellence) in the UK and are favoured by those who allocate health care funds (ISPOR, 2014). This approach produces assessments and recommendations that are reliable, robust, and relevant to the needs of decision-makers and stakeholders (HIQA, 2020). Furthermore, this study demonstrates how cost effectiveness analyses can be conducted early in an intervention’s life cycle, to provide evidence for policy makers and research funders to determine if a definitive trial is feasible for an intervention or to avoid further investment in interventions that are unlikely to be cost effective (Gannon, 2017).

Our reported analysis includes a value of information analysis, which indicated that there is value in collecting further information. Specifically, the expected value of perfect information at €45 000/QALY is €64 million across five years for the estimated population (50 500 children) or €6347 per person. The EVPI provides a maximum value for the return on further research for this population group, therefore providing an upper bound on the potential value of additional research in the Irish context and suggests there is value in collected additional evidence on Happy Talk compared to usual care. To the best of our knowledge this is the first economic evaluation of an SLP intervention to include Value of Information analysis, an important indicator of the value of further research.

While we advocate conducting cost effectiveness analyses in speech and language interventions we acknowledge there are challenges, and indeed limitations associated with the analysis presented here, such as sample size, potential differences in parental education levels, short follow-up in the trial and the aforementioned sensitivity of the health related quality of life measure. In contrast to previous costing studies in the field, we employed the perspective of the publicly-funded health and social care system in Ireland, so excluded parent costs. While narrow, the choice of perspective was guided by national guidelines (HIQA, 2020). In addition, interpreting results without estimates of what society is willing to pay for improvements in each effectiveness measure (such as an additional point change in total PLS-5) is challenging and impedes recommendations on cost effectiveness. To overcome this in this study, we employed a proxy using return on investment estimates from the UK, to make judgements regarding cost effectiveness, an imperfect solution albeit the best available.

Despite these challenges, this study demonstrates it is feasible to apply the standard methodologies for cost-effectiveness analysis used by decision makers in Ireland, and the UK, to SLCN interventions. In addition, there is potential for them to be used more as the number of randomised control trials in the field increases. This would provide robust evidence to support proposals for SLCN programmes/interventions and aid decision makers when choosing between competing alternatives for finite resources. Simultaneously, further investigations into how improvements in language score can be generalised into functional improvements that might give QALY gain are warranted. With respect to Happy Talk specifically, this short-term cost-effectiveness analysis demonstrates it is potentially cost effective and there is value in collecting further data. A priority of a larger scale evaluation should be on longer-term follow-up to examine if language improvements persist and to determine if there is an impact on health-related quality of life. Demonstrating unequivocally the cost-effectiveness of targeted selective intervention approaches, to improve language in socially disadvantaged populations, could have significant impact on policy and commissioning, and in turn on the life chances of children growing up in socially challenging circumstances.

Declarations of interest

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed at https://doi.org/10.1080/17549507.2021.1975815.

Additional information

Funding

The Happy Talk trial was carried out with the support of the Irish Research Council (IRC) Employment based programme [EBPPG/2018/196].

Notes

1 A staff grade speech-language pathologist is one who is qualified for a minimum of 3 years

2 This is a method of measuring the relative affluence or disadvantage of a particular geographical area in Ireland

3 Applying the Education Endowment Foundation toolkit guidance, this effect size can be considered ‘high impact’, in the context of educational interventions.