Effectiveness of Dementia Care Mapping™ to reduce agitation in care home residents with dementia: an open-cohort cluster randomised controlled trial

Abstract

Objectives

Agitation is common and problematic in care home residents with dementia. This study investigated the (cost)effectiveness of Dementia Care Mapping™ (DCM) for reducing agitation in this population.

Method

Pragmatic, cluster randomised controlled trial with cost-effectiveness analysis in 50 care homes, follow-up at 6 and 16 months and stratified randomisation to intervention (n = 31) and control (n = 19). Residents with dementia were recruited at baseline (n = 726) and 16 months (n = 261). Clusters were not blinded to allocation. Three DCM cycles were scheduled, delivered by two trained staff per home. Cycle one was supported by an external DCM expert. Agitation (Cohen-Mansfield Agitation Inventory (CMAI)) at 16 months was the primary outcome.

Results

DCM was not superior to control on any outcomes (cross-sectional sample n = 675: 287 control, 388 intervention). The adjusted mean CMAI score difference was –2.11 points (95% CI –4.66 to 0.44, p = 0.104, adjusted ICC control = 0, intervention 0.001). Sensitivity analyses supported the primary analysis. Incremental cost per unit improvement in CMAI and QALYs (intervention vs control) on closed-cohort baseline recruited sample (n = 726, 418 intervention, 308 control) was £289 and £60,627 respectively. Loss to follow-up at 16 months in the original cohort was 312/726 (43·0%) mainly (87·2%) due to deaths. Intervention dose was low with only a quarter of homes completing more than one DCM cycle.

Conclusion

No benefits of DCM were evidenced. Low intervention dose indicates standard care homes may be insufficiently resourced to implement DCM. Alternative models of implementation, or other approaches to reducing agitation should be considered.

Keywords:

• Alzheimer’s disease
• institutional care/residential care
• intervention
• long-term care
• person-centred care
• health economic evaluation
• practice development
• psychosocial interventions

Trial registration:

• Current Controlled Trials ISRCTN82288852

Introduction

Around 80% of care home residents have dementia (Alzheimer’s Society, 2013). Agitation is particularly common and clinically challenging, in this group, causing distress for the person and other residents, but is not an inevitable consequence of dementia, often being a response to unmet needs and psychological distress. Caring for people living with dementia who experience agitation is also time consuming and complex (Livingston et al., 2017). Thus, interventions to support care home staff to reduce agitation may produce considerable benefits. The limited efficacy and high risks of traditional pharmacological approaches to management of agitation (All-Party Parliamentary Group on Dementia, 2008; NICE, 2018) mean psychosocial responses are urgently required. Person-centred care (PCC) is a best practice psychosocial intervention for addressing unmet needs in people with dementia, demonstrated to be effective in reducing the incidence of agitation and improving quality of life of care home residents (Livingston, Kelly, et al., 2014; Livingston et al., 2017; NICE, 2018). However, training staff in PCC alone is unlikely to support sustained implementation (Chenoweth, Stein-Parbury, Lapkin, Wang, & Williams, 2019; Fossey et al., 2019). Therefore, methods to support care home staff to effectively embed and sustain PCC in practice are needed.

Dementia Care Mapping

Dementia Care Mapping™ (DCM) (Bradford Dementia Group, 2005) is an established, observational tool, set within a practice development cycle, used widely in care home settings (Barbosa, Lord, Blighe, & Mountain, 2017) as a whole home quality improvement intervention, to support the embedding of PCC in practice. It has been identified as a potentially effective method for reducing agitation in people with dementia in care home settings (Livingston, Kelly, et al., 2014). DCM places a focus on understanding the psychosocial factors, including care practices that may lead to unmet needs and their expression through behaviours such as agitation and can be used to develop individualised care plans and address broader care home practices (Barbosa et al., 2017). A logic model for DCM is published elsewhere (Surr, Shoesmith, et al., 2019).

However, robust evidence around its effectiveness and cost-effectiveness is limited and varied. Benefits of DCM for residents (reductions in resident agitation, depression, anxiety, neuropsychiatric symptoms, psychosis and falls, and improvements in quality of life) and staff (communication and connectedness, burnout) have been found in pilot studies (Chenoweth & Jeon, 2007; Kuiper, Dijkstra, Tuinstra, & Groothoff, 2009) and explanatory randomised controlled trials (RCTs) (Chenoweth et al., 2009; Jeon et al., 2012; Rokstad et al., 2013). However, an RCT (van de Ven et al., 2013) and a quasi-experimental pragmatic controlled trial (Dichter et al., 2015) found no significant benefits for DCM over control for resident outcomes, although staff reactions towards people living with dementia were improved. Cost-effectiveness was investigated in two RCTs with DCM not cost effective in one (Chenoweth et al., 2009) and cost-neutral in the other (van de Ven et al., 2014).

Limitations of the explanatory RCTs to date include implementation of only two DCM cycles and short follow-up periods (10–12 months), potentially reducing the opportunity to realise positive changes, which may take time to implement and effect resident outcomes. This also limits the period over which intervention sustainability can be monitored. Additionally, explanatory trial designs, while supporting understanding of intervention efficacy, do so under controlled, optimal conditions, often using specially selected sites and participants (Ford & Norrie, 2016), and with smaller samples, meaning the findings do not always translate into real-world practice. Inapplicability of trial findings to clinical practice, is a frequent criticism aimed at trials, by clinicians (Treweek & Zwarenstein, 2009). Pragmatic/effectiveness trials assess the benefit of interventions in ‘real world’ settings with a sample that is representative of the heterogenous population under study, permitting generalisability of the results to the wider population. They include the potential impact of external moderating factors that may impact intervention benefits in real-world situations (Singal, Higgins, & Waljee, 2014). As such they produce evidence of practical importance regarding whether particular interventions are accessible and relevant/applicable to the majority of potential beneficiaries (Tosh, Soares-Weiser, & Adams, 2011). Given the promising results of DCM for resident and staff outcomes in explanatory trials, in order to expand this to its applications in real-world situations, this trial aimed to conduct a timely definitive, pragmatic effectiveness trial adequately powered for generalisability and using methods aligned to routine practice (Roland & Torgerson, 1998). This article reports the results of this trial.

Aims

The DCM EPIC cluster RCT aimed to evaluate the clinical and cost-effectiveness of DCM alongside usual care (intervention), compared to usual care alone (control), for people living with dementia in care homes in the UK. The primary outcome was whether DCM reduced agitation at 16 months. Secondary outcomes included whether DCM reduced other behaviours staff may find difficult to support (e.g. apathy, anxiety, depression), improved quality of life, reduced the use of antipsychotic and other psychotropic medications and improved the quality of staff/resident interactions (e.g. more person-centred and less task focussed).

Research design and methods

Design

Trial design and procedures are reported in the original (Surr, Walwyn, et al., 2016) and revised (Surr, Ballard, et al., 2016) protocols. This pragmatic, multi-centre, cluster RCT recruited residential and nursing homes providing care to people living with dementia. It included an integral (published separately) cost-effectiveness analysis (Meads et al., 2020) and process evaluation (Griffiths, Kelley, et al., 2019; Surr, Griffiths, et al., 2019; Surr, Shoesmith, et al., 2019). Follow up was at 6 and 16 months.

The trial originally had a closed-cohort design, following all recruited residents in each care home from randomisation to final follow-up. However, there was unavoidably high loss to follow-up, mainly due to death, impacting the generalisability of the results to the intended population. To enable the original research question to be robustly addressed, in line with other examples (e.g. Underwood et al., 2013), we recruited additional residents in each care home at 16 months to allow a cross-sectional analysis at baseline and 16 months to be performed as our primary analysis. This also enabled us to assess the effects of the intervention on all eligible and consenting residents residing in the care home during the intervention period and not just those who had survived long enough to remain in the care home at 16 months. For robustness, we undertook sensitivity analyses on the original closed cohort.

Ethical issues

Ethical approval was granted by NRES Committee Yorkshire & The Humber - Bradford Leeds REC ref 13/YH/0016. The trial was registered with the International Standard Randomised Controlled Trial Register (ISRCTN) reference 82288852. All study participants provided written informed consent to take part. Where residents were assessed as lacking capacity, in accordance with law (Mental Capacity Act, 2005) a personal (family member or friend) or nominated (staff member independent of the research) consultee was appointed to provide advice on their wishes.

Study setting

To be eligible, care homes within three research recruitment hub areas (West Yorkshire, London, Oxfordshire) were required to: recruit a minimum of 10 residents with dementia and agree to release staff for intervention implementation in compliance with the trial protocol. Care homes were not eligible if: subject to care quality enforcement notices or admission bans; had used DCM within the previous 18 months; or had recently, were currently, or were planning to take part, in conflicting research.

All eligible care homes were approached (in random order) in batches of 10–20 by postal invite, followed up by researcher phone call, during which the trial was explained in detail and additional eligibility screening was conducted. A site visit, to discuss the trial with the manager, key staff and the owner/owners representative (if applicable) was arranged for homes expressing an interest. Where care homes were part of a group the Head Office was contacted concurrently to seek organisational support in approaching homes and, where consented, to ensure continued organisational support for participation. The manager and owner were encouraged to discuss the trial with the staff team to assess interest ahead of both providing written informed consent.

While this was a pragmatic trial, DCM use necessitates staff have an understanding of PCC. Therefore, dementia training was audited. Homes falling below a minimum standard (training content and reach) received a half-day dementia awareness and PCC training (Griffiths, Creese, Garrod, Chenoweth, & Surr, 2018) delivered by experienced trial facilitators, using materials adapted from those used by a large UK care home provider. Staff responsible for training within the care home was then able to cascade the training to additional staff. The training provided ensured all care homes had a minimum percentage of staff trained in dementia and PCC awareness immediately prior to randomisation.

Residents were recruited prior to care home randomisation (baseline) and also at 16-months post-randomisation. Resident eligibility screening was undertaken by researchers with the care home manager or senior staff member. All eligible residents within the care home at baseline and 16 months were approached. Baseline eligibility included permanent residency, a dementia diagnosis or score ≥4 or on the Functional Assessment Staging of Alzheimer’s Disease (FAST) (Reisberg, 1988) and sufficient proficiency in English, if self-consenting. Ineligibility included: terminal illness, permanent care in bed or participation in conflicting research. At 16 months, additional eligibility criteria were being resident in the care home for at least 3 months (to permit a period of settling in and for potential effects of any care home good practice related to intervention implementation to be realised) and not declining participation at baseline.

Permanent members of staff who knew the resident well, acted as proxy informants for data collection. Where possible the same proxy informant was used when obtaining individual resident data at each time point. Due to staff turnover, annual leave and shift patterns, this was not always possible.

Control

Routine care was delivered, with any changes recorded (e.g. new staff roles, ownership, practice initiatives or training).

Intervention

DCM was implemented pragmatically, using standard procedures (Bradford Dementia Group, 2005) and following the most common UK implementation model of staff-led use. Two staff members from each intervention care home (‘mappers’) attended a standard four-day training in DCM, including competency assessment, delivered by the University of Bradford. Mapper eligibility included: being a permanent staff member, having appropriate skills and qualities and designated time to undertake DCM (assessed by the home manager against a role descriptor, developed by the trial team based on advice from DCM experts) (see Figure 1) and willingness to undertake the role. Potential mappers were also asked to complete a short, written statement about their understanding of and desire to undertake the mapper role. Written informed consent was obtained from all mappers. Where a mapper withdrew during the trial, another mapper was identified, consented and trained, where possible.

Figure 1. Mapper role descriptor.

DCM is described elsewhere (Brooker & Surr, 2006). Briefly, it involves a five-phase cycle including holding briefing session(s) for the staff team; observing up to five residents with dementia for up to six consecutive hours; undertaking data analysis; writing a report summarising the data and providing feedback of findings to the staff team; and action planning. During observations the mapper records: resident behaviour every five-minutes through choice of a Behaviour Category Code from a possible list of 23 codes; and accompanying Mood and Engagement Value from a six-point scale (-5, −3, −1, +1, +3, +5); and instances of good and poor staff care and communication with residents in the form of Personal Enhancers and Detractors, as and when they occur. The mapper also records qualitative notes to supplement the coded data. To protect privacy and dignity, observations only occur in communal areas and not during provision of personal care. Since DCM is implemented as a whole home intervention, mappers and care home staff jointly selected residents to observe, including residents not participating in the trial.

Mappers were asked to implement three DCM cycles, scheduled 3-, 8- and 13-months post randomisation. To support intervention implementation and consistency, a number of measures was put in place. Mappers were provided with standardised templates based on DCM guidelines for completing each component of each cycle. While mappers would usually undertake DCM without further external input post-training, unless experienced mappers are available within an organisation, additional methods of support were put in place in this trial, taking into account the pragmatic trial design. They included sending SMS reminders ahead of each cycle and support from a DCM expert during cycle one (see Surr, Shoesmith, et al., 2019). Telephone and email support were provided thereafter, if required.

Adherence to intervention implementation (fidelity and dose) was monitored through feedback from the external experts, collection of standard DCM documentation from mappers following each cycle (plus two additional documents relating recording attendance at briefing and feedback sessions) and telephone contact with mappers by trial management staff. Only documented components/cycles were recorded as completed. Intervention fidelity is reported elsewhere (Surr, Griffiths, et al., 2019).

Randomisation and masking

Immediately following baseline data collection care homes were randomised to intervention or control (3:2 ratio) using 24-hour, automated, computer generated randomisation using minimisation. Arms were balanced for home/unit type (general residential/nursing, specialist dementia care); size (large ≥ 40 beds, medium/small < 40 beds); dementia awareness training provision by the research team (yes, no); and catchment area (West Yorkshire, London, Oxfordshire). To maintain researcher blinding, randomisation and site notification were performed by the trial data managers. Clusters were not blinded to allocation.

Sample size

Fifty care homes, each recruiting an average of 15 residents provided 90% power at the 5% significance level to detect a clinically important difference of 3 points on the Cohen-Mansfield Agitation Inventory (Cohen-Mansfield, 1991) (CMAI) (standard deviation (SD) 7.5 points), that is a moderate standardised effect size of 0·4. Calculations assumed 25% loss to follow-up (Chenoweth et al., 2009) and an inflation factor of 2.0 (i.e. cluster size of 11 analysable participants at follow-up and an intracluster correlation coefficient (ICC) no greater than 0.1 (Fossey et al., 2006)). Due to predicted sources of clustering (provision of care) and higher predicted ICC in the intervention arm, an allocation ratio of 3:2 was used, giving 30 (450) and 20 (300) care homes (residents) in intervention and control, respectively.

Loss to follow-up was substantially higher than the anticipated 25%, mainly due to resident deaths and to a lesser extent, residents leaving the care home. To preserve statistical power close to 90%, and the ability to detect a standardised effect size of 0.4 SD, additional residents were recruited at 16 months and a cross-sectional cohort analysis conducted, including all residents registered in the study at 16 months (recruited at either baseline or 16 months).

Data collection

Data collection commenced in June 2014 and was completed in May 2017.

Resident and care home characteristics

These were completed via researcher interview with the care home manager or other senior staff member.

Resident demographics and comorbidities collected via interview and care records review.

Clinical Dementia Rating Scale (CDR) (Hughes, Berg, Danziger, Coben, & Martin, 1982) rates dementia severity across six-categories and provides an overall severity rating.

Functional Assessment Staging (FAST) (Reisberg, 1988) records the functional severity of dementia, ranging from 1 (no dementia) to 7 (severe dementia).

Care Home Demographics including home (size, type, ownership, staff turnover, staff ratios, etc.) and manager (qualifications, length of time in post, etc.) information.

Outcome measures

Outcome measures were completed by researcher (blinded to intervention allocation) via interview with staff proxy informant, at baseline, 6 and 16 months, unless otherwise stated. Mappers were not permitted to be proxy informants.

Primary outcome

CMAI (Cohen-Mansfield, 1991) measures the frequency of 29 agitated behaviours on a seven-point scale, reported for the previous two-weeks providing a total score (range 29–203).

Secondary outcomes

Neuropsychiatric Inventory - Nursing Home (NPI-NH): (Cummings et al., 1994) 12-item measure of the frequency, severity and disruptiveness of neuropsychiatric symptoms (e.g. delusions, hallucinations, apathy) in care home populations.

DEMQoL-Proxy: (Smith et al., 2007) 32-item, dementia-specific, proxy quality of life questionnaire, modelled to enable the derivation of preference-based indices (utility values), the latter of which were employed in the secondary cost-utility analyses (Rowen et al., 2012).

EQ-5D-5L/EQ-5D-5L Proxy: (EuroQol Group, 1990) five-item measure of health outcome, providing a single index value for health status, including self-report and proxy versions (Herdman et al., 2011).

Quality of Life in Late Stage Dementia (QUALID): (Weiner et al., 2000) 11-item dementia specific proxy measure of quality of life rating the presence and frequency of QoL indicators over the previous seven days.

QOL-AD (care home): (Edelman, Fulton, Kuhn, & Chang, 2005) 15-item self-report measure of quality of life with adapted wording relevant to those living in long-term care and reported to be reliable in use with people with mild to severe dementia (Hoe, Katona, Roch, & Livingston, 2005).

Quality of Interactions Schedule (QUIS): (Dean, Proudfoot, & Lindesay, 1993) observational measure of the quality and quantity of staff interactions with residents, Researchers observed using 15-minute time-sampling over two, one-hour observation periods, on different days (one AM and one PM) within a week, during a period of activity. The proportion of positive to other interactions was compared.

Prescription Medications: within categories of interest (antipsychotic, benzodiazepine, non-benzodiazepine anxiolytic, Memantine, analgesic, anti-depressant) and administration of these (regular/as required (PRN)), via researcher review of the previous month’s medication records.

Sense of Competence in Dementia Care Staff (SCIDS): (Schepers, Orrell, Shanahan, & Spector, 2012) a 17-item scale measuring staff members’ feelings of competence in caring for people with dementia. It was not possible to analyse data from this measure due to poor return rates.

General Health Questionnaire (12-item) (GHQ-12): (Goldberg & Williams, 1988) a measure of stress/psychological well-being, used to assess staff work stress. Due to poor return rates, collection of this measure was ceased to support improved return rates of the SCIDS.

Supportive outcome data

As the primary outcome, data were provided by an un-blinded staff proxy, an independent researcher collected additional observational agitation data.

Cohen Mansfield Agitation Inventory - Observational (CMAI-O): (Griffiths, Albertyn, et al., 2019) an observational version of the CMAI, developed for use in this trial, collected on a single day (approx. 10:00–12:00 and 14:00–17:00) in communal areas.

Pittsburgh Agitation Scale (PAS): (Rosen et al., 1994) observational rating of agitation severity, collected in communal areas on a single day (approx. 10:00–12:00 and 14:00–17:00).

Cost-effectiveness analysis

Cost per unit of improvement in CMAI at 16 months and cost per QALY for residents recruited at baseline (i.e. closed cohort) using a health and personal social service provider perspective. Unit costs for health service staff and resources were obtained from national sources. Costs of the DCM intervention consisted of two components: (i) delivery and receipt of DCM training; and (ii) implementation of the DCM process in care homes. Discounting at the NICE preferred rate of 3.5% per annum for costs and effects were conducted for values post 12 months.

Missing data

Where there were no instructions by instrument authors on how to handle missing data items, prorating was used if ≤25% of items were missing, otherwise scores were assigned as missing. Complete cases analysis was the primary method for handling missing scale data in the cross-sectional analyses of primary, secondary and health economic outcomes, under the assumption that data are missing completely at random (MCAR). If data were not MCAR and predictors were identified, then analyses used multiple-imputed data, assuming that cross-sectional data were missing at random (MAR), and complete cases were reported as a sensitivity analysis.

Data analysis

A comprehensive Statistical and Health Economic Analysis Plan was developed and approved a priori. All analyses were performed in SAS software v9.4 (SAS Institute, Cary, NC) or Stata v14 (StataCorp, College Station, TX).

The continuous primary outcome of agitation (CMAI score) was analysed on an intention-to-treat (ITT) basis on the cross-sectional sample using a linear two-level heteroscedastic regression model that allowed the cluster and resident-level random effects to vary by arm. The model adjusted for minimisation factors (care home type, size, provision of dementia awareness training and recruiting hub) and average care home-level baseline characteristics (dementia severity via CDR, age and CMAI score) as fixed effects.

The robustness of the primary effectiveness analysis to the choice of covariates, the outcome measure and the sample was assessed via sensitivity analyses. The primary effectiveness analysis was repeated using the CMAI-O and PAS scores in place of the CMAI, using the closed-cohort sample, and including dementia severity, age and CMAI score as covariates at the resident-level.

An exploratory analysis of the effect of the intervention received (as opposed to what was intended) was undertaken using complier average causal effect (CACE) analyses (Angrist, Imbens, & Rubin, 1996) of the cross-sectional sample, via two-stage least square instrumental variable regressions, using robust standard errors to allow for clustering. In this analysis, compliers were defined as care homes that completed at least one cycle to an acceptable level. Thus, our CACE analysis estimated the effect of receiving at least one DCM cycle.

For secondary outcomes (behaviours staff find challenging, mood, resident quality of life, and the quality of staff/resident interactions), analyses were performed for the cross-sectional sample at 16 months and closed-cohort at 6 months and 16 months. Analyses were undertaken using the same principles as the primary analysis and the same covariates (for closed-cohort analyses, individual resident-level covariates were used as appropriate). Cluster-specific linear two-level heteroscedastic regressions were fitted where outcomes were continuous.

Cost-effectiveness analysis measured incremental costs, CMAI and QALYs for residents recruited at baseline (i.e. closed cohort, n = 726) using Seemingly Unrelated Regression (SUR) (Willan, Briggs, & Hoch, 2004), a system of regression equations that recognises individual costs and outcomes are correlated.

Results

Demographics

Across the 50 care homes, 726 residents were registered at randomisation. Loss to follow-up at 16 months in the original cohort was 312/726 (43.0%), with the majority 272/312 (87.2%) due to deaths. Participant loss to follow-up was comparable to other recently reporting care home trials (e.g. Ballard et al. (2018) 35% loss to follow-up at 9 months; Livingston et al. (2019) 21% at 8 months and Sackley et al. (2015) 30% at 12 months). At 16 months, 261 additional residents were recruited (99 control, 162 intervention), totalling 675 residents included in the primary analysis (see Figure 2).

Figure 2. Care home and resident CONSORT diagram.

Care home and resident characteristics are presented in Tables 1 and 2, respectively.

Table 1. Baseline care home demographics.

Characteristics	Control (n = 19)	Intervention (n = 31)
Unit type (N (%) missing) – General residential/nursing home	11 (57.9%) 0	20 (64.5%) 0
– Specialist dementia care home/unit	8 (42.1%) 0	11 (35.5%) 0
More than one unit included in study (N (%) missing)	3 (15.8%) 0	3 (9.7%) 0
DCM used within last 18-months to 5 years (N (%) missing)	11 (57.9%) 0	20 (64.5%) 0
Residents' meeting held within the last 6 months (N (%) missing)	17 (89.5%) 0	30 (100%) 1
Relatives' meeting held within the last 6 months (N (%) missing)	18 (94.7%) 0	29 (96.7%) 1
Number of beds in the care home (Mean (SD) missing)	28.8 (8.97) 2	36.8 (14.28) 1
Number of permanent residents (Mean (SD) missing)	30 (11.27) 0	32.9 (14.02) 1
Percentage of permanent residents with dementia (Mean (SD) missing)	83.1 (21.21) 0	74.2 (22.48) 1
Percentage of self-funded residents (Mean (SD) missing)	52.8 (28.12) 0	37.9 (21.12) 1
Cost of a self-funded place per year (£) (Mean (SD) missing)	44,553 (13,291) 0	41,638 (13,003) 1
Resident:staff ratio daytime (Median (Q1, Q3) Missing)	5.2 (3.8, 6.3) 0	4.7 (3.9, 6.4) 1
Resident:staff ratio night time (Median (Q1, Q3) Missing)	9.5 (7.5, 11.7) 0	9.7 (7.8, 11.6) 1

Table 2. Resident demographics.

	Closed cohort at baseline		Cross-section at 16 months
Resident characteristics	Control (n = 308)	Intervention (n = 418)	Control (n = 287)	Intervention (n = 388)
Age at randomisation Years (Mean (SD))	85.3 (7.38)	86 (7.83)	83.7 (7.77)	85.2 (7.79)
Gender: Male N (%)	64 (20.8%)	126 (30.1%)	71 (24.7%)	110 (28.4%)
Number of comorbidities per resident (Median (Q1, Q3))	2 (1, 4)	2 (1, 4)	2 (1, 3)	3 (1, 4)
Selected comorbidities ^a Anxiety	34 (11.0%)	23 (5.5%)	26 (9.1%)	27 (7.0%)
Depression	62 (20.1%)	55 (13.2%)	64 (22.3%)	66 (17.0%)
Psychosis	16 (5.2%)	24 (5.7%)	11 (3.8%)	21 (5.4%)
Sleep disturbance	6 (1.9%)	7 (1.7%)	2 (0.7%)	5 (1.3%)
Delirium	3 (1.0%)	2 (0.5%)	2 (0.7%)	2 (0.5%)
FAST dementia stage (out of completed scores) N (%)	(n = 306)	(n = 391)	(n = 284)	(n = 384)
No/early (score 1, 2, 3)	5 (1.7%)	1 (0.3%)
Mild (score 4)	39 (12.7%)	56 (14.3%)	22 (7.7%)	35 (9.1%)
Moderate (score 5)	26 (8.5%)	48 (12.3%)	20 (7.0%)	21 (5.5%)
Moderately severe (score 6)	166 (54.2%)	214 (54.7%)	168 (59.2%)	238 (62.0%)
Severe (score 7)	70 (22.9%)	72 (18.4%)	74 (26.1%)	90 (23.4%)

^a Not mutually exclusive.

Intervention fidelity was sub-optimal compared to protocol; 7/31 (22.6%) intervention care homes did not complete a full intervention cycle to an acceptable level, 16/31 (51.6%) completed one cycle, 4/31 (12.9%) completed two cycles and only 4/31 (12.9%) completed all three cycles (Surr, Shoesmith, et al., 2019). Likewise, monitoring indicated that there was poor consistency in the fidelity, dose and reach of DCM implementation in terms of the execution of components of each cycle (briefing, mapping, analysis, feedback and reporting, action planning). Mapper retention over the trial period was problematic with one or both mappers withdrawing from their role (mainly due to leaving the organisation or long-term absence [sickness/maternity]) in 54.8% of homes (Surr, Griffiths, et al., 2019). The process evaluation identified a range of barriers and facilitators to implementation of DCM at a mapper, care home, intervention and trial level (Griffiths, Kelley, et al., 2019). While mappers and managers could identify a range of ways in which they felt using DCM had led to positive changes in practice, overall homes reported a lack of time, resources, skills and confidence to implement DCM sustainably. The external experts were valued and many homes reported being unable to continue using DCM without the external input (Surr, Shoesmith, et al., 2019). In-depth discussion of intervention fidelity, barriers and facilitators to implementation and the role of external experts in supporting implementation can be found in the separate papers on these topics (as cited above).

Primary analysis

Analyses on the cross-sectional sample (primary) and the closed-cohort (see Table 3) identified no significant difference in mean agitation scores (CMAI) between arms at 16 months. The mean difference in total CMAI score from the two-level heteroscedastic linear regression model fitted to the multiply-imputed data (assuming data were MAR) was –2.11 points in intervention vs control (adjusted means 45.47 points in control; 43.35 points in intervention, 95% CI –4.66 to 0.44, p = 0.104). The unadjusted ICC was zero in the control and 0.058 in the intervention arm, but the adjusted ICC was zero in the control and 0.001 in the intervention arm, indicating that intervention arm between-cluster heterogeneity was explained by model covariates. Using the complete cases, the mean difference was –2.19 points for intervention versus control (95% CI –4.81 to 0.43) and the adjusted ICC was zero in both treatment arms, indicating the treatment effect was neither clinically meaningful nor statistically significant (p = 0.099).

Table 3. Primary, sensitivity and health economic analyses assuming missing data are MAR.

Analysis	Adjusted mean in control	Adjusted mean in intervention	Estimated mean difference	Lower 95% confidence limit	Upper 95% confidence limit	p-value	Adjusted ICC for intervention	Adjusted ICC for control	N
Primary analysis	45.47	43.35	–2.11	–4.66	0.44	0.104	0.001	0.000	675
Sensitivity analyses (cross-sectional sample)
Key sensitivity analysis (hub omitted from the model)	46.02	43.78	–2.24	−4.91	0.42	0.099	0.010		675
1. Adjusting for before-after eligibility change^a	44.82	42.69	–2.13	–4.71	0.45	0.105	0.002	0.000	675
2. Care home size as a continuous variable	45.59	43.21	–2.38	–5.00	0.25	0.076	0.000	0.000	675
3. Homogeneous clustering across arms	45.41	43.32	–2.09	−4·61	0.44	0.105	0.001		675
4a. CMAI-O (AM)	31.00	30.41	–0.58	–1.62	0.45	0.269	0.215	0.006	675
4b. CMAI on subset with CMAI-O (AM)	47.49	43.43	–4.06	–7.55	–0.57	0.023	0.016	0.001	365
4c. CMAI-O (PM)	31.34	31.11	–0.22	–1.52	1.08	0.737	0.220	0.013	675
4d. CMAI on subset with CMAI-O (PM)	47.49	43.43	–4.06	–7.55	–0.57	0.023	0.016	0.001	365
4e. PAS (AM)	0.93	0.73	–0.20	–0.67	0.27	0.402	0.166	0.011	675
4f. CMAI on subset with PAS (AM)	47.49	43.43	–4.06	–7.55	–0.57	0.023	0.016	0.001	365
4g. PAS (PM)	1.17	0.89	–0.28	–0.96	0.41	0.429	0.299	0.018	675
4h. CMAI on subset with PAS (PM)	47.49	43.43	–4.06	–7.55	–0.57	0.023	0.016	0.001	365
5. CMAI at 16 months (closed-cohort)	46.4	43.16	–3.25	–6.13	–0.37	0.027	0.013	0.001	726
Cost-effectiveness analyses (closed-cohort sample)	Mean costs in control	Mean costs in intervention	Adjusted mean difference	Mean outcome in control	Mean outcome in intervention	Adjusted mean difference	Incremental cost effectiveness ratio		N
EQ-5D-5L	£2060	£3539	£1479	0.708	0.718	0.024	£60,627		726
CMAI	£2345	£3318	£974	–0.557	–1.767	–3.37	£288.88		404

^a Eligibility changed in December 2014 after first two care homes randomised.

CMAI-O = Cohen Mansfield Agitation Inventory – Observational; PAS = Pittsburgh Agitation Scale; CMAI = Cohen Mansfield Agitation Inventory; EQ-5D-5L = 5 item measure of health status.

Analyses of the secondary outcomes

Analyses of NPI-NH, quality of life and quality of staff interactions on the closed-cohort at 6 months and on the cross-sectional sample and the closed-cohort at 16 months (see Supporting Information S4) showed no benefits of intervention over control.

At baseline, the proportions of residents recorded with behaviours reported on the NPI-NH were similar across the intervention and control arms (see Supporting Information S3). However, the average NPI-NH score was higher in the control arm. At 16 months, the proportion of residents experiencing one or more behaviours was smaller in the intervention arm compared to control for both the cross-sectional and closed-cohort samples. There were no significant differences between the arms at 6 or 16 months, with greater average NPI-NH score reductions in the control arm.

There were poor self and family carer proxy completion rates of the QOL-AD and QUALID respectively (see Supporting Information S3). No statistically significant differences were found in staff proxy QUALID scores, relative proxy QUALID or resident rated QOL-AD at baseline, 6 or 16 months in either sample.

The proportion of positive interactions as measured by the QUIS (see Supporting Information S3) differed between arms at baseline and at 6 months, with a higher proportion of positive interactions in the intervention group. These did not differ significantly by treatment arm at 16 months.

There were no clinically meaningful or statistically significant differences in PRN use of medications at 6 or 16 months (see Supporting Information S4), although PRN medication administration rates were low across both arms at all time-points, with wide confidence intervals making the results difficult to interpret (Surr et al., in press).

Supportive, sensitivity and CACE analyses

For supportive analyses (see Table 4), the overall CMAI-O and PAS scores were lower than the proxy reported scores (see Supporting Information S1), indicating a potential over-estimation of the treatment effect within the primary analysis. The mean differences between control and intervention groups were smaller using CMAI-O independent observations. Supportive analyses of the closed-cohort at 6 and 16 months indicated no differences in CMAI, CMAI-O or PAS scores at 6 months, and no differences in CMAI-O and PAS scores at 16 months, confirming that there was no evidence that the intervention was superior to control.

Table 4. Supportive analyses assuming missing data are MAR.

Analysis	Adjusted mean in control	Adjusted mean in intervention	Estimated mean difference	Lower 95% confidence limit	Upper 95% confidence limit	p-value	Adjusted ICC for intervention	Adjusted ICC for control	N
Supportive analyses (closed-cohort sample), multiple imputation
6 -months									726
CMAI	43.44	44.04	0.59	–1.98	3.17	0.653	0.049	0.001
CMAI-O (AM)	31.40	31.86	0.46	–0.37	1.30	0.276	0.019	0.000
CMAI-O (PM)	31.64	32.20	0.57	–0.27	1.40	0.182	0.023	0.001
PAS (AM)	1.04	1.18	0.14	–0.24	0.52	0.473	0.022	0.001
PAS (PM)	1.05	1.23	0.18	–0.20	0.57	0.350	0.021	0.001
16 months
CMAI-O (AM)	30.90	30.50	–0.40	–1.27	0.46	0.361	0.014	0.001
CMAI-O (PM)	31.17	31.05	–0.13	–1.09	0.84	0.795	0.012	0.001
PAS (AM)	0.91	0.79	–0.12	–0.52	0.28	0.547	0.008	0.001
PAS (PM)	1.08	0.91	–0.17	–0.67	0.33	0.502	0.018	0.001
CACE analyses, multiple imputation	Treatment effect (SE)
At least one cycle to an acceptable level			–2.5 (1.5)	–5.4	0.4	0.089

CMAI-O = Cohen Mansfield Agitation Inventory – Observational; PAS = Pittsburgh Agitation Scale; CMAI = Cohen Mansfield Agitation Inventory; CACE = Complier Average Causal Effect.

Sensitivity analyses conducted on the cross-sectional sample supported the findings of the primary ITT analysis. No statistically significant or clinically meaningful differences were found between the arms (see Table 4). The sensitivity analysis conducted on the closed-cohort gave a mean difference of –3.25 (95% CI –6.13 to –0.37, p = 0.027).

A CACE analysis of the cross-sectional sample, gave a mean difference in CMAI score at 16 months of –2.5 points (95% CI –5.4 to 0.4, p = 0.089) lower in ‘compliers’ compared to ‘non-compliers’ (see Table 3 and Supporting Information S2), indicating that the intention to treat estimate from the primary analysis was comparable to being a complier (completing at least one cycle to an acceptable level).

Analyses of safety

There were no reported unexpected serious adverse events. There were 272 (37.5%) reported deaths amongst residents comprising the original cohort (111/308 (36.0%) control and 161/418 (38.5%) intervention) (Surr et al., 2020). Most residents in the closed-cohort did not have any hospital admissions (231/308 (75.0%) control and 308/418 (73.7%) intervention).

Economic analysis

This is reported in detail elsewhere (Meads et al., 2020). In summary, the total cost of DCM was estimated to be £421.07 per resident (£9,290.30 per care home). Control arm intervention costs were assumed to be zero. Using the multiple-imputed data, assuming the cross-sectional data are MAR, incremental cost per unit improvement in CMAI and QALYs was £289 and £60,627, respectively, for intervention versus control (see Table 3).

Discussion and implications

This is the only pragmatic trial to be conducted of DCM internationally. It has permitted assessment of whether the promising results of efficacy trials translate into application of DCM in a real-world context, using a sample that is generalisable to the wider care home population. In this real-world context, DCM was not found to be effective versus control on the primary or any secondary outcomes, nor was it cost-effective. Sensitivity analysis gave a mean difference apparently contradicting the primary analysis conclusion. However, due to loss to follow-up and missing data, they rely on multiple imputed data for a large percentage of participants and thus are not reliable. The exploratory CACE analyses suggested a dose-response relationship, however, further research is required to explore this, particularly given so few care homes implemented more than one DCM cycle, meaning the results are difficult to interpret reliably. The study results may be explained by lower than expected intervention compliance, with only 8/31 (25.8%) completing more than the first external expert mapper supported DCM cycle (Surr, Griffiths, et al., 2019).

The cost per unit improvement in CMAI was higher than other recent evaluations of interventions involving staff training in PCC or communication skills with or without behavioural management training (£6 to £62, 2011 UK prices) (Livingston, Barber, et al., 2014). The cost per QALY was also higher than the upper bound of the threshold over which treatments are least likely to be funded in England, although some lower estimates were reported in sensitivity analyses (Meads et al., 2020). The findings align with one (Chenoweth et al., 2009) and contrast with another previous, non-UK, cost analysis of DCM (van de Ven et al., 2014). Compared to other recent UK trials of interventions aimed to reduce agitation in care home residents, DCM compares unfavourably (e.g. Ballard et al., 2018; Livingston et al., 2019) in terms of both effectiveness and cost-effectiveness of the interventions tested.

While retaining DCM use as close to real world practice as possible, we included a range of additional supports to optimise implementation, which could feasibly (and potentially affordably) be implemented in real-world practice (e.g. mapper selection, external expert support, ongoing telephone/email support, provision of standardised documentation and SMS reminders). Nevertheless, considerable DCM implementation challenges still occurred (Griffiths, Kelley, et al., 2019). Consequently, caution is needed in comparing these outcomes with previous DCM trial findings which did not utilise a pragmatic design. The current findings contrast with two previous explanatory trials, (Chenoweth et al., 2009; Rokstad et al., 2013) which found significant benefits of DCM for resident outcomes, but which adopted ideal case, non-standard, researcher-led DCM cycles. Our results concur with those of other studies using real-world, care home staff-led DCM cycles, but which did not utilise effectiveness trial designs (Dichter et al., 2015; van de Ven et al., 2013).

Robust research evidence on models of DCM implementation is limited, although staff-led implementation is reported to be challenging (Surr, Griffiths, & Kelley, 2018). In this trial, the greatest perceived barriers and facilitators to DCM implementation were at the mapper and care home levels (Griffiths, Kelley, et al., 2019), including lack of mapper time, skills and confidence to implement DCM and lack of resources and management support. It is possible additional training alongside ongoing, rather than time-limited support from an external expert might address mapper barriers. However, provision of additional training did not, improve DCM implementation or outcomes in an RCT where mappers also received three-day ‘Advanced User’ training (van de Ven et al., 2013). Lack of resources and management support was despite detailed discussions at care home recruitment about what trial involvement and DCM implementation would involve. However, there were manager changes in 42% of recruited homes over the trial period, (Surr et al., in press) which may account for some of the challenges in sustaining high level support for DCM over 16 months. Thus, even with additional information, screening and support to that which is provided as standard, many care homes appear to lack the financial, and human resources to successfully and sustainably implement DCM.

Rapaport, Livingston, Murray, Mulla, and Cooper (2017) identify a number of components that can support successful implementation of interventions in care home settings, these include interactive training, post-training support, retaining written materials post-training, aiming to train most staff and building interventions into routine care. DCM as implemented in this trial included all of these components with the exception of training most staff, which would not be feasible given the costs and complexity of the method and building the intervention into routine care. Our results indicate that embedding DCM within routine practice is challenging for many homes due to its complexity, time consuming nature and resource intensity. Therefore, the standard model of DCM implementation may not be fit for purpose in the majority of care home settings, without the provision of additional ongoing external support or an alternative external model of implementation. Alternative interventions for reducing agitation should be explored, which make careful consideration of the learning from this trial and the growing evidence-base on intervention implementation in care home settings.

Strengths and limitations

This is the largest and only definitive, pragmatic RCT worldwide of the effectiveness of DCM, in care home settings using real-world, routine practice conditions. Limitations include relying on unblinded staff proxy reports for primary and secondary outcome measure completion, owing to self-report difficulties for some participants with dementia. Blinded observational measures were also performed but these were limited by the proportion of missing values. Due to the large loss to follow up rates, there was considerable missing data to manage within the health economic analyses. The random selection of care homes ensured good representation and thus promoted generalisability of the trial in the UK. Exclusion of care homes that were subject to care quality admissions bans or other improvement measures, means poorly performing homes were not represented. While staff blinding to treatment allocation was not possible, researcher blinding was maintained throughout. Observational measures of agitation completed by a blinded researcher were used to detect any potential reporting bias. The fact observations of agitation did not include personal care is a limitation, since personal care is a time when agitation is likely to occur or be triggered. Given only DCM cycles where documentation had been received to evidence completion were recorded as completed, we may have under-reported actual implementation.

Conclusion and implications

This trial suggests that in the majority of UK care home settings using routine staff-led implementation, may not provide the right conditions for a complex system-level intervention like DCM. The similar findings in other trials that used staff-led implementation of DCM, suggests this is likely to be the case internationally. Either alternative interventions for addressing agitation in care home residents with dementia should be sought, or consideration needs to be given to different models that may optimise DCM implementation. The results have implications for the design and implementation of complex interventions in care homes more widely.

The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HTA, NIHR, NHS or the Department of Health and Social Care.

Authors Contribution

CS: the conception and design of the study, data acquisition, interpretation of data and drafting of this article.

IH: analysis of the statistical data, interpretation of data and drafting of this article.

RW: the conception and design of the study, analysis of the statistical data, interpretation of data and drafting of this article.

AG: acquisition of data, analysis of the qualitative data, interpretation of data and drafting of this article.

DM: design of the study, analysis of the health economic data, interpretation of data and drafting of this article.

AM: analysis of the health economic data, interpretation of data and drafting of this article.

RK: design of the process evaluation, analysis of the qualitative data, interpretation of data and commenting on the draft of this article.

CB: design of the study, data acquisition and commenting on the draft of this article.

JF: design of the study, data acquisition and commenting on the draft of this article.

NB: data acquisition and commenting on the draft of this article.

LC: design of the study and commenting on the draft of this article.

BC: data acquisition and commenting on the draft of this article.

MD: design of the study and commenting on the draft of this article.

LG: data acquisition and commenting on the draft of this article.

EG: study design, data acquisition and commenting on the draft of this article.

ALK: study design, data acquisition, management of the trial and commenting on the draft of this article.

JM: data acquisition and commenting on the draft of this article.

VM: data acquisition, management of the trial and commenting on a draft of this article.

HM: data acquisition and commenting on the draft of this article.

DP: data acquisition and commenting on the draft of this article.

LR: study design and commenting on the draft of this article.

OR: data acquisition, qualitative data analysis and commenting on the draft of this article.

ES: data acquisition, qualitative data analysis and commenting on the draft of this article.

NS: study design and commenting on the draft of this article.

GS: study design and commenting on the draft of this article.

DW: study design and commenting on the draft of this article.

AF: the conception and design of the study, interpretation of data and commenting on the draft of this article.

Acknowledgements

We would like to thank all the care homes, individuals with dementia, their family members and care home staff for taking part in this study and giving freely of their time and members of the Independent Trial Steering Committee and Data Monitoring Committee who provided oversight and gave expert advice throughout the trial. We would like to thank the following people who have contributed to the successful completion of this trial. Chris Albertyn, Marie Crabbe, Cara Gates, Stephanie Jones, Baber Malik, Harriet Maunsell, Kirsty Nash, Sahdia Parveen, Luisa Rabanal, Bina Sharma, Emily Standell, Miguel Vasconcelos Da Silva and other researchers who collected the data; Madeline Goodwin, Alison Fergusson, Laura Stubbs and others who undertook data management; Benjamin Thorpe who assisted with statistical programming; Sharon Jones, Lisa Heller, Juniper West, Judith Farmer, Maria Scurfield and others who supported DCM™ intervention implementation activities; Jan Leeks and Lindsey Collins who delivered Dementia Awareness training; Jane Ward and other members of the Lay Advisory Group and Susan Fortescue who sat on the Trial Management Group and Lay Advisory Group; Ian Wheeler who provided administrative support for the trial and Matt Murray from the Alzheimer’s Society who provided oversight for the Lay Advisory Group. GS would like to acknowledge Bupa UK, who were his employing organisation during the majority of the study period. Fossey receives support from the NIHR Oxford Health Biomedical Research Centre; a partnership between Oxford Health NHS Foundation Trust and the University of Oxford.

Disclosure statement

CS was previously employed by the University of Bradford, who own the IP to the DCM intervention tested in this trial. In this role, she held responsibility for DCM training and method development. She was a technical author on the British Standards Institute PAS 800 guide on implementing DCM in health and social care provider organizations. She reports personal fees from Hawker Publications outside of the submitted work.

DM reports being a member of the NIHR HTA Elective and Emergency Specialist Care Panel (EESC) being a member of the NIHR PGfAR funding panel.

CB reports grants and personal fees from Acadia pharmaceutical company, grants and personal fees from Lundbeck, personal fees from Roche, personal fees from Otusaka, personal fees from Novartis, personal fees from Eli Lilly, personal fees from Pfizer, outside the submitted work.

MD works at the University of Bradford which holds the Intellectual Property for DCM and runs courses for practitioners and professionals who wish to learn how to use the method.

LR reports grants from NIHR Senior Investigator Award, grants from NIHR Translational Professorship Award, outside the submitted work; and NIHR board membership of the Primary Care Themed Call board.

Tables and figures are reproduced from the study report Surr et al. (2020).

All other authors have no conflicts to declare.

Data availability statement

Data from this study for use for further research may be obtained by contacting the corresponding author.

Additional information

Funding

This work was supported by the National Institute for Health Research Health Technology Assessment programme (project number 11/15/13). The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HTA, NIHR, NHS or the Department of Health and Social Care.