Predicting good Active Support for people with intellectual disabilities in supported accommodation services: Key messages for providers, consumers and regulators

ABSTRACT

Background: There is strong evidence about the effectiveness of Active Support. Recent research has established predictors of good Active Support as staff training, practice leadership, and service setting size. This paper explores features of organisational leadership and structures predictive of Active Support.

Methods: Multilevel modelling with data from surveys, observations and interviews was used to identify predictors of Active Support at the levels of service users (n = 253), services (n = 71) and organisations (n = 14).

Results: Good Active Support was predicted by: (1) positive staff perceptions of management, (2) prioritisation of practice and Active Support by senior managers, (3) strong management support for practice leadership, (4) organisation of practice leadership close to everyday service delivery, and (5) concentration of practice leadership with frontline management.

Conclusion: These findings extend understanding of predictors of Active Support and provide indicators of service quality, with important implications for service providers, service users and those monitoring the quality of services.

KEYWORDS:

• Active Support
• practice leadership
• implementation, organisational and management structure
• quality of life
• supported accommodation

Australia is experiencing a unique period of disability service reform. Unlike the United Kingdom (UK) and other European countries, since 2013, there have been an unprecedented expansion of funding to support the social and economic participation of an estimated 480,000 people with severe disabilities (Miller & Hayward, 2017). The Australian National Disability Insurance Scheme (NDIS) provides individual funding for “reasonable and necessary” disability supports to eligible participants, as well as grant funding for capacity building initiatives (NDIS, 2013, Section 34). While it aligns closely with neo-liberal welfare state reforms characterised by marketisation and individualised funding, the NDIS goes much further than other schemes (Carey, Malbon, Olney, & Reeders, 2018). Unlike the UK, there is no centralised or local planning or commissioning of services; this function rests with the market and decisions of each individual through expenditure of funds within parameters of their plan. Unlike the Scandinavian countries, where small subgroups of people receive individualised funding, there is no opt in: individualised funding applies to all eligible participants across all States and Territories in Australia and there is no block funding of disability services.

People with intellectual disabilities face considerable challenges exercising the consumer power that underpins individualised funding systems, such as the NDIS (Churchill, Sotiri, & Rowe, 2017; O’Connor, 2014). One of these challenges is service choice, especially given the significant variability in quality across supported accommodation services (Bigby, Bould, & Beadle-Brown, 2019). Evidence-based indicators of service quality are one means of better equipping people with intellectual disabilities, their families or advocates to navigate service marketplaces.

Service quality indicators are also central for organisations responsible for regulating marketised service systems, such as the NDIS Quality and Safeguard Commission (Commonwealth of Australia, 2018) and UK Care Quality Commission (Key Lines of Enquiry u.d). However, quality indicators used by organisations such as these are often high level and generic, rather than tailored to reflect evidence relevant to particular groups of service users or types of service. People with intellectual disabilities are the largest group of adult participants in the NDIS, making up the majority of approximately 17,000 people with disabilities in supported accommodation services in Australia (NDIS, 2019). If service quality indicators are to be of value to this group, abstract concepts, such as person-centred practice, found in the NDIS Commission’s quality indicators, need to be translated into evidence-based behavioural expectations of good practice in supported accommodation services and descriptions of the organisational priorities, structures and processes necessary to deliver good practice.

Consistent use of Person-Centred Active Support (Active Support) is the most well-researched predictor of quality in supported accommodation services for people with intellectual disabilities (Bigby & Beadle-Brown, 2018). Active Support is a practice whereby staff use an enabling relationship to facilitate the engagement of people with intellectual disabilities in meaningful activities and social relationships (Mansell & Beadle-Brown, 2012). Based on theories of behaviour and communication, consistent use of Active Support significantly increases service user engagement, choice and control (Flynn et al., 2018). Active Support dovetails with the coherent, enabling, motivating and respectful cultures found in high performing services (Bigby & Beadle-Brown, 2016). Active Support also complements Positive Behaviour Support (PBS), providing the foundation for proactive strategies through targeting support that increases engagement and reduces reliance on challenging behaviour to gain choice and control (Ockenden, Ashman, & Beadle-Brown, 2014). Clear behavioural indicators of good Active Support practice are incorporated into research tools (Mansell, Elliott, & Beadle-Brown, 2005) and training materials (see e.g., Murphy, Bradshaw, & Beadle-Brown, 2017; Every Moment Has Potential).

Though widely adopted by disability services in Australia, Active Support has been difficult to embed in services (Flynn et al., 2018; Mansell & Beadle-Brown, 2012). Researchers have sought to identify predictors of good Active Support at the organisational, service and individual levels both to inform service design and provide indicators of service quality to support the exercise of choice by consumers and the work of service regulators. Flynn et al. (2018), in a review of 10 studies, found the strongest evidence about predictors of Active Support was in respect to training (classroom combined with in-situ methods), relatively low staff-to-service user ratios and larger services (maximum of six service users), and management processes, such as team meetings. More recently, a large Australian study that applied multi-level modelling found predictors of good Active Support were the individuals’ adaptive behaviour, strength of frontline practice leadership, staff training in Active Support, and time since Active Support was implemented (Bigby, Bould, Iacono, Kavangh, & Beadle-Brown, 2019a). Similar predictors were found in a study of increases in the quality of Active Support over time, which included repeated measures from the same 51 services in eight organisations over periods of two to seven years (Bould, Bigby, Iacono, & Beadle-Brown, 2019). Predictors of Active Support in these studies were predominantly characteristics at the level of the individual or service rather than organisation. Despite propositions about the importance of organisational level factors, such as commitment from senior managers, organisational readiness, and relevant policies and structures (Qian, Tichá, & Stancliffe, 2017; Mansell & Beadle-Brown, 2012), there is a dearth of evidence about them (Bigby & Beadle-Brown, 2018; Flynn et al., 2018). Qian et al. (2017) suggested one of the reasons for such limited evidence at this level was that organisational factors relevant to services in the disability sector had not been sufficiently well conceptualised or operationalised to enable measurement.

A qualitative analysis of interviews with senior leaders from 14 Australian disability organisations, which had adopted Active Support, identified four features of organisational management and structures (management features) that were common to the six of these organisations that delivered good Active Support to the majority of service user in 71% or more of their services (Bigby, Bould, Iacono, & Beadle-Brown, 2019b). These features were senior leaders who shared prioritisation of practice and Active Support and strongly supported practice leadership, and practice leadership organised close to every-day service delivery and concentrated in one position with frontline management. Table 1 provides definitions for each of these features. Bigby et al. (2019b) suggested the potential to statistically test whether the presence of these four features, along with other variables, were predictive of good Active Support, singly or in combination. Accordingly, the aim of the present study was to further explore predictors of good Active Support by including, in a multi-level model (MLM), results reported by Bigby et al. (2019b) about the presence or absence of the four organisational management features in 14 organisations, together with other individual and service level data for these organisations,

Table 1. Organisational management features.

Organisational management feature	Description
Shared prioritization of practice and active support	Senior leaders share a commitment to practice and active support, recognise significance of conceptual clarity about practice, demonstrate investment of resources to leadership of practice across services, symbolic use of language of practice and continuous reflection on progress.
Strongly supporting practice leadership	Senior leaders are concerned with strengthening and adjusting delivery of frontline practice leadership through structure, support and development to staff, and redistribution of tasks and resources.
Closeness of practice leadership to every-day service delivery	Frontline practice leadership functions reside with staff who are close to the everyday management and running of the service. They work alongside staff and have both formal and incidental opportunities for observation, modelling and coaching.
Concentration of practice leadership tasks	Frontline practice leadership functions are concentrated in one position rather than dispersed across positions.

Method

Design

This study was a cross-sectional design. Data were collected over the period of February 2017 to January 2018 from a cohort of services provided by 14 organisations. These data were in the form of surveys completed by staff about their own characteristics and work experiences, surveys completed by staff about service user characteristics, observations of service users, staff practice and practice leadership and interviews with practice leaders. Also included were the data on the organisational management features of each of these organisations reported by Bigby et al. (2019b).

Ethical considerations

The study received ethical approval from the La Trobe University Human Research Ethics Committee (HREC). Consent was obtained directly from staff and service users, or their proxy for those without capacity for consent, as approved by the HREC.

Participants and settings

As Table 2 shows, the number of services managed by each organisation varied from 5 to 34, the time since they first adopted Active Support varied from 1 to 15 years, and six had an annual turnover above $50 million. The 14 organisations managed a total of 272 services, supporting a total of 1112 service users. Table 3 shows the total number of services, service users and staff who participated from each organisation. The sample of 253 service users from 71 services were selected for representativeness on the basis of service user age, gender, adaptive behaviour, challenging behaviour, social impairment, physical disability and presence of autism. Comparisons between the selected and non-selected samples were conducted using Mann–Whitney U and chi-square. As Table 4 shows, there were no significant differences on any of these attributes (p < .01 level).

Table 2. Key characteristics of the 14 participating organisations

Organisation	Total number of services managed	Total number of service users	Number of years implementing active support	Annual turnover split into 2 groups
1	5	21	8	<50 million
2	15	28	14	>$50 million
3	5	18	13	>$50 million
4	34	155	12	>$50 million
5	25	100	6	<$50 million
6	7	29	5	>$50 million
7	10	62	5.5	<$50 million
8	33	138	11	<$50 million
9	27	140	2	>$50 million
10	38	131	9	>$50 million
11	23	66	2	<$50 million
12	7	42	1	<$50 million
13	16	78	1	<$50 million
14	31	142	1	>$50 million

Table 3. Number of services, consenting service users (SUs), staff surveys and practice leadership interviews from each organisation included in the analysis.

Organisation	Total number of services	Total number of service users	Total number of staff	Total number of practice leaders
1	5	13	18	4
2	6	12	20	6
3	5	14	17	4
4	4	17	13	4
5	5	16	15	5
6	7	25	25	7
7	7	35	25	7
8	6	21	36	6
9	5	19	18	5
10	3	13	11	3
11	5	11	15	5
12	5	28	20	5
13	4	17	12	4
14	4	12	18	4

Table 4. Comparison of the characteristics of the selected and non-selected service user samples.

		ALL (n = 1112)	2017–2018 sample (n = 253)	Non-selected sample (n = 859)	p
Age (years)	M	47	47	48	p = .188
	Range	20–84	21–81	20–84
Percentage male		52% (n = 578)	54% (n = 137)	52% (n = 441)	p = .483
Part I ABS score	M	150	147	151	p = .481
	Range	22–291	22–272	22–291
Total score on the ABC	M	28	25	29	p = .08
	Range	0–144	0–97	0–144
Percentage socially impaired		60% (n = 626)	61% (n = 144)	60% (n = 482)	p = .961
Percentage with autism spectrum disorder		17% (n = 187)	17% (n = 44)	17% (n = 143)	p = .804
Percentage with a physical impairment		35% (n = 392)	36% (n = 91)	35% (n = 301)	p = .823

Measure of the predicted variable – quality of active support

The quality of Active Support was determined using the Active Support Measure (ASM) (Mansell et al., 2005). This measure has been reported to have acceptable reliability and validity, and in most studies a Cronbach Alpha over 0.9 has been reported (see e.g., Beadle-Brown, Hutchinson, & Whelton, 2012; Mansell, Beadle-Brown, Macdonald, & Ashman, 2003). The ASM is completed for each consenting service user after 2-hours of observation, conducted between the hours of 4:00 and 6:00 pm. Each of 15 items addresses the quality of staff support to individual service users to enable them to be engaged in meaningful activities and relationships. The measure is completed according to guidelines provided by Mansell et al. (2005), with each item rated according to a scale anchored by 0 (poor, inconsistent support) and 3 (good, consistent support) to yield a maximum score of 45. If challenging behaviour has not been observed, it is scored as “not applicable,” resulting in the omission of two items and a maximum possible score of 39. Scores are then converted to a percentage, with a score over 66.66% designated as the threshold score indicative of a good level of Active Support (Mansell & Beadle-Brown, 2012).

Four observers were trained in the ASM by the second author using video material, then the ASM was completed at least twice alongside the second author in a service before collecting data alone. An average of 87% (range 69%–100%, n = 26) agreement and kappa of .73 (range .53–.100) were obtained across observers. In light of the low agreement for some ASM items, paired t-tests were performed for overall scores obtained across observers, which indicated that any differences did not reach significance (p = .271).

Measures of the predictor variables

Predictor variables included service user level factors (service user characteristics – gender, age, adaptive behaviour, aberrant behaviour and social impairment), 10 service level factors (staff-to-resident ratio, staff training, satisfaction, role clarity, perception of practice leadership, perception of quality of management, attitudes towards people with intellectual disabilities, observed measure of practice leadership, number of residents and their heterogeneity) and eight organisation level factors (presence or absence of the four organisational management features from Bigby et al. (2019b) (see Table 1); number of services managed, total number of service users, time since adoption of Active Support, and annual revenue turnover).

Service user characteristics questionnaire. Data about each service user were obtained from a staff-completed questionnaire. Items related to gender, date of birth, and other disabilities present. The questionnaire incorporated the Adaptive Behavior Scale (SABS) Part 1 (Hatton et al., 2001), the Aberrant Behavior Checklist (ABC) (Aman, Burrow, & Wolford, 1995) and the Quality of Social Impairment question from the Schedule of Handicaps Behaviours and Skills (HBS) (Wing & Gould, 1978). Authors of these measures have reported them to have acceptable reliability and validity.

Staff-to-resident ratio. A proforma completed by the observer (at the time of the ASM) was used to record the numbers of residents present and staff on duty during the 2-hour observation. The staff-to-resident ratio, a service level factor, was determined by dividing the number of staff by the number of residents.

Staff experiences and satisfaction survey. Staff in each service completed an adapted version of the Staff Experiences and Satisfaction Questionnaire (SESQ) (Beadle-Brown, Gifford, & Mansell, 2005). It includes three sections: (1) demographics and training; (2) experiences at work – satisfaction, role clarity and conflict, and perception of practice leadership and quality of management; and (3) a shortened 13-item version of the original Section D scale looking at attitudes towards people with intellectual disabilities. All scales along with their reliability and validity are described in detail in Mansell, Beadle-Brown, Whelton, Beckett, and Hutchinson (2008, pp. 401–402). Based on a large-scale evaluation involving 550 staff, the 13-item attitude scale was shown to have a Cronbach’s alpha of 0.856 (Mansell et al., 2008)

The Observed Measure of Practice Leadership. The quality of practice leadership in the service was measured using the Observed Measure of Practice Leadership (Beadle-Brown, Bigby, & Bould, 2015). Across several studies, this has been shown to be a valid and reliable measure, with good internal consistency (Cronbach Alpha over 0.9), acceptable inter-rater reliability (Kappa value over 0.6 on average across the five domains) and good construct validity in terms of discriminatory power for the predictor variable (the ASM) – better practice leadership was consistently associated with higher levels of active support (t (171) = 3.88, p < 0.001; in Beadle-Brown et al., 2015). An observer made an additional visit to the service, or one of the services supervised by a practice leader, for a 10–30 min observation, followed by an interview with the practice leader (approximately 1 h) and review of the paperwork associated with practice leadership, such as staff work allocation and team meeting minutes. The observer used these data to score five core aspects of practice leadership: (1) overall focus on service users’ quality of life (QoL); (2) allocating and organising staff to provide the support needed by service users to maximise their quality of life; (3) coaching, observing, modelling and giving feedback to staff about the quality of their support; (4) reviewing performance with individual staff during supervision; and (5) reviewing team performance in team meetings. Rating of each element was based on a 5-point scale, anchored by 1 (no or almost no evidence of the element being in place) and 5 (element was consistently in place). Scores were tallied across elements and divided by 5 to yield a mean score indicative of the overall strength of practice leadership provided by the service’s frontline manager.

Organisational management features. The data from findings reported by Bigby et al. (2019b) on the presence or absence of four organisational management features which are reproduced in Table 5 were used. Based on the presence (score of 1) or absence (score of 0) of each feature in each organisation, an organisational total score was calculated to yield a maximum score of 4 across the four items: (1) shared prioritisation of practice and Active Support by senior leaders (2) senior leaders strongly supporting practice leadership, (3) organisation of practice leadership close to every-day service delivery, and (4) concentration of practice leadership and front-line management tasks.

Table 5. Organisational management features across the 14 organisations (adapted from Bigby et al., 2019b).

Organisation	Services (SUs) receiving good active support^1	Shared prioritisation of practice and active support	Strongly supporting practice leadership	Practice leadership close to everyday service delivery	Concentration of practice leadership and line management tasks
1	100% (92%)	1	1	1	1
2	71% (62%)	1	1	1	1
3	100% (93%)	1	1	1	1
4	33% (35%)	0	0	0	0
5	40% (41%)	0	0	0	1
6	57% (48%)	0	1	1	0
7	71% (71%)	1	1	1	1
8	86% (88%)	1	1	1	1
9	50% (55%)	1	0	0	0
10	33% (31%)	0	0	1	1
11	29% (21%)	1	0	1	1
12	57% (42%)	0	0	1	1
13	83% (70%)	1	1	1	1
14	50% (62%)	1	0	0	0

Note: ¹Organisations with 71% or more services with the majority of service users (>61%) shared four these organisational management features.

Organisational size, turnover and time since adopting Active Support. Data were collected from each organisation on the number of services managed, total service users supported, time since adoption of Active Support, and annual turnover.

Procedure

An audit questionnaire was completed within each service for all service users for the purposes of selecting a sample, and ascertaining the total number of services managed and service users supported by each organisation. The audit questionnaire was combined with the service user characteristics questionnaire into one package. For each organisation, an audit database was created and sent to a contact person, with instructions to (1) complete the coding of service users identified within the database; (2) distribute questionnaires for all service users in the organisation, with requests for a staff member who knew the individual well to complete and return to the contact person; (3) remove the service user name on each questionnaire and leave only a unique code from the database; and (4) return completed audit questionnaires to the research team in the pre-paid envelopes provided. Participation in the study was dependant on completion of this audit for all service users with intellectual disability.

Once staff consent was gained, staff questionnaires were mailed to supervisory and managerial staff associated with each service, who were asked to give a copy to each consenting member of staff. Completed staff questionnaires were returned directly to the researchers by mail using a pre-paid envelope.

An observer then visited each service to conduct the 2-hour observation and complete the ASM for each consenting service user. On another day, an observer visited the service to complete the Observed Measure of Practice Leadership. Hence, two visits were made to each service, within 2–4 months, except for services that shared a practice leader, in which case only one visit was made across these services for the observed measure of practice leadership.

Analysis

Data were entered into IBM SPSS 24. The criterion for inclusion of data in the analysis was a minimum of three staff questionnaires returned for a service. Descriptive statistics and correlational analyses were conducted to examine relationships among predictors, with Cohen’s (1988) guidelines applied for effect sizes.

For the purpose of the MLM, data were organised across service user, service, and organisation levels. Service user level scores were the ASM percentage score, and the full-scale score for Part 1 of the Adaptive Behaviour Scale (ABS) was estimated from the SABS using the method described by Hatton et al. (2001). Service users were initially categorised into two groups: below 151 and 151 and above, a cut off used in other studies to indicate service users with more or less severe disability (see Mansell, Beadle-Brown, & Bigby, 2013). However, preliminary analysis indicated that all people supported by one organisation had an ABS of 80 or less, therefore the ability grouping was revised to include ABS scores of less than 80, 81–150 or 151 and above. These ABS groups were aggregated to the service level to represent the number of ABS groups supported by a service: for example, in a service with four service users, the number of ABS groups was two if three service users were in the less than 80 ABS category and one service user was in the 81–150 category. Other service level scores were the Observed Measure of Practice Leadership mean for each frontline manager and staff-to-resident ratios during the 2-hour observation. The unique codes provided from organisations using the audit database were used to ascertain the total number of service users in each service, which were grouped into two categories: 1–6 and 7+, a cut off based on studies by Tøssebro (1995) and Flynn et al. (2018). The aggregated data for size of setting, ability group, and practice leadership score were assigned to all the individual service users within the same service(s).

For the staff questionnaires, data were included in the analysis only if at least three staff surveys were returned for a service. Individual staff data on attitudes towards people with intellectual disabilities, perception of practice leadership, quality of senior management, role clarity and conflict, job satisfaction, and training in Active Support were aggregated to the service level through a mean score for each service, and subsequently assigned to all the individual service users within the same service.

Finally, the unique codes provided from organisations using the audit database were used to ascertain the total number of services and service users supported by the organisation. The annual turnover was grouped into greater (>) or less than (<) $50 million, and these data, along with the total score for presence of the four organisational management features taken from Bigby et al. (2019b), and number of years implementing Active Support were aggregated at the organisational level. These scores were subsequently assigned to all individual service users within the same organisation. Due to missing data, 16 services and 54 service users were excluded from the final analysis. The data structure for the MLM, which took into account the clustering, was 253 individual service users (level 1) nested within 71 supported accommodation services (level 2) from 14 organisations (level 3). The MLwiN program (Version 3.02; Charlton, Rasbash, Browne, Healy, & Cameron, 2017) was used for the MLM analysis. The Deviance Information Criterion (DIC) (Spiegelhalter, Best, Garlin, & van der Linde, 2002) statistics for model comparison was calculated using the Markov chain Monte Carlo (MCMC) (Browne, 2017) estimation, given the size and structure of the data set that required partitioning of the variance at three levels (Rodriguez, 2007). Any decrease in the goodness of fit diagnostic, DIC, suggests a better model. All models were estimated using non-informative priors (Browne, 2004) with a burn-on of 1000 and 20,000 iterations.

An initial null model was estimated, which also computed an intraclass correlation coefficient (ICC): that is, the proportion of the total residual variance attributable to differences between groups, referred to as the variance partition coefficient (VPC) (Goldstein, 2003). The formula for calculating the VPC is the ratio of the variance at each level to the total variance. Subsequently, a series of multi-level models were built using a bottom up approach (Hox, 2010; Raudenbush & Bryk, 2002). The fully adjusted model was:$\begin{array}{rl}\text{ASM Scor}{\text{e}}_{ijk}& ={\text{β}}_{ijk}+\text{AB}{\text{S}}_{ijk}+\mathrm{P}\mathrm{L}\mathrm{M}\mathrm{e}\mathrm{a}{\text{n}}_{ijk}\\ & +7\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{o}\mathrm{r}\mathrm{e}\mathrm{S}\mathrm{U}{\text{s}}_{ijk}\\ & +\text{Staff perception of managemen}{\text{t}}_{ijk}\\ & +{\text{Organisational management features}}_{ijk}+e_{ijk}\end{array}$

In this model, i refers to the service user, j the supported accommodation service, and k the organisation. β_ijk refers to the grand mean (i.e., average Active Support score of the 253 service users from 71 services from the 14 organisations) and e_ijk refers to a random effect.

Results were deemed significant if the estimates were more than twice their estimated empirical standard error. All predictors were grand mean centred (i.e., the intercept was centred around the mean of the sample) to facilitate the interpretation of the intercepts and slopes, and because of primary interest was the influence at the higher levels of service and organisation factors (Enders & Tofighi, 2007).

Results

Table 4 provides descriptive statistics for the service users (n = 253). Inspection of this table shows that they varied in terms of their characteristics and support needs. On average, the sample was relatively able compared to participant samples in other studies of Active Support (Mansell et al., 2013).

Table 6 provides the spearman correlations (at the service user level) used to examine relationships among predictors included in the final model. The largest correlation with the ASM was the level of adaptive behaviour (ρ = .432, n = 253, p < .001), with a medium effect size (Cohen, 1988). Small to medium correlations were found between the organisational management features, determined by tallying across the number of features present for each organisation (Table 5, with a total possible score of 4) and each of the ASM, ABS, mean practice leadership and the service size (Table 6).

Table 6. Spearman correlations between predictor variables.

	ABS	Mean Practice Leadership	Service size – total number of SUs	Staff perception of quality of management	Organisation score on management features present
ASM	.432^b	.295^b	−.255^b	0.061	.381^b
ABS		−0.117	.250^b	.170^b	.180^b
Mean Practice Leadership			−0.051	−0.080	.322^b
Service size – Total number of SUs				.147^a	.130^a
Staff perception of quality of management					0.009

Notes: ^aCorrelation is significant at the 0.05 level (2-tailed).

^bCorrelation is significant at the 0.01 level (2-tailed).

Other service user level data, such as socio-demographic or degree of social impairment, were not significantly correlated with the ASM. Other factors not found to correlate significantly with the ASM were service level data, including the number and heterogeneity of service users, staff-service user ratios, staff qualifications and attitudes, and organisational level data about characteristics, such as size, turnover and years implementing Active Support. Hence, these factors were not found to be predictors in the model.

Table 7 presents the modelling results as parameter (beta) coefficients and their standard errors, along with the model-fitted diagnostic DIC. Model 1 is the null, which includes no predictor variables, and the VPC indicated 8% of the variance in the ASM scores were accounted for by differences between organisations, 60% by differences between the supported accommodation services, and 32% by within individual service user differences. In Model 2, individual predictors were included, and only one predictor (ABS) was significant, indicating that individuals with greater adaptive behaviour received better quality of support as measured on the ASM. No other service user measures contributed to the model. Model 3 included variables associated with the service; services with higher practice leadership scores and a more positive perception of management by staff were associated with higher ASM scores. Conversely, services with 7+ service users were associated with lower ASM scores. No other service level variables contributed to the model. Model 4 allowed for examination of variables associated with the organisation. Inclusion of the score on organisational management features led to a further improvement in the model-fitted diagnostic DIC. No other organisation level variable contributed to the model.

Table 7. Parameter (beta) estimates of the multi-level models and deviance information criterion (MCMC).

	Model 1 (S.E)	Model 2 (S.E)	Model 3 (S.E)	Model 4 (S.E)
Fixed parameters
Constant	67.488 (2.496)	67.458 (2.157)	69.948 (1.765)	69.436 (1.511)
Individual predictors
ABS		0.134 (0.017)	0.154 (0.017)	0.146 (0.017)
Service level predictors
Mean practice leadership score			7.820 (1.846)	6.793 (1.714)
7 or more SUs in the service (6 or less base)			−28.673 (7.112)	−29.586 (6.587)
Staff perception of quality of management			0.274 (0.139)	0.267 (0.128)
Organisation level predictors
Organisation score of management features present				3.51 (1.132)
Random parameters
Level 3: Between organisations	34.253 (47.568)	22.529 (33.275)	13.406 (20.741)	3.676 (9.243)
Level 2: Between services	273.191 (54.823)	229.443 (44.678)	106.102 (28.444)	98.841 (24.298)
Level 1: Within individuals	144.704 (13.905)	120.094 (11.526)	117.663 (12.375)	116.949 (12.349)
Deviance Information Criterion (DIC)	2471.614	2407.568	1999.577	1975.018
Change in DIC		64.046	407.991	24.559

Note: All estimates are significant at 0.05 probability level or smaller.

Although there remained significant variance at each of the three levels, as indicated in Figure 1, the predictors included in Model 4 accounted for 88% of the between organisation variance, 64% of the between service variance, and 19% of the within individual service user variance.

Figure 1. Predictors of good active support.

Discussion

Extending findings from an earlier study, the present study demonstrated statistically, that combined, the four organisational management features identified by Bigby et al. (2019b) were predictors of the quality of Active Support. These features were senior leaders’ shared prioritisation of practice and Active Support, strong support for practice leadership by senior leaders, the organisation of practice leadership close to every-day service delivery and concentrated in one position with frontline management (Figure 1). For this study, the presence or absence of each of these features in an organisation was derived from interviews with senior organisational leaders previously reported in Bigby et al. (2019b). The present study has also confirmed findings from two earlier studies that good Active Support is predicted, at the individual level, by higher adaptive behaviour of service users, and, at the service level, by stronger practice leadership, and having six or fewer service users in a service (Bigby et al., 2019a; Bould et al., 2019). In addition, staff with a more positive perception of the quality of management was also predictive of good Active Support.

Somewhat unexpectedly, factors predictive of Active Support in previous studies drawn from overlapping data sets (Bigby et al., 2019a; Bould et al., 2019) were not predictive in this study: these were whether or not staff had been trained in Active Support and homogeneity of service users (the number of ABS groups represented within a service) at the service level, and organisational size and time since adopting Active Support, at the organisational level. The limited variability in this data set, compared to the two earlier ones, provides the most plausible explanations for these differences. Unlike in the previous studies (Bigby et al., 2019a; Bould et al., 2019), data were collected at one rather than multiple time points, and at a time when all organisations had been implementing Active Support and monitoring progress annually over a number of years.

This maturity in implementing Active Support may account for the high percentage of staff (82%) having been trained in Active Support in the cohort of organisations in the present study. In contrast, there was greater variability in training of staff across the organisations at the varied time points captured in the studies reported by Bigby et al. (2019a) and Bould et al., (2019). The findings of these two previous studies, together with those of earlier ones point strongly to the importance of Active Support training and its nature to the implementation of Active Support (Flynn et al., 2018). It would seem that variability in the number of staff with Active Support training will influence the quality of support provided, but only to a point, after which, it no longer has predictive influence. What that threshold point might be could not be determined from the present study, but the failure of this variable to predict Active Support quality suggests it was at least met, if not exceeded at the point of which 82% of staff had been trained.

Similarly, the absence of an association between homogeneity of service user needs in a service and the quality of Active Support in the present study may be accounted for by the fact that only 13 of the 71 services included service users who fell into three different ABS groups. There was greater variability in the data for this item in Bigby et al. (2019a) and Bould et al. (2019), in that the heterogeneity among service users was found to be associated with poor quality Active Support. Combined, the findings from the three studies suggests complementary evidence: that heterogeneity of service user support needs detracts from the provision of Active Support quality, while homogeneity removes this factor as an influence.

A further factor found predictive of Active Support in the previous studies Bigby et al., (2019a) and Bould et al., (2019), but not the present study, was time since implementing Active Support. Again, variability in these data evident in the previous studies was not found in this study, reflecting the fact that most organisations (n = 9) had been implementing Active Support for more than five years. This longer time of implementation may also account for the absence of an association between organisational size and quality of Active Support found by Bould et al., (2019). It is likely to take larger organisations longer than smaller ones to successfully implement Active Support. It may have been the relatively early success of smaller organisations in a short time period that was identified in the earlier studies (Bigby et al., 2019a), which disappeared as Active Support became embedded over longer periods.

A tentative hypothesis from the present study that is worthy of further investigation is that advantages of smaller organisations dissipate over time, and that, for larger organisations, a period of five years may be required to successfully implement and embed Active Support. It may also be that after five years, other organisational level factors confound the impact of time. For example, the qualitative data reported by Bigby et al., (2019b) indicates that disruption to the processes of implementation may result from changes to senior personnel or competing organisational priorities emanating from external factors. This proposition is supported by findings in earlier studies by Mansell and colleagues (2008, 2013) of low levels of good Active Support in organisations that had been implementing it for more than five years. They demonstrate the likely interaction between time since implementing Active Support, and other organisational or service level factors in achieving good support.

Implications for practice

The findings from the present study together with earlier studies of Active Support (Flynn et al., 2018; Bigby et al., 2019a; Bould et al., 2019; Bigby et al., 2019b) suggest the following features at the service and organisational levels are predictors of good Active Support: (1) staff trained in Active Support using classroom and in-situ methods; (2) strong practice leadership of individual direct support workers and their team through regular coaching, observation and feedback about their practice, discussion of Active Support in team meetings and individual supervision, shift planning, and support to maintain focus on the quality of life of the people they support as core to everything they do; (3) practice leadership structured so leaders are close to every-day practice, and their tasks are not split across different positions; (4) staff having confidence in the management of the organisation; (5) services not supporting more than six people under one roof; (6) people sharing accommodation having support needs that are not too different, and not all having challenging behaviour; and (7) senior leaders having a shared understanding of Active Support, and recognising and valuing high-quality practice. These features provide a blueprint for the design of services committed to delivering quality Active Support.

The definitions of each of these predictors and associated research measures could be translated into a set of evidence-based indicators of conditions, at the service and organisational levels, that are necessary for delivery of good quality Active Support. Indicators could be tailored to different audiences: consumers of services to assist in choice, the Commission to assist in service registration, and auditors or Commission for inspecting or monitoring services. Quality indicators of the nature described would build on, for example, the “What does good look like” checklist (Beadle-Brown & Ashman, 2016), with the resulting tool tested for reliability against research measures, such as the Active Support Measure. Importantly, however, this type of quality indicator is once removed from actual practice, and should not replace regular observation of frontline practice by supervisors and practice leaders, using a tool that is easy to complete in a practice context.

Conclusion

Active Support is one of few areas in disability practice with not only an evidence base but in which predictive factors have been explored internationally. This evidence provides behavioural indicators of good Active Support practice, benchmarks of good practice, and practice guidelines. The present study has contributed further evidence to support the development of a set of indicators of factors at the service and organisational levels necessary for delivery of good Active support.

Acknowledgements

Thanks are extended to the disability services participating in this study, and to research assistants Louise Phillips, Samuel Murray, Emma Caruana, Lincoln Humphreys, Rosa Solá Molina and Andrew Westle for support with data collection.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

Funding support was from the Australian Research Council Linkage Grant LP130100189 and partner disability services.