How leaders in day service organisations understand service quality

Abstract

Organisations for people with intellectual disabilities must comply with regulatory quality standards written by Australian governments. Standards are abstract and predominantly focus on paperwork and processes. In thinking about service quality, organisational leaders must decide where to focus their efforts and whether to look beyond compliance issues. This study aimed to identify how leaders in day-service organisations for people with intellectual disabilities perceived and monitored service quality, and what they thought influenced quality in their services. Using a constructivist grounded theory methodology, semi-structured interviews were conducted with eight leaders from three day-service organisations in Victoria, Australia. Interviews were recorded, transcribed, and thematically analysed using constant comparison and line-by-line coding. Overall, the leaders had two contrasting approaches to quality in their organisations. Four had a “process compliance” approach and the other four a “service user’s experience of support” approach. These two approaches to service quality mirrored the tensions between the process compliance approach used by Australian governments to regulate the quality of services provided to people with intellectual disabilities, and an approach preferred by researchers, which argues the importance of judging quality through observation of service users’ experience of support. Consideration should be given to merging these approaches and creating indicators that incorporate both observation and process review methods.

Keywords:

• Service quality
• intellectual disability
• day service

For several decades, Australian governments have prescribed quality standards to which organisations for people with intellectual disabilities must comply, in order to receive funding (McEwen et al., 2014). Quality standards specify the principles embedded in legislation that are considered to be evidence of “good” quality service provision (McEwen et al., 2014). Leadership teams in organisations for people with intellectual disabilities are responsible for ensuring compliance with quality standards. The primary mechanism used to determine whether or not organisations comply with quality standards are audits. Audits typically involve reviewing an organisation’s pre-existing records and observations of the way in which processes or systems are enacted, to judge whether or not they match the expectations set out within quality standards (Ellis & Whittington, 1993). The systematic identification and monitoring of processes against desired outcomes is commonly associated with Total Quality Management (Biazzo & Bernardi, 2003), a system developed in the 1980s to assure the quality of goods and services in the manufacturing and engineering sectors (Petersen, 1999). Recent studies have suggested that organisational policies, procedures, or records are the dominant form of evidence used during audits to determine compliance against quality standards (McEwen et al., 2014). With this in mind, organisational leaders must decide where to focus their efforts and whether to look beyond issues of compliance with paperwork and processes in thinking about quality. Research suggests an important indicator of service quality is the way staff provide support to people with intellectual disabilities; the nature of their interactions, and the behaviours, attitudes, and values they display towards the people they support (Windley & Chapman, 2010). In shared supported accommodation services particularly, researchers have argued the value of direct observation of staff practice (Mansell, 2011) as a way of judging quality and have illustrated discrepancies between conclusions about service quality based on compliance with standards and those based on direct observation (Beadle-Brown et al., 2008).

The way that leaders approach quality has the potential to influence the support people with intellectual disabilities receive, yet little is known about their understanding of quality, particularly in day services where, compared to shared supported accommodation, very limited research has been conducted. In the Australian context, day services have been designed to facilitate educational and recreational activities for people with intellectual disabilities who are unable to access mainstream education and employment (Australian Institute of Health and Welfare, 2008).

This study reports on a subset of data from a larger project that aimed to explore quality in day-service organisations for people with intellectual disabilities (McEwen et al., 2021). The present study aimed to identify how leaders in day-service organisations understand quality and had the following research questions: (i) how do leaders in day-service organisations perceive service quality? (ii) how do they monitor service quality? (iii) what do they think influences the quality of services?

Method

Study design

The study was guided by a grounded theory methodology and a constructivist epistemology, which allowed data to be collected and analysed using a systematic and inductive approach (Charmaz, 2014). The La Trobe University Human Ethics Committee approved the study, and all participants gave their informed consent to participate. All names of individual participants and organisations have been replaced with pseudonyms to protect anonymity.

The inclusion criteria were that organisations were located in Melbourne, provided day services to people with intellectual disabilities, and were of moderate size, employing between 20 and 50 employees. Purposeful sampling was used to invite organisations that had previously demonstrated interest in this research to participate in the study. All three organisations that were invited agreed to participate. Each of these day-service organisations provided support to over 70 adults with intellectual disabilities.

Leaders of the three day-service organisations were invited to participate via email and letter. Eight leaders agreed to participate, including two chief executive officers, two general managers, two quality managers, and two frontline managers. The chief executive officers were responsible for the overall management and strategic direction of their organisations, general managers oversaw operations and assured the quality of day services. Quality managers ensured all services provided by their organisation were compliant with regulatory standards. Front line managers were responsible for the day-to-day management and quality of services. Age, qualifications, and work experience varied across leaders; two had university level qualifications specialising in disability (a manager and a chief executive officer), and the remaining six held a mix of bachelor and master’s degrees in disciplines such as psychology, physical education, finance, and quality assurance. Four of the eight leaders, including a chief executive officer, quality manager, general manager, and frontline manager, had worked in day-service organisations for 15 years or more, and had significant experience of providing direct support to people with intellectual disabilities. The remaining four leaders, a chief executive officer, general manager, quality manager, and frontline manager, had worked in day services for 5 years or less, having previously worked in finance, psychology, and physical education. Five of the leaders were female; three were aged between 20 and 30 years, three between 35 and 50 years, and two between 50 and 60 years of age.

Data collection and analysis

Semi structured interviews, each lasting between 1 and 1.5 h, were conducted by the first author with each participant. Interviews were conducted in the day-service organisation leaders worked for, during the times they were available. Open ended interview questions were aimed at eliciting information about each leader’s perceptions of service quality. Questions included “what do you think good service quality looks like?” and “how do you know if staff have delivered a good quality service?”. The first author kept field note records of behaviours, activities, and practices she observed before, during, and after interviews. Field note records were collected to capture the context surrounding the data collected, and the possible meaning interviewees assigned to it. Interviews were recorded and transcribed.

Consistent with the study’s constructivist grounded theory methodological framework (Charmaz, 2014), analysis moved through a process of data-driven open and focused coding, identification of emergent categories, and exploration of relations between these categories using the method of constant comparison. Initially, the first author examined verbatim transcripts and field note records, and textual excerpts were coded to reflect the meaning of the words recorded. As the data from each successive interview were compared and contrasted in an iterative process, and refined through discussion with the second and third authors, codes became increasingly focused until clear categories emerged, as illustrated in Box 1. Categories were reviewed against the original transcripts to ensure they maintained the voice of the participants and were anchored in their experience.

Results

The eight leaders had differing views about how they perceived service quality, how they monitored service quality, and what they thought influenced service quality in their day-service organisation. For each of these topics their views fell into two contrasting categories, which are described in the following sections using illustrative quotes.

Perceptions of service quality: policies and procedures vs personal outcomes

Four of the eight leaders perceived service quality to be about system efficiencies and complying with policies, procedures, and standards. For example, they said: “Service quality is about making sure that we abide by certain standards, have procedures” (Nathan, general manager); “You have to know what it is you want to achieve, and the way to know it is to say it and to document it and then to make sure you’ve got a process in place” (Elle, quality manager). In contrast, the other four leaders believed service quality to be about the way service users experienced support, including the personal outcomes achieved for individuals as a consequence of support. For example, they said:

So, service quality for me is the quality of the relationship between staff, and how many times you can get a good outcome for a client. (Natasha, quality manager)

Service quality is, I think, participants first, is what I would say. If participants are enjoying it, gaining out of it, they’re achieving their goals. (Priya, frontline manager)

Monitoring quality: secondary vs primary evidence

Contrasting viewpoints amongst leaders were also apparent in the way they described how they monitored service quality in their organisations. Four leaders described methods that involved the collection and analysis of “secondary” evidence or data completed by another person. Examples included records written by staff about service users, data about complaints, compliments or incidents, internal audit reports or photographs taken by staff of participants engaging in activities or tasks. For example, they said:

Your complaints and your compliments processes. You've also got your Cat 1s, 2s, and 3s¹ and everything else that you have to organise and send up to the department as well. That's another feedback process in and of itself; a much more formal one. (Joe, chief executive officer)

So once a week I will go and have a scan of them [file notes] - just from random clients - to see what happened in the last week, and I think that’s also the way that I relate to staff. So I can go ‘well, I noticed that Bob had two cigarettes while you were around’. At least I can be in touch with the ground. (Nathan, general manager)

The other four leaders described using methods to monitor service quality such as direct observation of the way people were supported by frontline staff, and interviews with service users about the quality of the support they received. This evidence or data was “primary” in nature, as it was collected by the same person who reviewed it and used it to make determinations about service quality. For example, they said:

Observation for people that aren’t verbal and talking to lots of people. Because people [staff] can write something and then when you’re in conversation all of a sudden, it’s not verified. Also, how do you let people’s behaviour speak? Because we’ve got positive behaviour support but when an incident comes in, that lets them [service users] speak and be heard. (Natasha, quality manager)

Most communication is not verbal. If we’re relying on verbal communication there’s lots of people that don’t tell the truth when they’re talking to authority figures or tell lies or misinformation. So, what people say is less important than what they do. So, if people are being aggressive or agitated, they’re probably not living a great life. (Ron, chief executive officer)

Factors influencing the quality of services: indirect and external vs direct and internal

Contrasting viewpoints were also evident among leaders when they described the factors that contributed to good service quality in the day-service organisations they worked for. Four of the leaders spoke about “indirect and external” factors, such as a redesign of the service or the staffing structure or a resource acquired from an external provider such as staff training. “Training is very much part of how you instil a quality culture. We did that - came in in October, we had an all-in staff training day based on culture” (Joe, chief executive officer); “You have to know what it is you want to achieve, and the way to know it is to say it and to document it and then to make sure you’ve got a process in place” (Elle, quality manager); “For businesses to be able to provide quality services, we need funding. We need to be given money to be able to ensure that people are safe and we’re delivering quality services” (Amy, frontline manager).

The remaining four leaders spoke about “direct and internal” factors that they believed contributed to service quality. They described these as positive characteristics demonstrated by staff, such as a commitment to the people they supported and an ability to problem solve. Leaders also commented on the connection between good service quality and staff being happy in their role and with the tasks they performed. For example, they said:

I’d say putting staff in programs where they actually fit well. Like, a person who has a gardening background has to get into horticultural programs or gardening programs and all that. If staff are happy, that’s what’s passed on. I think if you’re not content inside, if you’re not happy inside, you will pass on that negative kind of thing. (Priya, frontline manager)

Staff who are able to strategise; it’s dealing with problems along the way that may occur, complex issues that may come up. (Tanya, general manager)

Whether or not they’re [staff are] committed to supporting the client through thick and thin. You definitely can recognise staff that are committed to a client. The supports different. The respects different. (Natasha, quality manager)

Overall approaches to quality: process compliance vs service users experience of support

When the viewpoint of each leader was mapped against each of the three topics, as illustrated in Table 1, a pattern of two overarching approaches to quality emerged. The first was a “process compliance approach” in which leaders perceived service quality to be about complying with procedures and standards, using secondary evidence such as records and statistics to monitor service quality and saw external and indirect factors as influencing quality. The second approach was “service user’s experience of support” in which leaders perceived service quality to be about the way staff engaged with the people they supported, used real time observations of staff practices or direct report from service users to monitor quality, and saw direct and internal factors as influencing service quality.

Table 1. Leaders’ perceptions of service quality, how it is monitored and what influences it.

Process compliance approach to service quality				Service users experience of support approach to service quality
	Perception of service quality Policies and procedures	Monitoring service quality Secondary evidence	Influences on service quality Indirect and external	Perception of service quality Personal outcomes	Monitoring service quality Primary evidence	Influences on service quality Direct and internal
Joe, chief executive officer (Org 3)	X	X	X
Nathan, general manager (Org 1)	X	X	X
Elle, quality manager (Org 3)	X	X	X
Amy, manager (Org 3)	X	X	X
Ron, chief executive officer, (Org 2)				X	X	X
Tanya, general manager (Org 2)				X	X	X
Natasha, quality manager, (Org 1)				X	X	X
Priya, manager (Org 3).				X	X	X

The overarching approach of leaders to service quality did not appear to be associated with formal organisational positions. As Table 1 shows, leaders occupying similar positions in different organisations held contrasting views. Furthermore, leader’s approach to quality did not appear to be associated with their particular organisations, as several individuals who worked in the same organisation held contrasting approaches.

Discussion

Findings revealed two overarching approaches to service quality amongst leaders: a process compliance approach and a service user’s experience of support approach. Leaders who were orientated towards a process compliance approach perceived service quality as something that could be broken down into measureable parts and monitored systematically through a process of comparing evidence to standards or “frameworks.” These findings reflect the type of reductive, systematic approach to service quality found in Total Quality Management. The Total Quality Management approach has been consistently used by the Australian government to monitor service quality in organisations for people with intellectual disabilities for decades (McEwen et al., 2014). The alignment between some leaders with a process compliance approach to service quality and government methods for regulating organisations for people with intellectual disabilities, suggests some day-service organisations have a strong compliance culture.

Nevertheless, half of the leaders who participated in this study saw service quality as broader than compliance. These leaders had a service user’s experience of support approach to quality and described using observational methods to monitor the support individuals receive. Their views were similar to those of some researchers (Mansell, 2011; Beadle-Brown et al., 2008; Schalock, 2001), who have advocated for the use of observational methods to monitor service quality and see these as being particularly effective for people who have difficulty communicating with others without assistance. Their approach to service quality was also similar to that found by Deveau and McGill (2016) among accommodation service leaders who believed using observational methods allowed them to both better understand service users’ needs and characteristics and to monitor the quality of the support they received.

In terms of characteristics, leaders with a service users experience of support approach to service quality had more direct experience of providing frontline support than those with a process compliance approach who had come into management from other sectors such as finance, psychology, and physical education. Having had greater direct experience of providing frontline support may have made leaders more aware of the impact that moments of support and interaction with staff have on service users’ quality of life. It is also possible that leaders with more frontline experience better understood the limitations of relying on documentation written by staff such as file notes as a record of what had actually happened during the course of service provision. Some research has suggested that such documents often fail to adequately capture people’s day-to-day experiences, needs, and characteristics (Poppes et al., 2014) and may portray what staff felt managers wanted to hear about what happened in services (Quilliam et al., 2018). Similarly, leaders with a service-users experience of support approach may also have invested in observational methods because they understood that organisational policies and procedures did not necessarily ensure that people with intellectual disabilities received good quality services (The Senate, Community Affairs References Committee, 2015).

Important limitations of this research were its small exploratory nature and that it was conducted in day-service organisations located across one Australian city. Further research is required to examine whether these findings resonate with leaders in other organisations for people with intellectual disabilities and are relevant to disability service organisations in other Australian states.

Conclusion

The two approaches to service quality identified among leaders in this study were not dissimilar to the differences between the way the Australian government has regulated service quality in disability service organisations, and the way some researchers have defined and monitored service quality. The approaches are at opposite ends of a spectrum, with process compliance at one end and the way service users experience support at the other. It is clear that service quality is a topic that requires more exploration across the disability sector, both at a practice and government level. Particular focus should be placed on how organisational processes and the experience of service users can be adequately monitored in ways that integrate a systems approach and best practice recommendations from research. Consideration should be given to merging these approaches and creating indicators that incorporate both observation and process review methods.

Acknowledgements

The authors would like to thank the organisations and individuals who participated in this research.

Notes

1 Cat 1s, 2s and 3s refers to a category reporting system where funded disability services were responsible for notifying the Department of Health (a government agency) of situations where service users had experienced an adverse outcome. For example, an injury or medication error (Department of Health, 2013).