Psychometric properties of a modification of the Safety Attitudes Questionnaire (SAQ) for child protection services

ABSTRACT

The concept of a safety culture is crucial to improving organisational risk management in several sectors, including health and aviation. However, social and welfare services are also sectors where organisations should be aware of the risk in their daily practice. The aim of the present article is to confirm the usability of a modified Safety Attitudes Questionnaire (SAQ) in the Norwegian child protection services CPS by checking a number of its psychometric properties. The SAQ is an instrument that has been widely applied and understood as valid and trustworthy for mapping employees’ views of their organisation’s safety culture. A confirmative factor analysis of data from a national survey of Norwegian CPSs demonstrated that the modified questionnaire has a factor structure which is internally consistent and matches the structure of the widely used SAQ health care questionnaire. The results indicate that the CPS-modified SAQ questionnaire could help understanding and mapping safety culture in the child protection services.

ABSTRAKT

Sikkerhetskultur har vist seg å kunne være avgjørende for å forbedre organisasjoners evne til å håndtere risiko i flere sektorer, som blant annet helsetjenester og luftfart. Sosial- og velferdstjenester er også sektorer der organisasjoner bør være oppmerksomme på risiko i daglige praksis. Målet med denne artikkelen er å vurdere psykometriske egenskaper i spørreskjemaet Safety Attitudes Questionnaire (SAQ) modifisert for norske barneverntjenester. SAQ er et instrument som har blitt mye brukt og som er vurdert som gyldig og pålitelig for å kartlegge ansattes syn på organisasjoners sikkerhetskultur. Konformativ faktoranalyse av data fra en nasjonal undersøkelse fra Norske barneverntjenester viste at det modifiserte spørreskjemaet har en faktor struktur som er internt konsistent og samsvarer med strukturen til det mye brukt SAQ spørreskjemaet for helsevesenet. Resultatene indikerer at det modifisert SAQ spørreskjema kan hjelpe forståelse og kartlegging av sikkerhetskultur i barnevernet.

KEYWORDS:

• Child protection services
• safety culture
• error management
• Safety Attitude Questionnaire
• Norway

NØKKELORD:

• Barnevern
• sikkerhetskultur
• håndtering av feil og mangler
• Safety Attitude Questionnaire
• Norge

Background

Daily, child protection services’ (CPS) employees find themselves in situations where they make decisions about children’s, youth’s and family’s lives. These decisions are always connected to the risk of making incorrect decisions that could lead to the complication of family relations, serious maltreatment, and even death. Unfortunately, there have been some cases in Norway where CPS decisions had fatal outcomes (Kirkengen & Thornquist, 2013).

Insufficient decisions can be misunderstood as reflecting individual social workers’ incompetence or being unfortunate single incidents (Munro, 2019a). According to Sicora (2018), problems in CPS practice are nearly always related to organisational culture and system issues (Munro, 2019b). This understanding is also supported by various reports from the Norwegian Board of Health Supervision, which has documented that mistakes in the CPS are often no single incidents but practices being repeated time and again (Slettebø et al., 2019). Consequently, improvements to make the actors perform better may turn the focus from individual performance in specific cases to an organisational culture of preventing and handling inefficacy and mistakes (Reason et al., 2001). The lack of attention to safety culture in practice and decision making may have negative consequences, such as an increased culture of blame, in which individual persons or single services are blamed (Lachman & Bernard, 2006). This undermines psychological safety, which is required to learn from adverse events and organisational failures (Edmondson, 2018). The relevance for focusing on CPS organisations and a safety culture is also supported by research from several other countries. Studies have described a lack of culture to learn from mistakes and accountability as an issue in CPS (Munro, 2019a; Sicora, 2018; Sicora et al., 2020; Vogus et al., 2016). In the development of CPS, internal control systems and capacity building have been discussed, but the development of supportive learning cultures is also key (Sicora, 2018), here with a focus on the safety and working conditions of employees (Munro, 2005). Drawing on the organisational safety culture literature from other high-risk sectors, Munro (2019a) calls for a positive and proactive culture to improve services in child protection, including proactive learning from mistakes.

Senge (2010) defines learning organisations as those that are adaptive and develop a culture of innovative learning that facilitates continuous reflection and growth. A learning culture in CPS implies that the services create, transfer, and use knowledge and experiences to achieve positive changes and realise their goals from practise and decisions that went wrong. We can find important support for the value of safety culture in a study from the US showing that young people who receive services from CPS with a well-functioning organisational culture increase the chances for better outcomes for youths (Glisson & Green, 2011).

In other sectors such as the military, aviation and hospitals, this perspective has been well established. It has been proven that organisations that systematically work to improve their safety culture have reduced risk for errors (Braithwaite et al., 2017). Many organisations apply surveys to map the safety culture as a part of their organisational development. However, CPS lacks the tools for evaluating safety cultures. There are several established instruments for mapping safety culture in services; one of the most well-known is the Safety Attitude Questionnaire (SAQ) (Sexton et al., 2006), which has been widely applied for building safety culture in hospitals and healthcare services (Deilkås & Hofoss, 2008; Nordén-Hägg et al., 2010). Internal validation of the SAQ has been performed in several studies. Some have also reported that the questionnaire is correlated with patient outcome (Colla et al., 2005; Deilkås & Hofoss, 2008). Therefore, questionnaires can be an important tool in health services to map safety concerns and improve safety culture. In the present article, we examine the validity and reliability (psychometric properties) of the SAQ adapted to CPS. The paper also discusses the possibilities of implementing the SAQ in CPS to strengthen the organisational learning culture and user quality of their services.

It seems that CPS services both in Norway and elsewhere lack knowledge of how to map safety concerns in services that can provide knowledge of how the organisation can facilitate systematic work and prevent recurring mistakes (Lachman & Bernard, 2006; Zuchowski et al., 2019).

Safety culture, knowledge from aviation and hospitals that can have relevance for CPS

Helmreich and Merritt’s (2001) book Culture at Work exemplifies how safety and error management can improve by working with different aspects of organisational culture and professional attitudes and values. Deilkås et al. (2019, p. 2) state that the safety culture as a culture ‘involves leader and staff interactions, attitudes, routines, awareness, and practices that influence risks of adverse events’ in a health service setting. Deilkås et al. (2019, p. 2) further describe, ‘A cultural trait may, for example, be how leaders facilitate dialogue with staff to uncover negative relationships and behaviour, promote mutual understanding of the causes of adverse events, and establish consensus regarding safety priorities’. This is also a relevant situation for CPS.

The report, To Err is Human: Building a Safer Health System (Kohn et al., 1999), explicitly describes how failures and mistakes in American health services lead to patient injuries. These failures are often connected to employees’ work situations and the factors connected to internal organisational services, such as lack of routines, work stress or conflicts, or lack of collaboration between professionals. The report has contributed to an intensified focus on patient safety in hospitals.

Aviation and surgery are probably more standardised globally than CPS. CPS has major differences in its mandate and structure, both within national communities and worldwide. However, the question of how to help organisations and professionals learn from unsuccessful interventions and build and maintain a safety culture is relevant across sectors and national borders. Health services have several instruments to measure employees’ safety concerns and attitudes (Singla et al., 2006) that might be useful for CPS. The measurements of perceptions of safety culture shared among staff in an organisational unit are referred to as safety climates (Zohar, 2014). Valid safety climate measurements can identify levels of staff perceptions, the extent to which they are shared and how they vary across organisational units (Alsalem et al., 2018). Variations in such assessments may predict variability in risk related to tasks, work environment, staff behaviour and the outcome for users (Braithwaite et al., 2017).

Safety Attitude Questionnaire (SAQ)

Sexton et al. (2006) work on organisational culture has been transformed into the SAQ (Sexton et al., 2006), a questionnaire for mapping employees’ and leaders’ views of organisational safety culture and concerns. The short version of the SAQ has been modified and applied globally in many services, but it is most commonly used to measure and facilitate safety culture within hospitals (Deilkås & Hofoss, 2008). The SAQ has also been applied and found to have acceptable validity in other health care services, for instance, in nursing homes (Bondevik et al., 2019; Deilkas et al., 2019), pharmacies (Nordén-Hägg et al., 2010) and general practitioner practices and emergency rooms (Deilkås et al., 2019).

The questionnaire focuses on six defined factors in organisational culture that have an impact on how employees manage situations in which there is a risk of defective or erroneous action: safety climate, teamwork climate, working conditions, job satisfaction, stress recognition and perceptions of safety management. Research has confirmed the relevance of the factor structure of the SAQ short version in Norwegian hospitals (Deilkås & Hofoss, 2008).

Parts of the SAQ questionnaire have also been applied earlier in CPS (Munro, 2019b; Vogus et al., 2016). However, to the best of our knowledge, there is no study in which the questionnaire has been systematically modified and tested in a CPS setting. Hence, the aim of the present study was to modify and adapt SAQ for CPS settings and to test its psychometric properties.

Materials and methods

Participants and setting

The present study was based on a study of system accountability in CPS, in which all Norwegian child protection workers working frontline were invited to participate. The SAQ items were added at the end of the questionnaire. This article focuses on confirmatory factor analysis and include only the SAQ-items included in the survey.

The Norwegian CPS is organised into municipalities, and no central registry of employees exists. Therefore, an inquiry was sent to all Norwegian chief municipal executives asking for permission to invite their CPS employees to participate. The municipality administrators sent our research team a list of the e-mail addresses of relevant employees. We received 2844 valid e-mail addresses of staff employed in professional positions in municipal CPS. All received invitations at their work e-mail address to participate in a Questback questionnaire. Reminders were sent to those who did not respond. Individual invitations to the questionnaire ensured that the appropriate individuals replied, that no one replied more than once and that participation was anonymous for CPS leaders and colleagues.

Measures

The SAQ short version

The items in the latest nongeneric short version of the SAQ consist of 36 items grouped into six factors (Sexton et al., 2006): teamwork climate (6 items), safety climate (7 items), job satisfaction (5 items), stress recognition (4 items), perceptions of management (5 items, measured at levels: unit and hospital) and working conditions (4 items). Five of the items in the questionnaire were not included in any factors and, therefore, were irrelevant to the confirmatory factor analysis. The complete questionnaire with factors is presented in the first column of Table 1.

Table 1. Original SAQ questionnaire and Norwegian CPS version (translated to English by the authors).

Original SAQ questionnaire (factor)	Norwegian CPS version translated into English (hypothesised factor)
1. Nurse input is well received in this clinical area (Teamwork Climate)	We have a culture where we all take the responsibility for difficult decisions (Teamwork Climate)
2. In this clinical area, it is difficult to speak up if I perceive a problem with patient care (Teamwork Climate)	At work, it is difficult to speak up if I perceive a problem with providing services that serve the best interest of the child (Safety Climate)
3. Disagreements in this clinical area are resolved appropriately, i.e. not who is right, but what is best for the patient (Teamwork Climate)	Disagreements between child protection workers are resolved appropriately, i.e. not who is right, but what is best for the clients (Safety Climate)
4. I have the support I need from other personnel to care for patients (Teamwork Climate)	I have the support I need from colleagues to handle my work obligations (Teamwork Climate)
5. It is easy for personnel here to ask questions when there is something that they do not understand (Teamwork Climate)	It is easy for personnel here to ask questions when there is something they do not understand (Safety Climate)
6. The physicians and nurses here work together as a well-coordinated team (Teamwork Climate)	The child protection workers work together as a well-coordinated service (Teamwork Climate)
7. I would feel safe being treated here as a patient (Safety Climate)	I would feel safe being treated here as a client (Safety Climate)
8. Medical errors are handled appropriately in this clinical area (Safety Climate)	Mistakes and shortcomings are handled appropriately in this CPS (Safety Climate)
9. I know the proper channels to direct questions regarding patient safety in this clinical area (Safety Climate)	I know the proper channels to direct questions regarding clients’ safety in this CPS (Safety Climate)
10. I receive appropriate feedback about my performance (Safety Climate)	I receive appropriate feedback about my performance (Safety Climate)
11. In this clinical area, it is difficult to discuss errors (Safety Climate)	In the CPS, it is difficult to discuss errors (Safety Climate)
12. I am encouraged by my colleagues to report any patient safety concerns I may have (Safety Climate)	I am encouraged by my colleagues to report any client safety concerns I may have (Safety Climate)
13. The culture in this clinical area makes it easy to learn from the errors of others (Safety Climate)	The culture in this child protection service makes it easy to learn from the errors of others (Safety Climate)
14. My suggestions about safety would be acted upon if I expressed them to management (Not part of the scale)	Question excluded, not part of the scale
15. I like my job (Job Satisfaction)	I like my job (Job Satisfaction)
16. Working here is like being part of a big family (Job Satisfaction)	This workplace offers a good social environment for employees (Job Satisfaction)
17. This is a good place to work (Job Satisfaction)	This is a good place to work (Job Satisfaction)
18. I am proud to work in this clinical area (Job Satisfaction)	I am proud to work in this CPS (Job Satisfaction)
19. Morale in this clinical area is high (Job Satisfaction)	Employees in this child protection service have high job morale (Teamwork Climate)
20. When my workload becomes excessive, my performance is impaired (Stress Recognition)	Question excluded; not found relevant for CPS
21. I am less effective at work when fatigued (Stress Recognition)	Question excluded; not found relevant for CPS
22. I am more likely to make errors in tense or hostile situations (Stress Recognition)	Question excluded; not found relevant for CPS
23. Fatigue impairs my performance during emergencies (Stress Recognition)	Question excluded; not found relevant for CPS
24. Management (unit/hospital management) supports my daily effort (Perceptions of Management)	Management (unit management/municipality level) supports my daily efforts (Perceptions of Management)
25. Management (unit/hospital management) does not knowingly compromise patient safety (Perceptions of Management)	Management (unit management/municipality level) does not knowingly compromise client safety/ the quality of the professional work (Perceptions of Management)
26. Management (unit/hospital management) is doing a good job (Perceptions of Management)	Management (unit management/municipality level) is doing a good job (Perceptions of Management)
27. Problem personnel are dealt with constructively by our management (unit/hospital management) (Perceptions of Management)	Management (unit management/municipality level) follows up employees who do not perform their duties satisfactorily (Perceptions of Management)
28. I get adequate, timely info about events that might affect my work from management (unit/hospital management) (Perceptions of Management)	I get adequate, timely information about events that might affect my work from management/from management (unit management/municipality level) about important work-related management conditions that might affect my work (Perceptions of Management)
29. The levels of staffing in this clinical area are sufficient to handle the number of patients (Working Conditions)	The levels of staffing in this service are sufficient to handle the workload (Working Conditions)
30. This hospital does a good job of training new personnel (Working Conditions)	This child protection service does a good job of training new personnel (Working Conditions)
31. All the necessary information for diagnostic and therapeutic decisions is routinely available to me (Working Conditions)	All the necessary information to make decisions in my work is routinely available to me (Working Conditions)
32. Trainees in my discipline are adequately supervised (Working Conditions)	We are adequately supervised (Working Conditions)
33. I experience good collaboration with nurses in this clinical area (Not part of the scale)	Question excluded; Not part of the scale
34. I experience good collaboration with staff physicians in this clinical area (Not part of the scale)	Question excluded; Not part of the scale
35. I experience good collaboration with pharmacists in this clinical area (Not part of the scale)	Question excluded; Not part of the scale
36. Communication breakdowns that lead to delays in delivery of care are common (Not part of the scale)	Question excluded; Not part of the scale

Modification of SAQ and hypothesised factor structure

Transforming a questionnaire from one sector to another includes changes in the terms and context-related items. Changes in factor structure could also be considered. Such changes should be theoretically based on arguments, previous research or advice from people with advanced knowledge of the field.

The first and second authors developed drafts for the questionnaire based on the English short version of the SAQ for hospitals. We consulted an expert panel comprising practitioners, bureaucrats, researchers, and service users from CPS to benefit from their experience. To modify the SAQ for use in a CPS context, particular emphasis was placed on adapting the wording. For example, we had to develop appropriate concepts for CPS and replace words from the health context, such as nurses, doctors, hospitals and patients. Some items were edited to fit the context.

In total, four items in the short version for hospitals of the SAQ were not found applicable for conversion to the CPS context and, therefore, were excluded. Those four items were all part of the ‘stress recognition’ factor, which neither the researchers nor the expert panel considered appropriate in the CPS context.

The questionnaire was also piloted in a child welfare service in which the employees had the opportunity to comment on content and structure. They also suggested removing the ‘stress recognition’ factor. In addition, their feedback was related to some minor language changes, and they stated that the questions were relevant and easy to answer for CPS workers.

Like the original short SAQ version for hospitals, our questionnaire applies a 5-point Likert-type scale that ranges from 1 = disagree strongly to 5 = agree strongly for all items. The participants were informed that they could leave the items blank.

The questionnaire presented in the present article was translated from Norwegian to English. The translation was done by the first author and verified by the other authors.

In the workshop groups, the practitioner was given the modified items and asked to place them in the right factor (teamwork climate, safety climate, job satisfaction, perceptions of management and working conditions). Most of the factor structure from health was confirmed to be relevant for CPS. The researchers discussed and negotiated where an item was scored as belonging to another factor than in the generic version. Item 19, which focuses on morale, was moved from the job satisfaction factor to teamwork climate. The researchers’ conclusion was to include items that refer to everyday collaboration in the teamwork climate factor and those that relate more specifically to managing or avoiding risk in the safety climate factor. This led to three items (2, 3 and 5) being moved from teamwork climate to safety climate because these were found to be more closely related to concerns about communication and risks in practice than to general team collaboration. In practice, our hypothesis is based on the idea that teamwork is not so much related to direct risk in CPS because there is probably less direct collaboration than in cockpits, hospital wards and operating rooms. Teamwork is still highly relevant for ensuring safety in CPS by involving colleagues as discussion partners or co-workers in difficult or risky cases. There was no need for any changes in the items related to the factors of working conditions and perceptions of management. The modified questionnaire with a hypothesised factor structure is presented in Table 1, second column.

Sociodemographic variables

The questionnaire includes several sociodemographic variables, such as gender, age, profession, education, position, CPS work experience and size and geographical spread of CPS. These items are applied in the analysis for knowledge about the dataset’s representativity.

We also asked questions about sick leave and future ambitions, which are relevant for a wider understanding of the respondents’ views and work situations. These items are not included in the current article but can be relevant data for further understanding the work and safety culture.

Data processing – confirmatory factor analysis: internal construct validity

The confirmatory analysis of the SAQ was performed using the software AMOS, which is an SPSS extension. SPSS was applied to estimate Cronbach’s alpha, item-to-own correlations, intercorrelations of factors, test–retest correlations and all item-descriptive statistics. An indicator of an item not belonging where it was hypothesised to be included is whether its removal would markedly improve that variable set’s Cronbach’s alpha. Cronbach’s alpha scores between 0.70 and 0.90 were considered good and above 0.60 as acceptable (Tavakol & Dennick, 2011).

The hypothesised five-factor model was tested by confirmatory factor analysis (CFA) in AMOS. Because factors reflect the correlation structure in the item responses, valid factors should reflect a thematic logic that is coherent with the purpose of the questionnaire. CFA provides goodness-of-fit indices that measure how well the factor pattern of the survey responses conformed to the prehypothesised factor model.

The following goodness-of-fit indices were calculated: the ratio of the Chi-square to the model’s degrees of freedom (χ²/d.f.), the model’s p-value, the root mean squared error of approximation (RMSEA), the comparative fit index (CFI), the Tucker–Lewis Index (TLI) and the Hoelter 0.05. The χ²/d.f. should be below 5 (Wheaton et al., 1977); the p-value should exceed 0.05 (Joreskog, 1969); the RMSEA should not exceed 0.08 (Browne & Cudeck, 1993); and the CFI and TLI should exceed 0.90 (Jackson et al., 2009). The Hoelter 0.05 – an estimate of the largest sample for which the model in question would fit the data – should exceed 200 (Joreskog, 1969).

Ethical considerations

The present study was approved by the Norwegian Social Science Data Services, which is the government agency responsible for protecting the privacy of respondents participating in survey research, here according to the Norwegian Personal Data Act (Ref. No. 2015/42892). Informed consent was obtained, and the participants could withdraw from the study at any time.

Results

Response rates and demographic variables

An invitation to participate was sent to 2844 individuals, and 1429 returned the web-based survey. A total of 149 responses were rejected because of incomplete questionnaires. The final nationwide sample comprised 1280 participants, which constituted 45% of the original number invited to participate.

The sample’s sociodemographic variables were compared with register data about CPS from Statistics Norway (Statistikkbanken - barnevern, 2017). Distribution of important parameters such as gender, education, number of employees at the offices and national geographic spread in the survey were found to be close to representative for the population.

Table 2 lists the respondents’ education data and a comparison with the data from Statistics Norway. From the table, it can be seen that the educational backgrounds of the sample and the population were almost identical.

Table 2. All CPS workers in Norway compared with the respondents.

	Population (Statistic Norway, 2017); N (%)	Sample; N^a (%)
Child welfare educators	2827 (50)	642 (51)
Social workers	1656 (29)	411 (33)
Other college or university education	992 (18)	197 (16)
Other education/no education	165 (3)	13 (1)
Total	5640 (100)	1263^a (100)

^a 17 of 1280 missing.

Understanding organisation mapping views from those individuals with different positions in the organisation is important. Figure 1 illustrates the respondents across positions in CPS. We have not been able to find data that give a fair comparison of responses compared with the general population of CP employees. However, analysis of the invited versus sample indicates that leaders have higher response rates than caseworkers.

Figure 1. Participants’ positions in CPS.

Data quality

The number of participants is considered acceptable for exploring the psychometric properties of the questionnaire. A commonly used rule of thumb is that adequately sized samples have cases-to-indicator ratios of at least 10 or 20 (Kyriazos, 2018) – our ratio was 1280/27 ≈ 47.

Missing answer was for all items, from 2.3% to 4.7%. The highest percentage of missing items was related to perceptions of management. This is not necessarily an indication that the question was unclear or that the informants would not score their management. It could also be a natural consequence that some employees probably never met their top leader and, therefore, did not answer this. It is known that turnover is high in CPS (Statistikkbanken - barnevern, 2017) and that many respondents (11.6%) have worked in the respective CPS for less than one year.

Cronbach’s alphas – internal consistency

The Cronbach’s alphas of our five factors are as follows: perception of management (5 items, alpha = .878), safety climate (10 items, alpha = .867), teamwork climate (4 items, alpha = .756), job satisfaction (4 items, alpha = .869) and work conditions (4 items, alpha = .733).

For no factor, the exclusion of any variable would noticeably increase the α-value.

Construct validity – confirmatory factor analysis (CFA)

We tested the factor structure using CFA. Our factor structure model is presented in Figure 2.

Figure 2. Factor structure of generic re-hypnotised version of SAQ for CPS (variable number in original SAQ questionnaire (see Table 1)).

CFA showed that the hypothesised total model, which grouped the 27 indicators under five factors, that is, teamwork climate, safety climate, job satisfaction, perceptions of management and working conditions, fit the data adequately. The goodness-of-fit indices for the model are presented in Table 3.

Table 3. Goodness-of-fit indices for generic re-hypnotised version of SAQ for CPS.

	χ^2/d.f.	p	RMSEA	Hoelter 0.05	CFI
Entire model	1898.674/314 = 6.047	<.000	.063	241	.908

Cut-off: χ²/d.f.: should be below 5; p: should exceed 0.05; RMSEA: root mean square error of approximation should not exceed 0.08; Hoelter 0.05: should exceed 200; CFI: should exceed 0.90.

Discussion

The current article has presented a psychometric analysis of the SAQ instrument, modified and translated to the Norwegian CPS context. The Cronbach’s alpha values from .733 to .869 are good. In general, the current paper shows acceptable Goodness-of-Fit values for the hypothesised five-factor model (perception of management, safety climate, teamwork climate, job satisfaction and work conditions). The sample, despite its low response rate of 1280/2844 = 45%, is probably close to being representative of important parameters, such as gender, education, number of employed and national geographic spread.

The results indicate that the questionnaire provides an opportunity to measure CPS employees’ views of organisational safety culture and concerns that are well-known in other sectors.

Today, hospitals and health services have established a common understanding of that patient safety (Halligan & Zecevic, 2011); this processes to build organisations that are aware of risks and avoid patient injuries are crucial for daily practice. Pronounced and active safety cultures probably also influence users’ trust in organisations services. In CPS, lack of clarity of risk could make it hard to identify and build learning organisations that handle risks and errors. In the modifying process of the questionnaire, with input from both an expert panel and a pilot test, we have seen that CPS also need concepts that differ from health services, although most of the items from the health service questionnaire were possible to modify. The questionnaire presented in this article could be relevant for providing knowledge for the ongoing discussions to obtain a more common understanding of how to define errors in CPS and the crucial elements for building a safety culture.

A main strength of the SAQ is the possibility of comparing different workplaces and institutions. This has been done across national borders and sectors. For instance, in Norwegian hospitals, the same questionnaire is used annually to map and compare different hospital and hospital wards (Deilkås & Hofoss, 2010). This annual comparison has aimed to understand which services are successfully building a safety culture and which are at a high risk. This has provided knowledge on how to learn from others and reduce the chance of risk in all national hospitals. Today, there is no instrument to conduct such a comparison in Norwegian CPS. The modified questionnaire presented in the present article could be a starting point to map and compare safety culture in CPS too.

The questionnaire could probably be improved by conducting further surveys and modifications. However, we have found it likely that the questionnaire in its current form could be applied in the organisational development process for understanding how CPS operates concerning the conditions for safeguarding quality and avoiding repetitive mistakes in client work.

An important lesson to be learned from safety culture management is that risk should be understood as an organisational issue (Helmreich & Merritt, 2001). Instead of blaming individuals for failures and mistakes, one should use the incidents and adverse events for organisational learning. From the dialogue with the expert panel in the modification process of the questionnaire, we were left with the impression that the responsibility for mistakes in CPS is unclear. In the evaluation of British CPS, Munro (2005) identifies a potential blame culture where case workers and services are at risk of being characterised by lack of competence, assessment ability or skills in following manuals, routines or the legal system when mistakes happened. These cultures do not stimulate organisational learning. Sicora (2018) also describes blame culture as an obstacle to building responsible services in Italy. Theory from safety culture and SAQ could be relevant in exploring how responsibility is placed in CPS and how this supports safety culture.

SAQ supports the improvement of organisational culture in an evidence-based way, as in other sectors. In CPS, a debate of the evidence has had a different orientation from country to country (Gilbert et al., 2011) and has often been focusing on measures to clients and less on organisation building. Research strongly indicates that organisation culture can impact client’s outcome from services (Glisson & Green, 2011; Williams & Glisson, 2013a, 2013b). To support more evidence of how to build well-functioning CPS, we need studies that openly share the questionnaire and give readers knowledge of how the questionnaire is organised, including its strengths and weaknesses. Regarding the fact that CPS can be seen as very different depending on the location, the country-to-country application of this questionnaire should be done with a critical stance. Evidence is context dependent, but we believe that more awareness of safety culture, including applying questionnaires like SAQ, has universal relevance for improving services. For leaders and employees in CPS, the present article and the validated questionnaire also give an evidence base for understanding and discussing organisation and safety concerns.

Limitations

The analysis confirmed the internal validity of the modified questionnaire. The external validity of the CPS instrument was not explored. Further research should explore external validity by acquiring knowledge about connections between cultural elements in the workplace and quality of services provided to users, hereby documenting the important key elements for facilitating a safety culture in CPS and by exploring the eventual correlation between these elements.

Conclusions

The current study has presented a modification of the SAQ for CPS. The analysis shows an acceptable factor structure. From a wider perspective, the presented paper discusses how knowledge from the safety culture in healthcare may improve CPS. The discussion illustrates that both the questionnaire and transformation of the framework are relevant to understanding and developing CPS.

Acknowledgements

We want to thank research studentassistant Simon Molvær and his team for distributing the questionnaire. Many thanks to the expert panel who participated in the quality assurance of the questionnaire and the employees who participated in the pilot survey.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings are available from the authors and VID Specialised University, subject to the restrictions laid down by the Norwegian Social Science Data Services.

Additional information

Funding

The Norwegian Directorate for Children, Youth and Family Affairs initiated and funded the survey development and data collection.

Notes on contributors

Erik Skjeggestad

Erik Skjeggestad, is an Associate Professor(PhD) and Programme Coordinator at the Faculty of Social Studies at VID scientific university. He has work in child protection services as social worker in several years. His field of research are child welfare and professional collaboration. He has a special interest in service and quality development in child protection services.

Torgeir Sørensen

Torgeir Sørensen is an Associate Professor at the at the Faculty of Health at VID Specialized University. He holds a PhD in the field of psychology of religion, and he has performed studies investigating relationships between religion and health, as well as qualitative studies on the significance of meaning-making for people in demanding life situations. Additionally, he has worked with translation and validation of standardized instruments for the Norwegian context.

Dag Hofoss

Dag Hofoss, PhD, Professor (University of Oslo (ret)). Health services research, sociology of professionalization, inequalities in health, quality of life, patient satisfaction and experiences with health care, standardization of care, safety culture in health care, psychometric properties of patient and employee questionnaires, service monitoring methods, quality improvement methods.

Tor Slettebø

Tor Slettebø is a professor in social work at the Faculty of Social Studies at VID scientific university with a long-lasting research background. His field of research is user participation in health and welfare services, and in particular in child protection. He has a special interest in service and quality development in child protection services and how they can develop into learning organisations. He was the leader of the project out of which this article stem from.

Ellen Tveter Deilkås

Ellen Tveter Deilkås, is a specialist in internal medicine with 30 years of clinical practice, and a PhD in patient safety culture. She works as a senior researcher on patient safety and part time as a clinical consultant in an ambulatory multidisciplinary rehabilitation clinic at Akershus University hospital. She has worked both full time and part time as an advisor for the Norwegian government since 2010, designing, establishing and developing national measurements for adverse events and patient safety culture.