Development of a new measurement scale for interprofessional collaborative competency: The Chiba Interprofessional Competency Scale (CICS29)

ABSTRACT

The purpose of this study was to refine the items on a scale measuring interprofessional collaborative competency that was developed by the authors in an earlier pilot study. A questionnaire-based study was conducted with a sample of 2133 health professionals using the reformulated questionnaire. Construct validity was tested by comparing the survey results with a covariance structure analysis and the domains of interprofessional collaboration competencies presented in previous studies. A second survey was conducted 2 weeks later with a sample of 571 nursing professionals, using the same survey form to test its reliability. We constructed a model comprising 29 observed variables and six latent variables (the Chiba Interprofessional Competency Scale: CICS29), and obtained the following values for the model’s goodness of fit: GFI = 0.925, AGFI = 0.908, CFI = 0.950, RMSEA = 0.049. With regard to reliability, we obtained scores ranging from 0.65 to 0.77 for the intraclass correlation coefficients of the six domains. Compared with the interprofessional collaboration scales indicated in previous studies, the CICS29 was found to have subsumed the key concepts that should be configured as interprofessional collaboration competencies. The CICS29 appeared to have satisfactory levels of reliability and validity and is recommended as a scale for measuring competencies of interprofessional practice.

KEYWORDS:

• Collaborative practice
• interprofessional education
• quantitative method
• scale development

Introduction

The value of interprofessional education (IPE) has been recognised globally. The Centre for the Advancement of Interprofessional Education (CAIPE) defines IPE as “occurring when two or more professions learn with, from and about each other to improve collaboration and the quality of care” (CAIPE, 2002). Highlighting the issue of “tribalism” among health professionals, Frenk et al. (2010) noted the pressing need to redesign professional health education from the perspective of opportunities for mutual learning and joint solutions to health problems in the 21st century. Both the formal and informal aspects of IPE should be continually studied across the full spectrum of undergraduate, graduate, and continuing professional development studies (Institute of Medicine, 2015). The learning outcomes from these studies must go beyond the mere acquisition of knowledge, and manifestation of competency is necessary in the form of practical behaviours.

The current forms of health-related professional learning have shifted towards being competency based, and the embodiment of each individual’s competency has become an important concern among educators.

The five cross-cutting strategies for reforming professional education in the healthcare system in the 21st century have led to a consensus on ‘what are the fundamental abilities’. Moreover, they have prompted the development of an educational approach and curriculum based on a foundation upon which these abilities can be acquired (Royeen, Jensen, & Harvan, 2009).

In the UK, Canada, the USA, and Japan, there has been a consensus on strategy- or value-based competencies in interprofessional collaborative practice. However, no inventory or scale currently exists wherein competency in interprofessional collaborative practice is expressed as a measurable objective indicator. Barr (1998) defined professional competency as comprising “common”, complementary, and collaborative competencies. The expert panel of the Interprofessional Education Collaborative (IPEC) published a report in 2011, entitled Core Competencies for Interprofessional Collaborative Practice, identifying four competency domains for interprofessional collaborative practice (IPEC Expert Panel, 2011). The Canadian Interprofessional Health Collaborative (CIHC) also published a report in 2010, entitled A National Interprofessional Competency Framework, identifying six domains of competency (CIHC, 2010). Thistlethwaite and Moran (2010) distilled six learning outcomes following a review of relevant studies. While these competencies and learning outcomes are very well organised at a conceptual level, none of them are constructed on the basis of objectively measured data. Therefore, we have developed a scale that is both valid and reliable, based on the assessment items we obtained concerning interprofessional work (IPW) by conducting behavioural result interviews with high-performing healthcare professionals (Yamamoto et al., 2014). This new inventory asks whether the participant ‘actually performs an action’, and it comprises objective questions as a part of the scale for self-assessment. Our previous pilot study examined the construct validity and surface validity of the scale and investigated its internal consistency (alpha reliability).

This study resolved the problems with the scale that were observed in the previous pilot study and refined the assessment items with the particular aim of increasing its reliability and construct validity. Of the 35 items, seven had a high factor loading over multiple factors. Therefore, we considered it especially necessary to revise these seven items. These aspects were verified using a relatively large-scale sample, thus ensuring the scale’s reliability and validity. This scale contributes to the creation of robust evidence related to the outcomes of IPE and IPW. The evidence is obtained by addressing the following questions: how the IPE continuum improves an individual’s competencies in interprofessional collaborative practice, how an increase in an individual’s competencies contributes to their team’s performance, and how the workplace or work environment in the healthcare system affects individual acquisition of the competencies. In this study, our aim was to reassess the contents of the 35-item scale that was formulated in the previous study, to confirm its reliability and validity through an analysis of covariance structure and further testing, and to formulate a practicable scale for measuring competencies in interprofessional collaborative practice.

Methods

Study A

Between December 2011 and January 2012, we administered a questionnaire survey to 2133 health professionals at five hospitals. These included three hospitals with approximately 400–600 beds that are primarily oriented towards providing acute care and one hospital with approximately 200 beds that specialises in rehabilitation between the acute and chronic phases of care. The fifth hospital was a facility comprising approximately 300 beds that primarily provides psychiatric care between the acute and chronic phases of care. Questionnaires were distributed by gatekeepers such as charge nurses in each section and were collected either by the gatekeepers or by post.

The assessment items that measured interprofessional collaborative competency comprised 35 items previously formulated by the authors (Yamamoto et al., 2014). A comprehensive review of the wording of these 35 items was conducted, in particular the semantic content of the items with a low factor loading. Five of the 35 items used in the pilot study were eliminated, and ten items (1, 2, 3, 5, 6, 25, 34, 35, 38, and 40) were added from the 255-item pool that was created prior to the pilot study. Thus, the revised questionnaire contained a total of 40 items. The response options were as follows: ‘Agree’ (5 points), ‘Agree to a certain extent’ (4 points), ‘Neither’ (3 points), ‘Disagree to a certain extent’ (2 points), and ‘Disagree’ (1 point). A high score implied that the respondent had relatively high competency. There were no reverse scoring items among the 40 items used.

The survey included items that measured interprofessional collaborative competency, demographic data, and working conditions. Demographic data collected included age, gender, profession, type of employment, job title, educational background, and service tenure. Questions regarding working conditions included participants’ overtime hours and how easy it was for them to take a break during work.

Interprofessional collaborative competency was assumed to be composed of the following six domains: ‘Respecting Patients,’ ‘Team management skills,’ ‘Fulfilling one’s role as a professional,’ ‘Attitudes and beliefs as a professional,’ ‘Attitudes that improve team cohesion,’ and ‘Taken to achieve the team’s goal.’ An initial model was tested through confirmatory factor analysis, domains were assumed as latent variables, the items that comprised each domain were assumed to be observed variables and covariance paths were set from the domains to the items. Furthermore, we set a covariance path between domains, given the moderate positive correlation between each domain confirmed in our previous study (Yamamoto et al., 2014). We investigated how to make the model more applicable by using five indices, including the comparative fit index (CFI), the goodness-of-fit index (GFI), the adjusted goodness-of-fit index (AGFI), the root mean square error of approximation (RMSEA) and the normed chi-square (χ²/df). An effective model is suggested when GFI is 0.90 or above, AGFI is 0.85 or above, CFI is 0.95 or above and RMSEA constitutes a reasonable standard at 0.05 or lower (Browne & Cudeck, 1993). The criterion for acceptance of χ²/df was less than 5 (Schumacker & Lomax, 2004). We constructed particular models by (1) eliminating items for which paths were suspected to exist from domains to which they did not belong, (2) eliminating items for which paths were suspected to exist from or to items belonging to other domains and (3) adding paths of covariance between errors to an extent that would not hinder the interpretability of the model. In terms of internal consistency, Cronbach’s alpha coefficients were calculated in each domain. In addition to analysing the total sample, models were tested for each of two sub-groups, namely, the ‘nurses’ and ‘other professionals’. The SPSS 19.0 and AMOS 19.0 statistical packages were used to analyse the data.

Study B

During January and February 2012, we administered a questionnaire survey to 571 healthcare professionals at two hospitals. The same questionnaire was administered 2 weeks later to test its reliability. Both hospitals had approximately 400 beds and provided acute care. Questionnaires were distributed by gatekeepers such as charge nurses in each section and were collected either by the gatekeepers or by post.

The survey items were identical to those in Study A. However, in Study B, it was necessary to link individual participants’ responses to both surveys while maintaining their anonymity. Therefore, at the time of the initial survey, participants were asked to create an eight-digit ID number for themselves and to record it on the survey form in both surveys. This enabled us to link the responses of individual participants in both surveys.

Analysis was based on the model ultimately constructed by the confirmatory factor analysis in Study A, and intraclass correlation coefficients (ICC_(2,1)) were calculated for each of the domains in the first and second surveys. The SPSS 19.0 statistical package was used to analyse the data.

Ethical considerations

The ethics review board of the Chiba University School of Nursing approved this study. Healthcare professionals were invited to participate in this study on a voluntary basis. Although the gatekeepers handled the questionnaires, the responses of each individual were unable to be identified by hospital staff. We provided each participating healthcare professional with a document explaining the ethical considerations of the study.

Results

Study A

A total of 1312 health-care professionals responded, which was a response rate of 61.5%. Sixty-seven respondents did not provide their age, sex or profession, or did not provide a response for four or more (≥ 10%) of the 40 items used to evaluate interprofessional collaboration competency. These respondents were excluded from the study, leaving a total of 1245 valid responses (response rate: 58.4%) for analysis.

The basic attributes of the study participants are presented in Table 1. Of the respondents, 74.8% were nurses, 83.2% were female, and the most common age group was 20–29 years (46.5%). More than two-thirds (69.5%) of respondents were graduates of vocational schools. Although there were variations in service length, more than 30% of the respondents had less than 5 years of work experience. A substantial majority of the respondents (91.3%) worked full-time, with a considerable majority (79%) working in positions without an official job title.

Table 1. Demographic characteristics of respondents (n = 1245).

Variable	Category	n	%
Job classification	Nursing professional (registered nurse, registered practical nurse, public health nurse, and midwife)	931	74.8
	Rehabilitation-related therapist (physiotherapist, occupational therapist, speech-language-hearing therapist, and orthoptist)	94	7.6
	Medical care-related professional (radiological technologist, clinical laboratory technician, and biomedical engineering technologist)	77	6.2
	Physician	69	5.5
	Pharmacist	27	2.2
	Social welfare-related professional (care worker, social worker, and care manager)	26	2.1
	Dietician and registrant dietician	14	1.1
	Clinical psychologists	7	0.6
Gender	Male	209	16.8
	Female	1036	83.2
Age (years)	20–29	452	46.5
	30–39	366	37.7
	40–49	218	22.4
	50 and above	209	21.5
Educational background	Vocational school	855	68.7
	Junior college	139	11.2
	University/Graduate school	237	19.0
	Unknown	14	1.1
Tenure of service	Less than 5 years	381	30.6
	More than 5 years and less than 10 years	264	21.2
	More than 10 years and less than 15 years	196	15.7
	More than 15 years and less than 20 years	119	9.6
	20 years or more	281	22.6
	Unknown	4	0.3
Type of employment	Full-time	1130	90.8
	Part-time	108	8.7
	Unknown	7	0.6
Job title	Section chief level/head nurse	108	8.7
	Subsection chief level/charge nurse	152	12.2
	Staff	977	78.5
	Unknown	8	0.6

In confirmatory factor analysis, an initial model was formulated using the 40 items measuring competency of collaborative practice as observed variables and the six domains as latent variables (Tables 1 and 2). Table 3 shows the goodness of fit of the initial models and the final models in relation to all respondents, divided into the ‘nursing professionals’ group and the ‘other professionals’ group. In the initial model for all the respondents, none of the five indices satisfied the necessary criteria. Using the modification indices, we eliminated 11 items to create a final model comprising 29 items, as shown in Figure 1. It can be seen from Table 3 that all five indices satisfied the necessary criteria in the final model. The 29 items adopted in the final model and the 11 items that were eliminated are shown in Table 2. Six of the items that were added for this study (items 1, 2, 3, 5, 6, and 25) were retained, while four items (items 34, 35, 38, and 40) were deleted. We have named the scale that was constructed from these six domains and 29 items the Chiba Interprofessional Competency Scale (29-item version), or CICS29. In the ‘nursing professionals’ group, the final model met the appropriate criteria in all the indices. However, in the ‘other professionals’ group, only one index (GFI = 0.895) did not satisfy the appropriate criteria.

Table 2. The 29 items that were included in and the 11 items that were excluded from the final model (CICS29).

Item number	Domain	Item
1	Attitudes and beliefs as a professional	I constantly strive to improve my performance
2		I always reflect on the care that I have provided
3		I strive to be a professional
4		I practice evidence-based care
5		I am able to explain the basis for care to anyone
6		I am able to apply updated expert knowledge to actual practice
*		I am proud of my professional work
*		I continue studies in order to improve my skills and knowledge
*		As a professional, I have acted so as to prioritise the interests of patients
7	Team management skills	I understand the scope and limits of my team members’ work
8		I respect my team members’ busy schedules and work pace
9		I cooperate with my team members to try to solve the problem when the team is not functioning well
10		I reconcile conflicts among team members
11		I know when problems within the team are likely to arise
*		I know what kinds of education other healthcare professionals have received and the processes they follow
12	Actions for accomplishing team goals	I am able to explain the results of my team’s initiatives
13		I am able to adjust my practices to achieve the team’s objectives
14		I am able to coordinate the opinions of myself and my team members in light of the team’s objective
15		I provide necessary support to my team members depending on their professional competency
16		I am able to evaluate whether the team is operating well objectively
*		I always confirm the goals set by the team
17	Providing care that respects patients	I respect not only the wishes of the patient but also those of their family
18		I keep patient independence in mind when providing care
19		I interact with patients to help them make their own decisions
20		I change my manner of interacting with patients on the basis of their characteristics and situations
21		I seek the best way to care for patients
*		I always act with due consideration for the privacy of the patient
*		I provide accurate and easy-to-understand information to patients
22	Attitudes and behaviours that improve team cohesion	I consciously create opportunities for communication with other professionals
23		I discuss ideal patient care daily with other professionals
24		I try to create a suitable atmosphere during meetings wherein it is easy for other professionals to speak
25		I strive daily to create good interpersonal relations between professionals
*		I find it enjoyable working with other healthcare professionals as a team
*		I provide other professionals with the information I deem necessary as a professional
26	Fulfilling one’s role as a professional	I am able to express opinions in front of other professionals on the basis of my expert knowledge
27		I fulfil my professional role as required by my team
28		I understand the scope of what can be accomplished by professional expertise and skills
29		I am able to state my opinions when necessary from the viewpoint of my professional expertise, even if doing so creates friction with other professionals
*		I have described my own professional expertise to other professionals
*		I am able to trust the other professionals and let them do their jobs

*Item excluded from the final model.

Table 3. Goodness of fit to the two models in each group.

		Group and model
		All respondents (n = 1199)		Nursing professionals (n = 894)		Other professionals (n = 305)
		Initial model	Final model	Initial model	Final model	Initial model	Final model
	GFI	0.827	0.925	0.814	0.928	0.781	0.895
	AGFI	0.804	0.908	0.790	0.909	0.752	0.866
	CFI	0.873	0.950	0.870	0.955	0.874	0.957
Indices	RMSEA	0.064	0.049	0.065	0.049	0.066	0.049
	(90% CI)	(0.052–0.066)	(0.046–0.052)	(0.063–0.067)	(0.045–0.053)	(0.062–0.070)	(0.042–0.056)
	χ^2/df	5.96	3.87	4.75	3.16	2.31	1.73

Figure 1. Confirmatory factor analysis of final model: 29 items solution (n = 1199). *p < 0.05, **p < 0.01, ***p < 0.0011). Standardised path coefficients were shown.

The Cronbach’s alpha coefficients exceeded 0.80 in all cases (Table 4). Similarly, in the analysis of the subsamples of ‘nursing professionals’ and ‘other professionals’, these coefficients exceeded 0.80 in all domains.

Table 4. Reliability of the six domains of the CICS29.

	Indices of reliability
	Alpha coefficient (Study A)
Domain	Allrespondents	Nursingprofessionals	Otherprofessionals	ICC(2,1)(Study B)
Attitudes and beliefs as a professional	0.89	0.88	0.90	0.75
Team management skills	0.86	0.86	0.85	0.71
Actions for accomplishing team goals	0.91	0.92	0.90	0.65
Providing care that respects patients	0.82	0.81	0.84	0.73
Attitudes and behaviours that improve team cohesion	0.84	0.84	0.83	0.72
Fulfilling one’s role as a professional	0.83	0.83	0.83	0.77

Study B

The two surveys were linked using the ID numbers provided by the 180 respondents. Two respondents did not provide their age, gender or profession, or gave no response for four or more (≥ 10%) of the 40 items used to evaluate interprofessional collaboration competency, leaving 178 valid responses for analysis (response rate: 31.2%).

Table 5 shows the basic attributes of the participants. More than 90% of the respondents were female, and the most common age group was 40–49 years (30.3%). Although there were variations in the tenure of service, 38.8% of respondents had 20 years or more of work experience. The majority of the participants (88.1%) worked full-time, with a considerable percentage (82.0%) working with no title.

Table 5. Demographic characteristics of the respondents in Study B (n = 178).

Variable	Category	n	%
Job classification	Registered nurse	136	76.4
	Registered practical nurse	31	17.4
	Public health nurse	9	5.1
	Midwife	2	1.1
Gender	Male	12	6.7
	Female	166	93.3
Age (years)	20–29	31	17.4
	30–39	52	29.2
	40–49	54	30.3
	50 and above	41	23.0
Tenure of service	Less than 5 years	20	11.2
	More than 5 years and less than 10 years	38	21.3
	More than 10 years and less than 15 years	29	16.3
	More than 15 years and less than 20 years	22	12.4
	20 years or more	69	38.8
Type of employment	Full-time	155	87.1
	Part-time	23	12.9
Job title	Section chief level/head nurse	18	10.1
	Subsection chief level/charge nurse	14	7.9
	Staff	146	82.0

The intraclass correlation coefficients (ICC_(2,1)) obtained for the total scores on the 29 items were as follows: 0.75 for ‘attitudes and beliefs as a professional’, 0.71 for ‘team management skills’, 0.65 for ‘actions for accomplishing team goals’, 0.73 for ‘providing care that respects patients’, 0.72 for ‘attitudes and behaviour that improve team cohesion’, and 0.77 for ‘fulfilling one’s role as a professional’.

Discussion

The purpose of this study was to modify and refine the measurement scale for interprofessional collaborative competency that was developed in an earlier pilot study. The intraclass correlation coefficient that calculates reliability was defined as either low (<0.75), moderate (0.75–0.9) or high (>0.9) (Wind, Gouttebarge, Kuijer, & Frings-Dresen, 2005), and the fact that domain values close to 0.75 were generally obtained in this study suggests an acceptable level of reliability. With regard to internal consistency, values exceeding 0.80 were calculated for all domains, indicating that a high level of internal consistency was achieved.

The Institute of Medicine (2015) states that the seamless development of continuing education that builds on fundamental education is important for IPE, and recommends that evidence measuring the effectiveness of learning be collected during that process. However, there is no standardised scale that can measure the effectiveness of learning. In the covariance structure analysis, we constructed a model comprising six latent variables as well as 29 observed variables, and the model demonstrated reasonable goodness of fit. The six domains used in this study replicated those that were identified in the previous study’s exploratory factor analysis, although the names of the domains have been partially revised, and have been confirmed by the construct validity.

Next, by comparing the results with the constructs of the four frequently employed scales Interdisciplinary Education Perception Scale (IEPS) (Luecht, Madsen, Taugher, & Petterson, 1990), Readiness for Interprofessional Learning Scale (RIPLS) (McFadyen et al., 2005), Interprofessional Attitudes Questionnaire (IAQ) (Robben et al., 2012), and Attitudes Towards Healthcare Teams Scale (ATHCTS) (Heinemann, Schmitt, Farrell, & Brallier, 1999), we identified six domains comprising the characteristics of the CICS29. The CICS29 domain ‘providing care that respects patients’ is a construct that does not appear in other scales. The IEPS, IAQ, and ATHCTS scales comprise items regarding ‘attitudes and beliefs as a professional’ and ‘attitudes that improve team cohesion’. These are not domains in actual behaviour when offering care to patients. RIPLS was developed through a deductive process, and although the construct of ‘patient’s well-being’ existed at the time of the scale’s creation, the factor of ‘patient’s well-being’ was not extracted in the results of the factorial analysis. Thus, there is a problem with its construct validity. The CICS29 evaluates competencies based on behaviour. We interviewed expert medical professionals who practice coordination with other experts and created the item pool based on these interviews. Coordination competencies were inductively modelled, and this structure has been verified using a sample of 1400 people. Other scales do not possess this kind of strict developmental process, and thus we can argue that the CICS29 possesses robust conceptual structures.

Based on the above findings, it was determined that the CICS29 maintained a certain degree of structural validity as a scale to assess the competencies in interprofessional collaborative practice of healthcare professionals based on their practice. We can also argue that the scale can be used to evaluate the degree of achievement of the learning objectives in fundamental education and IPE in postgraduate education.

In terms of the limitations of this study and future work, the CICS29 that has been developed in this study has been shown to be a high-utility scale for measuring competencies in interprofessional collaborative practice. However, the fact that the respondents are health professionals working in Japanese hospitals, primarily during the acute phase of care, indicates that further investigation is necessary with regard to its possible application as a scale in a cross-cultural environment or in a chronic care environment. Furthermore, the fact that most participants in the study were female nurses suggests the need to exercise caution when attempting to apply the scale in a different healthcare environment.

The challenges in relation to the refinement of the CICS29 are summarised from the perspective of guaranteeing the scale’s validity. First, we are yet to verify the cross-cultural validity of the scale. Furthermore, it will be necessary to examine the criterion-related validity in the future because there has been virtually no development of a scale of competency in interprofessional collaborative practice. Another theme for future investigation is reactivity, i.e., the extent to which the CICS29 score changes with participants’ professional development. Because this scale uses self-evaluation to measure IPE competencies, further discussion is required as to whether these indications can be used as an objective evaluation of performance.

The reliability of the CICS29 has been adequately proven, but it is essential to verify the test–retest reliability for healthcare professionals other than nurses.

Concluding comments

The CICS29 is a scale comprising items that were developed based on behavioural result interviews with high-performing healthcare workers. During the refinement process, responses regarding whether they ‘actually took some action’ were obtained from healthcare professionals and measured using Likert-scale options, and their competencies in interprofessional collaborative practice were assessed based on the scores that were obtained. Because this scale can assess actual behaviours, it can be used to assess the effectiveness of learning, which makes it possible to measure the effectiveness of education.

As the structural validity of the CICS29 has been verified, it is possible to obtain a multidimensional measurement of the competencies in interprofessional collaborative practice. This will contribute to the accumulation of IPW-related evidence such as how the expression of an individual’s competency in interprofessional collaborative practice is affected by their working environment.

Declaration of interest

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

Acknowledgements

The authors would like to express their sincere gratitude to all of the administrators and health professionals at the seven hospitals that participated in the study.

Funding

This study was supported by aid grants from the discretionary expenses of the president of Chiba University and a Grant-in-Aid for Scientific Research (C) (24590621) from the Ministry of Education, Culture, Sports, Science and Technology, Japan.

Additional information

Funding

This study was supported by aid grants from the discretionary expenses of the president of Chiba University and a Grant-in-Aid for Scientific Research (C) (24590621) from the Ministry of Education, Culture, Sports, Science and Technology, Japan.