The transferability of Western concepts to other cultures: Validation of the Zuckerman–Kuhlman Personality Questionnaire in a Saudi Arabic context

Abstract

Background: The importance of medical education research in Saudi Arabia has been acknowledged increasingly and a lot of concepts used have been derived from the Western world. The question arises, however, whether Western concepts and questionnaires are transferable to societies with different cultures. The aim of this study was to investigate the instrument structure and the reliability of the Arabic version of the Zuckerman–Kuhlman Personality Questionnaire-medium (ZKPQ-m).

Methods: Three statistical methods with decreased amount of strictness were used to analyse our data: Confirmatory Factor Analysis, Procrustes rotation and Principal Component Analysis.

Results: Our outcomes did not confirm the original instrument structure. Instead, we found four interpretable components: Emotional Instability, Impulse-seeking, Activeness and Self-Control. However, the amount of explained variance was not very high and the internal consistencies ranged from unsatisfactory to only moderate. The data showed a high percentage of respondents agreeing with more than three items of the Infrequency scale, which may be attributable to the collectivistic culture in Saudi Arabia.

Conclusion: We did not succeed in replicating the ZKPQ structure in the Arabic context. Social desirability, a common characteristic in collectivistic cultures, may have reduced the replicability of the internal structure of the ZKPQ-m. Different methods to measure concepts in collectivistic cultures may help to get round social desirability.

Introduction

Over the past years, the medical education field in the kingdom of Saudi Arabia has displayed a rapid development (AlFaris et al. 2006; Bin Abdulrahman 2011; Telmesani et al. 2011; Bin Abdulrahman et al. 2012; Tekian et al. 2014). A similar development seems to take place in the medical education research field: the importance of research in medical education has been acknowledged more and more and the number of Saudi Arabic research papers is increasing (Bin Abdulrahman 2011, 2012; Bin Abdulrahman et al. 2013). It appears that a lot of concepts and questionnaires used in Arabic medical education and education research have been derived from the Western world (Gallagher 1988). We wondered whether Western concepts and questionnaires are transferable to societies with a different culture and focused our study on constructing and validating an Arabic version of an instrument – the Zuckerman–Kuhlman Personality Questionnaire-medium (ZKPQ-m).

Cultural differences between Western countries and Arabic countries may affect the validity of Western questionnaires in the Arabic society. Whereas the cultures in most Western countries tend to be relatively low on power distance and high on individualism, the Saudi Arabian culture can be characterized as high on power distance and relatively low on individualism (or high on collectivism) (Hofstede & Hofstede 2005). Power distance is “the extent to which the less powerful members of institutions (institutions are the basic elements of society, e.g. family, school, community) or organizations within a country expect and accept that power is distributed unequally” (Hofstede & Hofstede 2005). In large-power-distance countries, subordinates depend considerably on their bosses, rely on authority figures to make decisions and their behaviours are characterized by obedience, submission to authority. In collectivistic societies, people emphasize the needs and goals of the group as a whole over the needs and wishes of each individual: “people… are integrated in strong, cohesive in-groups, which… continue to protect them in exchange for unquestioning loyalty” (Hofstede & Hofstede 2005). They value respect for and obedience to parents, avoid confrontations, conform themselves more often and are often patriotic (Hofstede & Hofstede 2005; Harzing 2006). Recent research has shown that the cultural values in Saudi Arabia may have implications for the applicability of Western frameworks in the Arabic context. For instance, a Western professionalism framework for healthcare providers needed adaptations in order to tailor it to the Arabic context (Al-Eraky et al. 2014). Culture may also affect the way in which instruments are filled out.

Table

Practice points

Questionnaires are not by definition transferable to other cultures. Cultural differences in social desirability may affect the replicability of instrument structure. Ensure that item content applies to the culture in which the questionnaire will be administered. Future research may focus on the use of different measurement approaches, like the forced-choice approach, to sidestep social desirability.

From literature, we know that cultural characteristics like power distance and collectivism may affect response behaviour (Harzing 2006). For instance, power distance seems to be related to the use of the neutral or middle response categories on rating scales, also called a middle response style. Likewise, collectivism has been found to go along with a preference for a middle response style and for acquiescence, which is the tendency to agree with an item regardless of the content. Such a response bias may obscure relationships between variables and influence the extent to which the internal structure of instruments can be replicated (e.g. Ganster et al. 1983). Research indicated, for instance, that acquiescent response behaviour may affect the psychometric properties of personality questionnaires in such a way that original instrument structures are not replicable (Rammstedt et al. 2010; Rammstedt & Kemper 2011). Since both power distance and collectivism may affect response behaviour in Saudi Arabic self-report instruments, we wondered whether the instrument structure and the reliability of an instrument were replicable in the Arabic translation of the instrument.

In this study, we focused on an instrument of which the English version has been used in the Arabic medical education context by Mehmood et al (2013), namely the Zuckerman–Kuhlman Personality Questionnaire. Considering the disadvantages of using non-native questionnaires (Gibbons et al. 1999; Harzing 2006), however, it seems better to use native questionnaires. Therefore, we developed an Arabic version of the Zuckerman–Kuhlman Personality Questionnaire and we investigated its instrument structure and reliability.

Methods

Participants and procedure

In 2010–2011, we performed a cross-sectional study among first, second, third, fourth and fifth-year students at the King Khalid University medical school in Saudi Arabia. We informed the students of the college of medicine about our study and invited them all to participate. The study took place before the start of a lecture near the end of first semester. Participation was voluntary and anonymous. We obtained ethical approval for our study. We adhered to ethical principles and no plausible harms to participating individuals arise from this study.

Instrument

The Zuckerman–Kuhlman Personality Questionnaire (ZKPQ) measures five dimensions of personality: Activity, Aggression-Hostility, Sociability, Impulsive Sensation Seeking and Neuroticism-Anxiety. Activity refers to the tendency to be active, to prefer challenging work and to be impatient or restless when there is nothing to do. Aggression-Hostility pertains to the tendency to express verbal aggression and show rudeness, lack of inconsideration, vindictiveness, a quick temper and an impatient attitude. Sociability is the tendency to interact well with others, to enjoy being with others and to have intolerance for social isolation. Impulsive Sensation Seeking is the tendency to act quickly on impulse without planning, often in response to the perceived need for thrills and excitement, change and novelty. Neuroticism-Anxiety concerns the tendency to be nervous, tense, sensitive to criticism, easily upset and obsessively indecisive.

There are several versions of the ZKPQ, which differ in questionnaire length: a long form (Zuckerman et al. 1993), a medium form (Aluja et al. 2006) and a short form (Zuckerman 2002). We developed an Arabic version of the ZKPQ-m, the medium form of the ZKPQ (Aluja et al. 2006). The ZKPQ-m contains 50 items on a Likert-type scale (1 – strongly disagree, 4 – strongly agree) that measure the personality dimensions. The items are equally distributed over the five scales (i.e. 10 items in each scale). In addition, the ZKPQ-m contains an Infrequency scale of 10 items. Agreement with more than three items of this scale is being suggested as indicating social desirable response behaviour (Zuckerman et al. 1993; Zuckerman 2002). We obtained written permission from the developers of the questionnaire to use the ZKPQ-m in our study. Professional translators at the colleges of medicine and education of King Khalid University translated and back-translated the ZKPQ-m.

Data analysis

We performed factor analysis to obtain an indication of the construct validity of the original instrument structure in our Saudi Arabic context. We successively used three statistical methods to analyse our data. These three methods are characterized by a decreased amount of strictness for testing correspondence between instrument structures. In the first method, Confirmatory Factor Analysis, confirmation of the instrument structure is restricted to finding the same pattern of primary loadings. In the second method, using a Procrustes rotation, all variables are allowed to load on all factors and the entire loading pattern is taken into account in determining the congruence between instrument structures found in different populations. The advantage of this method is that the pattern of primary loadings does not have to be exactly the same as in the original dataset; instead, the entire loading pattern is taken into account. This implies that correspondence can also be found if the primary loading pattern is not the same, but if – overall – loading patterns are fairly the same. In the last method, we performed a Principal Component Analysis with Varimax rotation, in which we did not restrict ourselves to the five-component solution, but explored several solutions to find the best interpretable instrument structure. These three methods are described in detail in the next three paragraphs.

Confirmatory factor analysis

To evaluate whether the original five-factor structure was supported in the Arabic version of the ZKPQ-m, we performed a Confirmatory Factor Analysis – the Oblique Multiple Group Method (OMG) (Stuive et al. 2008). To perform the OMG, we first formed scales containing those items that were originally found to belong together (see Aluja et al. 2006). Subsequently, we calculated the correlation of each item with each scale. After correction for self-correlation, items should correlate strongest with the scale they are assigned to. If an item turns out to be correlated stronger with another scale, this indicates that it was not assigned to the correct scale and that the item should be shifted to the scale with which it correlates most.

Procrustes rotation and congruence coefficients

We performed a Principal Component Analysis with Varimax rotation extracting five factors. To assess cross-language replicability, we applied an orthogonal Procrustes rotation to compare our Varimax rotated loading matrix with the matrices of two populations included in Aluja et al. (2006) (e.g. McCrae et al. 1996; Van de Vijver & Leung 1997). This method has been applied before to compare the instrument structures of different translations of the ZKPQ (e.g. Gomà-i-Freixanet et al. 2004; Aluja et al. 2006; Rossier et al. 2008). We obtained the loading matrices of the Spanish and the American populations included in Aluja et al. (2006) from the authors and used these as reference or “target” populations against which the loading matrix of the five-component solution of our Arabic sample was compared. We calculated factor congruence coefficients and total congruence coefficients to ascertain the cross-cultural replicability of the Spanish and the American instrument structures. Similarity indices of 0.85–0.94 imply fair similarity, whereas values higher than 0.95 imply that the two factors or components compared can be considered structurally equivalent across cultures (Lorenzo-Seva & Ten Berge 2006).

Principal Component Analysis

We performed Principal Component Analysis (PCA) as described by Schönrock-Adema et al. (2009) to further explore the instrument structure. We applied a combination of three statistical criteria and four interpretability criteria to identify the most sensible instrument structure (Hatcher 1994). The three statistical criteria were (a) the point of inflexion as displayed by the scree plot, (b) eigenvalues >1.5 and (c) each individual component explaining at least around 5% of the variance. The four interpretability criteria implied (i) minimally three variables loading 0.40 or higher on the component, (ii) variables loading highest on the same component measure the same construct, (iii) those loading on different components measure dissimilar constructs and (iv) the rotated factor pattern displays “simple structure”, which – broadly speaking – means that variables load high on only one component and low on the others. As in Schönrock-Adema et al. (2009), we decided to investigate the interpretability of several solutions. To determine which solutions should be included in our interpretation process, we applied the three statistical criteria. To find the best interpretable and theoretically sensible solution, we also decided to include solutions with up to two components more and less.

The interpretation process was performed using a Delphi-like procedure. The panel consisted of seven members – mainly psychologists and educationalists, who were all involved in research. In the first round, each panel member independently interpreted the components of all relevant solutions and noted their interpretations. Subsequently, one of the investigators (JSA) made an overview of all interpretations and submitted this overview in a second round to the panel members, together with the solutions. Each panel member was asked to individually review the interpretations noted by all members, compare these with the content of the solutions and note the – in their opinion – best interpretations. This cycle was repeated until the investigators agreed that there was large correspondence in the connotations of the component interpretations. The investigators made an overview of these interpretations along with a proposal for the final interpretations and asked the panel members in the final decision-making round whether they agreed with the proposed interpretations.

Internal consistency

Internal consistencies of the ZKPQ-m scales were determined by calculating Cronbach’s alpha. Values between 0.7 and 0.8 are considered indicative of modest internal consistency and values of 0.8 and above of good internal consistency (Nunnally & Bernstein 1994). In addition, the corrected item-total correlations of each item should at least be 0.30 (Field 2009).

Results

Of all 590 students, 556 (202 females and 354 males) completed the ZKPQ-m (a response rate of 94.2%). The mean age of the respondents was 21.1 years (SD 2.5 years). Inspection of the dataset showed that 73.6% of the respondents agreed with more than three items of the Infrequency scale.

Confirmatory factor analysis

The outcomes of the Confirmatory Factor Analysis did not support the original instrument structure (see Table 1). Sixteen out of 50 items loaded higher on another scale than the one they were assigned to, namely seven items from Aggression-Hostility, five items from Sociability, three items from Impulsive Sensation Seeking and one item from Neuroticism-Anxiety. The original ZKPQ-m structure explained 24.8% of the variance. Additional analyses showed that the internal consistencies of the original scales varied from 0.32 for Aggression-Hostility through 0.51 (Sociability), 0.52 (Impulsive Sensation Seeking) and 0.60 (Activity) to 0.70 for Neuroticism-Anxiety. The majority of the items, 33 out of 50, demonstrated corrected item-total correlations <0.30, which means that these items did not correlate very well with the overall scores on their scales (Field 2009).

Table 1. Item-rest correlations, after corrections for test-length and self-correlation^a,b.

	F1	F2	F3	F4	F5
01. I do not like to waste time just sitting around and relaxing	0.118	−0.025	−0.036	−0.030	−0.020
02. When I get mad, I say ugly things	−0.005	0.113	−0.013	0.027	0.108
03. It’s natural for me to curse when I am mad	0.003	0.091	0.001	0.016	0.041
04. I do not mind going out alone and usually prefer it to being out in a large group	−0.042	0.019	−0.078	−0.069	0.023
05. I lead a busier life than most people	0.089	−0.016	−0.035	0.050	0.050
06. I often do things on impulse	0.192	−0.083	−0.007	0.016	−0.110
07. I almost never feel like I would like to hit someone	−0.070	0.041	0.004	−0.048	0.000
08. I spend as much time with my friends as I can	−0.024	−0.012	0.094	0.092	−0.032
09. My body often feels all tightened up for no apparent reason	−0.021	0.037	−0.007	0.065	0.190
10. I frequently get emotionally upset	−0.014	0.048	−0.092	0.025	0.262
11. If someone offends me, I just try not to think about it	−0.057	0.036	0.010	−0.043	−0.027
12. I like to be doing things all of the time	0.189	−0.043	−0.032	0.068	−0.020
13. I would like to take off on a trip with no preplanned or definite routes or timetables	−0.037	0.021	0.037	0.125	0.084
14. I tend to be oversensitive and easily hurt by thoughtless remarks and actions of others	0.027	0.047	−0.055	0.057	0.151
15. I do not need a large number of casual friends	−0.053	0.042	0.069	−0.004	−0.036
16. I can enjoy myself just lying around and not doing anything active	0.108	−0.067	−0.024	−0.069	−0.070
17. I enjoy getting into new situations where you can’t predict how things will turn out	0.060	0.017	0.017	0.138	0.021
18. I am easily frightened	−0.076	0.065	−0.070	0.013	0.239
19. If people annoy me I do not hesitate to tell them so	0.017	0.014	−0.064	0.030	0.151
20. I tend to be uncomfortable at big parties	−0.002	−0.074	0.135	0.021	−0.151
21. I do not feel the need to be doing things all of the time	0.105	−0.022	−0.002	−0.010	−0.104
22. I sometimes feel panicky	−0.058	0.032	−0.067	0.026	0.235
23. At parties, I enjoy mingling with many people whether I already know them or not	0.066	−0.012	0.084	0.111	0.004
24. I sometimes like to do things that are a little frightening	0.102	0.012	0.066	0.133	−0.052
25. When on vacation I like to engage in active sports rather than just lie around	0.123	−0.026	0.074	0.080	−0.079
26. I’ll try anything once	−0.077	0.079	−0.030	0.017	0.110
27. I often feel unsure of myself	−0.052	0.068	−0.102	0.001	0.202
28. I would not mind being socially isolated in some place for some period of time	−0.024	−0.054	0.152	−0.038	−0.172
29. I like to wear myself out with hard work or exercise	0.154	−0.021	−0.017	0.096	−0.010
30. I would like the kind of life where one is on the move and travelling a lot, with lots of change and excitement	0.090	−0.055	0.018	0.113	−0.003
31. I often worry about things that other people think are unimportant	0.034	0.040	−0.027	−0.002	0.103
32. When people disagree with me I cannot help getting into an argument with them	−0.036	0.069	−0.075	0.021	0.149
33. Generally, I like to be alone so I can do things I want to do without social distractions	−0.006	−0.058	0.151	−0.019	−0.140
34. I sometimes do “crazy” things just for fun	0.015	0.063	0.030	0.171	0.053
35. I have a very strong temper	0.024	−0.070	0.008	0.056	−0.025
36. I like to be active as soon as I wake up in the morning	0.104	−0.036	−0.003	0.036	0.003
37. I can’t help being a little rude to people I do not like	−0.003	0.020	−0.018	0.028	0.042
38. I am a very sociable person	0.101	−0.022	0.105	0.101	−0.083
39. I prefer friends who are excitingly unpredictable	−0.031	0.058	0.074	0.042	−0.008
40. I often feel like crying sometimes without a reason	−0.017	0.043	−0.043	0.065	0.239
41. I like to keep busy all the time	0.133	0.022	−0.035	0.031	−0.023
42. I often get so carried away by new and exciting things and ideas that I never think of possible complications	−0.008	−0.005	−0.011	0.094	0.062
43. I don’t let a lot of trivial things irritate me	−0.060	0.024	−0.022	−0.104	0.031
44. I am always patient with others even when they are irritating	−0.091	0.040	0.020	−0.061	−0.006
45. I usually prefer to do things alone	−0.125	0.023	0.044	−0.066	−0.002
46. I often feel uncomfortable and ill at ease for no real reason	−0.044	0.067	−0.064	0.024	0.244
47. I probably spend more time than I should socializing with friends	0.008	0.057	0.076	0.110	0.041
48. When I do things, I do them with lots of energy	0.179	−0.028	0.007	0.112	−0.006
49. I like “wild” uninhibited parties	0.057	0.019	0.048	0.104	0.011
50. When people shout at me, I shout back	−0.045	0.064	0.037	0.099	0.039

^aF1 – Activity, F2 – Aggression-Hostility, F3 – Sociability, F4 – Impulsive Sensation Seeking, F5 – Neuroticism-Anxiety.

^bCorrelations of items with the scales to which they were assigned are underlined and – if the items correlate highest with the scales being assigned to – printed in italics.

Procrustes rotation and congruence coefficients

Exploratory Factor Analysis with orthogonal Procrustes rotation supported the original instrument structure as little as the Confirmatory Factor Analysis did. Comparison of the Arabic loading matrices with the American and with the Spanish reference group showed that neither the total congruence coefficients (0.65 for each of the comparisons) nor the factor congruence coefficients – ranging from 0.41 to 0.76 and 0.45 to 0.76 respectively – reached the lower limit of 0.85, which indicates that we did not find any factor similarity (Lorenzo-Seva & Ten Berge 2006).

Principal component analysis

The scree plot showed a first point of inflection after the fourth component and a second one after the sixth component (see Figure 1). Six components had an initial eigenvalue >1.5, with values ranging from 1.63 to 4.39. Of these, only the first three components accounted for more than 5% of the variance. The statistical criteria indicated that the interpretabilities of the 3-, 4-, 5-, and 6-component solutions should be investigated. Beforehand, we decided to base inclusion of solutions in our interpretation process on the three statistical criteria and to investigate the interpretability of solutions with up to two components more and less to find the best interpretable and theoretically sensible solution. As the three statistical criteria already resulted in a set of four potentially applicable solutions, we decided to add only the solutions with one component more and less. As a result, we included the solutions with two to seven components for further investigation. Since the solution with seven components included one component that did not contain three items with significant loadings, this solution was left out of the interpretation process. A 3-round Delphi-procedure led to the conclusion that the 5- and 6-factor solutions contained components that were not unambiguously interpretable. In addition, these solutions failed to meet the criteria that different components should measure different constructs and that variables loading on the same component should measure the same construct. The solutions with two and three components also contained components that seemed to measure more than one construct. The interpretation process showed that the solution with four components was the best interpretable solution (see Table 2). The components were interpreted as Emotional Instability, Impulse-seeking, Activeness and Self-Control. The 4-component solution explained 25.9% of the variance. The internal consistencies of the four components were 0.78 for Emotional Instability, 0.70 for Impulse-seeking, 0.62 for Activeness and 0.52 for Self-Control. Of the items included in these components, 3 (out of 15 items), 6 (out of 14 items), 9 (out of 13 items) and 6 (out of 8 items), respectively, had Corrected Item-Total Correlations below 0.30, which means that these items did not correlate very well with the overall scores on the components in question (Field 2009).

Figure 1. Scree plot of the eigenvalues obtained from principal components analysis.

Table 2. Factor loadings and communalities of the final 4-factor solution.

		Component
No.	Item	1	2	3	4	h^2
18	I am easily frightened	0.620	−0.024	−0.157	0.021	0.41
46	I often feel uncomfortable and ill at ease for no real reason	0.571	0.070	−0.101	−0.005	0.34
10	I frequently get emotionally upset	0.570	−0.026	0.027	−0.069	0.33
40	I often feel like crying sometimes without a reason	0.570	0.156	0.006	−0.018	0.35
28	I would not mind being socially isolated in some place for some period of time	0.556	−0.061	0.236	−0.027	0.37
27	I often feel unsure of myself	0.552	−0.090	−0.106	−0.052	0.33
22	I sometimes feel panicky	0.538	0.025	−0.059	0.002	0.29
33	Generally, I like to be alone so I can do things I want to do without social distractions	0.499	−0.133	0.143	−0.009	0.29
20	I tend to be uncomfortable at big parties	0.489	−0.250	0.134	0.035	0.32
19	If people annoy me I do not hesitate to tell them so	0.476	0.057	0.045	0.176	0.26
32	When people disagree with me I cannot help getting into an argument with them	0.451	−0.046	−0.118	0.011	0.22
9	My body often feels all tightened up for no apparent reason	0.413	0.150	−0.057	−0.194	0.23
14	I tend to be oversensitive and easily hurt by thoughtless remarks and actions of others	0.389	0.143	0.127	0.023	0.19
26	I’ll try anything once	0.346	0.030	−0.291	−0.006	0.21
31	I often worry about things that other people think are unimportant	0.256	−0.003	0.069	0.120	0.08
24	I sometimes like to do things that are a little frightening	−0.118	0.539	0.193	0.036	0.34
47	I probably spend more time than I should socializing with friends	0.088	0.536	−0.161	−0.064	0.32
34	I sometimes do “crazy” things just for fun	0.175	0.518	−0.004	−0.202	0.34
8	I spend as much time with my friends as I can	−0.079	0.489	−0.199	0.130	0.30
38	I am a very sociable person	−0.265	0.472	0.146	0.168	0.34
49	I like “wild” uninhibited parties	0.053	0.469	0.089	−0.052	0.23
23	At parties, I enjoy mingling with many people whether I already know them or not	−0.103	0.466	0.057	0.120	0.25
17	I enjoy getting into new situations where you can’t predict how things will turn out	0.089	0.442	0.167	−0.056	0.23
13	I would like to take off on a trip with no preplanned or definite routes or timetables	0.189	0.374	−0.156	−0.079	0.21
25	When on vacation I like to engage in active sports rather than just lie around	−0.210	0.354	0.285	0.161	0.28
42	I often get so carried away by new and exciting things and ideas that I never think of possible complications	0.224	0.347	−0.008	0.152	0.19
30	I would like the kind of life where one is on the move and travelling a lot, with lots of change and excitement	0.012	0.335	0.198	0.130	0.17
4	I do not mind going out alone and usually prefer it to being out in a large group	−0.081	0.328	0.097	0.066	0.13
43	I don’t let a lot of trivial things irritate me	−0.026	0.324	0.081	0.283	0.19
12	I like to be doing things all of the time	0.030	0.108	0.566	0.068	0.34
6	I often do things on impulse	−0.231	0.103	0.523	0.149	0.36
29	I like to wear myself out with hard work or exercise	−0.003	0.205	0.506	0.094	0.31
48	When I do things, I do them with lots of energy	0.031	0.328	0.504	0.008	0.36
1	I do not like to waste time just sitting around and relaxing	−0.040	−0.114	0.432	−0.004	0.20
45	I usually prefer to do things alone	0.116	0.121	0.414	0.218	0.25
16	I can enjoy myself just lying around and not doing anything active	0.189	0.265	−0.414	−0.082	0.28
41	I like to keep busy all the time	0.015	0.013	0.395	−0.140	0.18
5	I lead a busier life than most people	0.115	0.098	0.334	−0.073	0.14
36	I like to be active as soon as I wake up in the morning	0.023	0.086	0.333	0.244	0.18
21	I do not feel the need to be doing things all of the time	0.231	0.074	−0.313	0.119	0.17
15	I do not need a large number of casual friends	0.168	−0.069	0.268	0.220	0.15
39	I prefer friends who are excitingly unpredictable	−0.035	0.231	−0.250	0.044	0.12
2	When I get mad, I say ugly things	0.210	0.152	0.103	−0.633	0.48
44	I am always patient with others even when they are irritating	0.103	0.160	0.143	0.589	0.40
3	It’s natural for me to curse when I am mad	0.101	0.152	0.098	−0.519	0.31
35	I have a very strong temper	−0.006	0.159	−0.060	0.484	0.26
11	If someone offends me, I just try not to think about it	0.113	0.098	0.120	0.446	0.24
50	When people shout at me, I shout back	0.108	0.320	−0.182	−0.361	0.28
7	I almost never feel like I would like to hit someone	0.042	0.118	0.182	0.253	0.11
37	I can’t help being a little rude to people I do not like	0.166	0.040	−0.049	0.212	0.08
	Initial eigenvalues	4.39	3.88	2.69	1.98
	Explained variance after rotation	8.79%	7.77%	5.38%	3.97%

h²: communality estimates (estimates of the proportion of variance in a given variable explained by all components jointly). Each item's highest loading is highlighted in bold.

Discussion

The aim of this study was to develop and validate an Arabic version of the ZKPQ-m and to investigate the transferability of this Western instrument to the Arabic context. Our outcomes did not confirm the original instrument structure in the Arabic version of the ZKPQ-m. In particular, the Sociability scale was not replicable. Further analyses led to four interpretable components: Emotional Instability, Impulse-seeking, Activeness, and Self-Control. Closer inspection of the loading matrices revealed that most of the items that originally belonged to the scale Sociability loaded on other components. These transpositions were explicable however. For instance, the Sociability item “I would not mind being socially isolated in some place for some period of time” loaded positively on Emotional Instability. It seems to fit in this component, as persons high on Emotional Instability are more vulnerable and may feel comfortable being socially isolated as that may decrease the chance to get emotionally hurt. Other Sociability items, like “At parties, I enjoy mingling with many people whether I already know them or not” and “I do not mind going out alone and usually prefer it to being out in a large group” loaded on the Impulse-seeking scale, where they seem to fit considering that mingling with unknown people or going out alone rather than in a group has an exciting side that meets the need to seek for impulses. These activities may be exciting for Arabic people since these activities are individualistic rather than collectivistic: they imply that individuals step out of their safe, protective groups. Despite the fact that the transposition of most items was explicable, the amount of explained variance was not very high and the internal consistencies of the new scales ranged from unsatisfactory to only moderate.

We wondered why our outcomes differed from those of studies in which the original ZKPQ structure was replicated in translated versions of the ZKPQ (e.g. Gomà-i-Freixanet et al. 2004; Aluja et al. 2006; Rossier et al. 2008). A possible explanation for not finding support for the original instrument structure in the Arabic ZKPQ is method bias (Van de Vijver & Hambleton 1996). Method bias occurs when validity is reduced by instrument administration related factors, for instance, acquiescence or social desirability. From literature, we know that the tendency to respond in a socially desirable way is higher in collectivistic cultures (Bond & Smith 1996; Van Hemert et al. 2002; Bernardi 2006). We found a much higher percentage of respondents agreeing with more than three items of the Infrequency scale than other studies (Hojat et al. 2005, 2013; Rossier et al. 2008), which may be attributable to the collectivistic culture in Saudi Arabia. The fact that social desirability was stronger in our Arabic sample than in the samples used in previous studies may have made us find a different instrument structure (e.g. Rammstedt et al. 2010; Rammstedt & Kemper 2011). However, considering the fact that social desirability is characteristic for collectivistic cultures (Bond & Smith 1996; Van Hemert et al. 2002; Bernardi 2006), it can be argued that social desirability should not be considered as a methodological artefact, but rather as having substantial cultural meaning (see Smith 2004). Future research might focus on ways to overcome social desirability in collectivistic cultures.

Another explanation for not being able to replicate the original instrument structure in the Arabic version is item bias or differential item functioning (Van de Vijver & Hambleton 1996). This source of instrument anomalies refers to differences at the item level because of inaccurate translations, poor wording and inappropriateness of the item content in the culture of the population of interest. We realize that the translation of the items may have resulted in different connotations. However, we made an effort to carry out the translation process as conscientiously as possible to avoid poor wording and differences in connotation. As to the appropriateness of the item content in the Arabic setting, the question may arise whether all items are applicable to our Arabic culture, considering that the ZKPQ was developed in an individualistic country. In previous research in a collectivistic culture, some different personality constructs were found (Katigbak et al. 2002). This suggests that those culture specific constructs may be more salient in a collectivistic culture than in an individualistic culture, and, hence, that the appropriateness of items is different in different cultures, i.e. there is differential item functioning. Upon inquiry among some Saudi Arabian colleagues, who reviewed the items upon our request, it appeared that the ZKPQ-items were applicable to the Arabic culture, but we realize that social desirability may also have affected the outcomes of this survey and prevented finding evidence for differential item functioning.

Strengths and limitations

The strength of this study was the high response rate, which implies that our sample was representative of our entire medical student population. Moreover, we validated the instrument structure using a conscientious approach. Not only did we perform different kinds of analyses to evaluate whether the original five-factor structure was supported by the Arabic version of the ZKPQ-m, we also performed a thorough Principal Component Analysis using several statistical and interpretation criteria (Schönrock-Adema et al. 2009). Although we did find four interpretable components, the facts that two scales showed inacceptable internal consistencies and that no scale showed good internal consistency raise questions about the practical applicability of the Arabic version of the ZKPQ-m.

We realize that the current study was just a first step in validating the ZKPQ-m in the Arabic culture. Further validity evidence – beyond reliability and factorial structure – is needed, however, before instruments can be used in practice. Furthermore, we limited our study to the translation of the ZKPQ. The use of other kinds of personality instruments that rely on Likert-type answering scales, however, may also carry the risk of social desirability bias in collectivistic cultures. The same risk may be present when using self-report instruments with a focus on different concepts. Therefore, future attempts to develop (personality) instruments for collective cultures should take into account the potential influence of social desirability on the measurement outcomes. To get round social desirability, researchers might search for different ways to measure concepts, for instance, by using a multidimensional forced-choice method. In the organizational setting, such an approach has already proven effective in resisting response sets (Stark et al. 2012). Further investigation has to show whether a forced-choice approach is also effective in avoiding social desirability that is rooted in culture.

Implications

A practical implication of our study is that the current Arabic version of the ZKPQ-m does not (yet) seem to be applicable in the Arabic culture for, among other things, specialty choice counselling purposes. Despite the nature of our assessment method, being anonymous and voluntary, and despite the fact that the social value placed on the scale items was little – which, in general, decreases the tendency to answer in a socially desirable way (Van de Mortel 2008) – we found high social desirability scores. In situations in our Arabic culture that are of more importance and in which respondents are not anonymous, like applications, social desirability responding may even be stronger and, thus, further reduce the validity of the instrument. Based on literature, we surmise that in the Arabic culture, and maybe even in collectivistic cultures in general, Likert-type answering scales may not be adequate to measure self-perceptions. Possibly, other approaches to measuring may be more adequate to sidestep the problem of social desirability and to obtain self-perceptions that are more valid. Therefore, we recommend to investigate the validity of different answering formats in collectivistic cultures.

Conclusion

We conclude that we did not succeed in replicating the ZKPQ structure in the Arabic context. Although we did find a new instrument structure with four interpretable personality dimensions (Emotional Instability, Impulse-seeking, Activeness and Self-Control), the psychometric properties do not support our Arabic version of the ZKPQ as valid for specialty choice counselling purposes.

Note: A copy of the Arabic version of the ZKPQ-m can be obtained by contacting the author at ‘[email protected]’.

Notes on contributors

SYED IMRAN MAHMOOD, MBBS, MA, MMedEd, PhD, is a BMJ learning Champion and Associate Professor of Medical Education at the College of Medicine, Umm Al-Qura University, Makkah, Kingdom of Saudi Arabia.

SYED ABDUL DAIM, PhD, is an Educationist Researcher at the College of Education, King Khalid University, Abha, Kingdom of Saudi Arabia.

JAN C.C. BORLEFFS, MD, PhD, is an internist and Professor of Medical Education and Dean of Education at the University of Groningen, University Medical Center Groningen, The Netherlands.

MARJOLEIN HEIJNE-PENNINGA, PhD, is an educational psychologist and a researcher at the University Medical Center Groningen, and at the Research and Innovation Group Talent Development in Higher Education and Society, Hanze University of Applied Sciences, Groningen, The Netherlands.

JOHANNA SCHÖNROCK-ADEMA, PhD, is a psychologist and researcher at the Center for Research and Innovation in Medical Education, University of Groningen, University Medical Center Groningen, The Netherlands.

The publication of this supplement has been made possible with the generous financial support of the Dr Hamza Alkholi Chair for Developing Medical Education in KSA.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.